Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-20911

Implement salobj to Kafka feeder

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: ts_middleware
    • Labels:
    • Story Points:
      4
    • Epic Link:
    • Sprint:
      TSSW Sprint - Aug 19 - Aug 31
    • Team:
      Telescope and Site

      Description

      Implement the salobj to Kafka feeder as designed in DM-20910, including documentation and unit tests.

        Attachments

          Issue Links

          is triggered by

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-20910 Design salobj to Kafka feeder

          • Done
          relates to

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-21041 Test SAL Kafka Producer for AuxTel

          • Done

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            rowen Russell Owen added a comment - - edited

            salobj pull request: https://github.com/lsst-ts/ts_salobj/pull/56
            served documentation: https://ts-salkafka.lsst.io/v/DM-20911

            Show
            rowen Russell Owen added a comment - - edited salobj pull request: https://github.com/lsst-ts/ts_salobj/pull/56 served documentation: https://ts-salkafka.lsst.io/v/DM-20911
            Hide
            Permalink
            rowen Russell Owen added a comment - - edited

            Jonathan Sick please review my use of Kafka, including in mocks and unit test. Note that I use a real serializer and test that it fails in test_kafka_info.py

            Tiago Ribeiro please review the rest of the code, basically everything except the Kafka code (feel free to look at that, as well).

            Also I beg one of you to please run the producer to make sure that I have not broken anything. I tried running from home using my usual standard commands:

            run_salkafka_producer.py Test --broker efd0.lsst.codes:9094  --registry https://registry-efd.lsst.codes/ &

            followed by

            python make_sal_data.py

            (I will attach the latter file to this ticket; it is not in the repo)
            but the first command failed with:

            kafkit.registry.errors.RegistryBadRequestError: Registry error (409). 409 - Schema being registered is incompatible with an earlier schema

            Show
            rowen Russell Owen added a comment - - edited Jonathan Sick please review my use of Kafka, including in mocks and unit test. Note that I use a real serializer and test that it fails in test_kafka_info.py Tiago Ribeiro please review the rest of the code, basically everything except the Kafka code (feel free to look at that, as well). Also I beg one of you to please run the producer to make sure that I have not broken anything. I tried running from home using my usual standard commands: run_salkafka_producer.py Test --broker efd0.lsst.codes:9094 --registry https://registry-efd.lsst.codes/ & followed by python make_sal_data.py (I will attach the latter file to this ticket; it is not in the repo) but the first command failed with: kafkit.registry.errors.RegistryBadRequestError: Registry error (409). 409 - Schema being registered is incompatible with an earlier schema
            Hide
            Permalink
            jsick Jonathan Sick added a comment -

            This looks great from my perspective. I've got some comments on the PR at https://github.com/lsst-ts/ts_salkafka/pull/1

            Show
            jsick Jonathan Sick added a comment - This looks great from my perspective. I've got some comments on the PR at https://github.com/lsst-ts/ts_salkafka/pull/1
            Hide
            Permalink
            rowen Russell Owen added a comment -

            Tiago approved it on github

            Show
            rowen Russell Owen added a comment - Tiago approved it on github
            Hide
            Permalink
            rowen Russell Owen added a comment -

            Tiago added automatic checking and doc build.

            Merged to develop and master and released as v1.0b1

            Show
            rowen Russell Owen added a comment - Tiago added automatic checking and doc build. Merged to develop and master and released as v1.0b1

              People

              Assignee:
              rowen Russell Owen
              Reporter:
              rowen Russell Owen
              Reviewers:
              Tiago Ribeiro
              Watchers:
              Angelo Fausti, Jonathan Sick, Russell Owen, Tiago Ribeiro
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.