Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-40394

SAL Kafka testing continues

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: Middleware
    • Labels:
      None

      Description

      SAL Kafka testing continues......Tony has pointed me at the Simple SAL repo where he re-used bits of SAL in his code generation so I will see if those can be incorporated directly instead.
      Tiago will bundle the schema json generation and Avro registration tools into ts_xml, so I will test that when it's ready. Also work more with the DM container which Angelo will update to include a schema registry and Kafdrop monitoring tools

        Attachments

          Activity

          Hide
          dmills Dave Mills added a comment -

          Tested using ts_xml branch for schema generation, after a few iterations with Tiago we have a stable base for SAL code generation
          using that. More testing with upgraded test programs which allow for tuning number and frequency of commands, events and telemetry generated.
          Now getting sub 5msec roundtrip times for sample commands (eg MTMount_command_trackTarget) on a sustained basis (8+ hours continuous track).

          This is with the original 2-ack strategy, so with only 1-ack it would be faster still.

          Also did some historical data testing and once I realized that the Confluence Kafka had a default retention period of ~1 minute, it also went well.

          Next sprint will focus on generating the RPMs and loading/testing on the TTS/Jenkins environments.

          Show
          dmills Dave Mills added a comment - Tested using ts_xml branch for schema generation, after a few iterations with Tiago we have a stable base for SAL code generation using that. More testing with upgraded test programs which allow for tuning number and frequency of commands, events and telemetry generated. Now getting sub 5msec roundtrip times for sample commands (eg MTMount_command_trackTarget) on a sustained basis (8+ hours continuous track). This is with the original 2-ack strategy, so with only 1-ack it would be faster still. Also did some historical data testing and once I realized that the Confluence Kafka had a default retention period of ~1 minute, it also went well. Next sprint will focus on generating the RPMs and loading/testing on the TTS/Jenkins environments.
          Hide
          dmills Dave Mills added a comment -

          Still need updated container so I can switch from Confluent.
          And need to agree with Rob on RPM naming issue resolution

          Show
          dmills Dave Mills added a comment - Still need updated container so I can switch from Confluent. And need to agree with Rob on RPM naming issue resolution
          Hide
          aclements Andy Clements added a comment -

          Looks excellent Dave!  Glad to see the progress.

          Show
          aclements Andy Clements added a comment - Looks excellent Dave!  Glad to see the progress.

            People

            Assignee:
            dmills Dave Mills
            Reporter:
            dmills Dave Mills
            Reviewers:
            Andy Clements
            Watchers:
            Andy Clements, Dave Mills
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Jenkins

                No builds found.