Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-19705

Evaluate/Demonstrate DM-EFD capability to service TS&S specs

    Details

    • Type: Epic
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Epic Name:
      sqre-f19-tss-efd
    • Story Points:
      35
    • WBS:
      1.02C.10.02
    • Team:
      SQuaRE
    • Cycle:
      Fall 2019

      Description

      Following the work described in DM-16249 to provide a solution to expose telemetry data to the science platform using a kafka-avro-influxdb stack, we want to characterise the latency and throughput of this system to see if it can meet the 50Hz rate expected from the real summit systems to see if it could be used to provide a complete non-aggregated tap to the telemetry stream. 

        Attachments

          Activity

          Hide
          afausti Angelo Fausti added a comment - - edited

          We have tested the DM-EFD system with the M1M3 simulator and after optimizing SAL transform we showed that our solution is performant and in particular, can keep up with the 50Hz M1M3 telemetry stream.

          The results of those tests are reported in SQR-029.

          We deployed the DM-EFD at the Tucson test stand on a single machine using k3s (kubes) the deployment was successful but we could not test with the ATCam subsystem yet because the SAL version running at the lab does not include the kafka writers. We are waiting on SAL 3.9 or later to be deployed at the lab.

          Instructions for the DM-EFD deployment on k3s, operations and example of use are documented in SQR-031

          We also worked with NSCA to add the JDBC connector to our cluster and configure it to write to the Oracle database. That ticket is blocked on Oracle DB access.

          Show
          afausti Angelo Fausti added a comment - - edited We have tested the DM-EFD system with the M1M3 simulator and after optimizing SAL transform we showed that our solution is performant and in particular, can keep up with the 50Hz M1M3 telemetry stream. The results of those tests are reported in SQR-029 . We deployed the DM-EFD at the Tucson test stand on a single machine using k3s (kubes) the deployment was successful but we could not test with the ATCam subsystem yet because the SAL version running at the lab does not include the kafka writers. We are waiting on SAL 3.9 or later to be deployed at the lab. Instructions for the DM-EFD deployment on k3s, operations and example of use are documented in SQR-031 We also worked with NSCA to add the JDBC connector to our cluster and configure it to write to the Oracle database. That ticket is blocked on Oracle DB access.
          Hide
          frossie Frossie Economou added a comment -

          This epic was brought in early into S19 in order to take explore (ultimate successfully) the opportunity that our DM-EFD design was suitable for the TS&S usecase (ie retention of telemetry data at 50Hz).It displaced DM-17034 which will be completed in F19. 

          Show
          frossie Frossie Economou added a comment - This epic was brought in early into S19 in order to take explore (ultimate successfully) the opportunity that our DM-EFD design was suitable for the TS&S usecase (ie retention of telemetry data at 50Hz).It displaced DM-17034 which will be completed in F19. 

            People

            • Assignee:
              afausti Angelo Fausti
              Reporter:
              frossie Frossie Economou
              Reviewers:
              Frossie Economou
              Watchers:
              Angelo Fausti, Frossie Economou
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Summary Panel