Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-26824

Enhancements to the alert streaming system

    Details

    • Type: Epic
    • Status: To Do
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Epic Name:
      future-ap-alertstream
    • WBS:
      02C.03.02
    • Team:
      Alert Production
    • Cycle:
      Spring 2021

      Description

      This epic captures potential enhancements to the alert streaming system and associated simulator. These were initially captured as part of work on DM-25143, but were placed on hold due to staff availability in late calendar year 2020.

        Attachments

        Stories in Epic (Custom Issue Matrix)

        Key Summary Story Points Assignee Status
        DM-24858

        Pick a scheme for alert IDs

        1 Eric Bellm To Do
        DM-25281

        Use setuptools to install & build alert packet under EUPS

        1 Unassigned To Do
        DM-24647

        Add dashboards for alert stream simulator to measure Kafka throughput

        2 Spencer Nelson To Do
        DM-24306

        Add integration tests to alert_stream

        Unassigned To Do
        DM-24305

        Prototype a confluent schema registry instead of building our own registry

        Unassigned To Do
        DM-24304

        Investigate Kafka compression of the alert stream

        Unassigned To Do
        DM-24303

        Add automated linting/testing workflows to alert_stream

        Unassigned To Do
        DM-24301

        Rename sample-avro-alert

        Unassigned To Do
        DM-24293

        Convert alert schemas into canonical form before fingerprinting them

        Unassigned To Do
        DM-24291

        Make lsst.alert.packet pip-installable and conda-installable

        Unassigned To Do
        DM-24283

        Add documentation to the avro schema for alert distribution

        Eric Bellm To Do
        DM-24289

        Add automated linting/testing workflows to sample-avro-alert

        2 Unassigned To Do
        DM-24284

        Generate documentation on the alert distribution message schema

        Unassigned To Do
        DM-22007

        Design sketch for alert packet signing

        1 Unassigned To Do
        DM-19606

        Rethink alert distribution dockerization

        Unassigned To Do
        DM-19597

        Provide Avro-alert pretty-printer

        1 Unassigned To Do
        DM-19590

        Standardize alert distribution system deployment tools

        Unassigned To Do
        DM-19589

        Audit dependencies in lsst-dm/alert_stream/Dockerfile

        Unassigned To Do
        DM-19588

        Develop test mechanisms for alert distribution system

        8 Unassigned To Do
        DM-19587

        Overhaul lsst.alert.packet test suite

        5 Unassigned To Do
        DM-19586

        Adopt a plugin system for alert filters

        Unassigned To Do
        DM-19557

        Review docker-compose.yml settings for alert_stream

        1 Unassigned To Do
        DM-19257

        validateAvroRoundTrip.py should be a test case

        1 Unassigned To Do
        DM-19255

        Allow more flexibility and realism when generating pseudorandom alert packets

        1 Unassigned To Do
        DM-26063

        Create a user guide page for alert_packet on pipelines.lsst.io

        4 Eric Bellm To Do
        DM-17551

        Update example alert data to current schema

        2 Unassigned To Do
        DM-26099

        Review alert_packet schema handling

        6 Spencer Nelson To Do
        DM-24679

        Replace confluent_kafka with a python Kafka library that is more robust for alert-stream-simulator

        2 Spencer Nelson To Do
        DM-25103

        Distribute alert_stream via conda

        2 Spencer Nelson To Do
        DM-24648

        Validate that inputs match the right schema in alert stream simulator

        0.5 Spencer Nelson In Progress
         
        DM-19264

        Upstream fastavro schema resolution

        4 John Swinbank Won't Fix

          Activity

          There are no comments yet on this issue.

            People

            • Assignee:
              sullivan Ian Sullivan
              Reporter:
              swinbank John Swinbank
              Watchers:
              John Swinbank
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:

                Summary Panel