Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-15512

Provide estimated outbound bandwidth that may be allocated from the LDF for alert streams

    Details

    • Type: Story
    • Status: To Do
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: Alert Production, LDF
    • Labels:
      None
    • Templates:
    • Team:
      Data Facility

      Description

      The Alert Production pipelines will produce a real-time stream of transient alerts. The full stream will be sent to several community brokers. It will also be processed by the LSST mini-broker in the data facility, and the filtered outputs sent to individual science users.

      Both of these services rely on the availability of network bandwidth from NCSA to the public cloud. While the size of the alert stream produced by AP may be estimated to within perhaps 25%, there is currently no clear budget for the bandwidth that the Data Facility will reserve for alert stream services. This omission is now hindering both the selection of community brokers as well as capacity and operations planning for the mini-broker.

      Mario Juric reports that an original budget was developed by Mike Freemon of NCSA, perhaps based on the assumptions 2007-era Data Access Whitepaper (Document-5373). LSE-78 describes all user access to DAC services as sharing one 10 GbE link; it is unclear if this is inclusive of the full or filtered alert streams.

      This ticket is to provide an updated estimate of the outbound network bandwidth which may be reserved for near-real-time alert transport. I (Eric Bellm) suggest that the budget be shared between full streams to community brokers and the mini-broker, which will allow for future trades between the number of community brokers and the number of supported users of the LSST mini-broker.

        Attachments

          Container Issues

            Issue Links

              Activity

                People

                • Assignee:
                  mbutler Michelle Butler
                  Reporter:
                  ebellm Eric Bellm
                  Watchers:
                  Eric Bellm, John Swinbank, Maria Patterson [X] (Inactive)
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  3 Start watching this issue

                  Dates

                  • Created:
                    Updated:

                    Summary Panel