Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-28737

Write design document for alert database

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: alert_stream
    • Labels:
      None
    • Story Points:
      9
    • Epic Link:
    • Sprint:
      AP S21-6 (May)
    • Team:
      Alert Production
    • Urgent?:
      No

      Description

      Review: https://confluence.lsstcorp.org/display/~ebellm/Alert+Database+Use+Cases+and+Draft+Requirements
      Draft a tech note outlining potential technical implementations and tradeoffs in an alert database.

        Attachments

          Activity

          Ascending order - Click to sort in descending order
          Hide
          Permalink
          swnelson Spencer Nelson added a comment -

          I think this is ready for a look. It's pretty simple now - it could maybe be shortened, in fact.

          Show
          swnelson Spencer Nelson added a comment - I think this is ready for a look. It's pretty simple now - it could maybe be shortened, in fact.
          Hide
          Permalink
          swnelson Spencer Nelson added a comment -

          I was off by a factor of... er... 100... in data volume. It's about 390TB/yr to do naive storage of an Avro Object Container stored per alert (150TB of schema, 240TB of data) uncompressed. This might take a bit more time to think through - that's a lot more than 2.4TB/yr!

          Show
          swnelson Spencer Nelson added a comment - I was off by a factor of... er... 100... in data volume. It's about 390TB/yr to do naive storage of an Avro Object Container stored per alert (150TB of schema, 240TB of data) uncompressed. This might take a bit more time to think through - that's a lot more than 2.4TB/yr!
          Hide
          Permalink
          swnelson Spencer Nelson added a comment -

          (this is DMTN-183: https://github.com/lsst-dm/dmtn-183)

          Show
          swnelson Spencer Nelson added a comment - (this is DMTN-183: https://github.com/lsst-dm/dmtn-183 )
          Hide
          Permalink
          swnelson Spencer Nelson added a comment -

          I think this is ready for another look.

          Show
          swnelson Spencer Nelson added a comment - I think this is ready for another look.
          Hide
          Permalink
          ktl Kian-Tat Lim added a comment -

          Overall looks good. I have qualms about explicitly defining the Butler as the interface, however. Removing myself from the reviewer list.

          Show
          ktl Kian-Tat Lim added a comment - Overall looks good. I have qualms about explicitly defining the Butler as the interface, however. Removing myself from the reviewer list.
          Hide
          Permalink
          ebellm Eric Bellm added a comment -

          I had a couple minor suggestions but otherwise I think it looks good!

          Show
          ebellm Eric Bellm added a comment - I had a couple minor suggestions but otherwise I think it looks good!

            People

            Assignee:
            swnelson Spencer Nelson
            Reporter:
            sullivan Ian Sullivan
            Reviewers:
            Eric Bellm
            Watchers:
            Eric Bellm, Ian Sullivan, Kian-Tat Lim, Spencer Nelson
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Jenkins

                No builds found.