Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-14067

Incomplete flowdown of OSS provenance requirements to DMSR

    Details

    • Templates:
    • Sprint:
      Arch 2018-09-24, Arch 2018-10-08, Arch 2018-10-15, Arch 2018-10-22, Arch 2018-10-29, Arch 2018-11-05, Arch 2018-11-12, Arch 2018-11-19, Arch 2018-11-26
    • Team:
      Architecture

      Description

      In reviewing the flowdown for provenance requirements on SuperTask and the Butler-related middleware, I noticed that the high-level requirements OSS-REQ-0122, Provenance:

      The LSST Data Management system shall record provenance data on all its processing activities: all information necessary to reproduce computed data products from the associated raw data, and to understand the processing history of any data product.

      This shall include at least: software version and build information, settings of all configurable parameters, history of processing steps, identification of all calibration constants used in processing, and hardware and operating system configurations used.

      and OSS-REQ-0172, Provenance Archiving:

      The LSST project shall archive all processing provenance associated with archived data products.

      were never properly flowed down to the DMSR. There are numerous DMSR requirements that mention provenance - regarding maintaining it for specific data products, such as coadds (DMS-REQ-0106), or querying against it, and one that mandates a generic provenance mechanism usable in "Level 3" analyses. But there are no explicit functional requirements at DMSR on the creation of the mechanisms or their features.

      This makes it hard to verify their flowdown to the middleware requirements, LDM-556, at which level we currently cover some, but not all, of the features required to meet the OSS requirements.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                gcomoretto Gabriele Comoretto
                Reporter:
                gpdf Gregory Dubois-Felsmann
                Watchers:
                Brian Van Klaveren, Gabriele Comoretto, Gregory Dubois-Felsmann, Kian-Tat Lim, Leanne Guy, Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel