Uploaded image for project: 'Request For Comments'
  1. Request For Comments
  2. RFC-903

Archive / deprecate LDM-153 and update DPDD tables

    XMLWordPrintable

    Details

      Description

      RFC-894 proposes to remove the  LSST Baseline Schema from the Schema browser  and to archive sdm_schemas/yml/baseline.yaml as it is not used in any database. The question remains, what to do with LDM-153 and the DPDD. 
       
      The LSST Baseline Schema, rendered in LDM-153, is an aspirational schema that provides a  concrete realization that complies with the data products described in the  DPDD.  It was used by science pipelines to guide early development and  to produce the sizing model but is no longer up to date with the actual production data model and no longer necessary for the sizing model.  Keeping this aspirational schema up to date is a lot of work and provides diminishing return on investment.  We hence propose to freeze and archive baseline schema and LDM-153. LDM-153 is still informative as a historical document,  but should no longer be used and we should not refer people to it. 
       
      The DPDD is in essence a policy statement aimed at the community that describes what LSST data products they can expect. The tables in the DPDD describe conceptual schemas for the data products, i.e what we will deliver but not the implementation details.  These tables  are useful and we propose to keep these in the DPDD. To make the DPDD conceptual schemas easier to manage and more widely useful, we propose to take them out of their current latex format and store them in a machine-readable source, e.g yaml, and to generate the latex tables needed for the DPDD, and possibly other formats e.g HTML. We also propose to remove the "Type" column from the DPDD tables as it is not prescriptive nor useful to the science user, and can give the erroneous impression that that the conceptual schema represents a concrete implementation. ** 

        Attachments

          Issue Links

            Activity

            Hide
            Parejkoj John Parejko added a comment -

            A concern is that more than just the "type" column is going to be different between the DPDD and the production tables. Column names have changed from the DPDD, as have some units. I think it will be a challenge to make a yaml file that is generic enough.

            Show
            Parejkoj John Parejko added a comment - A concern is that more than just the "type" column is going to be different between the DPDD and the production tables. Column names have changed from the DPDD, as have some units. I think it will be a challenge to make a yaml file that is generic enough.
            Hide
            lguy Leanne Guy added a comment -

            Yes it is. The column names in the tables in the DPDD are not prescriptive either but we need  some name to describe the quantities that will be delivered. The 'Name' also serves to link information between tables in the DPDD. I could change 'Name' to 'Indicative Name' (or something to that effect). 

            Show
            lguy Leanne Guy added a comment - Yes it is. The column names in the tables in the DPDD are not prescriptive either but we need  some name to describe the quantities that will be delivered. The 'Name' also serves to link information between tables in the DPDD. I could change 'Name' to 'Indicative Name' (or something to that effect). 
            Hide
            lguy Leanne Guy added a comment -

            There are also several references to LDM-153 that would need to be removed from the DPDD if we archive LDM-153

            Show
            lguy Leanne Guy added a comment - There are also several references to LDM-153 that would need to be removed from the DPDD if we archive LDM-153
            Hide
            tjenness Tim Jenness added a comment -

            DM-CCB is happy to approve marking LDM-153 as "obsolete" (there is a formal option for that in lsst-texmf). We are happy for the DPDD changes to proceed to LCR.

            Show
            tjenness Tim Jenness added a comment - DM-CCB is happy to approve marking LDM-153 as "obsolete" (there is a formal option for that in lsst-texmf). We are happy for the DPDD changes to proceed to LCR.

              People

              Assignee:
              lguy Leanne Guy
              Reporter:
              lguy Leanne Guy
              Watchers:
              Colin Slater, Eric Bellm, John Parejko, Krzysztof Findeisen, Leanne Guy, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Planned End:

                  Jenkins

                  No builds found.