Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-13510

Correct inconsistencies in LDM-503 text and tables and improve auto-generation process

    Details

      Description

      The existing LDM-503 package has traces of a strategy to autogenerate substantial sections of the document from a milestone table maintained as the dmtestmilstones.csv file (as well as other portions from other .csv files).  In particular, from this file a dmtestmilstones.tex file is generated, for use in schedtab.tex in Section 6 of the document, as well as a file testsections.tex used as the body of Section 7 of the document.

      There are signs that both of those .tex files were hand-edited after the last time the autogeneration was performed, and these edits are at variance with each other, with the outcome that there are three versions, slightly differing, of the associated text.

      The auto-generation script is not part of the Makefile, and its outputs are part of the Github acquis for the package, or this would have turned out differently.

      Note also that the autogeneration script has a provision to substitute a longer description of a milestone into testsections.tex if one is available.  Currently this is only the case for LDM-503-2, for which there is an LDM-503-2.tex file in the package (and a much more extensive f17_drp.tex file which it in turn includes).  This longer description appears to predate the fuller, separate test specification for DRP and may be redundant or even in conflict with it; I have not checked that.

        Attachments

          Issue Links

            Activity

            Hide
            gpdf Gregory Dubois-Felsmann added a comment -

            I have done some of this work incidentally to trying to make some substantive changes to LDM-503.  It's gotten a bit beyond what I'm comfortable completing on my own, but I'll make a ticket branch for it as a starting point for the remaining work.

            Show
            gpdf Gregory Dubois-Felsmann added a comment - I have done some of this work incidentally to trying to make some substantive changes to LDM-503.  It's gotten a bit beyond what I'm comfortable completing on my own, but I'll make a ticket branch for it as a starting point for the remaining work.
            Hide
            swinbank John Swinbank added a comment -

            On tickets/DM-13510, we're now autogenerating the milestone table, the testsections.tex and even the associated Gantt chart. Before closing this issue, I want to do some sanity checking, and to include a way to reference the test specifications associated with each milestone.

            Show
            swinbank John Swinbank added a comment - On tickets/ DM-13510 , we're now autogenerating the milestone table, the testsections.tex and even the associated Gantt chart. Before closing this issue, I want to do some sanity checking, and to include a way to reference the test specifications associated with each milestone.
            Hide
            swinbank John Swinbank added a comment - - edited

            All changes on https://github.com/lsst/LDM-503/pull/10.

            Unfortunately, the update process is still fairly manual, as there's no API for extracting data from PMCS — instead, we have to export an Excel sheet, then convert it to CSV. However, I've automated things as much as I can.

            This includes generating the Gantt-like milestones chart on the fly. There's currently a version of that chart in the images repository, which is now obsolete — do you want to replace it with a version based on the changes here? Or remove it entirely? I'd rather not be trying to autogenerate from PMCS in two separate repositories...

            Show
            swinbank John Swinbank added a comment - - edited All changes on https://github.com/lsst/LDM-503/pull/10 . Unfortunately, the update process is still fairly manual, as there's no API for extracting data from PMCS — instead, we have to export an Excel sheet, then convert it to CSV. However, I've automated things as much as I can. This includes generating the Gantt-like milestones chart on the fly. There's currently a version of that chart in the images repository, which is now obsolete — do you want to replace it with a version based on the changes here? Or remove it entirely? I'd rather not be trying to autogenerate from PMCS in two separate repositories...
            Hide
            swinbank John Swinbank added a comment -

            Merged following review on GitHub.

            Also updated the DM master schedule in lsst-dm/images to use a PDF generated using the scripts in lsst/ldm-503.

            Show
            swinbank John Swinbank added a comment - Merged following review on GitHub. Also updated the DM master schedule in lsst-dm/images to use a PDF generated using the scripts in lsst/ldm-503 .

              People

              • Assignee:
                swinbank John Swinbank
                Reporter:
                gpdf Gregory Dubois-Felsmann
                Reviewers:
                Wil O'Mullane
                Watchers:
                Gregory Dubois-Felsmann, John Swinbank, Wil O'Mullane
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel