Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-26364

Science Pipelines Release v20 Acceptance Test Campaign

    XMLWordPrintable

    Details

      Description

      Carry out acceptance testing of v20.0 Science Pipelines Release.

        Attachments

          Issue Links

          is blocked by

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-25646 Create acceptance test plan for science pipelines R20

          • Done

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            jcarlin Jeffrey Carlin added a comment -

            DMTR-261 is at this link.

            Show
            jcarlin Jeffrey Carlin added a comment - DMTR-261 is at this link .
            Hide
            Permalink
            gcomoretto Gabriele Comoretto [X] (Inactive) added a comment -

            For extended comments on the content, please use the PR:

            https://github.com/lsst-dm/DMTR-261/pull/2

            Show
            gcomoretto Gabriele Comoretto [X] (Inactive) added a comment - For extended comments on the content, please use the PR: https://github.com/lsst-dm/DMTR-261/pull/2
            Hide
            Permalink
            lguy Leanne Guy added a comment -

            Nice work and great too see 14 newly verified test cases!

            For the recommended improvements, we should create tickets to schedule this work.

            Minor comments to address then this is done:

            • P1: This Acceptance Test campaign aims to verify a subset of DMSR (LSE-61) requirements related ...
              > This Acceptance Test campaign aims to verify a subset of DM subsystem requirements (LSE-61) related to ....
            • P1 readiness for LSST Operations => readiness for LSST commissioning .
              (commissioning verifies readiness for operations)
            • P1: The tests will be performed using the HSC-RC2 dataset (as defined in DM-11345 ).
              Reference should be DMTN-091: Test Datasets for Scientific Performance Monitoring. If DMTN-091 is not correct, we must update it.
            • P45: 5.1.3.11 LVV-T1757 – "Note that because the science pipelines are still in development, the actual resulting values for these metrics are not important." -> I would not phrase it as "not important", rather, that the values obtained are not the focus of this test and that the actual performance will be verified during commissioning on LSST data using this code. We do however expect to get reasonable results with precursor data and if, for example, we were seeing mag-level numbers for PA1 on HSC, we'd be looking for bugs.
            • Likewise for LVV-T1758, LVV-T1756, LVV-T1759
            • P 7, 56: A reference has not rendered "and ingested as part of the Operations Rehearsal detailed in ? : “LDM-503-11a: ComCam
              OPS Readiness Test Plan and Report”
            Show
            lguy Leanne Guy added a comment - Nice work and great too see 14 newly verified test cases! For the recommended improvements, we should create tickets to schedule this work. Minor comments to address then this is done: P1: This Acceptance Test campaign aims to verify a subset of DMSR (LSE-61) requirements related ... > This Acceptance Test campaign aims to verify a subset of DM subsystem requirements (LSE-61) related to .... P1 readiness for LSST Operations => readiness for LSST commissioning . (commissioning verifies readiness for operations) P1: The tests will be performed using the HSC-RC2 dataset (as defined in DM-11345 ). Reference should be DMTN-091: Test Datasets for Scientific Performance Monitoring. If DMTN-091 is not correct, we must update it. P45: 5.1.3.11 LVV-T1757 – "Note that because the science pipelines are still in development, the actual resulting values for these metrics are not important." -> I would not phrase it as "not important", rather, that the values obtained are not the focus of this test and that the actual performance will be verified during commissioning on LSST data using this code. We do however expect to get reasonable results with precursor data and if, for example, we were seeing mag-level numbers for PA1 on HSC, we'd be looking for bugs. Likewise for LVV-T1758, LVV-T1756, LVV-T1759 P 7, 56: A reference has not rendered "and ingested as part of the Operations Rehearsal detailed in ? : “LDM-503-11a: ComCam OPS Readiness Test Plan and Report”
            Hide
            Permalink
            jcarlin Jeffrey Carlin added a comment -

            I have updated the document(s) to address these comments, and created a ticket (DM-25786) to address the first of the recommended improvements. The second suggested improvement is being addressed by the new metrics tooling that is being developed, and the third recommendation (update LVV-T125 to test fake source insertion more carefully) has a ticket assigned to it (DM-25787).

            If the links to references render properly, then DMTR-261 should be complete.

            Show
            jcarlin Jeffrey Carlin added a comment - I have updated the document(s) to address these comments, and created a ticket ( DM-25786 ) to address the first of the recommended improvements. The second suggested improvement is being addressed by the new metrics tooling that is being developed, and the third recommendation (update LVV-T125 to test fake source insertion more carefully) has a ticket assigned to it ( DM-25787 ). If the links to references render properly, then DMTR-261 should be complete.
            Hide
            Permalink
            jcarlin Jeffrey Carlin added a comment - - edited

            Final version of DMTR-261 is available in Docushare.

            Show
            jcarlin Jeffrey Carlin added a comment - - edited Final version of DMTR-261 is available in Docushare.

              People

              Assignee:
              jcarlin Jeffrey Carlin
              Reporter:
              jcarlin Jeffrey Carlin
              Reviewers:
              Leanne Guy
              Watchers:
              Gabriele Comoretto [X] (Inactive), Jeffrey Carlin, Leanne Guy
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Due:
                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.