Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-37160

Update ts_wep test data

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: ts_aos
    • Labels:

      Description

      The test data for ts_wep suffered bit rot: sources for extra-focal corner sensors were simulated where no donuts appear given the WCS information. Image for one corner sensor using existing test data is attached. This ticket is to update the content of raw corner sensor images (in tests/testData/phosimOutput/realWfs/output/). Previously these files were a result of running imgCloseLoop (0-th iteration).

        Attachments

          Issue Links

            Activity

            No builds found.
            ksuberlak Krzysztof Suberlak created issue -
            ksuberlak Krzysztof Suberlak made changes -
            Field Original Value New Value
            Link This issue relates to DM-37063 [ DM-37063 ]
            ksuberlak Krzysztof Suberlak made changes -
            Link This issue relates to DM-37130 [ DM-37130 ]
            ksuberlak Krzysztof Suberlak made changes -
            Attachment extra_after_update.png [ 64914 ]
            Hide
            ksuberlak Krzysztof Suberlak added a comment - - edited

            Running imgCloseLoop with the existing `skyRefCat` as the instance catalog in phosimDir (slightly modified to allow for the ra range recognized by phosim: from 0:360 to -180:180 degrees) and substituting the fits files in ts_wep/tests/testData/phosimOutput/realWfs/output with those of phosimDir/iter0/img/wfs/ fixes the issue: running again content of test_cutOutDonutsCwfsTask we see that now sources are where they are expected to be, so that no donut stamps are empty (based on the WCS information):

            Show
            ksuberlak Krzysztof Suberlak added a comment - - edited Running imgCloseLoop with the existing `skyRefCat` as the instance catalog in phosimDir (slightly modified to allow for the ra range recognized by phosim: from 0:360 to -180:180 degrees) and substituting the fits files in ts_wep/tests/testData/phosimOutput/realWfs/output with those of phosimDir/iter0/img/wfs/ fixes the issue: running again content of test_cutOutDonutsCwfsTask we see that now sources are where they are expected to be, so that no donut stamps are empty (based on the WCS information):
            ksuberlak Krzysztof Suberlak made changes -
            Attachment Screen Shot 2022-11-30 at 2.38.36 PM.png [ 64913 ]
            ksuberlak Krzysztof Suberlak made changes -
            Attachment Screen Shot 2022-11-30 at 2.52.09 PM.png [ 64915 ]
            ksuberlak Krzysztof Suberlak made changes -
            Attachment Screen Shot 2022-11-30 at 2.52.09 PM.png [ 64915 ]
            ksuberlak Krzysztof Suberlak made changes -
            Attachment extra_before_update.png [ 64916 ]
            ksuberlak Krzysztof Suberlak made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            Hide
            ksuberlak Krzysztof Suberlak added a comment -

            Another concern is that this problem went undetected for an unknown amount of time. This is because the tests using this static version of phosim images are too porous, and empty donut stamps did not make the tests fail. 

            For instance, test_calcZernikesTaskCwfs.py  passes because :

            • testEstimateZernikes only checks if there's 19 Zks, and there is, even we pass two empty donut stamps
            • testEstimateCornerZernikes uses pre-computed R04_SW0_donutStamps.fits which are not a result of running the pipeline that includes generateDonutCatalogWcsTask
            • testGetCombinedZernikes again only checks if there's 19 Zernikes, but not if they make any sense

            Similarly, test_estimateZernikesCwfsTask.py passes because:

            • testTaskRunNormal /testTaskRunNormal only checks if running the task in real time) contains the same donut stamps / Zernikes as when reading from butler the results of running the task as a pipetask - it doesn't test if they make sense. Empty donut stamps pass this test.

            But this should be addressed as a new ticket.

            Show
            ksuberlak Krzysztof Suberlak added a comment - Another concern is that this problem went undetected for an unknown amount of time. This is because the tests using this static version of phosim images are too porous, and empty donut stamps did not make the tests fail.  For instance, test_calcZernikesTaskCwfs.py  passes because : testEstimateZernikes only checks if there's 19 Zks, and there is, even we pass two empty donut stamps testEstimateCornerZernikes uses pre-computed R04_SW0_donutStamps.fits which are not a result of running the pipeline that includes generateDonutCatalogWcsTask testGetCombinedZernikes again only checks if there's 19 Zernikes, but not if they make any sense Similarly, test_estimateZernikesCwfsTask.py passes because: testTaskRunNormal /testTaskRunNormal only checks if running the task in real time) contains the same donut stamps / Zernikes as when reading from butler the results of running the task as a pipetask - it doesn't test if they make sense. Empty donut stamps pass this test. But this should be addressed as a new ticket.
            Hide
            ksuberlak Krzysztof Suberlak added a comment -

            PR with updated test files for corner sensors merged on github.

            Show
            ksuberlak Krzysztof Suberlak added a comment - PR with updated test files for corner sensors merged on github.
            ksuberlak Krzysztof Suberlak made changes -
            Reviewers Te-Wei Tsai [ ttsai ]
            Status In Progress [ 3 ] In Review [ 10004 ]
            ksuberlak Krzysztof Suberlak made changes -
            Watchers Krzysztof Suberlak, Te-Wei Tsai [ Krzysztof Suberlak, Te-Wei Tsai ] Andrew Connolly, Bryce Kalmbach, Krzysztof Suberlak, Te-Wei Tsai [ Andrew Connolly, Bryce Kalmbach, Krzysztof Suberlak, Te-Wei Tsai ]
            Hide
            ttsai Te-Wei Tsai added a comment -

            Reviewed in GitHub.

            Show
            ttsai Te-Wei Tsai added a comment - Reviewed in GitHub.
            ttsai Te-Wei Tsai made changes -
            Status In Review [ 10004 ] Reviewed [ 10101 ]
            ksuberlak Krzysztof Suberlak made changes -
            Resolution Done [ 10000 ]
            Status Reviewed [ 10101 ] Done [ 10002 ]
            aclements Andy Clements made changes -
            Epic Link DM-35161 [ 1671607 ] DM-35163 [ 1671633 ]

              People

              Assignee:
              ksuberlak Krzysztof Suberlak
              Reporter:
              ksuberlak Krzysztof Suberlak
              Reviewers:
              Te-Wei Tsai
              Watchers:
              Andrew Connolly, Bryce Kalmbach, Krzysztof Suberlak, Te-Wei Tsai
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.