Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-37160

Update ts_wep test data

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: ts_aos
    • Labels:

      Description

      The test data for ts_wep suffered bit rot: sources for extra-focal corner sensors were simulated where no donuts appear given the WCS information. Image for one corner sensor using existing test data is attached. This ticket is to update the content of raw corner sensor images (in tests/testData/phosimOutput/realWfs/output/). Previously these files were a result of running imgCloseLoop (0-th iteration).

        Attachments

          Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. DM-37063 Fix donut template tests ts_wep, ts_phosim

          • Done

          Improvement - An improvement or enhancement to an existing feature or task. DM-37130 Update default imgCloseLoop ISR settings

          • Done

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            ksuberlak Krzysztof Suberlak added a comment - - edited

            Running imgCloseLoop with the existing `skyRefCat` as the instance catalog in phosimDir (slightly modified to allow for the ra range recognized by phosim: from 0:360 to -180:180 degrees) and substituting the fits files in ts_wep/tests/testData/phosimOutput/realWfs/output with those of phosimDir/iter0/img/wfs/ fixes the issue: running again content of test_cutOutDonutsCwfsTask we see that now sources are where they are expected to be, so that no donut stamps are empty (based on the WCS information):

            Show
            ksuberlak Krzysztof Suberlak added a comment - - edited Running imgCloseLoop with the existing `skyRefCat` as the instance catalog in phosimDir (slightly modified to allow for the ra range recognized by phosim: from 0:360 to -180:180 degrees) and substituting the fits files in ts_wep/tests/testData/phosimOutput/realWfs/output with those of phosimDir/iter0/img/wfs/ fixes the issue: running again content of test_cutOutDonutsCwfsTask we see that now sources are where they are expected to be, so that no donut stamps are empty (based on the WCS information):
            Hide
            Permalink
            ksuberlak Krzysztof Suberlak added a comment -

            Another concern is that this problem went undetected for an unknown amount of time. This is because the tests using this static version of phosim images are too porous, and empty donut stamps did not make the tests fail. 

            For instance, test_calcZernikesTaskCwfs.py  passes because :

            • testEstimateZernikes only checks if there's 19 Zks, and there is, even we pass two empty donut stamps
            • testEstimateCornerZernikes uses pre-computed R04_SW0_donutStamps.fits which are not a result of running the pipeline that includes generateDonutCatalogWcsTask
            • testGetCombinedZernikes again only checks if there's 19 Zernikes, but not if they make any sense

            Similarly, test_estimateZernikesCwfsTask.py passes because:

            • testTaskRunNormal /testTaskRunNormal only checks if running the task in real time) contains the same donut stamps / Zernikes as when reading from butler the results of running the task as a pipetask - it doesn't test if they make sense. Empty donut stamps pass this test.

            But this should be addressed as a new ticket.

            Show
            ksuberlak Krzysztof Suberlak added a comment - Another concern is that this problem went undetected for an unknown amount of time. This is because the tests using this static version of phosim images are too porous, and empty donut stamps did not make the tests fail.  For instance, test_calcZernikesTaskCwfs.py  passes because : testEstimateZernikes only checks if there's 19 Zks, and there is, even we pass two empty donut stamps testEstimateCornerZernikes uses pre-computed R04_SW0_donutStamps.fits which are not a result of running the pipeline that includes generateDonutCatalogWcsTask testGetCombinedZernikes again only checks if there's 19 Zernikes, but not if they make any sense Similarly, test_estimateZernikesCwfsTask.py passes because: testTaskRunNormal /testTaskRunNormal only checks if running the task in real time) contains the same donut stamps / Zernikes as when reading from butler the results of running the task as a pipetask - it doesn't test if they make sense. Empty donut stamps pass this test. But this should be addressed as a new ticket.
            Hide
            Permalink
            ksuberlak Krzysztof Suberlak added a comment -

            PR with updated test files for corner sensors merged on github.

            Show
            ksuberlak Krzysztof Suberlak added a comment - PR with updated test files for corner sensors merged on github.
            Hide
            Permalink
            ttsai Te-Wei Tsai added a comment -

            Reviewed in GitHub.

            Show
            ttsai Te-Wei Tsai added a comment - Reviewed in GitHub.

              People

              Assignee:
              ksuberlak Krzysztof Suberlak
              Reporter:
              ksuberlak Krzysztof Suberlak
              Reviewers:
              Te-Wei Tsai
              Watchers:
              Andrew Connolly, Bryce Kalmbach, Krzysztof Suberlak, Te-Wei Tsai
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.