Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-11594

test_distortion fails when run from pytest

    Details

      Description

      test_distortion.py fails when run from pytest while unpickling:

      >           savedData = pickle.load(dataFile)
      E           AttributeError: Can't get attribute 'CcdData' on <module '__main__' from '/Users/rowen/UW/LSST/lsstsw3/miniconda/bin/pytest'>
      

        Attachments

          Issue Links

            Activity

            Hide
            rowen Russell Owen added a comment -

            I hope there is some way to get pytest to work with pickle.

            If not, it will probably work to persist the data as a dict using json. There are two ways to do that:

            • Continue to use namedtuple or custom classes for convenient access to attributes (item.foo instead of item["foo"]) but write a translator between that and a dict.
            • Use a dict for everything.

            Either way, it'll be a headache when we switch from pupil to field angle unless we persist the json data using field angle now. That's important because I'd really like to continue to use the persisted test data for the new camera geometry distortion model in DM-5922 (which must use field angle).

            Show
            rowen Russell Owen added a comment - I hope there is some way to get pytest to work with pickle. If not, it will probably work to persist the data as a dict using json. There are two ways to do that: Continue to use namedtuple or custom classes for convenient access to attributes (item.foo instead of item ["foo"] ) but write a translator between that and a dict. Use a dict for everything. Either way, it'll be a headache when we switch from pupil to field angle unless we persist the json data using field angle now. That's important because I'd really like to continue to use the persisted test data for the new camera geometry distortion model in DM-5922 (which must use field angle).
            Hide
            rowen Russell Owen added a comment -

            Paul Price came up with a brilliantly simple fix: use lsst.pipe.base.Struct instead of named tuples. This class is in a known location so pytest can find it.

            I also made a few others changes:

            • Fixed the test for round tripping distortion computations so it tests round-tripped values against the original values instead of whatever the reverse computation happened to compute when the data was persisted. This exposed the fact that the round trip computation is only good to about 0.007 focal plane units (which are normally mm, but HSC uses pixels).
            • Changed coordinate system name "pupil" to "field angle" everywhere I could, since that is the new agreed-upon term and will allow the data to be used sensibly once we convert in DM-5922.
            • Changed one dict key from pixel position (as a tuple) to a string generated from pixel position, to aid future conversion to json (all dict keys are now strings). As part of this I added pixel position to the value of the dict in question.
            Show
            rowen Russell Owen added a comment - Paul Price came up with a brilliantly simple fix: use lsst.pipe.base.Struct instead of named tuples. This class is in a known location so pytest can find it. I also made a few others changes: Fixed the test for round tripping distortion computations so it tests round-tripped values against the original values instead of whatever the reverse computation happened to compute when the data was persisted. This exposed the fact that the round trip computation is only good to about 0.007 focal plane units (which are normally mm, but HSC uses pixels). Changed coordinate system name "pupil" to "field angle" everywhere I could, since that is the new agreed-upon term and will allow the data to be used sensibly once we convert in DM-5922 . Changed one dict key from pixel position (as a tuple) to a string generated from pixel position, to aid future conversion to json (all dict keys are now strings). As part of this I added pixel position to the value of the dict in question.
            Hide
            tjenness Tim Jenness added a comment -

            looks good to me. Thanks.

            Show
            tjenness Tim Jenness added a comment - looks good to me. Thanks.

              People

              • Assignee:
                rowen Russell Owen
                Reporter:
                rowen Russell Owen
                Reviewers:
                Tim Jenness
                Watchers:
                Paul Price, Russell Owen, Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel