Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-10105

Inconsistency in meas/forced wcs leads to CModel failure

    Details

      Description

      From Sogo Mineo:

      During processing a dataset with multiBandDriver.py in hscPipe 5.0-beta1, I ran into an error:
       
      Traceback (most recent call last):
        File "/data/mineo/opt/hscpipe/5/stack_5.0-beta1/Linux64/ctrl_pool/13.0-1-g0a48345/python/lsst/ctrl/pool/pool.py", line 113, in wrapper
          return func(*args, **kwargs)
        File "/data/mineo/opt/hscpipe/5/stack_5.0-beta1/Linux64/ctrl_pool/13.0-1-g0a48345/python/lsst/ctrl/pool/pool.py", line 1067, in run
          while not menu[command]():
        File "/data/mineo/opt/hscpipe/5/stack_5.0-beta1/Linux64/ctrl_pool/13.0-1-g0a48345/python/lsst/ctrl/pool/pool.py", line 237, in wrapper
          return func(*args, **kwargs)
        File "/data/mineo/opt/hscpipe/5/stack_5.0-beta1/Linux64/ctrl_pool/13.0-1-g0a48345/python/lsst/ctrl/pool/pool.py", line 1085, in reduce
          result = self._processQueue(context, func, [(index, data)], *args, **kwargs)[0]
        File "/data/mineo/opt/hscpipe/5/stack_5.0-beta1/Linux64/ctrl_pool/13.0-1-g0a48345/python/lsst/ctrl/pool/pool.py", line 544, in _processQueue
          return self._reduceQueue(context, None, func, queue, *args, **kwargs)
        File "/data/mineo/opt/hscpipe/5/stack_5.0-beta1/Linux64/ctrl_pool/13.0-1-g0a48345/python/lsst/ctrl/pool/pool.py", line 570, in _reduceQueue
          resultList = [func(self._getCache(context, i), data, *args, **kwargs) for i, data in queue]
        File "/data/mineo/opt/hscpipe/5/stack_5.0-beta1/Linux64/pipe_drivers/13.0-1-g07ace30+4/python/lsst/pipe/drivers/multiBandDriver.py", line 388, in runForcedPhot
          self.forcedPhotCoadd.run(dataRef)
        File "/data/mineo/opt/hscpipe/5/stack_5.0-beta1/Linux64/meas_base/13.0-3-gc489882+1/python/lsst/meas/base/forcedPhotImage.py", line 147, in run
          self.measurement.run(measCat, exposure, refCat, refWcs, exposureId=self.getExposureId(dataRef))
        File "/data/mineo/opt/hscpipe/5/stack_5.0-beta1/Linux64/meas_base/13.0-3-gc489882+1/python/lsst/meas/base/forcedMeasurement.py", line 330, in run
          beginOrder=beginOrder, endOrder=endOrder)
        File "/data/mineo/opt/hscpipe/5/stack_5.0-beta1/Linux64/meas_base/13.0-3-gc489882+1/python/lsst/meas/base/baseMeasurement.py", line 268, in callMeasure
          self.doMeasurement(plugin, measRecord, *args, **kwds)
        File "/data/mineo/opt/hscpipe/5/stack_5.0-beta1/Linux64/meas_base/13.0-3-gc489882+1/python/lsst/meas/base/baseMeasurement.py", line 287, in doMeasurement
          plugin.measure(measRecord, *args, **kwds)
        File "/data/mineo/opt/hscpipe/5/stack_5.0-beta1/Linux64/meas_modelfit/13.0-3-g177c5a2+1/python/lsst/meas/modelfit/cmodel.py", line 98, in measure
          "CModel forced measurement currently requires the measurement image to have the same"
      FatalAlgorithmError: CModel forced measurement currently requires the measurement image to have the same Wcs as the reference catalog (this is a temporary limitation).
      application called MPI_Abort(MPI_COMM_WORLD, 1) - process 10
       
      The exception was thrown from
      meas_modelfit/13.0-3-g177c5a2+1/python/lsst/meas/modelfit/cmodel.py:
       
          def measure(self, measRecord, exposure, refRecord, refWcs):
              if refWcs != exposure.getWcs(): ### Condition (A) ###
                  raise lsst.meas.base.FatalAlgorithmError(
                      "CModel forced measurement currently requires the measurement image to have the same"
                      " Wcs as the reference catalog (this is a temporary limitation)."
                  )
       
      `refWcs` and `exposure` in Condition (A) was from
      meas_base/13.0-3-gc489882+1/python/lsst/meas/base/forcedPhotImage.py:139
       
              refWcs = self.references.getWcs(dataRef)
              exposure = self.getExposure(dataRef)
       
      `self.references.getWcs` was probably defined in
      meas_base/13.0-3-gc489882+1/python/lsst/meas/base/references.py:202:
       
          def getWcs(self, dataRef):
              """Return the WCS for reference sources.  The given dataRef must include the tract in its dataId.
              """
              skyMap = dataRef.get(self.config.coaddName + "Coadd_skyMap", immediate=True)
              return skyMap[dataRef.dataId["tract"]].getWcs()
       
      Therefore, in Condition (A), one of the compared WCS was from skyMap,
      and the other was from a FITS file.
       
      I suspected that WCS had been changed when saved in the FITS file,
      due to limited number of digits written in FITS headers.
       
      To prove it, I wrote the following script. It actually proved
      that there are some tracts where Condition (A) is violated.
       
      ======================
      import lsst.afw.image as afwImage
      import cPickle as pickle
       
      def main():
          skyMap = pickle.load(open("skyMap.pickle", "rb"))
       
          for tract, tractInfo in enumerate(skyMap):
              testWcs(tract, tractInfo.getWcs())
       
      def testWcs(tract, wcs):
          exp = afwImage.ExposureF(0, 0)
          exp.setWcs(wcs)
       
          exp.writeFits("/dev/shm/foo.fits")
          exp = afwImage.ExposureF("/dev/shm/foo.fits")
       
          # This message must always be "{tract}, True"
          print tract, exp.getWcs() == wcs
       
      main()
      

        Attachments

          Issue Links

            Activity

            Hide
            jbosch Jim Bosch added a comment -

            Loss of precision appears to have been due to round-tripping the CRVALs through radians (implicitly, because we were using an API that uses Angle). Fix is easy, but I need to write a test.

            Show
            jbosch Jim Bosch added a comment - Loss of precision appears to have been due to round-tripping the CRVALs through radians (implicitly, because we were using an API that uses Angle). Fix is easy, but I need to write a test.
            Hide
            jbosch Jim Bosch added a comment -

            John Parejko, could you please take a quick look at this two-line bugfix (with a new ~25-line unit test), or ask me to reassign if you won't get to it today?

            Show
            jbosch Jim Bosch added a comment - John Parejko , could you please take a quick look at this two-line bugfix (with a new ~25-line unit test), or ask me to reassign if you won't get to it today?
            Hide
            Parejkoj John Parejko added a comment -

            Looks good.

            Show
            Parejkoj John Parejko added a comment - Looks good.
            Hide
            jbosch Jim Bosch added a comment -

            Merged to master. Thanks for the quick review.

            Show
            jbosch Jim Bosch added a comment - Merged to master. Thanks for the quick review.

              People

              • Assignee:
                jbosch Jim Bosch
                Reporter:
                jbosch Jim Bosch
                Reviewers:
                John Parejko
                Watchers:
                Jim Bosch, John Parejko
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel