Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-3442

Processing y-band HSC data fails in loading reference sources

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: meas_astrom, pipe_tasks
    • Labels:
      None

      Description

      processCcd.py /lsst3/HSC/data/ --output /raid/price/test --id visit=904400 ccd=50
      [...]
      processCcd.calibrate.astrometry.solver.loadAN: Loading reference objects using center (1023.5, 2091) pix = Fk5Coord(319.8934727, -0.0006943, 2000.00) sky and radius 0.111920792477 deg
      processCcd FATAL: Failed on dataId={'taiObs': '2013-11-03', 'pointing': 672, 'visit': 904400, 'dateObs': '2013-11-03', 'filter': 'HSC-Y', 'field': 'STRIPE82L', 'ccd': 50, 'expTime': 30.0}: Could not find flux field(s) y_camFlux, y_flux
      Traceback (most recent call last):
        File "/home/lsstsw/stack/Linux64/pipe_base/10.1-4-g6ba0cc7+15/python/lsst/pipe/base/cmdLineTask.py", line 320, in __call__
          result = task.run(dataRef, **kwargs)
        File "/home/lsstsw/stack/Linux64/pipe_base/10.1-4-g6ba0cc7+15/python/lsst/pipe/base/timer.py", line 118, in wrapper
          res = func(self, *args, **keyArgs)
        File "/home/lsstsw/stack/Linux64/pipe_tasks/10.1-28-gf9582e4+2/python/lsst/pipe/tasks/processCcd.py", line 85, in run
          result = self.process(sensorRef, postIsrExposure)
        File "/home/lsstsw/stack/Linux64/pipe_base/10.1-4-g6ba0cc7+15/python/lsst/pipe/base/timer.py", line 118, in wrapper
          res = func(self, *args, **keyArgs)
        File "/home/lsstsw/stack/Linux64/pipe_tasks/10.1-28-gf9582e4+2/python/lsst/pipe/tasks/processImage.py", line 160, in process
          calib = self.calibrate.run(inputExposure, idFactory=idFactory)
        File "/home/lsstsw/stack/Linux64/pipe_base/10.1-4-g6ba0cc7+15/python/lsst/pipe/base/timer.py", line 118, in wrapper
          res = func(self, *args, **keyArgs)
        File "/home/lsstsw/stack/Linux64/pipe_tasks/10.1-28-gf9582e4+2/python/lsst/pipe/tasks/calibrate.py", line 457, in run
          astromRet = self.astrometry.run(exposure, sources1)
        File "/home/lsstsw/stack/Linux64/pipe_base/10.1-4-g6ba0cc7+15/python/lsst/pipe/base/timer.py", line 118, in wrapper
          res = func(self, *args, **keyArgs)
        File "/home/lsstsw/stack/Linux64/meas_astrom/10.1-19-g6e01b25+5/python/lsst/meas/astrom/anetAstrometry.py", line 177, in run
          results = self.astrometry(sourceCat=sourceCat, exposure=exposure, bbox=bbox)
        File "/home/lsstsw/stack/Linux64/pipe_base/10.1-4-g6ba0cc7+15/python/lsst/pipe/base/timer.py", line 118, in wrapper
          res = func(self, *args, **keyArgs)
        File "/home/lsstsw/stack/Linux64/meas_astrom/10.1-19-g6e01b25+5/python/lsst/meas/astrom/anetAstrometry.py", line 292, in astrometry
          astrom = self.solver.determineWcs(sourceCat=sourceCat, exposure=exposure, bbox=bbox)
        File "/home/lsstsw/stack/Linux64/meas_astrom/10.1-19-g6e01b25+5/python/lsst/meas/astrom/anetBasicAstrometry.py", line 409, in determineWcs
          return self.determineWcs2(sourceCat=sourceCat, **margs)
        File "/home/lsstsw/stack/Linux64/meas_astrom/10.1-19-g6e01b25+5/python/lsst/meas/astrom/anetBasicAstrometry.py", line 437, in determineWcs2
          astrom = self.useKnownWcs(sourceCat, wcs=wcs, **kw)
        File "/home/lsstsw/stack/Linux64/meas_astrom/10.1-19-g6e01b25+5/python/lsst/meas/astrom/anetBasicAstrometry.py", line 308, in useKnownWcs
          calib = None,
        File "/home/lsstsw/stack/Linux64/pipe_base/10.1-4-g6ba0cc7+15/python/lsst/pipe/base/timer.py", line 118, in wrapper
          res = func(self, *args, **keyArgs)
        File "/home/lsstsw/stack/Linux64/meas_algorithms/10.1-15-g0d3ecf6/python/lsst/meas/algorithms/loadReferenceObjects.py", line 173, in loadPixelBox
          loadRes = self.loadSkyCircle(ctrCoord, maxRadius, filterName)
        File "/home/lsstsw/stack/Linux64/pipe_base/10.1-4-g6ba0cc7+15/python/lsst/pipe/base/timer.py", line 118, in wrapper
          res = func(self, *args, **keyArgs)
        File "/home/lsstsw/stack/Linux64/meas_astrom/10.1-19-g6e01b25+5/python/lsst/meas/astrom/loadAstrometryNetObjects.py", line 141, in loadSkyCircle
          fluxField = getRefFluxField(schema=refCat.schema, filterName=filterName)
        File "/home/lsstsw/stack/Linux64/meas_algorithms/10.1-15-g0d3ecf6/python/lsst/meas/algorithms/loadReferenceObjects.py", line 40, in getRefFluxField
          raise RuntimeError("Could not find flux field(s) %s" % (", ".join(fluxFieldList)))
      RuntimeError: Could not find flux field(s) y_camFlux, y_flux
      

      We should be able to fix this by setting config parameters (e.g., calibrate.astrometry.solver.defaultFilter or calibrate.astrometry.solver.filterMap), but how do we keep that synched with the choice of reference catalog? And once we get past astrometry, we also have the same problem in photocal.

        Attachments

          Container Issues

            Issue Links

              Activity

                People

                • Assignee:
                  price Paul Price
                  Reporter:
                  price Paul Price
                  Reviewers:
                  Russell Owen
                  Watchers:
                  Jim Bosch, Lauren MacArthur, Paul Price, Russell Owen
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  4 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Summary Panel