Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-12179

Identify changes in HSC processCcd matching between w_2017_39 and w_2017_38

    Details

    • Type: Story
    • Status: To Do
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Templates:
    • Team:
      Data Release Production

      Description

      Using the Python 2 shared stack on lsst-dev (either on /software or on /ssd), processCcd.py was able to process HSC data --id visit=9868 ccd=77 with w_2017_38 but failed with w_2017_39 at "Unable to match sources".

      The command to reproduce can be:

      processCcd.py /datasets/hsc/repo/ --rerun private/usename/ --id visit=9868 ccd=77
      

      With w_2017_38:

      processCcd.calibrate.astrometry INFO: Matched and fit WCS in 3 iterations; found 57 matches with scatter = 0.040 +- 0.026 arcsec
      processCcd.calibrate.photoRefObjLoader INFO: Loading reference objects using center Fk5Coord(34.7345186, -2.9528981, 2000.00) and radius 0.121483483029 deg
      processCcd.calibrate.photoRefObjLoader INFO: Loaded 3186 reference objects
      processCcd.calibrate.photoCal.matcher INFO: filterStars purged 0 reference stars, leaving 3186 stars
      processCcd.calibrate.photoCal.matcher INFO: Purged 778 unusable sources, leaving 101 usable sources
      processCcd.calibrate.photoCal.matcher INFO: Matched 72 sources
      processCcd.calibrate.photoCal INFO: Found 72 matches with scatter = 0.045 +- 0.032 arcsec; 
      processCcd.calibrate.photoCal WARN: No 'photometric' flag key found in reference schema.
      processCcd.calibrate.photoCal INFO: Applying color terms for filterName=u'g', config.photoCatName=ps1_pv3_3pi_20170110 because config.applyColorTerms is True
      processCcd.calibrate.photoCal INFO: Magnitude zero point: 32.157198 +/- 0.000578 from 19 stars
      processCcd.calibrate INFO: Photometric zero-point: 32.157198
      

      With w_2017_39:

      processCcd.calibrate.astrometry INFO: Matched and fit WCS in 3 iterations; found 56 matches with scatter = 0.039 +- 0.026 arcsec
      processCcd.calibrate.photoRefObjLoader INFO: Loading reference objects using center Fk5Coord(34.7345155, -2.9528997, 2000.00) and radius 0.121496056094 deg
      processCcd.calibrate.photoRefObjLoader INFO: Loaded 3184 reference objects
      processCcd.calibrate.photoCal.matcher INFO: filterStars purged 0 reference stars, leaving 3184 stars
      processCcd.calibrate.photoCal.matcher INFO: Purged 778 unusable sources, leaving 99 usable sources
      processCcd FATAL: Failed on dataId={'taiObs': '2014-10-01', 'pointing': 1004, 'visit': 9868, 'dateObs': '2014-10-01', 'filter': 'HSC-G', 'field': 'SSP_WIDE', 'ccd': 77, 'expTime': 150.0}: Unable to match sources
      Traceback (most recent call last):
        File "/ssd/lsstsw/stack_20170409/Linux64/pipe_base/13.0-14-g8b3bf66+27/python/lsst/pipe/base/cmdLineTask.py", line 407, in __call__
          result = task.run(dataRef, **kwargs)
        File "/ssd/lsstsw/stack_20170409/Linux64/pipe_base/13.0-14-g8b3bf66+27/python/lsst/pipe/base/timer.py", line 150, in wrapper
          res = func(self, *args, **keyArgs)
        File "/ssd/lsstsw/stack_20170409/Linux64/pipe_tasks/13.0-66-gfbf2f2ce+2/python/lsst/pipe/tasks/processCcd.py", line 199, in run
          icSourceCat=charRes.sourceCat,
        File "/ssd/lsstsw/stack_20170409/Linux64/pipe_base/13.0-14-g8b3bf66+27/python/lsst/pipe/base/timer.py", line 150, in wrapper
          res = func(self, *args, **keyArgs)
        File "/ssd/lsstsw/stack_20170409/Linux64/pipe_tasks/13.0-66-gfbf2f2ce+2/python/lsst/pipe/tasks/calibrate.py", line 433, in run
          icSourceCat=icSourceCat,
        File "/ssd/lsstsw/stack_20170409/Linux64/pipe_tasks/13.0-66-gfbf2f2ce+2/python/lsst/pipe/tasks/calibrate.py", line 538, in calibrate
          photoRes = self.photoCal.run(exposure, sourceCat=sourceCat, expId=exposureIdInfo.expId)
        File "/ssd/lsstsw/stack_20170409/Linux64/pipe_base/13.0-14-g8b3bf66+27/python/lsst/pipe/base/timer.py", line 150, in wrapper
          res = func(self, *args, **keyArgs)
        File "/ssd/lsstsw/stack_20170409/Linux64/pipe_tasks/13.0-66-gfbf2f2ce+2/python/lsst/pipe/tasks/photoCal.py", line 666, in run
          res = self.loadAndMatch(exposure, sourceCat)
        File "/ssd/lsstsw/stack_20170409/Linux64/pipe_base/13.0-14-g8b3bf66+27/python/lsst/pipe/base/timer.py", line 150, in wrapper
          res = func(self, *args, **keyArgs)
        File "/ssd/lsstsw/stack_20170409/Linux64/meas_astrom/13.0-16-g30f6da5+20/python/lsst/meas/astrom/ref_match.py", line 116, in loadAndMatch
          match_tolerance=None,
        File "/ssd/lsstsw/stack_20170409/Linux64/pipe_base/13.0-14-g8b3bf66+27/python/lsst/pipe/base/timer.py", line 150, in wrapper
          res = func(self, *args, **keyArgs)
        File "/ssd/lsstsw/stack_20170409/Linux64/meas_astrom/13.0-16-g30f6da5+20/python/lsst/meas/astrom/matchOptimisticB/matchOptimisticBContinued.py", line 276, in matchObjectsToSources
          raise RuntimeError("Unable to match sources")
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                hchiang2 Hsin-Fang Chiang
                Watchers:
                Hsin-Fang Chiang, Jim Bosch, John Swinbank, Lauren MacArthur, Tim Jenness, Yusra AlSayyad
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Summary Panel