Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-12180

Different processCcd matching outcomes using Py2 and Py3 stacks

    Details

    • Type: Story
    • Status: To Do
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: None
    • Templates:

      Description

      Running ProcessCcd with HSC data --id visit=9868 ccd=77 using Stack of the same tag w_2017_38 of the Python 2 shared stack at /ssd/lsstsw/stack and the Python 3 shared stack at /ssd/lsstsw/stack3/ gives different matching results. The Py2 stack passed but the Py3 stack failed with "Unable to match sources".

      The command to reproduce:

      processCcd.py /datasets/hsc/repo/ --rerun private/user/path --id visit=9868 ccd=77
      

      With Python 2 on /ssd/lsstsw/stack_20170409/:

      processCcd.calibrate.astrometry INFO: Matched and fit WCS in 3 iterations; found 57 matches with scatter = 0.040 +- 0.026 arcsec
      processCcd.calibrate.photoRefObjLoader INFO: Loading reference objects using center Fk5Coord(34.7345186, -2.9528981, 2000.00) and radius 0.121483483029 deg
      processCcd.calibrate.photoRefObjLoader INFO: Loaded 3186 reference objects
      processCcd.calibrate.photoCal.matcher INFO: filterStars purged 0 reference stars, leaving 3186 stars
      processCcd.calibrate.photoCal.matcher INFO: Purged 778 unusable sources, leaving 101 usable sources
      processCcd.calibrate.photoCal.matcher INFO: Matched 72 sources
      processCcd.calibrate.photoCal INFO: Found 72 matches with scatter = 0.045 +- 0.032 arcsec; 
      processCcd.calibrate.photoCal WARN: No 'photometric' flag key found in reference schema.
      processCcd.calibrate.photoCal INFO: Applying color terms for filterName=u'g', config.photoCatName=ps1_pv3_3pi_20170110 because config.applyColorTerms is True
      processCcd.calibrate.photoCal INFO: Magnitude zero point: 32.157198 +/- 0.000578 from 19 stars
      processCcd.calibrate INFO: Photometric zero-point: 32.157198
      

      With Python 3 on /ssd/lsstsw/stack3_2017-09-11/

      processCcd.calibrate.astrometry INFO: Matched and fit WCS in 3 iterations; found 60 matches with scatter = 0.044 +- 0.029 arcsec
      processCcd.calibrate.photoRefObjLoader INFO: Loading reference objects using center Fk5Coord(34.7345178, -2.9529006, 2000.00) and radius 0.12148793072780815 deg
      processCcd.calibrate.photoRefObjLoader INFO: Loaded 3187 reference objects
      processCcd.calibrate.photoCal.matcher INFO: filterStars purged 0 reference stars, leaving 3187 stars
      processCcd.calibrate.photoCal.matcher INFO: Purged 782 unusable sources, leaving 97 usable sources
      processCcd FATAL: Failed on dataId={'field': 'SSP_WIDE', 'ccd': 77, 'taiObs': '2014-10-01', 'dateObs': '2014-10-01', 'visit': 9868, 'expTime': 150.0, 'filter': 'HSC-G', 'pointing': 1004}: Unable to match sources
      Traceback (most recent call last):
        File "/ssd/lsstsw/stack3_2017-09-11/stack/miniconda3-4.2.12-7c8e67/Linux64/pipe_base/13.0-14-g8b3bf66+22/python/lsst/pipe/base/cmdLineTask.py", line 407, in __call__
          result = task.run(dataRef, **kwargs)
        File "/ssd/lsstsw/stack3_2017-09-11/stack/miniconda3-4.2.12-7c8e67/Linux64/pipe_base/13.0-14-g8b3bf66+22/python/lsst/pipe/base/timer.py", line 150, in wrapper
          res = func(self, *args, **keyArgs)
        File "/ssd/lsstsw/stack3_2017-09-11/stack/miniconda3-4.2.12-7c8e67/Linux64/pipe_tasks/13.0-64-gd461ca2f+2/python/lsst/pipe/tasks/processCcd.py", line 199, in run
          icSourceCat=charRes.sourceCat,
        File "/ssd/lsstsw/stack3_2017-09-11/stack/miniconda3-4.2.12-7c8e67/Linux64/pipe_base/13.0-14-g8b3bf66+22/python/lsst/pipe/base/timer.py", line 150, in wrapper
          res = func(self, *args, **keyArgs)
        File "/ssd/lsstsw/stack3_2017-09-11/stack/miniconda3-4.2.12-7c8e67/Linux64/pipe_tasks/13.0-64-gd461ca2f+2/python/lsst/pipe/tasks/calibrate.py", line 433, in run
          icSourceCat=icSourceCat,
        File "/ssd/lsstsw/stack3_2017-09-11/stack/miniconda3-4.2.12-7c8e67/Linux64/pipe_tasks/13.0-64-gd461ca2f+2/python/lsst/pipe/tasks/calibrate.py", line 538, in calibrate
          photoRes = self.photoCal.run(exposure, sourceCat=sourceCat)
        File "/ssd/lsstsw/stack3_2017-09-11/stack/miniconda3-4.2.12-7c8e67/Linux64/pipe_base/13.0-14-g8b3bf66+22/python/lsst/pipe/base/timer.py", line 150, in wrapper
          res = func(self, *args, **keyArgs)
        File "/ssd/lsstsw/stack3_2017-09-11/stack/miniconda3-4.2.12-7c8e67/Linux64/pipe_tasks/13.0-64-gd461ca2f+2/python/lsst/pipe/tasks/photoCal.py", line 667, in run
          res = self.loadAndMatch(exposure, sourceCat)
        File "/ssd/lsstsw/stack3_2017-09-11/stack/miniconda3-4.2.12-7c8e67/Linux64/pipe_base/13.0-14-g8b3bf66+22/python/lsst/pipe/base/timer.py", line 150, in wrapper
          res = func(self, *args, **keyArgs)
        File "/ssd/lsstsw/stack3_2017-09-11/stack/miniconda3-4.2.12-7c8e67/Linux64/meas_astrom/13.0-16-g30f6da5+15/python/lsst/meas/astrom/ref_match.py", line 116, in loadAndMatch
          match_tolerance=None,
        File "/ssd/lsstsw/stack3_2017-09-11/stack/miniconda3-4.2.12-7c8e67/Linux64/pipe_base/13.0-14-g8b3bf66+22/python/lsst/pipe/base/timer.py", line 150, in wrapper
          res = func(self, *args, **keyArgs)
        File "/ssd/lsstsw/stack3_2017-09-11/stack/miniconda3-4.2.12-7c8e67/Linux64/meas_astrom/13.0-16-g30f6da5+15/python/lsst/meas/astrom/matchOptimisticB/matchOptimisticBContinued.py", line 276, in matchObjectsToSources
          raise RuntimeError("Unable to match sources")
      RuntimeError: Unable to match sources
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                hchiang2 Hsin-Fang Chiang
                Watchers:
                Hsin-Fang Chiang, Tim Jenness, Yusra AlSayyad
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Summary Panel