Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-16717

processCcd HSC visit=36234 ccd=24 failed with w_2018_46 but passed with w_2018_47

    Details

    • Type: Story
    • Status: Won't Fix
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: Science Pipelines
    • Labels:
      None

      Description

      processCcd HSC data --id visit=36234 ccd=24 failed with w_2018_46 but passed with w_2018_47.

      It used to fail with

      processCcd.charImage.measurement INFO: Measuring 438 sources (438 parents, 0 children) 
      processCcd.charImage.measureApCorr INFO: Measuring aperture corrections for 23 flux fields
      processCcd.charImage.measureApCorr INFO: Aperture correction for ext_photometryKron_KronFlux: RMS 0.111340 from 19
      processCcd.charImage.measureApCorr INFO: Aperture correction for modelfit_CModel_dev: RMS 0.024882 from 2
      processCcd.charImage.measureApCorr INFO: Aperture correction for ext_convolved_ConvolvedFlux_2_kron: RMS 0.086138 from 19
      processCcd.charImage.measureApCorr INFO: Aperture correction for ext_convolved_ConvolvedFlux_3_3_3: RMS 0.265679 from 19
      processCcd.charImage.measureApCorr INFO: Aperture correction for ext_convolved_ConvolvedFlux_0_6_0: RMS 0.088057 from 19
      processCcd.charImage.measureApCorr INFO: Aperture correction for ext_convolved_ConvolvedFlux_1_4_5: RMS 0.122741 from 19
      processCcd.charImage.measureApCorr INFO: Aperture correction for ext_convolved_ConvolvedFlux_2_6_0: RMS 0.094711 from 19
      processCcd FATAL: Failed on dataId={'visit': 36234, 'ccd': 24, 'field': 'SSP_WIDE', 'dateObs': '2015-07-21', 'pointing': 1297, 'filter': 'HSC-I', 'taiObs': '2015-07-21', 'expTime': 200.0}: RuntimeError: Unable to measure aperture correction for required algorithm 'modelfit_CModel': only 1 sources, but require at least 2.
       
      Traceback (most recent call last):
        File "/software/lsstsw/stack_20181012/stack/miniconda3-4.5.4-fcd27eb/Linux64/pipe_base/16.0-16-ge6a35c8+3/python/lsst/pipe/base/cmdLineTask.py", line 388, in __call__
          result = self.runTask(task, dataRef, kwargs)
        File "/software/lsstsw/stack_20181012/stack/miniconda3-4.5.4-fcd27eb/Linux64/pipe_base/16.0-16-ge6a35c8+3/python/lsst/pipe/base/cmdLineTask.py", line 447, in runTask
          return task.runDataRef(dataRef, **kwargs)
        File "/software/lsstsw/stack_20181012/stack/miniconda3-4.5.4-fcd27eb/Linux64/pipe_base/16.0-16-ge6a35c8+3/python/lsst/pipe/base/timer.py", line 149, in wrapper
          res = func(self, *args, **keyArgs)
        File "/software/lsstsw/stack_20181012/stack/miniconda3-4.5.4-fcd27eb/Linux64/pipe_tasks/16.0-32-g044e2860+5/python/lsst/pipe/tasks/processCcd.py", line 188, in runDataRef
          doUnpersist=False,
        File "/software/lsstsw/stack_20181012/stack/miniconda3-4.5.4-fcd27eb/Linux64/pipe_base/16.0-16-ge6a35c8+3/python/lsst/pipe/base/timer.py", line 149, in wrapper
          res = func(self, *args, **keyArgs)
        File "/software/lsstsw/stack_20181012/stack/miniconda3-4.5.4-fcd27eb/Linux64/pipe_tasks/16.0-32-g044e2860+5/python/lsst/pipe/tasks/characterizeImage.py", line 349, in runDataRef
          background=background,
        File "/software/lsstsw/stack_20181012/stack/miniconda3-4.5.4-fcd27eb/Linux64/pipe_base/16.0-16-ge6a35c8+3/python/lsst/pipe/base/timer.py", line 149, in wrapper
          res = func(self, *args, **keyArgs)
        File "/software/lsstsw/stack_20181012/stack/miniconda3-4.5.4-fcd27eb/Linux64/pipe_tasks/16.0-32-g044e2860+5/python/lsst/pipe/tasks/characterizeImage.py", line 428, in run
          apCorrMap = self.measureApCorr.run(exposure=dmeRes.exposure, catalog=dmeRes.sourceCat).apCorrMap
        File "/software/lsstsw/stack_20181012/stack/miniconda3-4.5.4-fcd27eb/Linux64/meas_algorithms/16.0-19-gb830ed4e+13/python/lsst/meas/algorithms/measureApCorr.py", line 245, in run
          (name, len(subset2), self.config.minDegreesOfFreedom+1))
      RuntimeError: Unable to measure aperture correction for required algorithm 'modelfit_CModel': only 1 sources, but require at least 2.
      

      Now with w_2018_47:

      processCcd.charImage.measurement INFO: Measuring 438 sources (438 parents, 0 children) 
      processCcd.charImage.measureApCorr INFO: Measuring aperture corrections for 23 flux fields
      processCcd.charImage.measureApCorr INFO: Aperture correction for ext_convolved_ConvolvedFlux_3_3_3: RMS 0.265679 from 19
      processCcd.charImage.measureApCorr INFO: Aperture correction for ext_convolved_ConvolvedFlux_2_kron: RMS 0.086138 from 19
      processCcd.charImage.measureApCorr INFO: Aperture correction for ext_convolved_ConvolvedFlux_1_4_5: RMS 0.122741 from 19
      processCcd.charImage.measureApCorr INFO: Aperture correction for base_GaussianFlux: RMS 0.146412 from 19
      processCcd.charImage.measureApCorr INFO: Aperture correction for ext_convolved_ConvolvedFlux_0_4_5: RMS 0.114090 from 19
      processCcd.charImage.measureApCorr INFO: Aperture correction for ext_convolved_ConvolvedFlux_3_kron: RMS 0.110477 from 19
      processCcd.charImage.measureApCorr INFO: Aperture correction for modelfit_CModel_dev: RMS 0.023262 from 2
      processCcd.charImage.measureApCorr INFO: Aperture correction for base_PsfFlux: RMS 0.130425 from 19
      processCcd.charImage.measureApCorr INFO: Aperture correction for ext_photometryKron_KronFlux: RMS 0.111340 from 19
      processCcd.charImage.measureApCorr INFO: Aperture correction for ext_convolved_ConvolvedFlux_2_4_5: RMS 0.139504 from 19
      processCcd.charImage.measureApCorr INFO: Aperture correction for modelfit_CModel_exp: RMS 0.025076 from 2
      processCcd.charImage.measureApCorr INFO: Aperture correction for ext_convolved_ConvolvedFlux_2_3_3: RMS 0.218014 from 19
      processCcd.charImage.measureApCorr INFO: Aperture correction for ext_convolved_ConvolvedFlux_0_6_0: RMS 0.088057 from 19
      processCcd.charImage.measureApCorr INFO: Aperture correction for ext_convolved_ConvolvedFlux_3_6_0: RMS 0.101306 from 19
      processCcd.charImage.measureApCorr INFO: Aperture correction for ext_convolved_ConvolvedFlux_1_6_0: RMS 0.090117 from 19
      processCcd.charImage.measureApCorr INFO: Aperture correction for ext_convolved_ConvolvedFlux_2_6_0: RMS 0.094711 from 19
      processCcd.charImage.measureApCorr INFO: Aperture correction for ext_convolved_ConvolvedFlux_1_kron: RMS 0.084280 from 19
      processCcd.charImage.measureApCorr INFO: Aperture correction for ext_convolved_ConvolvedFlux_0_3_3: RMS 0.142056 from 19
      processCcd.charImage.measureApCorr INFO: Aperture correction for ext_convolved_ConvolvedFlux_1_3_3: RMS 0.173804 from 19
      processCcd.charImage.measureApCorr INFO: Aperture correction for ext_convolved_ConvolvedFlux_0_kron: RMS 0.100138 from 19
      processCcd.charImage.measureApCorr INFO: Aperture correction for modelfit_CModel: RMS 0.023711 from 2
      processCcd.charImage.measureApCorr INFO: Aperture correction for modelfit_CModel_initial: RMS 0.005822 from 8
      processCcd.charImage.measureApCorr INFO: Aperture correction for ext_convolved_ConvolvedFlux_3_4_5: RMS 0.159494 from 19
      processCcd.charImage.applyApCorr INFO: Applying aperture corrections to 23 instFlux fields
      processCcd.calibrate INFO: Processing {'visit': 36234, 'ccd': 24, 'field': 'SSP_WIDE', 'dateObs': '2015-07-21', 'pointing': 1297, 'filter': 'HSC-I', 'taiObs': '2015-07-21', 'expTime': 200.0}
      processCcd.calibrate.detection INFO: Detected 10847 positive peaks in 3097 footprints and 35 negative peaks in 34 footprints to 5 sigma
      processCcd.calibrate.detection INFO: Resubtracting the background after object detection
      processCcd.calibrate.deblend INFO: Deblending 3097 sources
      processCcd.calibrate.deblend INFO: Deblended: of 3097 sources, 420 were deblended, creating 2758 children, total 5855 sources
      processCcd.calibrate.measurement INFO: Measuring 5855 sources (3097 parents, 2758 children) 
      processCcd.calibrate.applyApCorr INFO: Applying aperture corrections to 3 instFlux fields
      processCcd.calibrate INFO: Copying flags from icSourceCat to sourceCat for 352 sources
      processCcd.calibrate.astromRefObjLoader INFO: Loading reference objects using center (339.345312, -0.103570) and radius 0.12781878507213473 deg
      processCcd.calibrate.astromRefObjLoader INFO: Loaded 1240 reference objects
      processCcd.calibrate.astrometry.matcher INFO: filterStars purged 0 reference stars, leaving 1240 stars
      processCcd.calibrate.astrometry.matcher INFO: Purged 5344 unusable sources, leaving 511 usable sources
      processCcd.calibrate.astrometry.matcher INFO: Matched 409 sources
      processCcd.calibrate.astrometry.matcher INFO: filterStars purged 0 reference stars, leaving 1240 stars
      processCcd.calibrate.astrometry.matcher INFO: Purged 5344 unusable sources, leaving 511 usable sources
      processCcd.calibrate.astrometry.matcher INFO: Matched 354 sources
      processCcd.calibrate.astrometry.matcher INFO: filterStars purged 0 reference stars, leaving 1240 stars
      processCcd.calibrate.astrometry.matcher INFO: Purged 5344 unusable sources, leaving 511 usable sources
      processCcd.calibrate.astrometry.matcher INFO: Matched 336 sources
      processCcd.calibrate.astrometry INFO: Matched and fit WCS in 3 iterations; found 336 matches with scatter = 0.113 +- 0.072 arcsec
      processCcd.calibrate.photoCal.match.sourceSelection INFO: Selected 459/5855 sources
      processCcd.calibrate.photoCal.match.referenceSelection INFO: Selected 243/1744 references
      processCcd.calibrate.photoCal.match INFO: Matched 21 from 459/5855 input and 243/1744 reference sources
      processCcd.calibrate.photoCal.reserve INFO: Reserved 0/21 sources
      processCcd.calibrate.photoCal INFO: Applying color terms for filterName='i', config.photoCatName=ps1_pv3_3pi_20170110 because config.applyColorTerms is True
      processCcd.calibrate.photoCal INFO: Magnitude zero point: 32.732448 +/- 0.001080 from 16 stars
      processCcd.calibrate INFO: Photometric zero-point: 32.732448
      

      This probably is just good, but I'm filing it just to note the change. Please close the ticket if you think no further action is needed.

        Attachments

          Issue Links

            Activity

            Hide
            lskelvin Lee Kelvin added a comment -

            Hsin-Fang Chiang, do you have any objections if we mark this ticket as 'Won't Fix'? The weekly it refers to is fairly old now, and we cannot easily reproduce on our side.

            Show
            lskelvin Lee Kelvin added a comment - Hsin-Fang Chiang , do you have any objections if we mark this ticket as 'Won't Fix'? The weekly it refers to is fairly old now, and we cannot easily reproduce on our side.
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            No objections. It was just meant to record the noticed change in case the team wanted to follow up.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - No objections. It was just meant to record the noticed change in case the team wanted to follow up.
            Hide
            lskelvin Lee Kelvin added a comment -

            Great, thanks Hsin-Fang - closing it now!

            Show
            lskelvin Lee Kelvin added a comment - Great, thanks Hsin-Fang - closing it now!

              People

              • Assignee:
                Unassigned
                Reporter:
                hchiang2 Hsin-Fang Chiang
                Watchers:
                Hsin-Fang Chiang, Lee Kelvin
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: