Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-5419

ci_hsc fails test requiring >95% of PSF stars to be stars on the coadd

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: ci_hsc
    • Labels:
      None

      Description

      Since the first week of March 2016, ci_hsc fails its test that requires that >95% of the PSF stars be identified as stars in the coadd. I suspect this is related to the DM-4692 merge.

      Here is a sample job that fails:
      https://ci.lsst.codes/job/stack-os-matrix/9084/label=centos-6/console

      The relevant snippet of the failure is:

      [2016-03-10T17:12:06.667778Z] : Validating dataset measureCoaddSources_config for {'filter': 'HSC-R', 'tract': 0, 'patch': '5,4'}
      [2016-03-10T17:12:06.697383Z] CameraMapper: Loading registry registry from /home/build0/lsstsw/build/ci_hsc/DATA/registry.sqlite3
      [2016-03-10T17:12:06.697615Z] CameraMapper: Loading calibRegistry registry from /home/build0/lsstsw/build/ci_hsc/DATA/CALIB/calibRegistry.sqlite3
      [2016-03-10T17:12:07.716310Z] CameraMapper: Loading registry registry from /home/build0/lsstsw/build/ci_hsc/DATA/registry.sqlite3
      [2016-03-10T17:12:07.716443Z] CameraMapper: Loading calibRegistry registry from /home/build0/lsstsw/build/ci_hsc/DATA/CALIB/calibRegistry.sqlite3
      [2016-03-10T17:12:08.663566Z] : measureCoaddSources_config exists: PASS
      [2016-03-10T17:12:08.721051Z] : measureCoaddSources_config readable (<class 'lsst.pipe.tasks.multiBand.MeasureMergedCoaddSourcesConfig'>): PASS
      [2016-03-10T17:12:08.721077Z] : Validating dataset measureCoaddSources_metadata for {'filter': 'HSC-R', 'tract': 0, 'patch': '5,4'}
      [2016-03-10T17:12:08.721249Z] : measureCoaddSources_metadata exists: PASS
      [2016-03-10T17:12:08.721663Z] : measureCoaddSources_metadata readable (<class 'lsst.daf.base.baseLib.PropertySet'>): PASS
      [2016-03-10T17:12:08.721715Z] : Validating dataset deepCoadd_meas_schema for {'filter': 'HSC-R', 'tract': 0, 'patch': '5,4'}
      [2016-03-10T17:12:08.721878Z] : deepCoadd_meas_schema exists: PASS
      [2016-03-10T17:12:08.726703Z] : deepCoadd_meas_schema readable (<class 'lsst.afw.table.tableLib.SourceCatalog'>): PASS
      [2016-03-10T17:12:08.726834Z] : Validating source output for {'filter': 'HSC-R', 'tract': 0, 'patch': '5,4'}
      [2016-03-10T17:12:10.203469Z] : Number of sources (7595 > 100): PASS
      [2016-03-10T17:12:10.204166Z] : calib_psfCandidate field exists in deepCoadd_meas catalog: PASS
      [2016-03-10T17:12:10.204772Z] : calib_psfUsed field exists in deepCoadd_meas catalog: PASS
      [2016-03-10T17:12:10.205468Z] : Aperture correction fields for base_PsfFlux are present.: PASS
      [2016-03-10T17:12:10.206159Z] : Aperture correction fields for base_GaussianFlux are present.: PASS
      [2016-03-10T17:12:10.207193Z]  FATAL: 95% of sources used to build the PSF are classified as stars on the coadd (0 > 0): FAIL
      [2016-03-10T17:12:10.207455Z] scons: *** [.scons/measure-HSC-R] AssertionError : Failed test: 95% of sources used to build the PSF are classified as stars on the coadd (0 > 0)
      [2016-03-10T17:12:10.207481Z] Traceback (most recent call last):
      [2016-03-10T17:12:10.207525Z]   File "/home/build0/lsstsw/stack/Linux64/scons/2.3.5/lib/scons/SCons/Action.py", line 1063, in execute
      [2016-03-10T17:12:10.207556Z]     result = self.execfunction(target=target, source=rsources, env=env)
      [2016-03-10T17:12:10.207593Z]   File "/home/build0/lsstsw/build/ci_hsc/python/lsst/ci/hsc/validate.py", line 133, in scons
      [2016-03-10T17:12:10.207611Z]     return self.run(*args, **kwargs)
      [2016-03-10T17:12:10.207646Z]   File "/home/build0/lsstsw/build/ci_hsc/python/lsst/ci/hsc/validate.py", line 122, in run
      [2016-03-10T17:12:10.207663Z]     self.validateSources(dataId)
      [2016-03-10T17:12:10.207732Z]   File "/home/build0/lsstsw/build/ci_hsc/python/lsst/ci/hsc/validate.py", line 191, in validateSources
      [2016-03-10T17:12:10.207749Z]     0.95*psfStars.sum()
      [2016-03-10T17:12:10.207786Z]   File "/home/build0/lsstsw/build/ci_hsc/python/lsst/ci/hsc/validate.py", line 52, in assertGreater
      [2016-03-10T17:12:10.207816Z]     self.assertTrue(description + " (%d > %d)" % (num1, num2), num1 > num2)
      [2016-03-10T17:12:10.207853Z]   File "/home/build0/lsstsw/build/ci_hsc/python/lsst/ci/hsc/validate.py", line 43, in assertTrue
      [2016-03-10T17:12:10.207877Z]     raise AssertionError("Failed test: %s" % description)
      [2016-03-10T17:12:10.207919Z] AssertionError: Failed test: 95% of sources used to build the PSF are classified as stars on the coadd (0 > 0)
      [2016-03-10T17:12:10.209935Z] scons: building terminated because of errors.
      

      This is the test that fails

      https://github.com/lsst/ci_hsc/blob/74303a818eb5049a2015b5e885df2781053748c9/python/lsst/ci/hsc/validate.py#L169

      class MeasureValidation(Validation):
          _datasets = ["measureCoaddSources_config", "measureCoaddSources_metadata", "deepCoadd_meas_schema"]
          _sourceDataset = "deepCoadd_meas"
          _matchDataset = "deepCoadd_srcMatch"
       
          def validateSources(self, dataId):
              catalog = Validation.validateSources(self, dataId)
              self.assertTrue("calib_psfCandidate field exists in deepCoadd_meas catalog",
                              "calib_psfCandidate" in catalog.schema)
              self.assertTrue("calib_psfUsed field exists in deepCoadd_meas catalog",
                              "calib_psfUsed" in catalog.schema)
              self.checkApertureCorrections(catalog)
              # Check that at least 95% of the stars we used to model the PSF end up classified as stars
              # on the coadd.  We certainly need much more purity than that to build good PSF models, but
              # this should verify that flag propagation, aperture correction, and extendendess are all
              # running and configured reasonably (but it may not be sensitive enough to detect subtle
              # bugs).
              psfStars = catalog.get("calib_psfUsed")
              extStars = catalog.get("base_ClassificationExtendedness_value") < 0.5
              self.assertGreater(
                  "95% of sources used to build the PSF are classified as stars on the coadd",
                  numpy.logical_and(extStars, psfStars).sum(),
                  0.95*psfStars.sum()
              )
      

      Note that the assertion failure messages is a bit confusing. It should say
      "Fewer than 95% of the sources used to build the PSF are classified as stars on the coadd."

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                jbosch Jim Bosch
                Reporter:
                wmwood-vasey Michael Wood-Vasey
                Reviewers:
                Russell Owen
                Watchers:
                Hsin-Fang Chiang, Jim Bosch, John Swinbank, Michael Wood-Vasey, Russell Owen
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel