Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-29296

Filter "unknown" Found in FGCM in RC Processing

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: To Do
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: None
    • Urgent?:
      Yes

      Description

      In RC2 w_2021_10
      /datasets/hsc/repo/rerun/RC/w_2021_10/DM-29074

      We are getting persistent error in FGCM fgcmFitCycle (which occurs after fgcmMakeLut and fgcmBuildStars, which run successfully).

      The bug is in /datasets/hsc/repo/rerun/RC/w_2021_10/DM-29074/logs/FGCM/fgcmFitCycle_0.log as

      fgcmFitCycle FATAL: Failed: Filter unknown in filterToBand not in LUT
      Traceback (most recent call last):
        File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/fgcmcal/21.0.0-10-g8d1d15d+069596c267/python/lsst/fgcmcal/fgcmFitCycle.py", line 881, in __call__
          task.runDataRef(butler)
        File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_base/21.0.0-10-g560fb7b+901427974f/python/lsst/pipe/base/timer.py", line 157, in wrapper
          res = func(self, *args, **keyArgs)
        File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/fgcmcal/21.0.0-10-g8d1d15d+069596c267/python/lsst/fgcmcal/fgcmFitCycle.py", line 981, in runDataRef
          fgcmDatasetDict = self._fgcmFitCycle(camera, dataRefDict)
        File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/fgcmcal/21.0.0-10-g8d1d15d+069596c267/python/lsst/fgcmcal/fgcmFitCycle.py", line 1109, in _fgcmFitCycle
          fgcmFitCycle = fgcm.FgcmFitCycle(configDict, useFits=False,
        File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/fgcm/lsst-dev-g14fa2066fb/lib/python/fgcm/fgcmFitCycle.py", line 76, in __init__
          self.fgcmConfig = FgcmConfig(configDict,
        File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/fgcm/lsst-dev-g14fa2066fb/lib/python/fgcm/fgcmConfig.py", line 355, in __init__
          raise ValueError("Filter %s in filterToBand not in LUT" % (filterName))
      ValueError: Filter unknown in filterToBand not in LUT
      

      I talked with Eli Rykoff, and we did an update on FGCM to no avail.

      I can confirm that the FGCM output in:
      /datasets/hsc/repo/rerun/RC/w_2021_10/DM-29074/FGCM/fgcm-process

      Does not contain any weird filter errors (PhysicalFilter is consistently just the 6 HSC filters across all images).

      I could not see anything weird in the repo.

      I suspect this might be fixed with a filter/band fix in a --config file.

        Attachments

          Activity

          Hide
          erykoff Eli Rykoff added a comment -

          I see what's going on. I need to fix an overly aggressive input check in fgcm. See DM-29299.

          In the meantime, the updated config file in /home/erykoff/rc2_files_for_monika/w_2021_07+/fgcmFitCycleRC2_cycle00_config.py has the following appended which should fix the problem:

          config.physicalFilterMap = {'HSC-G': 'g',
                                      'HSC-R': 'r',
                                      'HSC-I': 'i',
                                      'HSC-Z': 'z',
                                      'HSC-Y': 'y',
                                      'NB0921': 'N921'}
          

          Show
          erykoff Eli Rykoff added a comment - I see what's going on. I need to fix an overly aggressive input check in fgcm . See DM-29299 . In the meantime, the updated config file in /home/erykoff/rc2_files_for_monika/w_2021_07+/fgcmFitCycleRC2_cycle00_config.py has the following appended which should fix the problem: config.physicalFilterMap = { 'HSC-G' : 'g' , 'HSC-R' : 'r' , 'HSC-I' : 'i' , 'HSC-Z' : 'z' , 'HSC-Y' : 'y' , 'NB0921' : 'N921' }
          Hide
          emorganson Eric Morganson [X] (Inactive) added a comment - - edited

          Great. Cycle00 seems to be working (got off the ground at least). Do I need to alter the later cycle config files? Or should they be able to read everything they need from the outputs of cycle 00?

          Can confirm that cycle 01 superficially seems to be working.

          Show
          emorganson Eric Morganson [X] (Inactive) added a comment - - edited Great. Cycle00 seems to be working (got off the ground at least). Do I need to alter the later cycle config files? Or should they be able to read everything they need from the outputs of cycle 00? Can confirm that cycle 01 superficially seems to be working.
          Hide
          erykoff Eli Rykoff added a comment -

          Good news! You shouldn't need to make any further changes, these will get propagated forward.

          Show
          erykoff Eli Rykoff added a comment - Good news! You shouldn't need to make any further changes, these will get propagated forward.
          Hide
          emorganson Eric Morganson [X] (Inactive) added a comment -

          Seems that this worked. Thanks. Would appreciate any updates I need to make in a few weeks for w14.

          Show
          emorganson Eric Morganson [X] (Inactive) added a comment - Seems that this worked. Thanks. Would appreciate any updates I need to make in a few weeks for w14.

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            emorganson Eric Morganson [X] (Inactive)
            Watchers:
            Eli Rykoff, Eric Morganson [X] (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:

                Jenkins

                No builds found.