Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-29446

DC2 singleFrameDriver Failure: FilterLabel(physical="z_sim_1.4") has no band.

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: obs_lsst
    • Labels:
      None
    • Story Points:
      1
    • Team:
      Data Release Production
    • Urgent?:
      No

      Description

      singleFrameDriver failing for DC2 (with w_2021_08 and w_2021_12) with error
      FilterLabel(physical="z_sim_1.4") has no band.

      Command is:

      singleFrameDriver.py /datasets/DC2/repoRun2.2i --rerun w_2021_08/DM-28966/sfm --batch-type slurm --mpiexec='-bind-to socket' --job dc2sfm08 --cores 24 --time 900  --id visit=159471^159491^159507^183772^183773^183818^183912^193780^193781^193827^194862^221574^221575^221577^221614^221615^221616^254358^254359^254379 --configfile /home/emorgan2/HSC/DC2/filter_config.py --clobber-config;
      /

      filter_config.py text is:

      DC2_filterMap = {f'{_}_sim_1.4': f'lsst_{_}_smeared' for _ in 'ugrizy'}
      DC2_filterMap.update({_: f'lsst_{_}_smeared' for _ in 'ugrizy'})
      config.processCcd.calibrate.astromRefObjLoader.filterMap = DC2_filterMap
      config.processCcd.calibrate.photoRefObjLoader.filterMap = DC2_filterMap
      /

      or alternately:

      config.processCcd.calibrate.astromRefObjLoader.filterMap = {f'{_}_sim_1.4': f'lsst_{_}_smeared' for _ in 'ugrizy'}
      config.processCcd.calibrate.photoRefObjLoader.filterMap = {f'{_}_sim_1.4': f'lsst_{_}_smeared' for _ in 'ugrizy'}
      /

      Error is

      Traceback (most recent call last):
        File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_base/21.0.0-10-g560fb7b+5f2ca89aed/python/lsst/pipe/base/cmdLineTask.py", line 431, in __call__
          result = self.runTask(task, dataRef, kwargs)
        File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_base/21.0.0-10-g560fb7b+5f2ca89aed/python/lsst/pipe/base/cmdLineTask.py", line 495, in runTask
          return task.runDataRef(dataRef, **kwargs)
        File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_drivers/21.0.0-4-g3af6bfd+64d2251902/python/lsst/pipe/drivers/singleFrameDriver.py", line 83, in runDataRef
          result = self.processCcd.runDataRef(sensorRef)
        File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_base/21.0.0-10-g560fb7b+5f2ca89aed/python/lsst/pipe/base/timer.py", line 157, in wrapper
          res = func(self, *args, **keyArgs)
        File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_tasks/21.0.0-38-g070523fc+2384b0eea2/python/lsst/pipe/tasks/processCcd.py", line 191, in runDataRef
          calibRes = self.calibrate.runDataRef(
        File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_base/21.0.0-10-g560fb7b+5f2ca89aed/python/lsst/pipe/base/timer.py", line 157, in wrapper
          res = func(self, *args, **keyArgs)
        File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_tasks/21.0.0-38-g070523fc+2384b0eea2/python/lsst/pipe/tasks/calibrate.py", line 586, in runDataRef
          calRes = self.run(
        File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_base/21.0.0-10-g560fb7b+5f2ca89aed/python/lsst/pipe/base/timer.py", line 157, in wrapper
          res = func(self, *args, **keyArgs)
        File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_tasks/21.0.0-38-g070523fc+2384b0eea2/python/lsst/pipe/tasks/calibrate.py", line 729, in run
          astromRes = self.astrometry.run(
        File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_base/21.0.0-10-g560fb7b+5f2ca89aed/python/lsst/pipe/base/timer.py", line 157, in wrapper
          res = func(self, *args, **keyArgs)
        File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/meas_astrom/21.0.0-4-g591bb35+760f7344ed/python/lsst/meas/astrom/astrometry.py", line 152, in run
          res = self.solve(exposure=exposure, sourceCat=sourceCat)
        File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_base/21.0.0-10-g560fb7b+5f2ca89aed/python/lsst/pipe/base/timer.py", line 157, in wrapper
          res = func(self, *args, **keyArgs)
        File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/meas_astrom/21.0.0-4-g591bb35+760f7344ed/python/lsst/meas/astrom/astrometry.py", line 183, in solve
          expMd = self._getExposureMetadata(exposure)
        File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/meas_astrom/21.0.0-4-g591bb35+760f7344ed/python/lsst/meas/astrom/ref_match.py", line 257, in _getExposureMetadata
          filterName = filterLabel.bandLabel if filterLabel is not None else None
      RuntimeError: FilterLabel(physical="z_sim_1.4") has no band.
      /

        Attachments

          Issue Links

            Activity

            Hide
            lauren Lauren MacArthur added a comment -

            Yep...I confirm this change along with the filterMap config override worked for me on a processCcd.py run using w_2021_13 + your tickets/DM-29446 branch.  Presumably the filterMap issue needs resolving (on another ticket, though!)?

            Show
            lauren Lauren MacArthur added a comment - Yep...I confirm this change along with the filterMap  config override worked for me on a processCcd.py  run using w_2021_13 + your tickets/ DM-29446 branch.  Presumably the filterMap  issue needs resolving (on another ticket, though!)?
            Hide
            emorganson Eric Morganson [X] (Inactive) added a comment - - edited

            This fix has not worked for me.

            To show current setup:

            setup lsst_distrib -t w_2021_08 # also lsst-distrib w_2021_12
            setup -v -r /home/emorgan2/SOFTWARE/obs_lsst # Have also tried with -k and -j option
            singleFrameDriver.py /datasets/DC2/repoRun2.2i --rerun w_2021_08/DM-28966/sfm --batch-type slurm --mpiexec='-bind-to socket' --job dc2sfm08 --cores 24 --time 900  --id visit=159471^159491^159507^183772^183773^183818^183912^193780^193781^193827^194862^221574^221575^221577^221614^221615^221616^254358^254359^254379 --configfile /home/emorgan2/HSC/DC2/filter_config_2021_12.py --clobber-config --clobber-versions;
            

            filter_config_2021_12.py is:

            DC2_filterMap = {band:'lsst_%s_smeared' % (band) for band in 'ugrizy'}
            config.processCcd.calibrate.astromRefObjLoader.filterMap = DC2_filterMap
            config.processCcd.calibrate.photoRefObjLoader.filterMap = DC2_filterMap
            

            Have also tried:

            DC2_filterMap = {f'{_}_sim_1.4': f'lsst_{_}_smeared' for _ in 'ugrizy'}
            DC2_filterMap.update({_: f'lsst_{_}_smeared' for _ in 'ugrizy'})
            config.processCcd.calibrate.astromRefObjLoader.filterMap = DC2_filterMap
            config.processCcd.calibrate.photoRefObjLoader.filterMap = DC2_filterMap
            

            And

            config.processCcd.calibrate.astromRefObjLoader.filterMap = {f'{_}_sim_1.4': f'lsst_{_}_smeared' for _ in 'ugrizy'}
            config.processCcd.calibrate.photoRefObjLoader.filterMap = {f'{_}_sim_1.4': f'lsst_{_}_smeared' for _ in 'ugrizy'}
            

            Produces error:

            RuntimeError: FilterLabel(physical="g_sim_1.4") has no band.
            Traceback (most recent call last):
              File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_base/21.0.0-15-g361d225+ff3912ea15/python/lsst/pipe/base/cmdLineTask.py", line 431, in __call__
                result = self.runTask(task, dataRef, kwargs)
              File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_base/21.0.0-15-g361d225+ff3912ea15/python/lsst/pipe/base/cmdLineTask.py", line 495, in runTask
                return task.runDataRef(dataRef, **kwargs)
              File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_drivers/21.0.0-4-g3af6bfd+64d2251902/python/lsst/pipe/drivers/singleFrameDriver.py", line 83, in runDataRef
                result = self.processCcd.runDataRef(sensorRef)
              File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_base/21.0.0-15-g361d225+ff3912ea15/python/lsst/pipe/base/timer.py", line 157, in wrapper
                res = func(self, *args, **keyArgs)
              File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_tasks/21.0.0-58-g436064c8+549a115573/python/lsst/pipe/tasks/processCcd.py", line 181, in runDataRef
                exposure = self.isr.runDataRef(sensorRef).exposure
              File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_base/21.0.0-15-g361d225+ff3912ea15/python/lsst/pipe/base/timer.py", line 157, in wrapper
                res = func(self, *args, **keyArgs)
              File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/ip_isr/21.0.0-10-g17396bd+2c5eba4060/python/lsst/ip/isr/isrTask.py", line 1719, in runDataRef
                isrData = self.readIsrData(sensorRef, ccdExposure)
              File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/ip_isr/21.0.0-10-g17396bd+2c5eba4060/python/lsst/ip/isr/isrTask.py", line 1148, in readIsrData
                if self.config.doFringe and self.fringe.checkFilter(rawExposure)
              File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/ip_isr/21.0.0-10-g17396bd+2c5eba4060/python/lsst/ip/isr/fringe.py", line 257, in checkFilter
                return checkFilter(exposure, self.config.filters, log=self.log)
              File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/ip_isr/21.0.0-10-g17396bd+2c5eba4060/python/lsst/ip/isr/isrFunctions.py", line 897, in checkFilter
                elif thisFilter.bandLabel in filterList:
            

            Sorry I have been late responding. I have been tweaking a lot of knobs.

            Show
            emorganson Eric Morganson [X] (Inactive) added a comment - - edited This fix has not worked for me. To show current setup: setup lsst_distrib -t w_2021_08 # also lsst-distrib w_2021_12 setup -v -r /home/emorgan2/SOFTWARE/obs_lsst # Have also tried with -k and -j option singleFrameDriver.py /datasets/DC2/repoRun2.2i --rerun w_2021_08/DM-28966/sfm --batch-type slurm --mpiexec='-bind-to socket' --job dc2sfm08 --cores 24 --time 900 --id visit=159471^159491^159507^183772^183773^183818^183912^193780^193781^193827^194862^221574^221575^221577^221614^221615^221616^254358^254359^254379 --configfile /home/emorgan2/HSC/DC2/filter_config_2021_12.py --clobber-config --clobber-versions; filter_config_2021_12.py is: DC2_filterMap = {band:'lsst_%s_smeared' % (band) for band in 'ugrizy'} config.processCcd.calibrate.astromRefObjLoader.filterMap = DC2_filterMap config.processCcd.calibrate.photoRefObjLoader.filterMap = DC2_filterMap Have also tried: DC2_filterMap = {f'{_}_sim_1.4': f'lsst_{_}_smeared' for _ in 'ugrizy'} DC2_filterMap.update({_: f'lsst_{_}_smeared' for _ in 'ugrizy'}) config.processCcd.calibrate.astromRefObjLoader.filterMap = DC2_filterMap config.processCcd.calibrate.photoRefObjLoader.filterMap = DC2_filterMap And config.processCcd.calibrate.astromRefObjLoader.filterMap = {f'{_}_sim_1.4': f'lsst_{_}_smeared' for _ in 'ugrizy'} config.processCcd.calibrate.photoRefObjLoader.filterMap = {f'{_}_sim_1.4': f'lsst_{_}_smeared' for _ in 'ugrizy'} Produces error: RuntimeError: FilterLabel(physical="g_sim_1.4") has no band. Traceback (most recent call last): File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_base/21.0.0-15-g361d225+ff3912ea15/python/lsst/pipe/base/cmdLineTask.py", line 431, in __call__ result = self.runTask(task, dataRef, kwargs) File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_base/21.0.0-15-g361d225+ff3912ea15/python/lsst/pipe/base/cmdLineTask.py", line 495, in runTask return task.runDataRef(dataRef, **kwargs) File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_drivers/21.0.0-4-g3af6bfd+64d2251902/python/lsst/pipe/drivers/singleFrameDriver.py", line 83, in runDataRef result = self.processCcd.runDataRef(sensorRef) File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_base/21.0.0-15-g361d225+ff3912ea15/python/lsst/pipe/base/timer.py", line 157, in wrapper res = func(self, *args, **keyArgs) File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_tasks/21.0.0-58-g436064c8+549a115573/python/lsst/pipe/tasks/processCcd.py", line 181, in runDataRef exposure = self.isr.runDataRef(sensorRef).exposure File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_base/21.0.0-15-g361d225+ff3912ea15/python/lsst/pipe/base/timer.py", line 157, in wrapper res = func(self, *args, **keyArgs) File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/ip_isr/21.0.0-10-g17396bd+2c5eba4060/python/lsst/ip/isr/isrTask.py", line 1719, in runDataRef isrData = self.readIsrData(sensorRef, ccdExposure) File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/ip_isr/21.0.0-10-g17396bd+2c5eba4060/python/lsst/ip/isr/isrTask.py", line 1148, in readIsrData if self.config.doFringe and self.fringe.checkFilter(rawExposure) File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/ip_isr/21.0.0-10-g17396bd+2c5eba4060/python/lsst/ip/isr/fringe.py", line 257, in checkFilter return checkFilter(exposure, self.config.filters, log=self.log) File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/ip_isr/21.0.0-10-g17396bd+2c5eba4060/python/lsst/ip/isr/isrFunctions.py", line 897, in checkFilter elif thisFilter.bandLabel in filterList: Sorry I have been late responding. I have been tweaking a lot of knobs.
            Hide
            emorganson Eric Morganson [X] (Inactive) added a comment -

            Worked with Jim and my own git problems. Can confirm it works now.

            Show
            emorganson Eric Morganson [X] (Inactive) added a comment - Worked with Jim and my own git problems. Can confirm it works now.
            Hide
            lauren Lauren MacArthur added a comment -

            Phew...was just starting to try to reproduce (failing so far!)

            Show
            lauren Lauren MacArthur added a comment - Phew...was just starting to try to reproduce (failing so far!)
            Hide
            yusra Yusra AlSayyad added a comment -

            Preops because DC2-specific

            Show
            yusra Yusra AlSayyad added a comment - Preops because DC2-specific

              People

              Assignee:
              jbosch Jim Bosch
              Reporter:
              emorganson Eric Morganson [X] (Inactive)
              Reviewers:
              Lauren MacArthur
              Watchers:
              Eric Morganson [X] (Inactive), Jim Bosch, John Parejko, Krzysztof Findeisen, Lauren MacArthur, Tim Jenness, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.