Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-29335

FilterLabel of DC2 exposures are not populated with bandLabel

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Invalid
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: afw, obs_base, obs_lsst
    • Labels:
      None
    • Story Points:
      4
    • Sprint:
      AP S21-4 (March)
    • Team:
      Alert Production
    • Urgent?:
      No

      Description

      Trying to process DC2 data from /datasets/DC2/repoRun2.2i/ results in the following error:

      RuntimeError: FilterLabel(physical="r_sim_1.4") has no band.
      

      The first time this is encountered is in isr.fringe with a call to the checkFilters() function in isrFunctions.py :
      https://github.com/lsst/ip_isr/blob/master/python/lsst/ip/isr/isrFunctions.py#L870-L903
       
      There seems to be two issues here.  One is that the filterList being passed in in this case is empty, so there shouldn’t even be a check being made.  This begs the question of whether this function should start with a check along the lines (or an error and make the calling code responsible for passing a non-empty list?):

          if len(filterList) == 0:
              return False
      

      Even if I add this condition and get past isr, I then get a similar failure in meas_astrom at the filterLabel setting here: https://github.com/lsst/meas_astrom/blob/master/python/lsst/meas/astrom/ref_match.py#L257
      Indeed, the exposure in play here does not have a bandLabel associated with its FilterLabel:

      filterLabel = exposure.getFilterLabel()
      print(filterLabel)
      FilterLabel(physical="r_sim_1.4")
      

      so the attempt to call filterLabel.bandLabel is what is failing and causing the RuntimeError.  This ticket is to address the lack of bandLabel being associated
      with these exposures.  A separate ticket will address the issue of whether this lack of bandLabel should be caught earlier (and when/where – if at all – it would be appropriate to be checking FilterLabel.hasBandLabel() before trying to access bandLabel).

      This seems to be related to the issue on DM-29313.

        Attachments

          Issue Links

            Activity

            Hide
            lauren Lauren MacArthur added a comment - - edited

            Yep, you could try something like:

            $ processCcd.py /datasets/DC2/repoRun2.2i --rerun private/you/yourRerun --id visit=440938 filter=r detector=0
            root INFO: Loading config overrride file '/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/obs_lsst/21.0.0-28-g8f26b9b+fd2772fd94/config/processCcd.py'
            root INFO: Loading config overrride file '/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/obs_lsst/21.0.0-28-g8f26b9b+fd2772fd94/config/imsim/processCcd.py'
            CameraMapper INFO: Loading exposure registry from /datasets/DC2/repoRun2.2i/registry.sqlite3
            CameraMapper INFO: Loading calib registry from /datasets/DC2/repoRun2.2i/CALIB/calibRegistry.sqlite3
            CameraMapper INFO: Loading calib registry from /datasets/DC2/repoRun2.2i/CALIB/calibRegistry.sqlite3
            root WARN: Unexpected ID visit; guessing type is "str"
            root WARN: Unexpected ID filter; guessing type is "str"
            root INFO: Running: /software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_tasks/21.0.0-58-g436064c8+549a115573/bin/processCcd.py /datasets/DC2/repoRun2.2i --rerun private/lauren/junk --no-versions --id visit=440938 filter=r detector=0
            processCcd INFO: Processing {'visit': 440938, 'filter': 'r', 'detector': 0, 'run': '440938', 'raftName': 'R01', 'expId': 440938, 'detectorName': 'S00'}
            processCcd.isr INFO: Performing ISR on sensor {'visit': 440938, 'filter': 'r', 'detector': 0, 'run': '440938', 'raftName': 'R01', 'expId': 440938, 'detectorName': 'S00'}.
            processCcd FATAL: Failed on dataId={'visit': 440938, 'filter': 'r', 'detector': 0, 'run': '440938', 'raftName': 'R01', 'expId': 440938, 'detectorName': 'S00'}: RuntimeError: FilterLabel(physical="r_sim_1.4") has no band.
            Traceback (most recent call last):
              File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_base/21.0.0-15-g361d225+ff3912ea15/python/lsst/pipe/base/cmdLineTask.py", line 431, in __call__
                result = self.runTask(task, dataRef, kwargs)
              File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_base/21.0.0-15-g361d225+ff3912ea15/python/lsst/pipe/base/cmdLineTask.py", line 495, in runTask
                return task.runDataRef(dataRef, **kwargs)
              File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_base/21.0.0-15-g361d225+ff3912ea15/python/lsst/pipe/base/timer.py", line 157, in wrapper
                res = func(self, *args, **keyArgs)
              File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_tasks/21.0.0-58-g436064c8+549a115573/python/lsst/pipe/tasks/processCcd.py", line 181, in runDataRef
                exposure = self.isr.runDataRef(sensorRef).exposure
              File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_base/21.0.0-15-g361d225+ff3912ea15/python/lsst/pipe/base/timer.py", line 157, in wrapper
                res = func(self, *args, **keyArgs)
              File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/ip_isr/21.0.0-10-g17396bd+2c5eba4060/python/lsst/ip/isr/isrTask.py", line 1719, in runDataRef
                isrData = self.readIsrData(sensorRef, ccdExposure)
              File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/ip_isr/21.0.0-10-g17396bd+2c5eba4060/python/lsst/ip/isr/isrTask.py", line 1148, in readIsrData
                if self.config.doFringe and self.fringe.checkFilter(rawExposure)
              File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/ip_isr/21.0.0-10-g17396bd+2c5eba4060/python/lsst/ip/isr/fringe.py", line 257, in checkFilter
                return checkFilter(exposure, self.config.filters, log=self.log)
              File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/ip_isr/21.0.0-10-g17396bd+2c5eba4060/python/lsst/ip/isr/isrFunctions.py", line 897, in checkFilter
                elif thisFilter.bandLabel in filterList:
            RuntimeError: FilterLabel(physical="r_sim_1.4") has no band.
            

            Show
            lauren Lauren MacArthur added a comment - - edited Yep, you could try something like: $ processCcd.py / datasets / DC2 / repoRun2. 2i - - rerun private / you / yourRerun - - id visit = 440938 filter = r detector = 0 root INFO: Loading config overrride file '/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/obs_lsst/21.0.0-28-g8f26b9b+fd2772fd94/config/processCcd.py' root INFO: Loading config overrride file '/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/obs_lsst/21.0.0-28-g8f26b9b+fd2772fd94/config/imsim/processCcd.py' CameraMapper INFO: Loading exposure registry from / datasets / DC2 / repoRun2. 2i / registry.sqlite3 CameraMapper INFO: Loading calib registry from / datasets / DC2 / repoRun2. 2i / CALIB / calibRegistry.sqlite3 CameraMapper INFO: Loading calib registry from / datasets / DC2 / repoRun2. 2i / CALIB / calibRegistry.sqlite3 root WARN: Unexpected ID visit; guessing type is "str" root WARN: Unexpected ID filter ; guessing type is "str" root INFO: Running: / software / lsstsw / stack_20210211 / stack / miniconda3 - py38_4. 9.2 - 0.4 . 1 / Linux64 / pipe_tasks / 21.0 . 0 - 58 - g436064c8 + 549a115573 / bin / processCcd.py / datasets / DC2 / repoRun2. 2i - - rerun private / lauren / junk - - no - versions - - id visit = 440938 filter = r detector = 0 processCcd INFO: Processing { 'visit' : 440938 , 'filter' : 'r' , 'detector' : 0 , 'run' : '440938' , 'raftName' : 'R01' , 'expId' : 440938 , 'detectorName' : 'S00' } processCcd.isr INFO: Performing ISR on sensor { 'visit' : 440938 , 'filter' : 'r' , 'detector' : 0 , 'run' : '440938' , 'raftName' : 'R01' , 'expId' : 440938 , 'detectorName' : 'S00' }. processCcd FATAL: Failed on dataId = { 'visit' : 440938 , 'filter' : 'r' , 'detector' : 0 , 'run' : '440938' , 'raftName' : 'R01' , 'expId' : 440938 , 'detectorName' : 'S00' }: RuntimeError: FilterLabel(physical = "r_sim_1.4" ) has no band. Traceback (most recent call last): File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_base/21.0.0-15-g361d225+ff3912ea15/python/lsst/pipe/base/cmdLineTask.py" , line 431 , in __call__ result = self .runTask(task, dataRef, kwargs) File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_base/21.0.0-15-g361d225+ff3912ea15/python/lsst/pipe/base/cmdLineTask.py" , line 495 , in runTask return task.runDataRef(dataRef, * * kwargs) File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_base/21.0.0-15-g361d225+ff3912ea15/python/lsst/pipe/base/timer.py" , line 157 , in wrapper res = func( self , * args, * * keyArgs) File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_tasks/21.0.0-58-g436064c8+549a115573/python/lsst/pipe/tasks/processCcd.py" , line 181 , in runDataRef exposure = self .isr.runDataRef(sensorRef).exposure File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_base/21.0.0-15-g361d225+ff3912ea15/python/lsst/pipe/base/timer.py" , line 157 , in wrapper res = func( self , * args, * * keyArgs) File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/ip_isr/21.0.0-10-g17396bd+2c5eba4060/python/lsst/ip/isr/isrTask.py" , line 1719 , in runDataRef isrData = self .readIsrData(sensorRef, ccdExposure) File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/ip_isr/21.0.0-10-g17396bd+2c5eba4060/python/lsst/ip/isr/isrTask.py" , line 1148 , in readIsrData if self .config.doFringe and self .fringe.checkFilter(rawExposure) File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/ip_isr/21.0.0-10-g17396bd+2c5eba4060/python/lsst/ip/isr/fringe.py" , line 257 , in checkFilter return checkFilter(exposure, self .config.filters, log = self .log) File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/ip_isr/21.0.0-10-g17396bd+2c5eba4060/python/lsst/ip/isr/isrFunctions.py" , line 897 , in checkFilter elif thisFilter.bandLabel in filterList: RuntimeError: FilterLabel(physical = "r_sim_1.4" ) has no band.
            Hide
            krzys Krzysztof Findeisen added a comment -

            Perfect, thanks!

            Show
            krzys Krzysztof Findeisen added a comment - Perfect, thanks!
            Hide
            jbosch Jim Bosch added a comment -

            I think I just fixed this on tickets/DM-29446 (still on a branch); didn't know about this ticket until I pushed.

            Show
            jbosch Jim Bosch added a comment - I think I just fixed this on tickets/ DM-29446 (still on a branch); didn't know about this ticket until I pushed.
            Hide
            krzys Krzysztof Findeisen added a comment -

            Lauren MacArthur, should this be closed as a duplicate of DM-29446?

            Show
            krzys Krzysztof Findeisen added a comment - Lauren MacArthur , should this be closed as a duplicate of DM-29446 ?
            Hide
            lauren Lauren MacArthur added a comment -

            Looks like it (I haven't followed if the fix there is considered a band-aid, or if it's the correct and final solution).

            Show
            lauren Lauren MacArthur added a comment - Looks like it (I haven't followed if the fix there is considered a band-aid, or if it's the correct and final solution).

              People

              Assignee:
              krzys Krzysztof Findeisen
              Reporter:
              lauren Lauren MacArthur
              Watchers:
              Jim Bosch, John Parejko, Krzysztof Findeisen, Lauren MacArthur, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.