[DM-29335] FilterLabel of DC2 exposures are not populated with bandLabel - Jira

Details

Type: Bug
Status: Invalid
Resolution: Done
Fix Version/s: None
Component/s: afw, obs_base, obs_lsst
Labels:
None

Story Points:
4
Epic Link:
s21b-ap-emergent
Sprint:
AP S21-4 (March)
Team:
Alert Production
Urgent?:
No

Description

Trying to process DC2 data from /datasets/DC2/repoRun2.2i/ results in the following error:

RuntimeError: FilterLabel(physical="r_sim_1.4") has no band.

The first time this is encountered is in isr.fringe with a call to the checkFilters() function in isrFunctions.py :
https://github.com/lsst/ip_isr/blob/master/python/lsst/ip/isr/isrFunctions.py#L870-L903

There seems to be two issues here. One is that the filterList being passed in in this case is empty, so there shouldn’t even be a check being made. This begs the question of whether this function should start with a check along the lines (or an error and make the calling code responsible for passing a non-empty list?):

    if len(filterList) == 0:

        return False

Even if I add this condition and get past isr, I then get a similar failure in meas_astrom at the filterLabel setting here: https://github.com/lsst/meas_astrom/blob/master/python/lsst/meas/astrom/ref_match.py#L257
Indeed, the exposure in play here does not have a bandLabel associated with its FilterLabel:

filterLabel = exposure.getFilterLabel()

print(filterLabel)

FilterLabel(physical="r_sim_1.4")

so the attempt to call filterLabel.bandLabel is what is failing and causing the RuntimeError. This ticket is to address the lack of bandLabel being associated
with these exposures. A separate ticket will address the issue of whether this lack of bandLabel should be caught earlier (and when/where – if at all – it would be appropriate to be checking FilterLabel.hasBandLabel() before trying to access bandLabel).

This seems to be related to the issue on ~~DM-29313~~.

Attachments

Issue Links

blocks

DM-25629 Do release 22.0.0 of science pipelines

Done

duplicates

DM-29446 DC2 singleFrameDriver Failure: FilterLabel(physical="z_sim_1.4") has no band.

Done

relates to

DM-29313 FilterLabel is not properly filled in for CFHT raws

Done

Activity

Ascending order - Click to sort in descending order

5 older comments

Hide

Permalink

Lauren MacArthur added a comment - 25/Mar/21 11:18 PM - edited

Yep, you could try something like:

$ processCcd.py /datasets/DC2/repoRun2.2i --rerun private/you/yourRerun --id visit=440938 filter=r detector=0

root INFO: Loading config overrride file '/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/obs_lsst/21.0.0-28-g8f26b9b+fd2772fd94/config/processCcd.py'

root INFO: Loading config overrride file '/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/obs_lsst/21.0.0-28-g8f26b9b+fd2772fd94/config/imsim/processCcd.py'

CameraMapper INFO: Loading exposure registry from /datasets/DC2/repoRun2.2i/registry.sqlite3

CameraMapper INFO: Loading calib registry from /datasets/DC2/repoRun2.2i/CALIB/calibRegistry.sqlite3

CameraMapper INFO: Loading calib registry from /datasets/DC2/repoRun2.2i/CALIB/calibRegistry.sqlite3

root WARN: Unexpected ID visit; guessing type is "str"

root WARN: Unexpected ID filter; guessing type is "str"

root INFO: Running: /software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_tasks/21.0.0-58-g436064c8+549a115573/bin/processCcd.py /datasets/DC2/repoRun2.2i --rerun private/lauren/junk --no-versions --id visit=440938 filter=r detector=0

processCcd INFO: Processing {'visit': 440938, 'filter': 'r', 'detector': 0, 'run': '440938', 'raftName': 'R01', 'expId': 440938, 'detectorName': 'S00'}

processCcd.isr INFO: Performing ISR on sensor {'visit': 440938, 'filter': 'r', 'detector': 0, 'run': '440938', 'raftName': 'R01', 'expId': 440938, 'detectorName': 'S00'}.

processCcd FATAL: Failed on dataId={'visit': 440938, 'filter': 'r', 'detector': 0, 'run': '440938', 'raftName': 'R01', 'expId': 440938, 'detectorName': 'S00'}: RuntimeError: FilterLabel(physical="r_sim_1.4") has no band.

Traceback (most recent call last):

  File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_base/21.0.0-15-g361d225+ff3912ea15/python/lsst/pipe/base/cmdLineTask.py", line 431, in __call__

    result = self.runTask(task, dataRef, kwargs)

  File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_base/21.0.0-15-g361d225+ff3912ea15/python/lsst/pipe/base/cmdLineTask.py", line 495, in runTask

    return task.runDataRef(dataRef, **kwargs)

  File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_base/21.0.0-15-g361d225+ff3912ea15/python/lsst/pipe/base/timer.py", line 157, in wrapper

    res = func(self, *args, **keyArgs)

  File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_tasks/21.0.0-58-g436064c8+549a115573/python/lsst/pipe/tasks/processCcd.py", line 181, in runDataRef

    exposure = self.isr.runDataRef(sensorRef).exposure

  File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_base/21.0.0-15-g361d225+ff3912ea15/python/lsst/pipe/base/timer.py", line 157, in wrapper

    res = func(self, *args, **keyArgs)

  File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/ip_isr/21.0.0-10-g17396bd+2c5eba4060/python/lsst/ip/isr/isrTask.py", line 1719, in runDataRef

    isrData = self.readIsrData(sensorRef, ccdExposure)

  File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/ip_isr/21.0.0-10-g17396bd+2c5eba4060/python/lsst/ip/isr/isrTask.py", line 1148, in readIsrData

    if self.config.doFringe and self.fringe.checkFilter(rawExposure)

  File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/ip_isr/21.0.0-10-g17396bd+2c5eba4060/python/lsst/ip/isr/fringe.py", line 257, in checkFilter

    return checkFilter(exposure, self.config.filters, log=self.log)

  File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/ip_isr/21.0.0-10-g17396bd+2c5eba4060/python/lsst/ip/isr/isrFunctions.py", line 897, in checkFilter

    elif thisFilter.bandLabel in filterList:

RuntimeError: FilterLabel(physical="r_sim_1.4") has no band.

Show

Lauren MacArthur added a comment - 25/Mar/21 11:18 PM - edited Yep, you could try something like: $ processCcd.py / datasets / DC2 / repoRun2. 2i - - rerun private / you / yourRerun - - id visit = 440938 filter = r detector = 0 root INFO: Loading config overrride file '/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/obs_lsst/21.0.0-28-g8f26b9b+fd2772fd94/config/processCcd.py' root INFO: Loading config overrride file '/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/obs_lsst/21.0.0-28-g8f26b9b+fd2772fd94/config/imsim/processCcd.py' CameraMapper INFO: Loading exposure registry from / datasets / DC2 / repoRun2. 2i / registry.sqlite3 CameraMapper INFO: Loading calib registry from / datasets / DC2 / repoRun2. 2i / CALIB / calibRegistry.sqlite3 CameraMapper INFO: Loading calib registry from / datasets / DC2 / repoRun2. 2i / CALIB / calibRegistry.sqlite3 root WARN: Unexpected ID visit; guessing type is "str" root WARN: Unexpected ID filter ; guessing type is "str" root INFO: Running: / software / lsstsw / stack_20210211 / stack / miniconda3 - py38_4. 9.2 - 0.4 . 1 / Linux64 / pipe_tasks / 21.0 . 0 - 58 - g436064c8 + 549a115573 / bin / processCcd.py / datasets / DC2 / repoRun2. 2i - - rerun private / lauren / junk - - no - versions - - id visit = 440938 filter = r detector = 0 processCcd INFO: Processing { 'visit' : 440938 , 'filter' : 'r' , 'detector' : 0 , 'run' : '440938' , 'raftName' : 'R01' , 'expId' : 440938 , 'detectorName' : 'S00' } processCcd.isr INFO: Performing ISR on sensor { 'visit' : 440938 , 'filter' : 'r' , 'detector' : 0 , 'run' : '440938' , 'raftName' : 'R01' , 'expId' : 440938 , 'detectorName' : 'S00' }. processCcd FATAL: Failed on dataId = { 'visit' : 440938 , 'filter' : 'r' , 'detector' : 0 , 'run' : '440938' , 'raftName' : 'R01' , 'expId' : 440938 , 'detectorName' : 'S00' }: RuntimeError: FilterLabel(physical = "r_sim_1.4" ) has no band. Traceback (most recent call last): File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_base/21.0.0-15-g361d225+ff3912ea15/python/lsst/pipe/base/cmdLineTask.py" , line 431 , in __call__ result = self .runTask(task, dataRef, kwargs) File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_base/21.0.0-15-g361d225+ff3912ea15/python/lsst/pipe/base/cmdLineTask.py" , line 495 , in runTask return task.runDataRef(dataRef, * * kwargs) File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_base/21.0.0-15-g361d225+ff3912ea15/python/lsst/pipe/base/timer.py" , line 157 , in wrapper res = func( self , * args, * * keyArgs) File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_tasks/21.0.0-58-g436064c8+549a115573/python/lsst/pipe/tasks/processCcd.py" , line 181 , in runDataRef exposure = self .isr.runDataRef(sensorRef).exposure File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/pipe_base/21.0.0-15-g361d225+ff3912ea15/python/lsst/pipe/base/timer.py" , line 157 , in wrapper res = func( self , * args, * * keyArgs) File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/ip_isr/21.0.0-10-g17396bd+2c5eba4060/python/lsst/ip/isr/isrTask.py" , line 1719 , in runDataRef isrData = self .readIsrData(sensorRef, ccdExposure) File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/ip_isr/21.0.0-10-g17396bd+2c5eba4060/python/lsst/ip/isr/isrTask.py" , line 1148 , in readIsrData if self .config.doFringe and self .fringe.checkFilter(rawExposure) File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/ip_isr/21.0.0-10-g17396bd+2c5eba4060/python/lsst/ip/isr/fringe.py" , line 257 , in checkFilter return checkFilter(exposure, self .config.filters, log = self .log) File "/software/lsstsw/stack_20210211/stack/miniconda3-py38_4.9.2-0.4.1/Linux64/ip_isr/21.0.0-10-g17396bd+2c5eba4060/python/lsst/ip/isr/isrFunctions.py" , line 897 , in checkFilter elif thisFilter.bandLabel in filterList: RuntimeError: FilterLabel(physical = "r_sim_1.4" ) has no band.

Hide

Permalink

Krzysztof Findeisen added a comment - 25/Mar/21 11:22 PM

Perfect, thanks!

Show

Krzysztof Findeisen added a comment - 25/Mar/21 11:22 PM Perfect, thanks!

Hide

Permalink

Jim Bosch added a comment - 26/Mar/21 11:02 PM

I think I just fixed this on tickets/~~DM-29446~~ (still on a branch); didn't know about this ticket until I pushed.

Show

Jim Bosch added a comment - 26/Mar/21 11:02 PM I think I just fixed this on tickets/ DM-29446 (still on a branch); didn't know about this ticket until I pushed.

Hide

Permalink

Krzysztof Findeisen added a comment - 29/Mar/21 5:07 PM

Lauren MacArthur, should this be closed as a duplicate of ~~DM-29446~~?

Show

Krzysztof Findeisen added a comment - 29/Mar/21 5:07 PM Lauren MacArthur , should this be closed as a duplicate of DM-29446 ?

Hide

Permalink

Lauren MacArthur added a comment - 29/Mar/21 6:34 PM

Looks like it (I haven't followed if the fix there is considered a band-aid, or if it's the correct and final solution).

Show

Lauren MacArthur added a comment - 29/Mar/21 6:34 PM Looks like it (I haven't followed if the fix there is considered a band-aid, or if it's the correct and final solution).

Data Management

FilterLabel of DC2 exposures are not populated with bandLabel

Details

Description

Attachments

Issue Links

Activity

People

Dates

Jenkins