# butler.get('raw',...) raises a lsst::pex::exceptions::NotFoundError for BOT data at lsst-dev

XMLWordPrintable

#### Details

• Type: Bug
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s: None
• Labels:
None
• Story Points:
6
• Team:
Data Release Production

#### Description

For the BOT data at lsst-dev, the following code raises a NotFoundError:

 from lsst.daf.persistence import Butler   BUTLER_BOT_REPO = '/lsstdata/offline/teststand/BOT/gen2repo' butler = Butler(BUTLER_BOT_REPO)   datarefs = butler.subset('raw', run='6545D', raftName='R10',  imagetype='BIAS', testtype='DARK')   for dataref in datarefs:  print(dataref.dataId)  exp = butler.get('raw', dataref.dataId) 

Here is the tail-end of the error output:

  File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/obs_base/18.1.0-11-g311e899+3/python/lsst/obs/base/cameraMapper.py", line 1076, in _standardizeExposure  self._setFilter(mapping, item, dataId)  File "/software/lsstsw/stack_20190330/stack/miniconda3-4.5.12-1172c30/Linux64/obs_base/18.1.0-11-g311e899+3/python/lsst/obs/base/cameraMapper.py", line 1030, in _setFilter  item.setFilter(afwImage.Filter(filterName)) lsst.pex.exceptions.wrappers.NotFoundError:   File "src/image/Filter.cc", line 335, in static int lsst::afw::image::Filter::_lookup(const string&, bool)  Unable to find filter SDSSu+ND_OD4.0 {0} lsst::pex::exceptions::NotFoundError: 'Unable to find filter SDSSu+ND_OD4.0' 

This is using lsst_distrib w_2019_33.

Camera I&T will start producing lots of BOT data with 9 science rafts and 4 corner rafts starting the second week of September 2019, so it would be good to have a fix for this before then.

#### Activity

Hide
Eli Rykoff added a comment -

After James Chiang told me about this, I took a quick look, and it seems that we should be able to leave the filter as UNKNOWN and log a warning and continue with reading the raw without crashing. I tried this on a user branch on obs_base and it seems to work. Is there any reason I can't turn this into a ticket branch to fix this? It would certainly be useful for all sorts of testing situations.

https://github.com/lsst/obs_base/commit/8f9f4da566e951fa8daf63b21e42e8e0e935ac65

Show
Eli Rykoff added a comment - After James Chiang told me about this, I took a quick look, and it seems that we should be able to leave the filter as UNKNOWN and log a warning and continue with reading the raw without crashing. I tried this on a user branch on obs_base and it seems to work. Is there any reason I can't turn this into a ticket branch to fix this? It would certainly be useful for all sorts of testing situations. https://github.com/lsst/obs_base/commit/8f9f4da566e951fa8daf63b21e42e8e0e935ac65
Hide
Tim Jenness added a comment -

That might mess up gen 3 butler ingest but I'll let Jim Bosch comment on that one.

Show
Tim Jenness added a comment - That might mess up gen 3 butler ingest but I'll let Jim Bosch comment on that one.
Hide
Jim Bosch added a comment -

Probably prudent to come back to this after DM-20763 lands (I think that should happen sometime today).  That changes the way filters are defined in obs packages quite a bit (in order to make it so Gen2 and Gen3 can use the same declarations).

Show
Jim Bosch added a comment - Probably prudent to come back to this after DM-20763 lands (I think that should happen sometime today).  That changes the way filters are defined in obs packages quite a bit (in order to make it so Gen2 and Gen3 can use the same declarations).
Hide
Eli Rykoff added a comment -

I will watch for that. But scanning the PR right now, I don't think it'll change that particular aspect of the reading of raws (but might change what tests are done).

Show
Eli Rykoff added a comment - I will watch for that. But scanning the PR right now, I don't think it'll change that particular aspect of the reading of raws (but might change what tests are done).
Hide
Eli Rykoff added a comment -

I've rebased against DM-20763 and things work just the same. Anybody want to review https://github.com/lsst/obs_base/pull/168 ?

Show
Eli Rykoff added a comment - I've rebased against DM-20763 and things work just the same. Anybody want to review https://github.com/lsst/obs_base/pull/168 ?

#### People

Assignee:
Eli Rykoff
Reporter:
James Chiang
Reviewers:
Jim Bosch
Watchers:
Eli Rykoff, Eric Charles, James Chiang, Jim Bosch, Kian-Tat Lim, Robert Lupton, Tim Jenness