Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: obs_decam
-
Labels:None
-
Story Points:13
-
Epic Link:
-
Team:Data Facility
Description
The current obs_decam expects instrument calibrated data from the community pipeline, i.e. it requires matching instcal (Instrument Calibrated), dqmask (the associated mask file), and wtmap (weight map) data from the same visit. This issue is to add functionality so that raw DECam images can be ingested into registry and retrieved by the data butler.
Practically, this will create new or expand existing sub-classes of CameraMapper and IngestTask.
A brief summary of changes:
- The unit test getRaw.py is updated and should pass, with
DM-3196 - Working testdata_decam for the unit test is currently at lsst-dev /lsst8/testdata_decam and https://uofi.box.com/testdata-decam
- DecamInstcalMapper is renamed to DecamMapper, to reflect that Butler can also get "raw" now besides "instcal". Please update _mapper in your data repositories.
- To create a registry for raw data, run
ingestImagesDecam.py /path/to/repo --mode=link --filetype="raw" /path/*.fits.fz
- The default filetype is "instcal" for ingestImagesDecam.py, so previous use for instcal stays.
Attachments
Issue Links
Activity
Thank you for your review!
Current CameraMapper standardize the input DecoratedImage into Exposure. So I guess when DecoratedImage is actually deprecated (DM-3190, DM-3191), something will be done with CameraMapper as well. DM-3191 seems to suggest there might be a new type to use.
Simply using Exposure in the policy file doesn't work. I will investigate more what else is needed to make it work (probably standardizing the raw imgaes in obs_decam).
Hsin-Fang Chiang
added a comment - - edited Thank you for your review!
Current CameraMapper standardize the input DecoratedImage into Exposure. So I guess when DecoratedImage is actually deprecated ( DM-3190 , DM-3191 ), something will be done with CameraMapper as well. DM-3191 seems to suggest there might be a new type to use.
Simply using Exposure in the policy file doesn't work. I will investigate more what else is needed to make it work (probably standardizing the raw imgaes in obs_decam). By Paul Price,
DecoratedImage may be deprecated, but it's the only thing we have that fits here. The Exposure constructor will now produce a warning because they are not actually Exposures (e.g., no separate Wcs extension), and plain Image doesn't give us the FITS header.
I would leave it as DecoratedImage for now.
If it all looks good, I'll merge it to master.
Hsin-Fang Chiang
added a comment - By Paul Price ,
DecoratedImage may be deprecated, but it's the only thing we have that fits here. The Exposure constructor will now produce a warning because they are not actually Exposures (e.g., no separate Wcs extension), and plain Image doesn't give us the FITS header.
I would leave it as DecoratedImage for now.
If it all looks good, I'll merge it to master. Sounds good. Merge away.
Merged to master.
README.md | 6 +- |
config/ingest.py | 1 + |
decam/camGeom/camera.py | 2630 +++++++++++++++++----------------- |
policy/DecamMapper.paf | 6 +- |
python/lsst/obs/decam/decamMapper.py | 16 +- |
python/lsst/obs/decam/ingest.py | 120 +- |
tests/getRaw.py | 73 +- |
7 files changed, 1435 insertions(+), 1417 deletions(-) |
Hsin-Fang Chiang
added a comment - Merged to master.
README.md | 6 +-
config/ingest.py | 1 +
decam/camGeom/camera.py | 2630 +++++++++++++++++-----------------
policy/DecamMapper.paf | 6 +-
python/lsst/obs/decam/decamMapper.py | 16 +-
python/lsst/obs/decam/ingest.py | 120 +-
tests/getRaw.py | 73 +-
7 files changed, 1435 insertions(+), 1417 deletions(-)
This looks good to me. I like having one ingester to ingest both types of data. I also think we should use Exposure instead of DecoratedImage, unless it won't work otherwise.
Some minor comments at: https://github.com/lsst/obs_decam/pull/2
I didn't run the unit tests, but I can if you'd like me to for completeness.