Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: obs_decam
-
Labels:
-
Story Points:12
-
Epic Link:
-
Sprint:AP F19-2
-
Team:Alert Production
Description
In order to best support AP PipelineTask conversion work at PCW2019, we need to have a Gen3 data repository for ap_verify_hits2015 (or maybe ap_verify_ci_hits2015, or both?).
Anyhow, first step towards that is Gen3 Butler support for obs_decam, which is what this ticket is about. That includes:
- Writing an Instrument subclass for DECam. The docs for the base class should tell you most of what you need to do (though I've just discovered they're not appearing in the Sphinx HTML for some reason), but I'm sure a concrete example will be even more useful, which HSC provides. Note that because this will be only the second Instrument subclass, it's probable we'll find we want to adjust the ABC-subclass boundary a bit.
- Writing a Formatter for raw DECam data (or if necessary, different formatters for different CCDs), probably by inheriting from FitsRawFormatterBase. Once again, you can use HSC as an example. This should do the same thing as the Gen2 CameraMapper's std_raw method. A test that shows the kind of consistency we expect (for HSC) can be found in ci_hsc_gen2 - note that the Gen3 version generally tries much harder to clean things up and make things are components are internally consistent, so it may not be possible to make them entirely consistent with the Gen2 version. We may not have a good place to put such a test for DECam until we're further along converting an actual test dataset, but it'd be great to write one for one-off testing now and attach it to this ticket, with the expectation that we'll put it in a CI'd package in the future.
After those steps (and after running the Instrument class's register method on a Gen3 repo), I think it should be possible to run the Gen3 raw ingest task on DECam, and I consider that the end point for this ticket as long as any unexpected problems aren't huge. This will rely a lot on the DECam translator in astro_metadata_translator as well as the above, but I think that's already working.
Note that there's no command-line driver script for that task yet, but it's pretty straightforward to run from an interactive Python prompt. Given that we know that DECam has buggy boresight information in the headers, I don't expect it to populate the database with good spatial regions yet, and I'm okay with fixing that on another ticket (probably by using astro_metadata_translator's on-the-fly header-patching system to just fix the boresight values to something we've fit for the test data we care about).
Please ping #dm-middleware on Slack with any questions.
Attachments
Issue Links
- blocks
-
DM-16297 Add Gen3 Instrument and Formatter classes for obs_lsst
- Done
- is triggering
-
DM-20994 Write how to make gen3 obs package guide
- Done
-
DM-22708 Fix decam gen3 ingest
- Done
-
DM-21016 Handle DECam instrument signature data in gen3
- Done
- relates to
-
DM-20842 Move FileDescriptor to a property in Formatter
- Done
-
DM-28460 HSC ENG-R1 filter seems to have wrong band
- Done
A standard Jenkins run (without ci_hsc_gen3) passed with this version: I've made a tweak to jointcal's tests so that it behaves better with the FilterDefinitions. I think any better solution will have to wait on the implementation of
RFC-624.Could someone please look at the jointcal PR? https://github.com/lsst/jointcal/pull/147
We'll have to decide whether getting ci_hsc_gen3 to pass is a requirement for merging this or not.