Further investigation shows that, in fact, ImageDifferenceTask cannot handle DECam data if the user doesn't provide a filter. Since it's not clear how or even whether to fix this behavior in ImageDifferenceTask and/or obs_decam (see below), and I can't think of a workaround in ap_verify that wouldn't require special-case code, I'm marking this issue as Won't Fix and opening
DM-14359 to address the data ID handling problems I found in the meantime.
For the record, the process by which image differencing runs without an explicit filter fail on DECam is the following:
- ImageDifferenceTask's argument parser interprets data IDs as references to calexps.
- obs_decam gives calexps a URI that depends only on the visit and ccd.
- Based on parsing the mapping information, obs_base considers the "keys" for calexp data IDs to be visit and ccd.
- The Butler defines complete dataIds as those that have values for all their keys.
- Therefore, data IDs for DECam calexps never have a filter added, even though this information can be extracted from the repository. (Note that HSC's mapper does include the filter in a calexp's URI, so by the preceding logic a complete HSC calexp dataref has an explicit filter.)
- ip_diffim tries to look up templates (in my case, a deepCoaddPsfMatched) based on the calexp dataref, supplemented with tract and patch IDs.
- obs_base identifies coadds using tract, patch, and filter. The Butler tries to look up the missing filter information (based on all available information, i.e. visit, ccd, patch, and tract), but there are no tables listed in which to do the lookup. The information is provided in raw, but coadds are not, themselves, raw data, and not all lookups for coadds are guaranteed to have a visit in their dataId.
This is a contract mismatch problem: each step makes sense in isolation, but the pieces don't work together. Since there's no single piece that's obviously wrong, and changing any of the behaviors I listed above will almost certainly have side effects, I defer the question of what, if anything, can be done about it to people more familiar with the framework than I.