Converting HiTS2015 repo fails on bad filter

XMLWordPrintable

Details

• Type: Bug
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
• Story Points:
3
• Sprint:
AP S21-4 (March)
• Team:
• Urgent?:
No

Description

While updating the ap_verify_hits2015 repo on DM-28438, I got the following error:

 convertRepo WARN: Skipping ingestion for '/tmp/tmpnw_qx4on/calibingested/cpFLAT/2015-02-23/Y/FLAT-2015-02-23.fits': Could not fetch record for required dimension physical_filter via keys {'instrument': 'DECam', 'detector': 62, 'physical_filter': 'Y'}. 

This looks suspiciously related to the ongoing filter conversion. Investigate and fix.

Activity

Hide
John Parejko added a comment - - edited

'Y' is not a DECam physical filter. "Y DECam c0005 10095.0 1130.0" is, however. Looks like this is one of the few cases where an alias from the FilterDefinitions was actually used!

Show
John Parejko added a comment - - edited 'Y' is not a DECam physical filter. "Y DECam c0005 10095.0 1130.0" is, however. Looks like this is one of the few cases where an alias from the FilterDefinitions was actually used!
Hide
Krzysztof Findeisen added a comment -

The culprit is obs.base.gen2to3.BandToPhysicalFilterKeyHandler (and higher-level code) considering only the bands and physical filters of any filter definition. Unfortunately, the way the existing code works looks like it would be very hard to generalize without unexpected side effects, so we decided on #dm-middleware-dev to just add some special-case code to obs_decam.

Show
Krzysztof Findeisen added a comment - The culprit is obs.base.gen2to3.BandToPhysicalFilterKeyHandler (and higher-level code) considering only the bands and physical filters of any filter definition. Unfortunately, the way the existing code works looks like it would be very hard to generalize without unexpected side effects, so we decided on #dm-middleware-dev to just add some special-case code to obs_decam .
Hide
Krzysztof Findeisen added a comment -

In addition to the code changes in obs_decam, I've re-converted ap_verify_hits2015 (to make the Y-band files available in Gen 3) and ap_verify_ci_hits2015 (to include the post-DM-23985 linearizer files as curated calibs).

Show
Krzysztof Findeisen added a comment - In addition to the code changes in obs_decam , I've re-converted ap_verify_hits2015 (to make the Y-band files available in Gen 3) and ap_verify_ci_hits2015 (to include the post- DM-23985 linearizer files as curated calibs).
Hide
Meredith Rawls added a comment -

Thanks for clearly explaining the situation and for fixing it!

Show
Meredith Rawls added a comment - Thanks for clearly explaining the situation and for fixing it!

People

Assignee:
Krzysztof Findeisen
Reporter:
Krzysztof Findeisen
Reviewers:
Meredith Rawls
Watchers:
John Parejko, Krzysztof Findeisen, Meredith Rawls