Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-28439

Converting HiTS2015 repo fails on bad filter

    XMLWordPrintable

    Details

    • Story Points:
      3
    • Sprint:
      AP S21-4 (March)
    • Team:
      Alert Production
    • Urgent?:
      No

      Description

      While updating the ap_verify_hits2015 repo on DM-28438, I got the following error:

      convertRepo WARN: Skipping ingestion for '/tmp/tmpnw_qx4on/calibingested/cpFLAT/2015-02-23/Y/FLAT-2015-02-23.fits': Could not fetch record for required dimension physical_filter via keys {'instrument': 'DECam', 'detector': 62, 'physical_filter': 'Y'}.
      

      This looks suspiciously related to the ongoing filter conversion. Investigate and fix.

        Attachments

          Issue Links

            Activity

            Hide
            Parejkoj John Parejko added a comment - - edited

            'Y' is not a DECam physical filter. "Y DECam c0005 10095.0 1130.0" is, however. Looks like this is one of the few cases where an alias from the FilterDefinitions was actually used!

            Show
            Parejkoj John Parejko added a comment - - edited 'Y' is not a DECam physical filter. "Y DECam c0005 10095.0 1130.0" is, however. Looks like this is one of the few cases where an alias from the FilterDefinitions was actually used!
            Hide
            krzys Krzysztof Findeisen added a comment -

            The culprit is obs.base.gen2to3.BandToPhysicalFilterKeyHandler (and higher-level code) considering only the bands and physical filters of any filter definition. Unfortunately, the way the existing code works looks like it would be very hard to generalize without unexpected side effects, so we decided on #dm-middleware-dev to just add some special-case code to obs_decam.

            Show
            krzys Krzysztof Findeisen added a comment - The culprit is obs.base.gen2to3.BandToPhysicalFilterKeyHandler (and higher-level code) considering only the bands and physical filters of any filter definition. Unfortunately, the way the existing code works looks like it would be very hard to generalize without unexpected side effects, so we decided on #dm-middleware-dev to just add some special-case code to obs_decam .
            Hide
            krzys Krzysztof Findeisen added a comment -

            In addition to the code changes in obs_decam, I've re-converted ap_verify_hits2015 (to make the Y-band files available in Gen 3) and ap_verify_ci_hits2015 (to include the post-DM-23985 linearizer files as curated calibs).

            Show
            krzys Krzysztof Findeisen added a comment - In addition to the code changes in obs_decam , I've re-converted ap_verify_hits2015 (to make the Y-band files available in Gen 3) and ap_verify_ci_hits2015 (to include the post- DM-23985 linearizer files as curated calibs).
            Hide
            mrawls Meredith Rawls added a comment -

            Thanks for clearly explaining the situation and for fixing it!

            Show
            mrawls Meredith Rawls added a comment - Thanks for clearly explaining the situation and for fixing it!

              People

              Assignee:
              krzys Krzysztof Findeisen
              Reporter:
              krzys Krzysztof Findeisen
              Reviewers:
              Meredith Rawls
              Watchers:
              John Parejko, Krzysztof Findeisen, Meredith Rawls
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.