Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-28236

get('calexp_filterLabel') does not return a full label for pre-FilterLabel data

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: obs_base
    • Labels:

      Description

      butler.get('calexp').getFilterLabel() returns the correct patched value for pre-FilterLabel data (e.g. testdata_jointcal/hsc), but butler.get('calexp_filterLabel') does not. The problem is that CameraMapper._initMappings().getFilterLabel() does not call the new _setFilter() that has extra logic to fill out the filter label from the FilterDefinitions.

      One question is what happens with old calexps after we remove afw.Filter. Currently, _setFilter uses the FilterDefinitions, which should be entirely independent of afw.Filter and is fairly conservative in how it determines whether to replace the as-read FilterLabel. Do we want that to be the process in the future, or do we want to e.g. modify calexps during gen2->gen3 conversion so that they have the correct FilterLabel on disk, so we don't have to have any gen3 FilterLabel logic for processed data?

      I'm adding FilterDefinitionCollection.physical_to_band (dict) in DM-27170, but I don't think helps you because your findAll is much broader in what it searches. That dict is exactly what we want for raw files, but doesn't help for calexps that could have either band or physical or some other alias in their `FILTER` header key.

        Attachments

          Issue Links

            Activity

            Hide
            Parejkoj John Parejko added a comment -

            I'm marking this a blocker for DM-27170, because I discovered it while trying to remove afw.Filter from jointcal but the calexps in testdata_jointcal are only giving `band`, not `physical`.

            Show
            Parejkoj John Parejko added a comment - I'm marking this a blocker for DM-27170 , because I discovered it while trying to remove afw.Filter from jointcal but the calexps in testdata_jointcal are only giving `band`, not `physical`.
            Hide
            tjenness Tim Jenness added a comment -

            To be clear, this is gen2 failure? Does gen3 work?

            Show
            tjenness Tim Jenness added a comment - To be clear, this is gen2 failure? Does gen3 work?
            Hide
            krzys Krzysztof Findeisen added a comment - - edited

            The bug was detected in Gen 2, yes.

            Currently there is no filter patching done in Gen 3, aside from the creation of the original FilterLabel object for raws. So unless the Gen 3 files themselves have been corrected, I would expect both get('calexp').getFilterLabel() and get('calexp.filterLabel') to potentially return incomplete information.

            Show
            krzys Krzysztof Findeisen added a comment - - edited The bug was detected in Gen 2, yes. Currently there is no filter patching done in Gen 3, aside from the creation of the original FilterLabel object for raws. So unless the Gen 3 files themselves have been corrected, I would expect both get('calexp').getFilterLabel() and get('calexp.filterLabel') to potentially return incomplete information.
            Hide
            krzys Krzysztof Findeisen added a comment -

            John Parejko, since you discovered and reported the bug, can you review it? The fixes themselves are small, but some refactoring pushes it up to 130 lines.

            Show
            krzys Krzysztof Findeisen added a comment - John Parejko , since you discovered and reported the bug, can you review it? The fixes themselves are small, but some refactoring pushes it up to 130 lines.
            Hide
            Parejkoj John Parejko added a comment -

            Thank you for the quick fix. I made one more reply about adding a comment, but otherwise this is good.

            Show
            Parejkoj John Parejko added a comment - Thank you for the quick fix. I made one more reply about adding a comment, but otherwise this is good.

              People

              Assignee:
              krzys Krzysztof Findeisen
              Reporter:
              Parejkoj John Parejko
              Reviewers:
              John Parejko
              Watchers:
              Eli Rykoff, John Parejko, Krzysztof Findeisen, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins Builds

                  No builds found.