Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-16339

Remove camFlux fields from LoadReferenceObjectsTask

    Details

    • Story Points:
      2
    • Sprint:
      AP F18-6, AP S19-1, AP S19-2, AP S19-3
    • Team:
      Alert Production

      Description

      The various camFlux fields in the existing reference catalogs are unused and their purpose is unclear (especially now that we have color term maps). We should remove them entirely from meas_algorithms.

        Attachments

          Issue Links

            Activity

            Hide
            Parejkoj John Parejko added a comment -

            Switching this back to `ToDo`. Per my initial work, and a conversation with Simon Krughoff, Russell Owen and Jim Bosch, it seems that everyone's assumptions about camFlux not being necessary were unfounded. LoadReferenceObjectsTask._addFluxAliases() uses the filterMap to create camFlux_X->flux_Y aliases that are used by getRefFluxField to determine the closest flux field to a given filter for that camera. This approach is rather non-obvious (as evidenced by the people who wrote it being confused by it) and makes it very clumsy to create refcats for testing purposes, but we were not able to come up with an obviously better solution.

            We may want to wait for gen2 butler to go away and then RFC a full rethink of how filterMaps are handled.

            Show
            Parejkoj John Parejko added a comment - Switching this back to `ToDo`. Per my initial work, and a conversation with Simon Krughoff , Russell Owen and Jim Bosch , it seems that everyone's assumptions about camFlux not being necessary were unfounded. LoadReferenceObjectsTask._addFluxAliases() uses the filterMap to create camFlux_X->flux_Y aliases that are used by getRefFluxField to determine the closest flux field to a given filter for that camera. This approach is rather non-obvious (as evidenced by the people who wrote it being confused by it) and makes it very clumsy to create refcats for testing purposes, but we were not able to come up with an obviously better solution. We may want to wait for gen2 butler to go away and then RFC a full rethink of how filterMaps are handled.
            Hide
            Parejkoj John Parejko added a comment -

            A further thought on this: when a user requests the "flux" for their camera's filter from a reference catalog, what they almost certainly want is the flux in their filter's system. This suggests that we might be better off totally removing the camFlux aliases and filterMap and moving the colorterm calculations into the object returned by LoadReferenceObjectsTask, so that refcat.getFlux(filter) (or whatever) just gives you the corrected flux.

            This would require quite a bit of rethinking, but the way colorterms are managed right now is also rather clumsy.

            Show
            Parejkoj John Parejko added a comment - A further thought on this: when a user requests the "flux" for their camera's filter from a reference catalog, what they almost certainly want is the flux in their filter's system. This suggests that we might be better off totally removing the camFlux aliases and filterMap and moving the colorterm calculations into the object returned by LoadReferenceObjectsTask , so that refcat.getFlux(filter) (or whatever) just gives you the corrected flux. This would require quite a bit of rethinking, but the way colorterms are managed right now is also rather clumsy.
            Hide
            jbosch Jim Bosch added a comment -

            I would ultimately like to replace colorterms with a combination of TransmissionCurves and best-guess SEDs (inferred from colors).  We will certainly ultimately need to do TransmissionCurves and SEDs in some places, but I have not thought deeply about whether they will allow us to remove all use of color terms.

            Show
            jbosch Jim Bosch added a comment - I would ultimately like to replace colorterms with a combination of TransmissionCurves and best-guess SEDs (inferred from colors).  We will certainly ultimately need to do TransmissionCurves and SEDs in some places, but I have not thought deeply about whether they will allow us to remove all use of color terms.
            Hide
            Parejkoj John Parejko added a comment -

            I wrote up a Community post with a sketch of some ideas here: https://community.lsst.org/t/reference-catalogs-camfluxes-and-colorterms/3578/2

            Show
            Parejkoj John Parejko added a comment - I wrote up a Community post with a sketch of some ideas here: https://community.lsst.org/t/reference-catalogs-camfluxes-and-colorterms/3578/2
            Hide
            Parejkoj John Parejko added a comment -

            Given the discussion on that Community post and the lessons learned while trying to implement this ticket, I'm closing this as "won't fix". The redesigned reference catalog API may or may not use camFlux fields in the future, but a simple removal of those aliases is definitely not the correct approach.

            Show
            Parejkoj John Parejko added a comment - Given the discussion on that Community post and the lessons learned while trying to implement this ticket, I'm closing this as "won't fix". The redesigned reference catalog API may or may not use camFlux fields in the future, but a simple removal of those aliases is definitely not the correct approach.

              People

              • Assignee:
                Parejkoj John Parejko
                Reporter:
                Parejkoj John Parejko
                Watchers:
                Jim Bosch, John Parejko, John Swinbank, Russell Owen, Simon Krughoff
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel