Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-13493

BaseSourceSelectorConfig should not filter on "interpolated"

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: meas_algorithms
    • Labels:
      None

      Description

      Given the recent change to how interpolated is used in the stack, it appears that BaseSourceSelectorConfig should not include interpolated in its list of default badFlags. We may want to add any new "why interplated" flags instead.

        Attachments

          Issue Links

            Activity

            Hide
            swinbank John Swinbank added a comment -
            Show
            swinbank John Swinbank added a comment - As promised... PR here: https://github.com/lsst/meas_algorithms/pull/103 Jenkins here: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/27469/pipeline/ (not finished at time of posting, but I have faith...)
            Hide
            swinbank John Swinbank added a comment -

            Jenkins failed in Jointcal.

            Looks like that's because the Jointcal tests hard-code exactly how many stars should be selected. Of course, one might expect that those numbers would be different given this change, so I think it's ok to just change the test values... but I shall defer to my distinguished reviewer before doing that. John Parejko, any thoughts?

            Show
            swinbank John Swinbank added a comment - Jenkins failed in Jointcal. Looks like that's because the Jointcal tests hard-code exactly how many stars should be selected . Of course, one might expect that those numbers would be different given this change, so I think it's ok to just change the test values... but I shall defer to my distinguished reviewer before doing that. John Parejko , any thoughts?
            Hide
            Parejkoj John Parejko added a comment -

            Yes, those tests are there specifically to catch changes in the source selectors. I think you're going to want to run jointcal yourself and look at the output to make sure the number of sources is still reasonable in the tests, otherwise we may have to reprocess the test data to bring it in line with the new definition of interpolated.

            The fact that the number of sources has changed will likely also affect the chi2, ndof and other values, which is why I recommend running jointcal yourself to catch all the changes.

            Show
            Parejkoj John Parejko added a comment - Yes, those tests are there specifically to catch changes in the source selectors. I think you're going to want to run jointcal yourself and look at the output to make sure the number of sources is still reasonable in the tests, otherwise we may have to reprocess the test data to bring it in line with the new definition of interpolated . The fact that the number of sources has changed will likely also affect the chi2, ndof and other values, which is why I recommend running jointcal yourself to catch all the changes.
            Hide
            swinbank John Swinbank added a comment -

            Unfortunately, this is no longer a ticket I can handle in between agenda and milestone munging. Back on the backlog!

            Show
            swinbank John Swinbank added a comment - Unfortunately, this is no longer a ticket I can handle in between agenda and milestone munging. Back on the backlog!
            Hide
            swinbank John Swinbank added a comment -

            I realised that, rather than updating all the test results to reflect the fact that we're now accepting interpolated pixels, we can simply update the tests to continue to reject them. I think this is appropriate, but perhaps it should be accompanied by another ticket to refresh the test data and then revert this change to the tests.

            Anyway:

            Whatcha reckon, John Parejko?

            Show
            swinbank John Swinbank added a comment - I realised that, rather than updating all the test results to reflect the fact that we're now accepting interpolated pixels, we can simply update the tests to continue to reject them. I think this is appropriate, but perhaps it should be accompanied by another ticket to refresh the test data and then revert this change to the tests. Anyway: PR on meas_algorithms: https://github.com/lsst/meas_algorithms/pull/103 PR on Jointcal: https://github.com/lsst/jointcal/pull/68 Success on Jenkins: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/27473/ Whatcha reckon, John Parejko ?
            Hide
            Parejkoj John Parejko added a comment -

            Sorry for the delay in reviewing. These look fine: please file a ticket on refreshing the testdata (minor priority, probably).

            I did see the interpolated flag show up in pipe.tasks.photoCal.py and meas.algorithms.starSelector: it should definitely be removed from the latter, but I'm not sure about the former.

            Show
            Parejkoj John Parejko added a comment - Sorry for the delay in reviewing. These look fine: please file a ticket on refreshing the testdata (minor priority, probably). I did see the interpolated flag show up in pipe.tasks.photoCal.py and meas.algorithms.starSelector : it should definitely be removed from the latter, but I'm not sure about the former.
            Hide
            Parejkoj John Parejko added a comment -

            Pinging this ticket: it's reviewed and can probably be merged after it's rebased to master.

            Show
            Parejkoj John Parejko added a comment - Pinging this ticket: it's reviewed and can probably be merged after it's rebased to master.
            Hide
            swinbank John Swinbank added a comment -

            Sorry for slowness; now merged.

            Show
            swinbank John Swinbank added a comment - Sorry for slowness; now merged.

              People

              Assignee:
              swinbank John Swinbank
              Reporter:
              Parejkoj John Parejko
              Reviewers:
              John Parejko
              Watchers:
              Chris Morrison [X] (Inactive), Jim Bosch, John Parejko, John Swinbank, Lauren MacArthur, Paul Price, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.