Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-16301

Integrate DESC's checkCcdAstrometry into ProcessCcd itself

    XMLWordPrintable

    Details

    • Team:
      Data Release Production

      Description

      Dominique Boutigny et al wrote a script to identify silent catastrophic astrometry outliers for their DC2 processing:

      https://github.com/LSSTDESC/ImageProcessingPipelines/blob/master/python/util/checkCcdAstrometry.py

      It just re-matches to the reference catalog with no offset, applies some very simple cuts, and tests the astrometric scatter against a configurable threshold.  We should include this logic in AstrometryTask itself.

      This will probably break HSC RC processing (but in a good way).

        Attachments

          Issue Links

            Activity

            Hide
            mfisherlevine Merlin Fisher-Levine added a comment -

            What's the default value for maxScatter though? It might also just be as simple as turning that down a bit too, given that the only real-world examples where this is actually a problem seem to be around the 4 to 5 mark, and I contend that the appropriate value might actually be more like 1" (that's what I've been using for my sanity checks, and it's been working great, albeit on a very small dataset on a very non-representative instrument)

            Show
            mfisherlevine Merlin Fisher-Levine added a comment - What's the default value for maxScatter though? It might also just be as simple as turning that down a bit too, given that the only real-world examples where this is actually a problem seem to be around the 4 to 5 mark, and I contend that the appropriate value might actually be more like 1" (that's what I've been using for my sanity checks, and it's been working great, albeit on a very small dataset on a very non-representative instrument)
            Hide
            lskelvin Lee Kelvin added a comment -

            The default value for maxScatterArcsec is currently 10. We could turn it down (to 5 maybe). I think this wouldn't quite get at the heart of the issue originally reported by DESC though - they're telling us that the reported number is incorrect (maybe). As I think we've decided, it might be best to wait until we have some more concrete examples of this to test.

            Show
            lskelvin Lee Kelvin added a comment - The default value for maxScatterArcsec is currently 10. We could turn it down (to 5 maybe). I think this wouldn't quite get at the heart of the issue originally reported by DESC though - they're telling us that the reported number is incorrect (maybe). As I think we've decided, it might be best to wait until we have some more concrete examples of this to test.
            Hide
            cmorrison Chris Morrison added a comment - - edited

            There is no final check on the final max scatter currently. The matcher works adaptability from the data to set an initial guess on what is safe to set on max scatter. There is not current default for what you are talking about, that is a post scatter test. That is part of what this ticket was adding. The fitter I believe does have a max scatter value for saying the fit failed. That is what the iteration failure in the data above is caused by.

            Show
            cmorrison Chris Morrison added a comment - - edited There is no final check on the final max scatter currently. The matcher works adaptability from the data to set an initial guess on what is safe to set on max scatter. There is not current default for what you are talking about, that is a post scatter test. That is part of what this ticket was adding. The fitter I believe does have a max scatter value for saying the fit failed. That is what the iteration failure in the data above is caused by.
            Hide
            cmorrison Chris Morrison added a comment -

            Lee Kelvin is their reason for saying it's "incorrect" because the rematch number doesn't match the result from the match-fit loop?

            Show
            cmorrison Chris Morrison added a comment - Lee Kelvin is their reason for saying it's "incorrect" because the rematch number doesn't match the result from the match-fit loop?
            Hide
            lskelvin Lee Kelvin added a comment -

            Chris Morrison, as far as I understand at least, that's the original motivation behind the ticket as reported to DM by Dominique and others in DESC. Sadly however, no specific data IDs were mentioned at the time which exhibit this effect.

            Show
            lskelvin Lee Kelvin added a comment - Chris Morrison , as far as I understand at least, that's the original motivation behind the ticket as reported to DM by Dominique and others in DESC. Sadly however, no specific data IDs were mentioned at the time which exhibit this effect.

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              jbosch Jim Bosch
              Reviewers:
              Chris Morrison
              Watchers:
              Chris Morrison, Chris Walter, Jim Bosch, John Parejko, Lee Kelvin, Merlin Fisher-Levine, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Dates

                Created:
                Updated:

                  CI Builds

                  No builds found.