Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-27547

Geometrically check that PsfCandidates are sufficiently far from edge

    XMLWordPrintable

    Details

    • Story Points:
      8
    • Sprint:
      AP S23-2 (January), AP S23-3 (February)
    • Team:
      Alert Production
    • Urgent?:
      No

      Description

      DM-5684 Removed a try block that served to check that the PsfCandidate was sufficiently far from the edge to make a postage stamp.
      DM-27535 Added it back in to unblock the release ASAP

      Geometrically testing that there's enough space to make a stamp is better than checking if the stamp created succeeds.

      See DM-27535 for a CCD with a specific known visit/ccd and how to reproduce.

        Attachments

          Issue Links

            Activity

            Hide
            Parejkoj John Parejko added a comment -

            Note to self: What about using the EDGE mask plane for this check?

            Show
            Parejkoj John Parejko added a comment - Note to self: What about using the EDGE mask plane for this check?
            Hide
            Parejkoj John Parejko added a comment - - edited

            Now that I finally have some useful string output for the psfCandidates and SpatialCells and an understanding of the code, it looks like makePsfCandidates can catch some of these, because it similarly calls psfCandidate.getMaskedImage(). It doesn't always catch them, because it uses the default stamp size, whereas the code block described on this ticket uses the psfex image kernel size.

            Since this doesn't seem to happen very often, I propose adding a `log.warning` to the exception block in psfexPsfDeterminer describing the failing candidate and expected stamp size, and leaving it as a try:catch. We're catching a specific exception here, so I'm not sure that replacing said exception with a check on the stamp bbox gains us anything.

            Note that this argument is against what I argue in my comment on DM-27535:

            Show
            Parejkoj John Parejko added a comment - - edited Now that I finally have some useful string output for the psfCandidates and SpatialCells and an understanding of the code, it looks like makePsfCandidates can catch some of these, because it similarly calls psfCandidate.getMaskedImage() . It doesn't always catch them, because it uses the default stamp size, whereas the code block described on this ticket uses the psfex image kernel size. Since this doesn't seem to happen very often, I propose adding a `log.warning` to the exception block in psfexPsfDeterminer describing the failing candidate and expected stamp size, and leaving it as a try:catch. We're catching a specific exception here, so I'm not sure that replacing said exception with a check on the stamp bbox gains us anything. Note that this argument is against what I argue in my comment on DM-27535 :
            Hide
            Parejkoj John Parejko added a comment - - edited

            Jim Bosch: do you mind reviewing these three small (~100 lines total) PRs? I did not make the change as requested here; see my comment above for my reasoning. I could still do that, but I think letting the psfCandidate handle bounds checking is more robust.

            In the end, I did add useful stringification of some otherwise opaque types, so it's not a complete waste. Those str outputs should make the error messages here clearer.

            afw: https://github.com/lsst/afw/pull/680
            meas_algorithms: https://github.com/lsst/meas_algorithms/pull/307
            meas_extensions_psfex: https://github.com/lsst/meas_extensions_psfex/pull/67

            Jenkins (with ci_hsc and ci_imsim): https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/38074/pipeline

            Show
            Parejkoj John Parejko added a comment - - edited Jim Bosch : do you mind reviewing these three small (~100 lines total) PRs? I did not make the change as requested here; see my comment above for my reasoning. I could still do that, but I think letting the psfCandidate handle bounds checking is more robust. In the end, I did add useful stringification of some otherwise opaque types, so it's not a complete waste. Those str outputs should make the error messages here clearer. afw: https://github.com/lsst/afw/pull/680 meas_algorithms: https://github.com/lsst/meas_algorithms/pull/307 meas_extensions_psfex: https://github.com/lsst/meas_extensions_psfex/pull/67 Jenkins (with ci_hsc and ci_imsim): https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/38074/pipeline
            Hide
            jbosch Jim Bosch added a comment -

            Additions look good, but I'm not sure the error message is actually better - maybe I'm just not familiar enough with what's wrong with the current one. A few minor comments on the PRs.

            Show
            jbosch Jim Bosch added a comment - Additions look good, but I'm not sure the error message is actually better - maybe I'm just not familiar enough with what's wrong with the current one. A few minor comments on the PRs.
            Hide
            Parejkoj John Parejko added a comment -

            One of the things that confused me when digging into all of this is that I could not find any examples of the makePsfCandidates version of the warning, but I did find a half dozen examples of the psfex warning (the subject of this ticket) in an rc2_subset run. I believe this is because the stamp size we use in MakePsfCandidates is smaller than the one in PsfexPsfDeterminer, so a candidate can get through the first (when close to an edge) but fail the second.

            Show
            Parejkoj John Parejko added a comment - One of the things that confused me when digging into all of this is that I could not find any examples of the makePsfCandidates version of the warning, but I did find a half dozen examples of the psfex warning (the subject of this ticket) in an rc2_subset run. I believe this is because the stamp size we use in MakePsfCandidates is smaller than the one in PsfexPsfDeterminer , so a candidate can get through the first (when close to an edge) but fail the second.
            Hide
            jbosch Jim Bosch added a comment -

            That sounds related to an RFC Arun Kannawadi filed back on config fields for PSF sizes, so he may have a useful opinion on what we should do about it (if anything).

            Show
            jbosch Jim Bosch added a comment - That sounds related to an RFC Arun Kannawadi filed back on config fields for PSF sizes, so he may have a useful opinion on what we should do about it (if anything).
            Show
            Parejkoj John Parejko added a comment - Post cleanup Jenkins run:  https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/38097/pipeline

              People

              Assignee:
              Parejkoj John Parejko
              Reporter:
              yusra Yusra AlSayyad
              Reviewers:
              Jim Bosch
              Watchers:
              Eric Bellm, Ian Sullivan, Jim Bosch, John Parejko, Nate Lust, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.