Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-37411

Add visit-level PSF model robustness metrics

    XMLWordPrintable

    Details

    • Story Points:
      10
    • Epic Link:
    • Team:
      Data Release Production
    • Urgent?:
      No

      Description

      In recent processing runs, detectors have been failing with issues relating to the PSF evaluation at certain locations. There is a config to "catch" these errors and allow the SFP to continue. The question is then whether it is safe to turn on this config (i.e. if the calexp is truly a hopeless case, can/would it get flagged and not included in the coadd based on our input selection criteria). Investigations on DM-36843 revealed that these detectors actually do pass our current checks. This and the analysis on DM-36930 have highlighted the need to add additional metrics to be thresholded for consideration in coaddition (and potentially some robustification of PIFF itself).

      One situation where the PSF modeling can go awry is if there is not good coverage of PSF model stars over the entire unmasked region of the detector (due to the resulting need for extrapolation of the PSF model, which seem to be quite unstable, see also this comment on DM-36930). This ticket is to add a metric that computes the maximum distance from an unmasked pixel to a PSF model star in a given detector. Its effectiveness as a metric for identifying problematic detectors that should not be included in the coadds will start with a close look at all the failed detectors noted and fully described on DM-36930.

      A code snippet for the computation of this metric was provided in DM-36930. This will be adapted and added into the ComputeExposureSummaryStatsTask in pipe_tasks (noting that it may get moved around pending the implementation of DM-35207).

        Attachments

          Issue Links

            Activity

            Hide
            yusra Yusra AlSayyad added a comment -

            The ci packages need to be tagged manually because they're not part of lsst_distrib, but are needed to test the release of lsst_distrib. Hypothesizing that Matthias Wittgen forgot this step while releasing v24.0.0.

            Show
            yusra Yusra AlSayyad added a comment - The ci packages need to be tagged manually because they're not part of lsst_distrib, but are needed to test the release of lsst_distrib. Hypothesizing that Matthias Wittgen forgot this step while releasing v24.0.0.
            Hide
            wittgen Matthias Wittgen added a comment - - edited

            My mistake. I retagged 24.0.0 to v24.0.0.rc1 in ci_hsc_gen3
            The jenkins build does tag this to the current HEAD unless the tag already was manually created. 
            I'll check the rest of the CI packages as well.

            Show
            wittgen Matthias Wittgen added a comment - - edited My mistake. I retagged 24.0.0 to v24.0.0.rc1 in ci_hsc_gen3 The jenkins build does tag this to the current HEAD unless the tag already was manually created.  I'll check the rest of the CI packages as well.
            Hide
            yusra Yusra AlSayyad added a comment -

            and the tags for the other rc's you've built too. for example for ci_hsc_gen3 I see (tag: v24.0.0.rc4, tag: v24.0.0.rc3, tag: v24.0.0.rc2, tag: v23.0.3.rc3, tag: 24.0.0) all on 993ccb99379f19c96bf6c046ff77ff55530bfb34 after Oct 4.

            Show
            yusra Yusra AlSayyad added a comment - and the tags for the other rc's you've built too. for example for ci_hsc_gen3 I see (tag: v24.0.0.rc4, tag: v24.0.0.rc3, tag: v24.0.0.rc2, tag: v23.0.3.rc3, tag: 24.0.0) all on 993ccb99379f19c96bf6c046ff77ff55530bfb34 after Oct 4.
            Hide
            jbosch Jim Bosch added a comment -

            Thanks, all.  Tags for ci_hsc_gen3 look good (as do those for ci_imsim, though I haven't looked as hard at those).  I'm kicking off a new Jenkins run now.

            Show
            jbosch Jim Bosch added a comment - Thanks, all.  Tags for ci_hsc_gen3 look good (as do those for ci_imsim, though I haven't looked as hard at those).  I'm kicking off a new Jenkins run now.
            Hide
            tjenness Tim Jenness added a comment -

            In terms of release tags, I see that rc2_subset repo is not getting release tags either. This one is important because we directly reference it from the tutorial and people get really confused running a v24 tutorial but it not working because they are using main and not the 24.0.0 tag.

            Show
            tjenness Tim Jenness added a comment - In terms of release tags, I see that rc2_subset repo is not getting release tags either. This one is important because we directly reference it from the tutorial and people get really confused running a v24 tutorial but it not working because they are using main and not the 24.0.0 tag.

              People

              Assignee:
              lauren Lauren MacArthur
              Reporter:
              lauren Lauren MacArthur
              Reviewers:
              Jim Bosch
              Watchers:
              Eric Bellm, Jim Bosch, John Parejko, Joshua Meyers, Lauren MacArthur, Matthias Wittgen, Tim Jenness, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.