Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-32625

Activate guards on SFM PSF quality for inclusion in coaddition for LSSTCam-imSim

    XMLWordPrintable

    Details

    • Story Points:
      5
    • Epic Link:
    • Team:
      Data Release Production
    • Urgent?:
      No

      Description

      The investigation on PREOPS-885 has highlighted the need to pay attention to the quality of the SFM processing PSF modeling and to impose criteria on certain metrics such that those calexps that fail to meet them do not get included in the coadds.  Such a guard is available via the PsfWcsSelectImagesTask and is, in fact, in use in HSC processing via the retargeting of the selection task in the makeWarp step of the DRP.yaml pipeline in obs_subaru here.  This PSF quality filtering should be activated for LSSTCam-imSim processing as well (ideally in time for the coaddition processing phase for the DP0.2 run!)  Some thought should be made as to the appropriate values of the two thresholds – maxEllipResidual and maxScaledSizeScatter -- to set for these data (this will be provided on this ticket for currently available data).

        Attachments

        1. DM-32625_medianE_merian.png
          DM-32625_medianE_merian.png
          100 kB
        2. DM-32625_psfStar_merian.png
          DM-32625_psfStar_merian.png
          106 kB
        3. exampleTapTool.png
          exampleTapTool.png
          170 kB
        4. HSC_visitSummary_RC2_w50.html
          5.44 MB
        5. LSSTCam-imSim_visitSummary_focalPlane_DP0.2_v23.png
          LSSTCam-imSim_visitSummary_focalPlane_DP0.2_v23.png
          10.92 MB
        6. LSSTCam-imSim_visitSummary_raDec_DP0.2_v23.png
          LSSTCam-imSim_visitSummary_raDec_DP0.2_v23.png
          16.74 MB
        7. LSSTCam-imSim_visitSummary_test-med-1_w03.html
          4.26 MB
        8. LSSTCam-imSim_VisitSummaryDistributions_medianE_DP0.2_v23_log.png
          LSSTCam-imSim_VisitSummaryDistributions_medianE_DP0.2_v23_log.png
          64 kB
        9. LSSTCam-imSim_VisitSummaryDistributions_psfStarScaledDeltaSizeScatter_DP0.2_v23_log.png
          LSSTCam-imSim_VisitSummaryDistributions_psfStarScaledDeltaSizeScatter_DP0.2_v23_log.png
          64 kB
        10. PSFmedianE_RC2_w50_log.png
          PSFmedianE_RC2_w50_log.png
          67 kB
        11. PSFmedianE_RC2_w50.png
          PSFmedianE_RC2_w50.png
          70 kB
        12. PSFmedianE_test-med-1_w48_log.png
          PSFmedianE_test-med-1_w48_log.png
          96 kB
        13. PSFmedianE_test-med-1_w48.png
          PSFmedianE_test-med-1_w48.png
          84 kB
        14. PSFmedianE_tract4431_w40_log.png
          PSFmedianE_tract4431_w40_log.png
          92 kB
        15. PSFmedianE_tract4431_w40.png
          PSFmedianE_tract4431_w40.png
          86 kB
        16. psfMetric_vs_psfSigma_DP0.2_v23.png
          psfMetric_vs_psfSigma_DP0.2_v23.png
          2.08 MB
        17. psfMetric_vs_psfSigma_HSC-RC2_w50.png
          psfMetric_vs_psfSigma_HSC-RC2_w50.png
          312 kB
        18. psfMetric_vs_psfSigma_test-med-1_w_2022_03.png
          psfMetric_vs_psfSigma_test-med-1_w_2022_03.png
          450 kB
        19. PSFpsfStarScaledDeltaSizeScatter_RC2_w50_log.png
          PSFpsfStarScaledDeltaSizeScatter_RC2_w50_log.png
          71 kB
        20. PSFpsfStarScaledDeltaSizeScatter_RC2_w50.png
          PSFpsfStarScaledDeltaSizeScatter_RC2_w50.png
          77 kB
        21. PSFpsfStarScaledDeltaSizeScatter_test-med-1_w48_log.png
          PSFpsfStarScaledDeltaSizeScatter_test-med-1_w48_log.png
          99 kB
        22. PSFpsfStarScaledDeltaSizeScatter_test-med-1_w48.png
          PSFpsfStarScaledDeltaSizeScatter_test-med-1_w48.png
          90 kB
        23. PSFpsfStarScaledDeltaSizeScatter_tract4431_w40_log.png
          PSFpsfStarScaledDeltaSizeScatter_tract4431_w40_log.png
          96 kB
        24. PSFpsfStarScaledDeltaSizeScatter_tract4431_w40.png
          PSFpsfStarScaledDeltaSizeScatter_tract4431_w40.png
          88 kB
        25. visitSummaryCorrelations_HSC-RC2_w50.png
          visitSummaryCorrelations_HSC-RC2_w50.png
          1.01 MB
        26. visitSummaryCorrelations_test-med-1_w_2022_03.png
          visitSummaryCorrelations_test-med-1_w_2022_03.png
          1.12 MB

          Issue Links

            Activity

            Hide
            lauren Lauren MacArthur added a comment -

            A community post announcing this change has been added.

            Show
            lauren Lauren MacArthur added a comment - A community post announcing this change has been added.
            Hide
            lskelvin Lee Kelvin added a comment - - edited

            I ran Lauren MacArthur's notebook to generate similar plots for some of the Merian survey DECam data we've recently been reducing. Plots attached below:


            These data reductions took place on the tiger2-sumire machine at Princeton, using w_2022_02 and the $OBS_SUBARU_DIR/pipelines/DRP.yaml pipeline. The above plots make use of data in the DECam/runs/merian/w_2022_02/202111 collection.

            For these data, it seems the maxEllipResidual and maxScaledSizeScatter thresholds are perfectly reasonable for DECam data processing - at least in the two narrow-bands (N540 and N708) explored here. If more DECam bands become available, it might be useful to check this again at that time.

            Show
            lskelvin Lee Kelvin added a comment - - edited I ran Lauren MacArthur 's notebook to generate similar plots for some of the Merian survey DECam data we've recently been reducing. Plots attached below: These data reductions took place on the tiger2-sumire machine at Princeton, using w_2022_02 and the $OBS_SUBARU_DIR/pipelines/DRP.yaml pipeline. The above plots make use of data in the DECam/runs/merian/w_2022_02/202111 collection. For these data, it seems the maxEllipResidual and maxScaledSizeScatter thresholds are perfectly reasonable for DECam data processing - at least in the two narrow-bands (N540 and N708) explored here. If more DECam bands become available, it might be useful to check this again at that time.
            Hide
            lauren Lauren MacArthur added a comment -

            That's awesome (and a huge relief)!  Thanks for looking into this so quickly, Lee Kelvin !!

            Show
            lauren Lauren MacArthur added a comment - That's awesome (and a huge relief)!  Thanks for looking into this so quickly, Lee Kelvin !!
            Hide
            lauren Lauren MacArthur added a comment -

            We now have the SFP outputs for the full DP0.2 dataset (PREOPS-905), so I have created the distribution plots for them:

            Seen in this full context, I'm feeling like the thresholds may be a bit tight, but not alarmingly so. In the most extreme case (g-band psfScaledDeltaSizeScatter), we are loosing ~3% of the detectors for a given band. By contrast, our highest rejection fraction in the HSC-RC2 dataset is ~6%.

            Show
            lauren Lauren MacArthur added a comment - We now have the SFP outputs for the full DP0.2 dataset (PREOPS-905), so I have created the distribution plots for them: Seen in this full context, I'm feeling like the thresholds may be a bit tight, but not alarmingly so. In the most extreme case (g-band psfScaledDeltaSizeScatter ), we are loosing ~3% of the detectors for a given band. By contrast, our highest rejection fraction in the HSC-RC2 dataset is ~6%.
            Hide
            lauren Lauren MacArthur added a comment -

            Yusra AlSayyad had some concern (in the context of "bestSeeing" selections) that the failures would be preferentially selecting away the "good seeing" images.  The initial trends from the smaller datasets indicated this should not be a worry, but it's definitely worth looking into this for the full DP0.2 dataset.  The following shows the PSF metrics as a function of psfSigma.  The shaded area represents the detectors that will get rejected with current thresholds:

            Again, we see a slight trend of worse psfStarScaledDeltaSizeScatter with smaller psfSigma, but we are certainly not loosing a vast majority of the best seeing data, so I don't think this should be a worry.

            To dig slightly deeper, the following shows the distribution of various parameters with RA/Dec. The rows are by band and the columns are for different parameters. Note that for all but the zeroPoint and skyBg panels (rightmost two columns) the colourbars are matched, so one can directly discern trends with bandpass. Of particular note here, I set the maximum value for the medianE and psfStarScaledDeltaSizeScatter (middle two columns) to their thresholds such that those not passing are in red. I sorted the tables by the latter, so the failures really "rise to the top" on those panels.

            They seem to be reasonably distributed on the sky, so I don't think we will disproportionally lose depth in a given area of the sky (I do note that g-band is a real outlier in psfStarScaledDeltaSizeScatter failures).

            Here is the same but as a function of Focal Plane coordinates:

            Again, they seem to be reasonably distributed.

            Show
            lauren Lauren MacArthur added a comment - Yusra AlSayyad  had some concern (in the context of "bestSeeing" selections) that the failures would be preferentially selecting away the "good seeing" images.  The initial trends from the smaller datasets indicated this should not be a worry, but it's definitely worth looking into this for the full DP0.2 dataset.  The following shows the PSF metrics as a function of psfSigma .  The shaded area represents the detectors that will get rejected with current thresholds: Again, we see a slight trend of worse psfStarScaledDeltaSizeScatter with smaller psfSigma , but we are certainly not loosing a vast majority of the best seeing data, so I don't think this should be a worry. To dig slightly deeper, the following shows the distribution of various parameters with RA/Dec. The rows are by band and the columns are for different parameters. Note that for all but the zeroPoint and skyBg panels (rightmost two columns) the colourbars are matched, so one can directly discern trends with bandpass. Of particular note here, I set the maximum value for the medianE and psfStarScaledDeltaSizeScatter (middle two columns) to their thresholds such that those not passing are in red. I sorted the tables by the latter, so the failures really "rise to the top" on those panels. They seem to be reasonably distributed on the sky, so I don't think we will disproportionally lose depth in a given area of the sky (I do note that g-band is a real outlier in psfStarScaledDeltaSizeScatter failures). Here is the same but as a function of Focal Plane coordinates: Again, they seem to be reasonably distributed.

              People

              Assignee:
              lauren Lauren MacArthur
              Reporter:
              lauren Lauren MacArthur
              Reviewers:
              Eli Rykoff
              Watchers:
              Colin Slater, Eli Rykoff, Jim Bosch, Lauren MacArthur, Lee Kelvin, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins Builds

                  No builds found.