Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-20775

Modify compareWarp default configs fix clipping stars (when one visit has bad seeing)

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Tanaka-san found a field in VVDS with bad stellar photometric matching to panstarrs. He found that compareWarp was clipping stars, despite the filter that rescues artifact candidates from the chopping block if they completely overlap a source detected in the PSF-Matched sigma-clipped template.

              self.detect.nSigmaToGrow = 2
              self.detectTemplate.nSigmaToGrow = 2.0
      

      These current defaults assumed that footprints of the detections in the PSF-matched template are larger than those in the warps. Incorrect.

      Fiddling with configs fixed the problem in the VVDS field: dataId=

      {'tract': 10181, 'patch': '2,4', 'filter': 'HSC-G', 'visit': 82508}

      ). I'm currently running a few configuration options through a full RC2 to determine with there are any unexpected side effects and metrics.

      So far changing to detect.nSigmaToGrow = 1 looks promising.

        Attachments

        1. izy_std_fit.pdf
          16 kB
        2. unmatchedDet2.pdf
          18 kB
        3. unmatchedPosDet_083019.pdf
          19 kB
        4. unmatchedPosDet.pdf
          19 kB
        5. w_std_fit_083019.pdf
          18 kB
        6. w_std_fit.pdf
          18 kB
        7. w_std_fit.pdf
          19 kB

          Activity

          Hide
          yusra Yusra AlSayyad added a comment -

          Changing the config improves both false positives and false negative (stellar locus and missed artifacts), but there are tradeoffs in changing the size of the artifact footprints.
          Smaller = better stellar locus
          Bigger = fewer missed.

          Based on the following, I propose:

                  self.detect.nSigmaToGrow = 0.6
                  self.detectTemplate.nSigmaToGrow = 2.4  # detection default so we remove config override
          

          unmatchedDet2.pdf w_std_fit.pdf izy_std_fit.pdf

          Show
          yusra Yusra AlSayyad added a comment - Changing the config improves both false positives and false negative (stellar locus and missed artifacts), but there are tradeoffs in changing the size of the artifact footprints. Smaller = better stellar locus Bigger = fewer missed. Based on the following, I propose: self.detect.nSigmaToGrow = 0.6 self.detectTemplate.nSigmaToGrow = 2.4 # detection default so we remove config override unmatchedDet2.pdf w_std_fit.pdf izy_std_fit.pdf
          Hide
          yusra Yusra AlSayyad added a comment -

          Going to try to merge this week so we can cross this off the HSC development list.

          TL DR; I'm recommending a config change:

          detect.nSigmaToGrow = 0.4
          detectTemplate.nSigmaToGrow = 2.4 
          

          As described in the tickets description, the problem was that the template footprints were too small relative to the footprints on the warps. Link to the debug plots: https://paper.dropbox.com/doc/Debug-bright-star-clipping-in-VVDS--Amt_xZhChOrfWcBd5GxmcyUUAg-KqvG00xL6k3m35byAeWpx

          The solution is a config change to grow artifacts less. This has the effect of adding some depth because we’re clipping less. I make a handful or reruns to test the effects on both stellar locus and a rough estimate of false positives (to be used only to additive relative comparison between reruns)

          w_std_fit_083019.pdf unmatchedPosDet_083019.pdf

          Parsing of the rerun names:
          For example /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold04_24
          w_2019_26: analysis was done with weekly 26
          04 = detect.nSigmaToGrow = 0.4
          24 = detectTemplate.nSigmaToGrow = 2.4

          Reruns can be found:
          /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold02_20
          /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold08_20
          /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold1_20

          /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold0_24
          /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold02_24
          /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold04_24 (9615, 9813)
          /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold04_24_v2 (9697)
          /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold06_24
          /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold08_24
          /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold1_24

          pipe_analysis plots can be found here: https://lsst-web.ncsa.illinois.edu/~yusra/RC_QA/

          Show
          yusra Yusra AlSayyad added a comment - Going to try to merge this week so we can cross this off the HSC development list. TL DR; I'm recommending a config change: detect.nSigmaToGrow = 0.4 detectTemplate.nSigmaToGrow = 2.4 As described in the tickets description, the problem was that the template footprints were too small relative to the footprints on the warps. Link to the debug plots: https://paper.dropbox.com/doc/Debug-bright-star-clipping-in-VVDS--Amt_xZhChOrfWcBd5GxmcyUUAg-KqvG00xL6k3m35byAeWpx The solution is a config change to grow artifacts less. This has the effect of adding some depth because we’re clipping less. I make a handful or reruns to test the effects on both stellar locus and a rough estimate of false positives (to be used only to additive relative comparison between reruns) w_std_fit_083019.pdf unmatchedPosDet_083019.pdf Parsing of the rerun names: For example /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold04_24 w_2019_26 : analysis was done with weekly 26 04 = detect.nSigmaToGrow = 0.4 24 = detectTemplate.nSigmaToGrow = 2.4 Reruns can be found: /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold02_20 /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold08_20 /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold1_20 /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold0_24 /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold02_24 /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold04_24 (9615, 9813) /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold04_24_v2 (9697) /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold06_24 /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold08_24 /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold1_24 pipe_analysis plots can be found here: https://lsst-web.ncsa.illinois.edu/~yusra/RC_QA/
          Hide
          jbosch Jim Bosch added a comment -

          - plots are pretty convincing that this is a substantial improvement.

           

          Only question is about the explanation:

          These current defaults assumed that footprints of the detections in the PSF-matched template are larger than those in the warps. Incorrect.

          I'd certainly have made that same incorrect assumption, because the PSF-matched template should both be deeper and have a larger PSF than any single warp.  Do you understand why that assumption was wrong?  Was it just that the footprints in the PSF-matched template weren't much larger than those in the warps (i.e. they were closer to the same size)?

          Show
          jbosch Jim Bosch added a comment - - plots are pretty convincing that this is a substantial improvement.   Only question is about the explanation: These current defaults assumed that footprints of the detections in the PSF-matched template are larger than those in the warps. Incorrect. I'd certainly have made that same incorrect assumption, because the PSF-matched template should both be deeper and have a larger PSF than any single warp.  Do you understand why that assumption was wrong?  Was it just that the footprints in the PSF-matched template weren't much larger than those in the warps (i.e. they were closer to the same size)?
          Hide
          mrawls Meredith Rawls added a comment -

          In principle this looks fine, although Jim raises a good question above. My primary concerns are implementation. Two questions:

          1. In assembleCoadd.py, I see no mention of the default detectTemplate.nSigmaToGrow = 2.4 you refer to. Where is that set, and would it hurt to set it here explicitly too?
          2. In assembleCoadd.py, there is one more instance of an nSigmaToGrow config, clipDetection.nSigmaToGrow = 2. Should this be changed as well, or is it an entirely different operation?
          Show
          mrawls Meredith Rawls added a comment - In principle this looks fine, although Jim raises a good question above. My primary concerns are implementation. Two questions: In assembleCoadd.py , I see no mention of the default detectTemplate.nSigmaToGrow = 2.4  you refer to. Where is that set, and would it hurt to set it here explicitly too? In assembleCoadd.py , there is one more instance of an nSigmaToGrow config, clipDetection.nSigmaToGrow = 2 . Should this be changed as well, or is it an entirely different operation?
          Hide
          yusra Yusra AlSayyad added a comment -

          Jim Bosch That sentence was a unclear. I mean "...detections in the PSF-matched template are larger than those in the PSF-MATCHED warps" I had assumed that the footprints in the psfmatched coadd would be bigger than those in the psfmatched warps, just because they're deeper. But the important thing is the size of the footprints in the each PSF-matched WarpDIFFS. Those footprints in the warpDiffs, as you can see in the field that Tanaka-san found, can be funky looking on bright stars if the FWHM in one visit is not like the other N-1, or it has a FWHM > than the PSF-Matched FWHM. (I defer to Meredith Rawls and Gabor Kovacs [X] on funky looking subtractions of bright stars. I'm skeptical that we'll get great subtractions of bright stars, and don't want to depend on them)

          Meredith Rawls
          (1) That's the default in `SourceDetectionTask`, but I'm not opposed to putting an explicit `detectTemplate.nSigmaToGrow = 2.4`. I'll add it since it won't hurt.
          (2) `clipDetection` is the subtask in `SafeClipAssembleCoaddTask` which doesn't use PSF-Matched anything, and I don't want to change the behavior of it.

          Show
          yusra Yusra AlSayyad added a comment - Jim Bosch That sentence was a unclear. I mean "...detections in the PSF-matched template are larger than those in the PSF-MATCHED warps " I had assumed that the footprints in the psfmatched coadd would be bigger than those in the psfmatched warps, just because they're deeper. But the important thing is the size of the footprints in the each PSF-matched WarpDIFFS. Those footprints in the warpDiffs, as you can see in the field that Tanaka-san found, can be funky looking on bright stars if the FWHM in one visit is not like the other N-1, or it has a FWHM > than the PSF-Matched FWHM. (I defer to Meredith Rawls and Gabor Kovacs [X] on funky looking subtractions of bright stars. I'm skeptical that we'll get great subtractions of bright stars, and don't want to depend on them) Meredith Rawls (1) That's the default in `SourceDetectionTask`, but I'm not opposed to putting an explicit `detectTemplate.nSigmaToGrow = 2.4`. I'll add it since it won't hurt. (2) `clipDetection` is the subtask in `SafeClipAssembleCoaddTask` which doesn't use PSF-Matched anything, and I don't want to change the behavior of it.
          Hide
          mrawls Meredith Rawls added a comment -

          OK thanks. I will mark the PR as Approved too when you put back the line explicitly setting detectTemplate.nSigmaToGrow.

          Show
          mrawls Meredith Rawls added a comment - OK thanks. I will mark the PR as Approved too when you put back the line explicitly setting detectTemplate.nSigmaToGrow .
          Hide
          yusra Yusra AlSayyad added a comment -

          Thanks for the reviews. I edited the commit to include the
          self.detectTemplate.nSigmaToGrow = 2.4 with inline commit on why I'm restating a default, and kicked off a final Jenkins.

          Show
          yusra Yusra AlSayyad added a comment - Thanks for the reviews. I edited the commit to include the self.detectTemplate.nSigmaToGrow = 2.4 with inline commit on why I'm restating a default, and kicked off a final Jenkins.

            People

            Assignee:
            yusra Yusra AlSayyad
            Reporter:
            yusra Yusra AlSayyad
            Reviewers:
            Meredith Rawls
            Watchers:
            Ian Sullivan, Jim Bosch, Meredith Rawls, Michael Wood-Vasey, Yusra AlSayyad
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Jenkins

                No builds found.