# Modify compareWarp default configs fix clipping stars (when one visit has bad seeing)

XMLWordPrintable

#### Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s: None
• Labels:
None
• Story Points:
8
• Sprint:
DRP F19-5
• Team:
Data Release Production

#### Description

Tanaka-san found a field in VVDS with bad stellar photometric matching to panstarrs. He found that compareWarp was clipping stars, despite the filter that rescues artifact candidates from the chopping block if they completely overlap a source detected in the PSF-Matched sigma-clipped template.

 self.detect.nSigmaToGrow = 2 self.detectTemplate.nSigmaToGrow = 2.0

These current defaults assumed that footprints of the detections in the PSF-matched template are larger than those in the warps. Incorrect.

Fiddling with configs fixed the problem in the VVDS field: dataId=

{'tract': 10181, 'patch': '2,4', 'filter': 'HSC-G', 'visit': 82508}

). I'm currently running a few configuration options through a full RC2 to determine with there are any unexpected side effects and metrics.

So far changing to detect.nSigmaToGrow = 1 looks promising.

#### Attachments

1. izy_std_fit.pdf
16 kB
2. unmatchedDet2.pdf
18 kB
3. unmatchedPosDet_083019.pdf
19 kB
4. unmatchedPosDet.pdf
19 kB
5. w_std_fit_083019.pdf
18 kB
6. w_std_fit.pdf
18 kB
7. w_std_fit.pdf
19 kB

#### Activity

No builds found.
Field Original Value New Value
Epic Link DM-20164 [ 323028 ]
 Status To Do [ 10001 ] In Progress [ 3 ]
 Attachment unmatchedDet2.pdf [ 39607 ] Attachment w_std_fit.pdf [ 39608 ] Attachment izy_std_fit.pdf [ 39609 ]
Hide

Changing the config improves both false positives and false negative (stellar locus and missed artifacts), but there are tradeoffs in changing the size of the artifact footprints.
Smaller = better stellar locus
Bigger = fewer missed.

Based on the following, I propose:

 self.detect.nSigmaToGrow = 0.6 self.detectTemplate.nSigmaToGrow = 2.4 # detection default so we remove config override

Show
Yusra AlSayyad added a comment - Changing the config improves both false positives and false negative (stellar locus and missed artifacts), but there are tradeoffs in changing the size of the artifact footprints. Smaller = better stellar locus Bigger = fewer missed. Based on the following, I propose: self.detect.nSigmaToGrow = 0.6 self.detectTemplate.nSigmaToGrow = 2.4 # detection default so we remove config override unmatchedDet2.pdf w_std_fit.pdf izy_std_fit.pdf
 Watchers Ian Sullivan, Meredith Rawls, Yusra AlSayyad [ Ian Sullivan, Meredith Rawls, Yusra AlSayyad ] Ian Sullivan, Meredith Rawls, Michael Wood-Vasey, Yusra AlSayyad [ Ian Sullivan, Meredith Rawls, Michael Wood-Vasey, Yusra AlSayyad ]
 Epic Link DM-20164 [ 323028 ] DM-21252 [ 414682 ]
 Sprint DRP F19-2 [ 939 ] DRP F19-2, DRP F19-4 [ 939, 955 ]
 Sprint DRP F19-2, DRP F19-4 [ 939, 955 ] DRP F19-5 [ 965 ]
 Attachment unmatchedPosDet.pdf [ 40619 ]
 Attachment w_std_fit.pdf [ 40620 ]
 Attachment w_std_fit_083019.pdf [ 40625 ]
 Attachment unmatchedPosDet_083019.pdf [ 40632 ]
Hide

Going to try to merge this week so we can cross this off the HSC development list.

TL DR; I'm recommending a config change:

 detect.nSigmaToGrow = 0.4 detectTemplate.nSigmaToGrow = 2.4

As described in the tickets description, the problem was that the template footprints were too small relative to the footprints on the warps. Link to the debug plots: https://paper.dropbox.com/doc/Debug-bright-star-clipping-in-VVDS--Amt_xZhChOrfWcBd5GxmcyUUAg-KqvG00xL6k3m35byAeWpx

The solution is a config change to grow artifacts less. This has the effect of adding some depth because we’re clipping less. I make a handful or reruns to test the effects on both stellar locus and a rough estimate of false positives (to be used only to additive relative comparison between reruns)

Parsing of the rerun names:
For example /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold04_24
w_2019_26: analysis was done with weekly 26
04 = detect.nSigmaToGrow = 0.4
24 = detectTemplate.nSigmaToGrow = 2.4

Reruns can be found:
/datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold02_20
/datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold08_20
/datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold1_20

/datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold0_24
/datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold02_24
/datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold04_24 (9615, 9813)
/datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold04_24_v2 (9697)
/datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold06_24
/datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold08_24
/datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold1_24

pipe_analysis plots can be found here: https://lsst-web.ncsa.illinois.edu/~yusra/RC_QA/

Show
Yusra AlSayyad added a comment - Going to try to merge this week so we can cross this off the HSC development list. TL DR; I'm recommending a config change: detect.nSigmaToGrow = 0.4 detectTemplate.nSigmaToGrow = 2.4 As described in the tickets description, the problem was that the template footprints were too small relative to the footprints on the warps. Link to the debug plots: https://paper.dropbox.com/doc/Debug-bright-star-clipping-in-VVDS--Amt_xZhChOrfWcBd5GxmcyUUAg-KqvG00xL6k3m35byAeWpx The solution is a config change to grow artifacts less. This has the effect of adding some depth because we’re clipping less. I make a handful or reruns to test the effects on both stellar locus and a rough estimate of false positives (to be used only to additive relative comparison between reruns) w_std_fit_083019.pdf unmatchedPosDet_083019.pdf Parsing of the rerun names: For example /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold04_24 w_2019_26 : analysis was done with weekly 26 04 = detect.nSigmaToGrow = 0.4 24 = detectTemplate.nSigmaToGrow = 2.4 Reruns can be found: /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold02_20 /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold08_20 /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold1_20 /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold0_24 /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold02_24 /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold04_24 (9615, 9813) /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold04_24_v2 (9697) /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold06_24 /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold08_24 /datasets/hsc/repo/rerun/private/yusra/RC2/w_2019_26_threshold1_24 pipe_analysis plots can be found here: https://lsst-web.ncsa.illinois.edu/~yusra/RC_QA/
 Reviewers Meredith Rawls [ mrawls ] Status In Progress [ 3 ] In Review [ 10004 ]
Hide
Jim Bosch added a comment -

- plots are pretty convincing that this is a substantial improvement.

Only question is about the explanation:

These current defaults assumed that footprints of the detections in the PSF-matched template are larger than those in the warps. Incorrect.

I'd certainly have made that same incorrect assumption, because the PSF-matched template should both be deeper and have a larger PSF than any single warp.  Do you understand why that assumption was wrong?  Was it just that the footprints in the PSF-matched template weren't much larger than those in the warps (i.e. they were closer to the same size)?

Show
Jim Bosch added a comment - - plots are pretty convincing that this is a substantial improvement.   Only question is about the explanation: These current defaults assumed that footprints of the detections in the PSF-matched template are larger than those in the warps. Incorrect. I'd certainly have made that same incorrect assumption, because the PSF-matched template should both be deeper and have a larger PSF than any single warp.  Do you understand why that assumption was wrong?  Was it just that the footprints in the PSF-matched template weren't much larger than those in the warps (i.e. they were closer to the same size)?
Hide
Meredith Rawls added a comment -

In principle this looks fine, although Jim raises a good question above. My primary concerns are implementation. Two questions:

1. In assembleCoadd.py, I see no mention of the default detectTemplate.nSigmaToGrow = 2.4 you refer to. Where is that set, and would it hurt to set it here explicitly too?
2. In assembleCoadd.py, there is one more instance of an nSigmaToGrow config, clipDetection.nSigmaToGrow = 2. Should this be changed as well, or is it an entirely different operation?
Show
Meredith Rawls added a comment - In principle this looks fine, although Jim raises a good question above. My primary concerns are implementation. Two questions: In assembleCoadd.py , I see no mention of the default detectTemplate.nSigmaToGrow = 2.4  you refer to. Where is that set, and would it hurt to set it here explicitly too? In assembleCoadd.py , there is one more instance of an nSigmaToGrow config, clipDetection.nSigmaToGrow = 2 . Should this be changed as well, or is it an entirely different operation?
Hide

Jim Bosch That sentence was a unclear. I mean "...detections in the PSF-matched template are larger than those in the PSF-MATCHED warps" I had assumed that the footprints in the psfmatched coadd would be bigger than those in the psfmatched warps, just because they're deeper. But the important thing is the size of the footprints in the each PSF-matched WarpDIFFS. Those footprints in the warpDiffs, as you can see in the field that Tanaka-san found, can be funky looking on bright stars if the FWHM in one visit is not like the other N-1, or it has a FWHM > than the PSF-Matched FWHM. (I defer to Meredith Rawls and Gabor Kovacs [X] on funky looking subtractions of bright stars. I'm skeptical that we'll get great subtractions of bright stars, and don't want to depend on them)

Meredith Rawls
(1) That's the default in SourceDetectionTask, but I'm not opposed to putting an explicit detectTemplate.nSigmaToGrow = 2.4. I'll add it since it won't hurt.
(2) clipDetection is the subtask in SafeClipAssembleCoaddTask which doesn't use PSF-Matched anything, and I don't want to change the behavior of it.

Show
Yusra AlSayyad added a comment - Jim Bosch That sentence was a unclear. I mean "...detections in the PSF-matched template are larger than those in the PSF-MATCHED warps " I had assumed that the footprints in the psfmatched coadd would be bigger than those in the psfmatched warps, just because they're deeper. But the important thing is the size of the footprints in the each PSF-matched WarpDIFFS. Those footprints in the warpDiffs, as you can see in the field that Tanaka-san found, can be funky looking on bright stars if the FWHM in one visit is not like the other N-1, or it has a FWHM > than the PSF-Matched FWHM. (I defer to Meredith Rawls and Gabor Kovacs [X] on funky looking subtractions of bright stars. I'm skeptical that we'll get great subtractions of bright stars, and don't want to depend on them) Meredith Rawls (1) That's the default in SourceDetectionTask, but I'm not opposed to putting an explicit detectTemplate.nSigmaToGrow = 2.4. I'll add it since it won't hurt. (2) clipDetection is the subtask in SafeClipAssembleCoaddTask which doesn't use PSF-Matched anything, and I don't want to change the behavior of it.
Hide
Meredith Rawls added a comment -

OK thanks. I will mark the PR as Approved too when you put back the line explicitly setting detectTemplate.nSigmaToGrow.

Show
Meredith Rawls added a comment - OK thanks. I will mark the PR as Approved too when you put back the line explicitly setting detectTemplate.nSigmaToGrow .
 Status In Review [ 10004 ] Reviewed [ 10101 ]
Hide

Thanks for the reviews. I edited the commit to include the
self.detectTemplate.nSigmaToGrow = 2.4 with inline commit on why I'm restating a default, and kicked off a final Jenkins.

Show
Yusra AlSayyad added a comment - Thanks for the reviews. I edited the commit to include the self.detectTemplate.nSigmaToGrow = 2.4 with inline commit on why I'm restating a default, and kicked off a final Jenkins.
 Status Reviewed [ 10101 ] In Review [ 10004 ]
 Status In Review [ 10004 ] Reviewed [ 10101 ]
 Resolution Done [ 10000 ] Status Reviewed [ 10101 ] Done [ 10002 ]
 Story Points 4 8

#### People

Assignee:
Reporter:
Reviewers:
Meredith Rawls
Watchers:
Ian Sullivan, Jim Bosch, Meredith Rawls, Michael Wood-Vasey [X] (Inactive), Yusra AlSayyad