Details
-
Type:
Story
-
Status: Won't Fix
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: ip_diffim
-
Labels:None
-
Story Points:8
-
Epic Link:
-
Team:Alert Production
Description
Create a suite of test images that exercise different parts of subtraction and diaSrc detection and characterization. These will likely be a mix of real images and simulated images.
Having such a suite of tests will help provide feedback when investigating individual tricky cases to make sure that the default algorithms and configurations still do reasonable jobs for other types of objects and subtractions.
I list a few cases here, but part of this ticket will be coming up with a more exhaustive list
1. Isolated transient point source against sky background.
- This should be trivial, should always be captured correctly, and should be a useful exploration of recovery as a function of counts relative to sky noise.
2. Varying isolated point source. - In principle should be easy. Compare forced photometry light curve on difference images to the photometry from the science images.
3. Transient point source on top of extended source. - Classic SN+galaxy case. Tests noise background decorrelation, object identification, merging, de-blending.
4. Transient point source overlapping with footprint of isolated point source. (a) bright static source; (b) faint static source - Tests ability to separate, sensitivity to detecting sources near potential dipoles, reliability of photometry.
- There could be additional levels of spatial PSF variation, spatial background level variation, spatial noise level variation.
Attachments
Activity
Suggestion: repurpose this ticket to set up the mechanism for such a suite. All we need is one example test case, and a repository for managing them; then we can add more as they show up.
We (Eric, Gabor and I) discussed this on 2019-02-19.
We agreed not to act on this ticket directly, but to keep it in mind as we look at DM-17825 — would any of the issues encountered there be appropriate for some sort of rogues gallery?
We also mused, a bit inconclusively, about what the “meaning” of a rogues gallery would be — would there be some quota of cases required to pass?
Closing as “won't fix”, on the basis that these tests should emerge naturally form ongoing development work, but that we're not going to sit down and bash out a test suite in isolation.
Eric Bellm, Gabor Kovacs [X] — this ticket has been hanging out on the backlog for a couple of years now. We should discuss whether it's something we want to address as written, or whether we want to do this on a more ad-hoc basis (e.g. building up a library of test cases based on real-world failures from Meredith Rawls's ongoing testing). Let's discuss this at our planning meeting of 2019-02-19.