Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: ip_isr, obs_subaru
-
Labels:None
-
Story Points:5
-
Epic Link:
-
Sprint:DRP S17-4
-
Team:Data Release Production
Description
As noted in DM-6817 and highlighted/further explored in DM-9411, there is a trend of increasing difference in trace radii with increasing magnitude when directly comparing outputs from the HSC vs. LSST stacks. A likely culprit is slight differences in the implementations of the Brighter-Fatter corrections between the stacks. This ticket is to assess whether this is contributing to the trace radii differences.
Attachments
Issue Links
- relates to
-
- Done
-
- Done
-
- Done
-
- Done
-
- Done
Activity
Bob Armstrong, would you mind having a look and seeing if you agree with my assessment?
Ping Bob Armstrong. Your review of DM-9553 would indicate you do agree, but can you please confirm when you have a moment?
Yes, I agree that the HSC version is doing better and that we should be using that for the time being.
So...reviewed?!
Thanks Bob Armstrong!
The speculation in this ticket is officially confirmed. The difference in the implementations is largely in the computation of the second derivatives. The original HSC method tries to avoid edge effects by explicitly omiting the outer ring of pixels as follows:
# Second derivative termdiffOut20 = numpy.diff(outArray,2,0)[startY:endY, startX+1:endX-1]diffOut21 = numpy.diff(outArray,2,1)[startY+1:endY-1, startX:endX]second = tmpArray[startY+1:endY-1, startX+1:endX-1]*(diffOut20 + diffOut21)The LSST approach was to use numpy.gradient and trust it to handle the edges elegantly:
diffOut20 = numpy.gradient(gradOut[0])diffOut21 = numpy.gradient(gradOut[1])second = tmpArray[startY:endY, startX:endX]*(diffOut20[0] + diffOut21[1])To test the hypothesis that this difference was causing the difference in trace radii of PSF model stars I ran the LSST stack using both versions (with all other settings being equal) and compared the trace radii between them:
Indeed, the above difference is causing the difference in trace radii.
To further confirm, the following is the same comparison but between and LSST stack run using the HSC second derivative algorithm vs the HSC stack run of
DM-9028:and they are in agreement.
This concludes this investigation, but it begs the question of how we should be computing the derivatives. This issue will be the subject of another ticket (see
DM-9553), but for the time being, we must select one to use for consistent stack run comparisons. The following diagnostic plots show the difference between the source trace radii and the PSF model trace radii (at the position of the source) for stars used in the PSF modeling (calib_psfUsed=True):LSST's implementation:
HSC's implementation:
It seems the HSC method is doing a slightly better job (there appears to be a mild upturn in the differences toward brighter magnitudes in the LSST version) , so this one will be adopted for another run of the HSC RC dataset with the LSST stack.