Details
-
Type:
RFC
-
Status: Implemented
-
Resolution: Done
-
Component/s: DM
-
Labels:None
Description
The linearity correction as it currently exists is confusing, and has an implementation that prevents easy creation and updating of linearity values. DM-23023 will be the implementation ticket for these changes.
1) Allow a linearity correction to override what is declared by the lsst.afw.cameraGeom.amplifier.getLinearityType() value. If a linearity correction is supplied by the user, that should be used.
2) Allow linearity corrections to be persisted in a useful format. Currently, only lookup tables exist as an on-disk objects, the rest existing as cameraGeom values. This means there are no clear targets for linearity generation, nor for the overrides necessary to test new linearity models. The proposal is that the linearity correction will "look like" a detector, and support an iterable containing getLinearityType, getLinearityCoeffs properties that can be passed along with an optional lookup table to the linearity correction.
3) Squared and LookupTable corrections do not change.
4) Add generic polynomial correction method. Upcoming linearity measurement code in cp_pipe will be able to fit arbitrary polynomials:
corrected = measured * (1 + a2 * measured + a3 * measured^2 + ...)
This matches the form of the Squared and LookupTable LinearityTypes (corrected = measured * f(measured)), allowing higher order terms. The set of [a2, a3, ...] will be stored in the amplifier.linearityCoeffs vector.
a0 must be equal to zero, as any deviation from this is a bias level offset. Similarly, a1 must be equal to one, as changes in this are identical to gain changes.
5) amplifier.linearityThreshold, amplifier.linearityMaximum, and amplifier.linearityUnits have never been supported, and should be deprecated. linearityMaximum should be handled with the suspectLevel value, and Threshold has been defined to always be zero. With the above definition the correction, the linearityUnits must be identical between pre- and post- corrected values, and therefore are in DN.
6) Reduce the potential confusion by suggesting strongly that all cameras with no linearity correction convert to use lsst.afw.cameraGeom.amplifier.getLinearityType() == "None". This will use lsst.afw.cameraGeom.NullLinearityType, resulting in no linearity correction. This will replace the current default of "Proportional". "Proportional" will be assumed to be equal to "None", but will be explicitly checked.
Attachments
Issue Links
Activity
A suggestion from Jim Bosch was to make a new linearization data class where the code gets the parameters, and to let this class be initialized from the detector and/or be persisted/retrieved to allow user overrides.
As I rework the code, I have a clearer picture of what I think I want the code to look like. The current linearity functors iterate over the amplifiers supplied, but have that parameter as optional. This could result in errors. In the interest of fixing this error and reducing duplication, I plan on demoting the current functors to simply apply a math operation to an image, called from the new LinearizeData class that can be constructed from either detectors/amplifiers (the default case) or from a persisted yaml file (the new path).
There should be minimal changes on the calling side, as the code will simply instantiate the new class instead of the old.
Thanks for doing this. As we have talked, I believe this looks good. I think it’s a good idea to have a generic polynomial as an option, and then leave for the user to decide whether that particular correction is precise enough or not. For example, we might find in the future using a different polynomial basis is better (c.f., https://arxiv.org/pdf/1902.02312.pdf for classical non-linearity correction in NIR detectors). Also, we have the LUT, so that would cover general cases. I think that the creation of a class will make things more modular and easier to understand and use.
Definitely support this reworking - the above makes it very clear, and sounds great!
PRs with concrete code changes pending no further discussion:
https://github.com/lsst/ip_isr/pull/126
https://github.com/lsst/obs_decam/pull/127
https://github.com/lsst/obs_subaru/pull/253
This sounds fine to me, although I'd like a bit more specificity about "looking like a detector" for persistence/retrieval and how a user would supply an override (ingesting a new calibration dataset?).