Details

Type: RFC

Status: Implemented

Resolution: Done

Component/s: DM

Labels:None
Description
As we start supporting nonlinearity correction for more cameras I would like to propose some rules, to keep things simple and allow a default version of nonlinaerity correction in IsrTask.
We need to support correction based on lookup tables and have existing code that uses polynomials. We may want to support variations of these or other techniques. Thus I propose to make nonlinearity correction a subtask of IsrTask that can be retargeted. I further propose that default task support polynomial correction, because it is simple and all the necessary data exists in the AmpInfo table.
For the polynomialbased correction there are a few issues:
 We have four coefficients available. This can easily be increased if needed, but there is no sign of that, rather the opposite...
 Unused coefficients are often set to nan (e.g. in obs_subaru and obs_test)
 The settings for various cameras have some quirks:
 obs_subaru, which has working nonlinearity correction code, sets linearityType= "PROPORTIONAL" (or "NONE" if no correction is wanted) in the AmpInfo table, and the associated linearityCoeffs are as follows. This seems a very unnatural order, and I dislike forcing a value to 0 and raising an exception if it is not:
 coeff[0] sqareCoeff: corrIm = uncorrIm + sqareCoeff*uncorrIm^2
 coeff[1] threshold: this must be 0 else an exception is raised
 coeff[2] maxUncorr: if maxUncorr > 0 then flag pixels as SUSPECT where uncorrIm > maxUncorr
 lsstSim sets linearityType="polynomial" and linearityCoeffs=(0, 1, 0, 0), presumably the first four polynomial coefficients. This data is ignored.
 The concept of maxUncorr from obs_subaru seems useful, but it is not a natural fit to a lookup table.
I propose the following:
 Add field suspectLevel to the AmpInfo table and set the SUSPECT flag using specific code for it, rather than relying on nonlinearity correction code to do it. This makes the field more obvious than tucking it away as an element of linearityCoeff and also simplifies writing variant tasks to correct nonlinearity, since they can focus on doing one thing well.
 Offer constants for linearityType names and use those in code. This catches typos earlier. Suggested names include NONE, POLYNOMIAL and LOOKUP_TABLE. These can be enums in python 3.
 The standard nonlinearity correction tasks must accept NONE and do nothing for such detectors. Mosaic cameras may have detectors for which we don't want this correction.
 The standard nonlinearity correction tasks will only support one linearityType or NONE. More heterogeneous cameras will require specialized tasks.
 Encode the number of coefficients by adding an extra field, e.g. numLinearityCoeffs. The specified number of fields are required to be finite. Yes we could (and perhaps should) also force the camera to make all unused fields 0, but this is an extra step and is clearly easy to forget. In any case, it is useful information; for instance it can save time to know the order of a polynomial in advance.
 For the standard polynomial nonlinear correction task: the coefficient array starts with the x^2 coefficient and goes up from there. Constant and scale correction are explicitly omitted because they are not useful.
Attachments
Issue Links
 is triggering

DM5462 Add nonlinearity correction to ISR task
 Done
Paul Price asks "aren't the images in floatingpoint". Yes, by the time we apply nonlinearity correction they are. The plan is to truncate each pixel value, use that to determine the index in the lookup table (e.g. by adding an offset if needed) and adding (or subtracting, if you prefer) the lookedup value to the pixel data. The lookup table will presumably contain floats.