Details
-
Type:
RFC
-
Status: Withdrawn
-
Resolution: Done
-
Component/s: DM
-
Labels:None
Description
1) measureCrosstalk.py should live in cp_pipe now that it exists, as it's creating a calibration product, so step 1 is to simply move it there, and move the writeCrosstalkCoeffs method as well, as that really belongs with the measuring code.
2) Update the data format for the crosstalk coefficients. Current format written is a 4-deep nested dict, with the two outer layers being 100% superfluous. The proposed change to the format will either be to just rip these out, or better, make a python defects class holding the coefficients so that they can be used by ampName (currently they're stored like that, but during the loading the names are discarded and they're thrown into a numpy array, losing the safety that the dict keys provide), which would involve making the requisite changes to the isrTask to enable their use.
Attachments
Issue Links
Activity
At the moment, neither. There's no class, and a dict of dicts of dicts of dicts is written manually in yaml, and when retrieved, the two outer layers are discarded, and the inner two, which give the coefficients by name, have their naming-protection discarded, and the values shoved into a naked numpy array.
Example:
/project/shared/auxTel/CALIB/crosstalk/crosstalk-S00-det000.yaml
|
{'crosstalk': {'auxTel': {'C10': {'C10': 0.0,
|
'C11': 0.00041452,
|
'C12': 5.3125e-05,
|
'C13': 1.0496e-05,
|
'C14': -7.5229e-06,
|
'C15': -2.0852e-05,
|
'C16': -3.0844e-05,
|
'C17': -2.7888e-05,
|
'C07': -3.1578e-05,
|
'C06': -3.738e-05,
|
'C05': -3.806e-05,
|
'C04': -4.1003e-05,
|
'C03': -3.189e-05,
|
'C02': -3.4353e-05,
|
'C01': -3.5075e-05,
|
'C00': -3.8668e-05},
|
'C11': {'C10': 0.00029295,
|
'C11': 0.0,
|
'C12': 0.00039701,
|
'C13': 0.0001004,
|
I am 100% onboard with #1. I think that's the only thing keeping calibration production and image processing from being correctly separated.
I'm working on what I'm tentatively calling a "gen3 crosstalk" for DECam, and my plan is to move to a Defects-like class. I think this simplifies the on-disk representation by pushing common values/layers into metadata entries. The new class should still be able to read the currently generated versions though, so the cp_pipe measure side can be done independently.
Meredith Rawls / DECam folk, I/we will obviously translate any DECam crosstalk objects to the new format, or make sure the old ones still work, of course.
I am familiar with CrosstalkTask, subtractCrosstalk, getCrosstalk, and even prepCrosstalk, but not measureCrosstalk. I gather from the discussion that it is being used to generate crosstalk coefficients from some kind of calibration data, and the question is (1) where to put that operation and (2) how to store and use those coefficients?
Assuming that is roughly correct, having (1) be cp_pipe seems like the right place. I support Christopher Waters's plan for (2) to have a Defects-like "curated calibration" class for storing crosstalk data, provided it can handle inter-CCD crosstalk (i.e., be DECam-friendly) and interface nicely with ip_isr.
It turns out that because Christopher Waters is overhauling all calibrations, creating a base class from which all calibrations products will inherit (which is AWESOME) this RFC is now superseded by a much better and more general plan, so is being withdrawn.
Are these implemented in a CrossTalk class that has read/write methods for serialization? Or are they used as python dicts directly?