# Validate auxTel crosstalk matrix from DM-18050 and DM-23623

XMLWordPrintable

#### Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Story Points:
5
• Team:
Data Release Production
• Urgent?:
No

#### Description

Apply crosstalk correction to auxTel data with visible crosstalk, and see if it disappears.

#### Attachments

1. crosstalk_DM_T90K_magic_coeff_10_11.png
2.47 MB
2. DM_crosstalk_90K.png
2.44 MB
3. histogram_crosstalk_measurements_DM_stack.png
70 kB
4. no_xtalk_correction.png
2.45 MB
5. R01_S11_R01_S11_crosstalk_results.fits
45 kB
6. xtalk_corrected_SLAC_magic_number_C05.png
2.46 MB
7. xtalk_corrected_SLAC.png
2.47 MB
8. xtalk_corrected_threshold_120000.png
2.46 MB
9. xtalk_corrected_threshold_30000.png
2.47 MB
10. xtalk_ghost_optical_ghost.png
1.26 MB
11. xtalk_slac.txt
4 kB
12. xtalk_stack_threshold_120000.txt
9 kB
13. xtalk_stack_threshold_30000.txt
10 kB

#### Activity

Hide
Merlin Fisher-Levine added a comment -

Just making you jpegs of all the images with amp overlays so you can pick

Show
Merlin Fisher-Levine added a comment - Just making you jpegs of all the images with amp overlays so you can pick
Hide
Andrés Alejandro Plazas Malagón added a comment -

For the moment, the SLAC matrix from above (including the coefficient [10, 11] = 15.8821e-4) is in

 /project/shared/auxTel/calibrations/crosstalk/crosstalk-det000.yaml 

and it can be retrieved via

 import lsst.daf.persistence as dafPersist butler = dafPersist.Butler("/project/shared/auxTel/")  xtalkDictionary = butler.get("crosstalk", detector=0) 

Show
Andrés Alejandro Plazas Malagón added a comment - For the moment, the SLAC matrix from above (including the coefficient [10, 11] = 15.8821e-4) is in /project/shared/auxTel/calibrations/crosstalk/crosstalk-det000.yaml and it can be retrieved via import lsst.daf.persistence as dafPersist butler = dafPersist.Butler( "/project/shared/auxTel/" ) xtalkDictionary = butler.get( "crosstalk" , detector = 0 )
Hide
Andrés Alejandro Plazas Malagón added a comment - - edited

Adding more data (using the movies made by Merlin: https://lsst-web.ncsa.illinois.edu/~mfl/) and raising the threshold (to 90K) does get more sources and decrease the errors for the matrix derived with measureCrosstalk.py, but the errors are still somewhat comparable to the coefficients, and the crosstalk does not completely goes away:

Command (w_18)

  measureCrosstalk.py /project/shared/auxTel/ --rerun plazas/crosstalk-auxTel-2020MAY05-T90K --crosstalkName auxTel --outputFileName /project/shared/auxTel/rerun/plazas/crosstalk-auxTel-2020MAY05-T90K/ctValues.yaml --dump-ratios /project/shared/auxTel/rerun/plazas/crosstalk-auxTel-2020MAY05-T90K/ctRatios.pkl -c isr.doBias=True isr.doLinearize=False -c isr.doDefect=True --id detector=0 expId=2020021700240..2020021700261^2020021700274^2020021700275^2020021700279^2020021700286^2020021700332..2020021700339^2020021700361..2020021700377^2020021800027..2020021800087^2020021800090..2020021800104^2020021800145..2020021800156^2020021800179..2020021800230^2020021800252..2020021800270^2020021800282^2020021800330^2020021900135..2020021900140^2020021900153^2020021900180^2020021900186^2020021900211^2020021900223^2020021900229^2020021900274..2020021900304^2020021900315..2020021900329^2020022000141..2020022000148^2020022000152..2020022000158^2020022000161..2020022000167^2020022000307..2020022000320^2020022000341..2020022000420^2020022000473..2020022000478^2020022100729..2020022100734^2020022100747..2020022100755^2020022100774..2020022100778^2020031200154..2020031200164^2020031200189^2020031300120..2020031300140^2020031300280..2020031300320^2020031400041..2020031400046^2020031400080..2020031400095^2020031400135^2020031400324..2020031400330^2020031500125..2020031500140 -c threshold=90000 --clobber-versions --clobber-config 

row 11 of matrix:

[-2.54445149e-05 4.87099351e-06 -1.63118166e-05 -1.69873056e-05
-1.23233058e-05 -1.28576867e-05 -1.14204813e-05 -1.04377168e-05
-6.19106095e-06 4.97129877e-05 6.05482492e-04 0.00000000e+00
1.74665664e-04 9.71086192e-05 1.67969029e-05 -3.13741621e-05]

errors for row 11:

[7.74531072e-05 8.57302366e-05 7.48624734e-05 7.74247528e-05
7.82358620e-05 7.73914217e-05 7.81350172e-05 7.76592715e-05
7.89291589e-05 8.03140283e-05 8.96375859e-05 nan
9.55726136e-05 8.55294129e-05 8.33563608e-05 7.62277195e-05]

Using Chris's magic number for [10,11] ([10,11] = 15.8821e-4 ):

Increasing threshold to 110k does not change things much:

row 11, coeff:

[-2.35623447e-05 1.06210500e-05 -1.67456965e-05 -1.46037573e-05
7.61219708e-04 -1.19054484e-05 -8.67544259e-06 -1.41007686e-05
1.66061018e-05 2.91818928e-04 0.00000000e+00 1.71138858e-03
5.02814801e-05 3.42305430e-05 -9.75924559e-06 5.82532994e-05]

row 11, errors:

[6.97678697e-05 7.81832205e-05 6.60678270e-05 6.85879131e-05
7.17506555e-05 6.90882371e-05 6.70679365e-05 6.91467430e-05
7.14215785e-05 7.01673271e-05 7.91788334e-05 nan
8.44053284e-05 8.31981888e-05 7.99703557e-05 6.83048711e-05]

Show
Andrés Alejandro Plazas Malagón added a comment - - edited Adding more data (using the movies made by Merlin: https://lsst-web.ncsa.illinois.edu/~mfl/ ) and raising the threshold (to 90K) does get more sources and decrease the errors for the matrix derived with measureCrosstalk.py, but the errors are still somewhat comparable to the coefficients, and the crosstalk does not completely goes away: Command (w_18) measureCrosstalk.py /project/shared/auxTel/ --rerun plazas/crosstalk-auxTel-2020MAY05-T90K --crosstalkName auxTel --outputFileName /project/shared/auxTel/rerun/plazas/crosstalk-auxTel-2020MAY05-T90K/ctValues.yaml --dump-ratios /project/shared/auxTel/rerun/plazas/crosstalk-auxTel-2020MAY05-T90K/ctRatios.pkl -c isr.doBias=True isr.doLinearize=False -c isr.doDefect=True --id detector=0 expId=2020021700240..2020021700261^2020021700274^2020021700275^2020021700279^2020021700286^2020021700332..2020021700339^2020021700361..2020021700377^2020021800027..2020021800087^2020021800090..2020021800104^2020021800145..2020021800156^2020021800179..2020021800230^2020021800252..2020021800270^2020021800282^2020021800330^2020021900135..2020021900140^2020021900153^2020021900180^2020021900186^2020021900211^2020021900223^2020021900229^2020021900274..2020021900304^2020021900315..2020021900329^2020022000141..2020022000148^2020022000152..2020022000158^2020022000161..2020022000167^2020022000307..2020022000320^2020022000341..2020022000420^2020022000473..2020022000478^2020022100729..2020022100734^2020022100747..2020022100755^2020022100774..2020022100778^2020031200154..2020031200164^2020031200189^2020031300120..2020031300140^2020031300280..2020031300320^2020031400041..2020031400046^2020031400080..2020031400095^2020031400135^2020031400324..2020031400330^2020031500125..2020031500140 -c threshold=90000 --clobber-versions --clobber-config row 11 of matrix: [-2.54445149e-05 4.87099351e-06 -1.63118166e-05 -1.69873056e-05 -1.23233058e-05 -1.28576867e-05 -1.14204813e-05 -1.04377168e-05 -6.19106095e-06 4.97129877e-05 6.05482492e-04 0.00000000e+00 1.74665664e-04 9.71086192e-05 1.67969029e-05 -3.13741621e-05] errors for row 11: [7.74531072e-05 8.57302366e-05 7.48624734e-05 7.74247528e-05 7.82358620e-05 7.73914217e-05 7.81350172e-05 7.76592715e-05 7.89291589e-05 8.03140283e-05 8.96375859e-05 nan 9.55726136e-05 8.55294129e-05 8.33563608e-05 7.62277195e-05] Using Chris's magic number for [10,11] ( [10,11] = 15.8821e-4 ): Increasing threshold to 110k does not change things much: row 11, coeff: [-2.35623447e-05 1.06210500e-05 -1.67456965e-05 -1.46037573e-05 7.61219708e-04 -1.19054484e-05 -8.67544259e-06 -1.41007686e-05 1.66061018e-05 2.91818928e-04 0.00000000e+00 1.71138858e-03 5.02814801e-05 3.42305430e-05 -9.75924559e-06 5.82532994e-05] row 11, errors: [6.97678697e-05 7.81832205e-05 6.60678270e-05 6.85879131e-05 7.17506555e-05 6.90882371e-05 6.70679365e-05 6.91467430e-05 7.14215785e-05 7.01673271e-05 7.91788334e-05 nan 8.44053284e-05 8.31981888e-05 7.99703557e-05 6.83048711e-05]
Show
Andrés Alejandro Plazas Malagón added a comment - SLAC crosstalk measurements: https://confluence.slac.stanford.edu/pages/viewpage.action?pageId=268380244
Hide
Craig Lage added a comment -

Andrés, nice summary of correcting the LATISS crosstalk.  I just wanted to add that the ribbon cables on the LATISS Dewar that connect the CCD to the WREB board are much longer than the ribbon cables in the production rafts that Adam Snyder measured at SLAC.  So to me it is not surprising that there is more corsstalk in LATISS than in the production rafts.  With the ITL CCD at UC Davis, which has a similar layout to LATISS, we measured crosstalk coefficients to nearest neighbors as high as 1.7E-3, which is close to the value of 1.59E-3 that you found as the "magic number" that best corrected the nearest neighbor crosstalk.

Show
Craig Lage added a comment - Andrés, nice summary of correcting the LATISS crosstalk.  I just wanted to add that the ribbon cables on the LATISS Dewar that connect the CCD to the WREB board are much longer than the ribbon cables in the production rafts that Adam Snyder measured at SLAC.  So to me it is not surprising that there is more corsstalk in LATISS than in the production rafts.  With the ITL CCD at UC Davis, which has a similar layout to LATISS, we measured crosstalk coefficients to nearest neighbors as high as 1.7E-3, which is close to the value of 1.59E-3 that you found as the "magic number" that best corrected the nearest neighbor crosstalk.

#### People

Assignee:
Andrés Alejandro Plazas Malagón
Reporter:
Andrés Alejandro Plazas Malagón
Reviewers:
Christopher Waters
Watchers:
Andrés Alejandro Plazas Malagón, Christopher Waters, Craig Lage, John Swinbank, Merlin Fisher-Levine