Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-18050

Calculate crosstalk coeffs for AuxTel sensor

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Deliverable is the crosstalk matrix for the AuxTel sensor in its currently frozen configuration, in the right format for it to be applied by ISR. Having some code/a notebook to easily recalculate it for when they change the config would be nice, but note this does not require writing this as a proper task, but some reusable code would be nice outcome.

      I believe Robert has code that should be able to do this from cosmics in darks, and we have some long darks on lsst-dev (though not ingested, and lacking the headers to ingest them without a little hacking).

      Could also do this from some of the spots.

      Should NOT be done from the noise correlation - we know there is a lot of pickup, and this isn't the same thing anyway.

        Attachments

          Issue Links

            Activity

            Hide
            plazas Andrés Alejandro Plazas Malagón added a comment -

            Most of the work for this ticket was done in DM-23623.

            Now, using the work from DM-23044 and DM-23045, the crosstalk coefficient matrix can be recalculated applying a non-linearity correction (4th order polynomial).

            • Command (w_2020_15):

               measureCrosstalk.py /project/plazas/auxTel-local-copy-2020MAR30/ --rerun plazas/crosstalk-auxTel-2020APR17 --crosstalkName auxTel --outputFileName /project/plazas/auxTel-local-copy-2020MAR30/rerun/plazas/crosstalk-auxTel-2020APR17/ctValues.yaml --dump-ratios /project/plazas/auxTel-local-copy-2020MAR30/rerun/plazas/crosstalk-auxTel-2020APR17/ctRatios.pkl -c isr.doBias=True isr.doLinearize=True -c isr.doDefect=False --id detector=0 expId=2020021900288..2020021900304 -c threshold=30000 

            • Coefficients (dictionary): crosstalk-det000.yaml
            • Put the coefficients under

              calibrations

              directory in repo (not in

              CALIB

              , see

              /obs_lsst/policy/lsstCamMapper.py

              for the crosstalk template) with a name that matches the expected template:

              /project/plazas/auxTel-local-copy-2020MAR30/calibrations/crosstalk/crosstalk-det000.yaml
              

            • Having done the above, you should be able to get the crosstalk dictionary from the butler:

            import lsst.daf.persistence as dafPersist   
            butler = dafPersist.Butler("/project/plazas/auxTel-local-copy-2020MAR30/")  
            xtalk = butler.get ("crosstalk", detector=0) 
            

            • Code to read matrix from above dictionary:

              import yaml
              import numpy as np
              stream=open ("/project/plazas/auxTel-local-copy-2020MAR30/calibrations/crosstalk/crosstalk-det000.yaml", 'r')
              dictionary = yaml.load (stream)
              #for key, value in dictionary.items():
              d=dictionary ['crosstalk']['auxTel'] 
              full_matrix = []
              for key in d:
                  vec = []
                  for sec_key in d[key]:
                      value = d[key][sec_key]
                      vec.append(value)
                  full_matrix.append(vec)
              full_matrix = np.array (full_matrix)
              print (full_matrix)
              

            • Plot of crosstalk matrix:
            • Ratio to SLAC ITL matrix:
            • Scatter plot nearest neighbors:
            Show
            plazas Andrés Alejandro Plazas Malagón added a comment - Most of the work for this ticket was done in DM-23623 . Now, using the work from DM-23044 and DM-23045 , the crosstalk coefficient matrix can be recalculated applying a non-linearity correction (4th order polynomial). Data: auxTel data taken on 2020-02-19 (ID's 288-304: https://confluence.lsstcorp.org/pages/viewpage.action?spaceKey=LSSTCOM&title=2020-02-19 ) in which a star (aggressor) moves around the detector covering all the amps. Command (w_2020_15): measureCrosstalk.py /project/plazas/auxTel-local-copy-2020MAR30/ --rerun plazas/crosstalk-auxTel-2020APR17 --crosstalkName auxTel --outputFileName /project/plazas/auxTel-local-copy-2020MAR30/rerun/plazas/crosstalk-auxTel-2020APR17/ctValues.yaml --dump-ratios /project/plazas/auxTel-local-copy-2020MAR30/rerun/plazas/crosstalk-auxTel-2020APR17/ctRatios.pkl -c isr.doBias=True isr.doLinearize=True -c isr.doDefect=False --id detector=0 expId=2020021900288..2020021900304 -c threshold=30000 Coefficients (dictionary): crosstalk-det000.yaml Put the coefficients under calibrations directory in repo (not in CALIB , see /obs_lsst/policy/lsstCamMapper.py for the crosstalk template) with a name that matches the expected template: /project/plazas/auxTel-local-copy-2020MAR30/calibrations/crosstalk/crosstalk-det000.yaml Having done the above, you should be able to get the crosstalk dictionary from the butler: import lsst.daf.persistence as dafPersist butler = dafPersist.Butler( "/project/plazas/auxTel-local-copy-2020MAR30/" ) xtalk = butler.get ( "crosstalk" , detector = 0 ) Code to read matrix from above dictionary: import yaml import numpy as np stream = open ( "/project/plazas/auxTel-local-copy-2020MAR30/calibrations/crosstalk/crosstalk-det000.yaml" , 'r' ) dictionary = yaml.load (stream) #for key, value in dictionary.items(): d = dictionary [ 'crosstalk' ][ 'auxTel' ] full_matrix = [] for key in d: vec = [] for sec_key in d[key]: value = d[key][sec_key] vec.append(value) full_matrix.append(vec) full_matrix = np.array (full_matrix) print (full_matrix) Plot of crosstalk matrix: Ratio to SLAC ITL matrix: Scatter plot nearest neighbors:

              People

              • Assignee:
                plazas Andrés Alejandro Plazas Malagón
                Reporter:
                mfisherlevine Merlin Fisher-Levine
                Watchers:
                Andrés Alejandro Plazas Malagón, John Swinbank, Merlin Fisher-Levine
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel