Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-18849

Make crosstalk coefficient calculation code in obs_subaru generic

    Details

    • Type: Improvement
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: cp_pipe, obs_subaru
    • Labels:
      None

      Description

      Take the code in https://github.com/lsst/obs_subaru/blob/master/python/lsst/obs/subaru/crosstalk.py and:

      • Make it camera-agnostic
      • Make it comply with coding standards
      • Put it in cp_pipe (the old code can be removed later, so this needn't be considered a move, it's just a new cp_pipe task for now)

      Then, use this task and some darks from the AuxTel chip to calc xtalk coefficients for DM-18050.

      If it looks like the code if specifically for working with darks then the task should be named as such. If it looks like it will be sufficiently generic to work with saturated stars/CBP exposures then it should have a more generic name.

      Output should be a written with a butler.put() with an appropriate dataset type defined in obs_base datasets.yaml.

        Attachments

          Issue Links

            Activity

            Hide
            plazas Andrés Alejandro Plazas Malagón added a comment -

            isr already has something generic in "measureCrosstalk.py" and "crosstalk.py" that I have been using to get the crosstalk matrix for AuxTel data, with the help of Chris Waters.  Is there anything else that should be done for this ticket? 

            Show
            plazas Andrés Alejandro Plazas Malagón added a comment - isr already has something generic in "measureCrosstalk.py" and "crosstalk.py" that I have been using to get the crosstalk matrix for AuxTel data, with the help of Chris Waters.  Is there anything else that should be done for this ticket? 
            Hide
            mfisherlevine Merlin Fisher-Levine added a comment -

            No, if that code is running for AuxTel then that means it's already camera-agnostic, so just maybe note exactly how you're running it so that we can reproduce this stuff easily.

            Show
            mfisherlevine Merlin Fisher-Levine added a comment - No, if that code is running for AuxTel then that means it's already camera-agnostic, so just maybe note exactly how you're running it so that we can reproduce this stuff easily.
            Hide
            plazas Andrés Alejandro Plazas Malagón added a comment - - edited

            The following command is an example of how to run `measureCrosstalk.py`

            (lsst_distrib 19.0.0+65 w_2020_10 current setup)

            measureCrosstalk.py /project/shared/auxTel --rerun plazas/crosstalk_test --crosstalkName ciCppTestAndres --outputFileName /project/shared/auxTel/rerun/plazas/crosstalk_test/ctValues.yaml --dump-ratios /project/shared/auxTel/rerun/plazas/crosstalk_test/ctRatios.pkl -c isr.doLinearize=False -c isr.doDefect=False --id detector=0 expId=2020022000426..2020022000432

            I vereified that the coefficients were produced in /project/shared/auxTel/rerun/plazas/crosstalk_test/ctValues.yaml (this particular matrix has some invalid values, but that might be a matter of modifying the parameters of the task, or to use a different set of input images).

            If we have a matrix with the crosstalk coefficients in some file, a way to apply the correction is through something like this:

            repo=/project/shared/auxTel'
            butler = dafPersist.Butler(inputs=[repo])
            dataId = dict(dayObs="2020-02-20", seqnum=433)
            bias = butler.get('bias', dataId)
            dataRef = butler.dataRef('raw', dataId=dataId)
            hdu = pf.open("./R01_S11_R01_S11_crosstalk_results.fits")
            xtalk_slac_itl = hdu[0].data
            for i in np.arange(0,16):
            xtalk_slac_itl[i,i] = 0.0
            new_coeff_1d= np.ravel(xtalk_slac_itl)
            isr_config = IsrTask.ConfigClass()
            isr_config.doDark=False
            isr_config.doFlat=False
            isr_config.doBias=True
            isr_config.doFringe=False
            isr_config.doDefect=False
            isr_config.doWrite=False
            #isr_config.doAddDistortionModel=False
            isr_config.doLinearize=False
            isr_config.doSaturation=False
            #doSaturationInterpolation=True
            isr_config.doSaturation=True
            doSaturationInterpolation=True
            isr_config.doCrosstalk=True
            isr_config.crosstalk.useConfigCoefficients=True
            isr_config.crosstalk.crosstalkBackgroundMethod=None
            isr_config.crosstalk.crosstalkValues=new_coeff_1d
            isr_config.crosstalk.crosstalkShape=(16, 16)
            isr = IsrTask(config=isr_config)
            postIsrImage_slac_yes = isr.runDataRef(dataRef).exposure

            Show
            plazas Andrés Alejandro Plazas Malagón added a comment - - edited The following command is an example of how to run `measureCrosstalk.py` (lsst_distrib 19.0.0+65 w_2020_10 current setup) measureCrosstalk.py /project/shared/auxTel --rerun plazas/crosstalk_test --crosstalkName ciCppTestAndres --outputFileName /project/shared/auxTel/rerun/plazas/crosstalk_test/ctValues.yaml --dump-ratios /project/shared/auxTel/rerun/plazas/crosstalk_test/ctRatios.pkl -c isr.doLinearize=False -c isr.doDefect=False --id detector=0 expId=2020022000426..2020022000432 I vereified that the coefficients were produced in /project/shared/auxTel/rerun/plazas/crosstalk_test/ctValues.yaml (this particular matrix has some invalid values, but that might be a matter of modifying the parameters of the task, or to use a different set of input images). If we have a matrix with the crosstalk coefficients in some file, a way to apply the correction is through something like this: repo=/project/shared/auxTel' butler = dafPersist.Butler(inputs= [repo] ) dataId = dict(dayObs="2020-02-20", seqnum=433) bias = butler.get('bias', dataId) dataRef = butler.dataRef('raw', dataId=dataId) hdu = pf.open("./R01_S11_R01_S11_crosstalk_results.fits") xtalk_slac_itl = hdu [0] .data for i in np.arange(0,16): xtalk_slac_itl [i,i] = 0.0 new_coeff_1d= np.ravel(xtalk_slac_itl) isr_config = IsrTask.ConfigClass() isr_config.doDark=False isr_config.doFlat=False isr_config.doBias=True isr_config.doFringe=False isr_config.doDefect=False isr_config.doWrite=False #isr_config.doAddDistortionModel=False isr_config.doLinearize=False isr_config.doSaturation=False #doSaturationInterpolation=True isr_config.doSaturation=True doSaturationInterpolation=True isr_config.doCrosstalk=True isr_config.crosstalk.useConfigCoefficients=True isr_config.crosstalk.crosstalkBackgroundMethod=None isr_config.crosstalk.crosstalkValues=new_coeff_1d isr_config.crosstalk.crosstalkShape=(16, 16) isr = IsrTask(config=isr_config) postIsrImage_slac_yes = isr.runDataRef(dataRef).exposure

              People

              • Assignee:
                plazas Andrés Alejandro Plazas Malagón
                Reporter:
                mfisherlevine Merlin Fisher-Levine
                Watchers:
                Andrés Alejandro Plazas Malagón, John Swinbank, Merlin Fisher-Levine
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel