Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-23159

Make PTC task compatible with RFC-352

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Make the PTC measurement task compatible with RFC-352, and move all the algorithmic code to a Task.run() method which returns to the Task.runDataRef() all the products to be saved. This would move all the dataRef.get() and dataRef.put() code to runDataRef(), letting run() operate only on python objects. The benefit of this is that gen3 conversion is then much easier (and can focus on gen3/middleware code instead of algorithmic code).

        Attachments

          Issue Links

            Activity

            Hide
            plazas Andrés Alejandro Plazas Malagón added a comment - - edited

            Suggestion by C. Waters:

            • L339-356 move to a run method, which is the "algorithmic" section. The idea being that run( list(postISRCCD), config) yields Struct(plots=list(ptcFitPlots), ptcDataset=dataset, linearizer=LinearizerLookupTable, etcDataToSave=somethingElse) , which runDataRef then calls butler.put on.
            Show
            plazas Andrés Alejandro Plazas Malagón added a comment - - edited Suggestion by C. Waters: Move the dataset into a new run() method, that expects a list of ISR processed exposures, and returns a struct containing all of the output products that will be written to disk. L328-337 ( https://github.com/lsst/cp_pipe/blob/tickets/DM-21221/python/lsst/cp/pipe/ptc.py ) stay in runDataRef, as they're doing the ISR processing to generate the postISRCCD files that will be passed to run. L339-356 move to a run method, which is the "algorithmic" section. The idea being that run( list(postISRCCD), config) yields Struct(plots=list(ptcFitPlots), ptcDataset=dataset, linearizer=LinearizerLookupTable, etcDataToSave=somethingElse) , which runDataRef then calls butler.put on.
            Hide
            plazas Andrés Alejandro Plazas Malagón added a comment - - edited

            ci_cpp_gen2 build (w_2020_43 setup): ptc_ci_cpp_gen2_2020OCT27.txt

            Show
            plazas Andrés Alejandro Plazas Malagón added a comment - - edited ci_cpp_gen2 build ( w_2020_43 setup ): ptc_ci_cpp_gen2_2020OCT27.txt
            Hide
            plazas Andrés Alejandro Plazas Malagón added a comment - - edited

            The PTC code has been split into two tasks: extract and solve. The first task receives as input a list of flats, and sorts these flats in pairs taken at the same time (if there’s a different number of flats, those flats are discarded). The mean, variance (using afw), and covariances (as in Astier+19) are measured from the difference of the flat pairs at a given time. The measured covariances at that time (along with other quantities such as the mean) are stored in a PTC dataset object (PhotonTransferCurveDataset), which gets partially filled. The number of partially-filled PTC dataset objects will be less than the number of input exposures, but gen3 requires/assumes that the number of input dimensions matches bijectively the number of output dimensions. Therefore, a number of “dummy” PTC dataset are inserted in the output list that has the partially-filled PTC datasets with the covariances.

            The solve task assembles all the individual PTC datasets into a final PTC dataset. Then, it fits the measured (co)variances to a polynomial model or to the models described in equations 16 and 20 of Astier+19 (referred to as POLYNOMIAL,EXPAPPROXIMATION, and FULLCOVARIANCE in the configuration options of the solve task, respectively).

            The doPhotodiode option (to calibrate signal-chain non-linearity) is currently implemented for gen2 only.

            Example of constructing a gen3 repo and running the PTC task on BOT data, detector 94 (in /project/plazas/WORK/DM-23159/TEST_BOT with this branch and w_2020_52):

             butler create TEST_BOT
             butler register-instrument TEST_BOT/ lsst.obs.lsst.LsstCam 
             butler write-curated-calibrations TEST_BOT/ lsst.obs.lsst.LsstCam
            for i in $(ls -d /project/shared/BOT/_parent/raw/2020-10-08/*/); butler ingest-raws TEST_BOT/ $i; done
            

            Bias:

            pipetask run -j 1 -d "detector=94 AND exposure IN (3020100800090, 3020100800091, 3020100800092, 3020100800093, 3020100800094) and instrument = 'LSSTCam' " -b /project/plazas/WORK/DM-23159/TEST_BOT/butler.yaml -i LSSTCam/raw/all,LSSTCam/calib -o test_bias2 -p /home/plazas/lsst_devel/LSST/cp_pipe/pipelines/cpBias.yaml -c isr:doDefect=False --register-dataset-types
             
            butler certify-calibrations /project/plazas/WORK/DM-23159/TEST_BOT test_bias2 calib/v00 --begin-date 1980-01-01 --end-date 2050-01-01 bias
            

            Dark:

            pipetask run -j 1 -d "detector=94 AND exposure IN (3020100800045) and instrument = 'LSSTCam' " -b /project/plazas/WORK/DM-23159/TEST_BOT/butler.yaml -i LSSTCam/raw/all,LSSTCam/calib,calib/v00 -o test_dark6 -p /home/plazas/lsst_devel/LSST/cp_pipe/pipelines/cpDark.yaml -c isr:doDefect=False -c isr:doCrosstalk=False -c isr:overscan.fitType=MEDIAN -c isr:overscan.order=1 --register-dataset-types
             
            butler certify-calibrations /project/plazas/WORK/DM-23159/TEST_BOT test_dark6 calib/v00 --begin-date 1980-01-01 --end-date 2050-01-01 dark
            

            Flat:

            pipetask run -j 1 -d "detector=94 AND exposure IN (3020100800140, 3020100800141) and instrument = 'LSSTCam' " -b /project/plazas/WORK/DM-23159/TEST_BOT/butler.yaml -i LSSTCam/raw/all,LSSTCam/calib,calib/v00 -o test_flat5 -p /home/plazas/lsst_devel/LSST/cp_pipe/pipelines/cpFlat.yaml -c isr:doDefect=False -c isr:doCrosstalk=False -c isr:overscan.fitType=MEDIAN -c isr:overscan.order=1 -c cpFlatMeasure:doVignette=False --register-dataset-types
             
            butler certify-calibrations /project/plazas/WORK/DM-23159/TEST_BOT test_flat5 calib/v00 --begin-date 1980-01-01 --end-date 2050-01-01 flat
            

            Defects:

            pipetask run -j 1 -d "detector=94 AND exposure IN (3020100800140,3020100800141,3020100800145) and instrument = 'LSSTCam' " -b /project/plazas/WORK/DM-23159/TEST_BOT/butler.yaml -i LSSTCam/raw/all,LSSTCam/calib,calib/v00 -o test_defects -p /home/plazas/lsst_devel/LSST/cp_pipe/pipelines/findDefects.yaml  -c measureDefects:badOnAndOffPixelColumnThreshold=50 --register-dataset-types
             
            butler certify-calibrations /project/plazas/WORK/DM-23159/TEST_BOT test_defects calib/v00 --begin-date 1980-01-01 --end-date 2050-01-01 defects
            

            ISR:

            pipetask run -j 1 -d "detector=94 AND exposure IN (3020100800155,3020100800156,3020100800158,3020100800159,3020100800185,3020100800186,3020100800161,3020100800162,3020100800188,3020100800189,3020100800164,3020100800165,3020100800191,3020100800192,3020100800167,3020100800168,3020100800194,3020100800195,3020100800170,3020100800171,3020100800197,3020100800198,3020100800173,3020100800174,3020100800200,3020100800201,3020100800176,3020100800177,3020100800203,3020100800204,3020100800179,3020100800180,3020100800206,3020100800207,3020100800182,3020100800183,3020100800209,3020100800210,3020100800212,3020100800213,3020100800215,3020100800216,3020100800218,3020100800219,3020100800221,3020100800222) and instrument = 'LSSTCam' " -b /project/plazas/WORK/DM-23159/TEST_BOT/butler.yaml -i LSSTCam/raw/all,LSSTCam/calib,calib/v00 -o ptcTestPostIsr.02 -p /home/plazas/lsst_devel/LSST/cp_pipe/pipelines/isrForPtc.yaml -c isr:overscan.fitType=MEDIAN -c isr:overscan.order=1 -c isr:doFringe=False -c isr:doSuspect=True -c isr:edgeMaskLevel=AMP -c isr:numEdgeSuspect=10 -c isr:doFlat=False --register-dataset-types
            

            with /home/plazas/lsst_devel/LSST/cp_pipe/pipelines/isrForPtc.yaml

            description: cp_pipe PTC calibration construction.
            tasks:
              isr:
                class: lsst.ip.isr.isrTask.IsrTask
                config:
                  connections.ccdExposure: 'raw'
                  connections.outputExposure: 'cpPtcProc'
                  doWrite: True
                  doOverscan: True
                  doAssembleCcd: True
                  doBias: True
                  doVariance: True
                  doLinearize: False
                  doCrosstalk: False
                  doBrighterFatter: False
                  doDark: True
                  doStrayLight: False
                  doFlat: True
                  doFringe: False
                  doApplyGains: False
                  doDefect: True
                  doNanMasking: True
                  doInterpolate: False
                  doSaturation: False
                  doSaturationInterpolation: False
                  growSaturationFootprintSize: 0
            

            PTC (for some reason it could not find the flat, so I set it to False):

            pipetask run -j 1 -d "detector=94 AND exposure IN (3020100800155,3020100800156,3020100800158,3020100800159,3020100800185,3020100800186,3020100800161,3020100800162,3020100800188,3020100800189,3020100800164,3020100800165,3020100800191,3020100800192,3020100800167,3020100800168,3020100800194,3020100800195,3020100800170,3020100800171,3020100800197,3020100800198,3020100800173,3020100800174,3020100800200,3020100800201,3020100800176,3020100800177,3020100800203,3020100800204,3020100800179,3020100800180,3020100800206,3020100800207,3020100800182,3020100800183,3020100800209,3020100800210,3020100800212,3020100800213,3020100800215,3020100800216,3020100800218,3020100800219,3020100800221,3020100800222) and instrument = 'LSSTCam' " -b /project/plazas/WORK/DM-23159/TEST_BOT/butler.yaml -i ptcTestPostIsr.02 -o ptcTestResultado.06 -p /home/plazas/lsst_devel/LSST/cp_pipe/pipelines/measurePhotonTransferCurve.yaml -c ptcSolve:ptcFitType=EXPAPPROXIMATION --register-dataset-types
            

            (with ISR options: )

            pipetask run -j 1 -d "detector=94 AND exposure IN (3020100800155,3020100800156,3020100800158,3020100800159,3020100800185,3020100800186,3020100800161,3020100800162,3020100800188,3020100800189,3020100800164,3020100800165,3020100800191,3020100800192,3020100800167,3020100800168,3020100800194,3020100800195,3020100800170,3020100800171,3020100800197,3020100800198,3020100800173,3020100800174,3020100800200,3020100800201,3020100800176,3020100800177,3020100800203,3020100800204,3020100800179,3020100800180,3020100800206,3020100800207,3020100800182,3020100800183,3020100800209,3020100800210,3020100800212,3020100800213,3020100800215,3020100800216,3020100800218,3020100800219,3020100800221,3020100800222) and instrument = 'LSSTCam' " -b /project/plazas/WORK/DM-23159/TEST_BOT/butler.yaml -i ptcTestPostIsr.02 -o ptcTestProbando.04 -p /home/plazas/lsst_devel/LSST/cp_pipe/pipelines/measurePhotonTransferCurve.yaml -c ptcSolve:ptcFitType=EXPAPPROXIMATION -c isr:doFlat=False -c isr:doCrosstalk=False --register-dataset-types
            

            Output dataset: ptcTestResultado.06/20201231T020616Z/ptcDataset/ptcDataset_94_LSSTCam_ptcTestResultado_06_20201231T020616Z.fits

            from lsst.ip.isr import PhotonTransferCurveDataset 
            filename="ptcTestResultado.06/20201231T020616Z/ptcDataset/ptcDataset_94_LSSTCam_ptcTestResultado_06_20201231T020616Z.fits"
            fromFits = PhotonTransferCurveDataset.readFits(filename) 
            fromFits.gain 
            fromFits.noise
            

            Gains:

             
            {'C00': 1.062342253140052,
             'C01': 1.069284400977535,
             'C02': 1.0718946810066032,
             'C03': 1.0698720790589018,
             'C04': 1.0755230961249225,
             'C05': 1.0697500124030388,
             'C06': 1.0696513340670983,
             'C07': 1.0618395593772259,
             'C10': 1.0788131187964267,
             'C11': 1.0833694635298263,
             'C12': 1.095240826031433,
             'C13': 1.0914294799719313,
             'C14': 1.0958563288441774,
             'C15': 1.0985139310664527,
             'C16': 1.0973343675070544,
             'C17': 1.0969255452082705}
            
            

            Noise:

            {'C00': 6.151925075768491,
             'C01': 5.97459431206064,
             'C02': 5.6993306138734114,
             'C03': 5.621638005069704,
             'C04': 5.6358216929068226,
             'C05': 6.082843390559317,
             'C06': 5.569225331081291,
             'C07': 6.158968155376583,
             'C10': 5.876614760829971,
             'C11': 6.036380486530242,
             'C12': 5.772919322451841,
             'C13': 5.896424344627466,
             'C14': 6.068517828241981,
             'C15': 6.7266935200050675,
             'C16': 6.674482599560533,
             'C17': 5.951403680148852}
            

            Using FULLCOVARIANCE: ptcTestResultado.07/20201231T024110Z/ptcDataset/ptcDataset_94_LSSTCam_ptcTestResultado_07_20201231T024110Z.fits

            Gain:

            {'C00': 1.0425526047034526,
             'C01': 1.0530311663384373,
             'C02': 1.0540685272603145,
             'C03': 1.0386811075221716,
             'C04': 1.061475832585829,
             'C05': 1.0555755173001389,
             'C06': 1.0434694202440942,
             'C07': 1.0356749845692936,
             'C10': 1.0582041818437944,
             'C11': 1.0645796838195933,
             'C12': 1.0819382322040194,
             'C13': 1.0939012841298579,
             'C14': 1.0848968736492128,
             'C15': 1.078412587502557,
             'C16': 1.0788399491643776,
             'C17': 1.0824094906159403}
            

            Noise:

            {'C00': 5.033597545607389,
             'C01': 5.1658357180823,
             'C02': 4.7660533194003785,
             'C03': 4.077912281257251,
             'C04': 4.852569375938531,
             'C05': 5.323858607073495,
             'C06': 4.4042938592755725,
             'C07': 5.685987684065248,
             'C10': 4.857324917128721,
             'C11': 5.2376226461227455,
             'C12': 5.9238080874005545,
             'C13': 6.717944248640032,
             'C14': 5.572157959987069,
             'C15': 6.026442101939837,
             'C16': 5.9025554026867795,
             'C17': 5.471607360413247}
            

            Show
            plazas Andrés Alejandro Plazas Malagón added a comment - - edited The PTC code has been split into two tasks: extract and solve . The first task receives as input a list of flats, and sorts these flats in pairs taken at the same time (if there’s a different number of flats, those flats are discarded). The mean, variance (using afw), and covariances (as in Astier+19) are measured from the difference of the flat pairs at a given time. The measured covariances at that time (along with other quantities such as the mean) are stored in a PTC dataset object ( PhotonTransferCurveDataset ), which gets partially filled. The number of partially-filled PTC dataset objects will be less than the number of input exposures, but gen3 requires/assumes that the number of input dimensions matches bijectively the number of output dimensions. Therefore, a number of “dummy” PTC dataset are inserted in the output list that has the partially-filled PTC datasets with the covariances. The solve task assembles all the individual PTC datasets into a final PTC dataset. Then, it fits the measured (co)variances to a polynomial model or to the models described in equations 16 and 20 of Astier+19 (referred to as POLYNOMIAL , EXPAPPROXIMATION , and FULLCOVARIANCE in the configuration options of the solve task, respectively). The doPhotodiode option (to calibrate signal-chain non-linearity) is currently implemented for gen2 only. Example of constructing a gen3 repo and running the PTC task on BOT data, detector 94 (in /project/plazas/WORK/ DM-23159 /TEST_BOT with this branch and w_2020_52 ): butler create TEST_BOT butler register-instrument TEST_BOT/ lsst.obs.lsst.LsstCam butler write-curated-calibrations TEST_BOT/ lsst.obs.lsst.LsstCam for i in $(ls -d /project/shared/BOT/_parent/raw/2020-10-08/*/); butler ingest-raws TEST_BOT/ $i; done Bias: pipetask run -j 1 -d "detector=94 AND exposure IN (3020100800090, 3020100800091, 3020100800092, 3020100800093, 3020100800094) and instrument = 'LSSTCam' " -b /project/plazas/WORK/DM-23159/TEST_BOT/butler.yaml -i LSSTCam/raw/all,LSSTCam/calib -o test_bias2 -p /home/plazas/lsst_devel/LSST/cp_pipe/pipelines/cpBias.yaml -c isr:doDefect=False --register-dataset-types   butler certify-calibrations /project/plazas/WORK/DM-23159/TEST_BOT test_bias2 calib/v00 --begin-date 1980-01-01 --end-date 2050-01-01 bias Dark: pipetask run -j 1 -d "detector=94 AND exposure IN (3020100800045) and instrument = 'LSSTCam' " -b /project/plazas/WORK/DM-23159/TEST_BOT/butler.yaml -i LSSTCam/raw/all,LSSTCam/calib,calib/v00 -o test_dark6 -p /home/plazas/lsst_devel/LSST/cp_pipe/pipelines/cpDark.yaml -c isr:doDefect=False -c isr:doCrosstalk=False -c isr:overscan.fitType=MEDIAN -c isr:overscan.order=1 --register-dataset-types   butler certify-calibrations /project/plazas/WORK/DM-23159/TEST_BOT test_dark6 calib/v00 --begin-date 1980-01-01 --end-date 2050-01-01 dark Flat: pipetask run -j 1 -d "detector=94 AND exposure IN (3020100800140, 3020100800141) and instrument = 'LSSTCam' " -b /project/plazas/WORK/DM-23159/TEST_BOT/butler.yaml -i LSSTCam/raw/all,LSSTCam/calib,calib/v00 -o test_flat5 -p /home/plazas/lsst_devel/LSST/cp_pipe/pipelines/cpFlat.yaml -c isr:doDefect=False -c isr:doCrosstalk=False -c isr:overscan.fitType=MEDIAN -c isr:overscan.order=1 -c cpFlatMeasure:doVignette=False --register-dataset-types   butler certify-calibrations /project/plazas/WORK/DM-23159/TEST_BOT test_flat5 calib/v00 --begin-date 1980-01-01 --end-date 2050-01-01 flat Defects: pipetask run -j 1 -d "detector=94 AND exposure IN (3020100800140,3020100800141,3020100800145) and instrument = 'LSSTCam' " -b /project/plazas/WORK/DM-23159/TEST_BOT/butler.yaml -i LSSTCam/raw/all,LSSTCam/calib,calib/v00 -o test_defects -p /home/plazas/lsst_devel/LSST/cp_pipe/pipelines/findDefects.yaml -c measureDefects:badOnAndOffPixelColumnThreshold=50 --register-dataset-types   butler certify-calibrations /project/plazas/WORK/DM-23159/TEST_BOT test_defects calib/v00 --begin-date 1980-01-01 --end-date 2050-01-01 defects ISR: pipetask run -j 1 -d "detector=94 AND exposure IN (3020100800155,3020100800156,3020100800158,3020100800159,3020100800185,3020100800186,3020100800161,3020100800162,3020100800188,3020100800189,3020100800164,3020100800165,3020100800191,3020100800192,3020100800167,3020100800168,3020100800194,3020100800195,3020100800170,3020100800171,3020100800197,3020100800198,3020100800173,3020100800174,3020100800200,3020100800201,3020100800176,3020100800177,3020100800203,3020100800204,3020100800179,3020100800180,3020100800206,3020100800207,3020100800182,3020100800183,3020100800209,3020100800210,3020100800212,3020100800213,3020100800215,3020100800216,3020100800218,3020100800219,3020100800221,3020100800222) and instrument = 'LSSTCam' " -b /project/plazas/WORK/DM-23159/TEST_BOT/butler.yaml -i LSSTCam/raw/all,LSSTCam/calib,calib/v00 -o ptcTestPostIsr.02 -p /home/plazas/lsst_devel/LSST/cp_pipe/pipelines/isrForPtc.yaml -c isr:overscan.fitType=MEDIAN -c isr:overscan.order=1 -c isr:doFringe=False -c isr:doSuspect=True -c isr:edgeMaskLevel=AMP -c isr:numEdgeSuspect=10 -c isr:doFlat=False --register-dataset-types with /home/plazas/lsst_devel/LSST/cp_pipe/pipelines/isrForPtc.yaml description: cp_pipe PTC calibration construction. tasks: isr: class: lsst.ip.isr.isrTask.IsrTask config: connections.ccdExposure: 'raw' connections.outputExposure: 'cpPtcProc' doWrite: True doOverscan: True doAssembleCcd: True doBias: True doVariance: True doLinearize: False doCrosstalk: False doBrighterFatter: False doDark: True doStrayLight: False doFlat: True doFringe: False doApplyGains: False doDefect: True doNanMasking: True doInterpolate: False doSaturation: False doSaturationInterpolation: False growSaturationFootprintSize: 0 PTC (for some reason it could not find the flat, so I set it to False): pipetask run -j 1 -d "detector=94 AND exposure IN (3020100800155,3020100800156,3020100800158,3020100800159,3020100800185,3020100800186,3020100800161,3020100800162,3020100800188,3020100800189,3020100800164,3020100800165,3020100800191,3020100800192,3020100800167,3020100800168,3020100800194,3020100800195,3020100800170,3020100800171,3020100800197,3020100800198,3020100800173,3020100800174,3020100800200,3020100800201,3020100800176,3020100800177,3020100800203,3020100800204,3020100800179,3020100800180,3020100800206,3020100800207,3020100800182,3020100800183,3020100800209,3020100800210,3020100800212,3020100800213,3020100800215,3020100800216,3020100800218,3020100800219,3020100800221,3020100800222) and instrument = 'LSSTCam' " -b /project/plazas/WORK/DM-23159/TEST_BOT/butler.yaml -i ptcTestPostIsr.02 -o ptcTestResultado.06 -p /home/plazas/lsst_devel/LSST/cp_pipe/pipelines/measurePhotonTransferCurve.yaml -c ptcSolve:ptcFitType=EXPAPPROXIMATION --register-dataset-types (with ISR options: ) pipetask run -j 1 -d "detector=94 AND exposure IN (3020100800155,3020100800156,3020100800158,3020100800159,3020100800185,3020100800186,3020100800161,3020100800162,3020100800188,3020100800189,3020100800164,3020100800165,3020100800191,3020100800192,3020100800167,3020100800168,3020100800194,3020100800195,3020100800170,3020100800171,3020100800197,3020100800198,3020100800173,3020100800174,3020100800200,3020100800201,3020100800176,3020100800177,3020100800203,3020100800204,3020100800179,3020100800180,3020100800206,3020100800207,3020100800182,3020100800183,3020100800209,3020100800210,3020100800212,3020100800213,3020100800215,3020100800216,3020100800218,3020100800219,3020100800221,3020100800222) and instrument = 'LSSTCam' " -b /project/plazas/WORK/DM-23159/TEST_BOT/butler.yaml -i ptcTestPostIsr.02 -o ptcTestProbando.04 -p /home/plazas/lsst_devel/LSST/cp_pipe/pipelines/measurePhotonTransferCurve.yaml -c ptcSolve:ptcFitType=EXPAPPROXIMATION -c isr:doFlat=False -c isr:doCrosstalk=False --register-dataset-types Output dataset: ptcTestResultado.06/20201231T020616Z/ptcDataset/ptcDataset_94_LSSTCam_ptcTestResultado_06_20201231T020616Z.fits from lsst.ip.isr import PhotonTransferCurveDataset filename = "ptcTestResultado.06/20201231T020616Z/ptcDataset/ptcDataset_94_LSSTCam_ptcTestResultado_06_20201231T020616Z.fits" fromFits = PhotonTransferCurveDataset.readFits(filename) fromFits.gain fromFits.noise Gains:   {'C00': 1.062342253140052, 'C01': 1.069284400977535, 'C02': 1.0718946810066032, 'C03': 1.0698720790589018, 'C04': 1.0755230961249225, 'C05': 1.0697500124030388, 'C06': 1.0696513340670983, 'C07': 1.0618395593772259, 'C10': 1.0788131187964267, 'C11': 1.0833694635298263, 'C12': 1.095240826031433, 'C13': 1.0914294799719313, 'C14': 1.0958563288441774, 'C15': 1.0985139310664527, 'C16': 1.0973343675070544, 'C17': 1.0969255452082705} Noise: {'C00': 6.151925075768491, 'C01': 5.97459431206064, 'C02': 5.6993306138734114, 'C03': 5.621638005069704, 'C04': 5.6358216929068226, 'C05': 6.082843390559317, 'C06': 5.569225331081291, 'C07': 6.158968155376583, 'C10': 5.876614760829971, 'C11': 6.036380486530242, 'C12': 5.772919322451841, 'C13': 5.896424344627466, 'C14': 6.068517828241981, 'C15': 6.7266935200050675, 'C16': 6.674482599560533, 'C17': 5.951403680148852} Using FULLCOVARIANCE : ptcTestResultado.07/20201231T024110Z/ptcDataset/ptcDataset_94_LSSTCam_ptcTestResultado_07_20201231T024110Z.fits Gain: {'C00': 1.0425526047034526, 'C01': 1.0530311663384373, 'C02': 1.0540685272603145, 'C03': 1.0386811075221716, 'C04': 1.061475832585829, 'C05': 1.0555755173001389, 'C06': 1.0434694202440942, 'C07': 1.0356749845692936, 'C10': 1.0582041818437944, 'C11': 1.0645796838195933, 'C12': 1.0819382322040194, 'C13': 1.0939012841298579, 'C14': 1.0848968736492128, 'C15': 1.078412587502557, 'C16': 1.0788399491643776, 'C17': 1.0824094906159403} Noise: {'C00': 5.033597545607389, 'C01': 5.1658357180823, 'C02': 4.7660533194003785, 'C03': 4.077912281257251, 'C04': 4.852569375938531, 'C05': 5.323858607073495, 'C06': 4.4042938592755725, 'C07': 5.685987684065248, 'C10': 4.857324917128721, 'C11': 5.2376226461227455, 'C12': 5.9238080874005545, 'C13': 6.717944248640032, 'C14': 5.572157959987069, 'C15': 6.026442101939837, 'C16': 5.9025554026867795, 'C17': 5.471607360413247}
            Hide
            plazas Andrés Alejandro Plazas Malagón added a comment - - edited

            Comparing the results in the comment above for the same detector with the gen2 command in this branch (DM-23159):

            measurePhotonTransferCurve.py /project/shared/BOT/rerun/cslage/PTC_LSSTCAM_New_12606 --rerun plazas/DM-23159/2021JAN02 --id detector=94 expId=3020100800155^3020100800156^3020100800158^3020100800159^3020100800185^3020100800186^3020100800161^3020100800162^3020100800188^3020100800189^3020100800164^3020100800165^3020100800191^3020100800192^3020100800167^3020100800168^3020100800194^3020100800195^3020100800170^3020100800171^3020100800197^3020100800198^3020100800173^3020100800174^3020100800200^3020100800201^3020100800176^3020100800177^3020100800203^3020100800204^3020100800179^3020100800180^3020100800206^3020100800207^3020100800182^3020100800183^3020100800209^3020100800210^3020100800212^3020100800213^3020100800215^3020100800216 -c solve.ptcFitType=EXPAPPROXIMATION doPhotodiode=False --clobber-config --clobber-version -j 1
            

            Using EXPAPPROXIMATION: /project/shared/BOT/rerun/cslage/PTC_LSSTCAM_New_12606/rerun/plazas/DM-23159/2021JAN02/calibrations/ptc/ptcDataset-det094.fits

            Gain:

            {'C00': 1.0623665682015864,
             'C01': 1.0692736458129461,
             'C02': 1.0719061777555439,
             'C03': 1.0698186646485832,
             'C04': 1.0755065917205915,
             'C05': 1.0697223373796645,
             'C06': 1.0696511819243888,
             'C07': 1.0618560986812993,
             'C10': 1.0788291305881819,
             'C11': 1.0833586774001116,
             'C12': 1.0952379465349606,
             'C13': 1.0914619568425337,
             'C14': 1.0958556943098245,
             'C15': 1.0984705558827348,
             'C16': 1.0973459162125423,
             'C17': 1.096906929928102}
            

            Noise:

            {'C00': 6.153475805831076,
             'C01': 5.973331468670271,
             'C02': 5.699858878211565,
             'C03': 5.61691772662747,
             'C04': 5.635055718329871,
             'C05': 6.079625027880279,
             'C06': 5.569378782470496,
             'C07': 6.159928134084513,
             'C10': 5.875580728681517,
             'C11': 6.0344819056114,
             'C12': 5.772831136821642,
             'C13': 5.899530865591045,
             'C14': 6.069236296852714,
             'C15': 6.724938160329195,
             'C16': 6.67640273657015,
             'C17': 5.950307038961287}
            

            Using FULLCOVARIANCE: /project/shared/BOT/rerun/cslage/PTC_LSSTCAM_New_12606/rerun/plazas/DM-23159/2021JAN02/FULLCOVARIANCE/calibrations/ptc/ptcDataset-det094.fits

            Gain:

            {'C00': 1.0425531053098844,
             'C01': 1.0530460028237154,
             'C02': 1.0540841995693877,
             'C03': 1.0386628419980093,
             'C04': 1.061489431439738,
             'C05': 1.0556060905559954,
             'C06': 1.0434438508943065,
             'C07': 1.0356673273507941,
             'C10': 1.0582499068756281,
             'C11': 1.0645580775004337,
             'C12': 1.081944596613428,
             'C13': 1.0939098534071015,
             'C14': 1.084900009252566,
             'C15': 1.0783756852548805,
             'C16': 1.0788008311368777,
             'C17': 1.0809517945824252}
            

            Noise:

            {'C00': 5.033844945221069,
             'C01': 5.16586296071402,
             'C02': 4.767118832864183,
             'C03': 4.076103010864511,
             'C04': 4.853851376347369,
             'C05': 5.324508253437716,
             'C06': 4.403355871485114,
             'C07': 5.684202672212702,
             'C10': 4.859855629246495,
             'C11': 5.235422775870591,
             'C12': 5.925012945621021,
             'C13': 6.718464542041558,
             'C14': 5.5733037153519325,
             'C15': 6.025369660453583,
             'C16': 5.901638560696901,
             'C17': 5.429647760406786}
            

            Show
            plazas Andrés Alejandro Plazas Malagón added a comment - - edited Comparing the results in the comment above for the same detector with the gen2 command in this branch ( DM-23159 ): measurePhotonTransferCurve.py /project/shared/BOT/rerun/cslage/PTC_LSSTCAM_New_12606 --rerun plazas/DM-23159/2021JAN02 --id detector=94 expId=3020100800155^3020100800156^3020100800158^3020100800159^3020100800185^3020100800186^3020100800161^3020100800162^3020100800188^3020100800189^3020100800164^3020100800165^3020100800191^3020100800192^3020100800167^3020100800168^3020100800194^3020100800195^3020100800170^3020100800171^3020100800197^3020100800198^3020100800173^3020100800174^3020100800200^3020100800201^3020100800176^3020100800177^3020100800203^3020100800204^3020100800179^3020100800180^3020100800206^3020100800207^3020100800182^3020100800183^3020100800209^3020100800210^3020100800212^3020100800213^3020100800215^3020100800216 -c solve.ptcFitType=EXPAPPROXIMATION doPhotodiode=False --clobber-config --clobber-version -j 1 Using EXPAPPROXIMATION : /project/shared/BOT/rerun/cslage/PTC_LSSTCAM_New_12606/rerun/plazas/ DM-23159 /2021JAN02/calibrations/ptc/ptcDataset-det094.fits Gain: {'C00': 1.0623665682015864, 'C01': 1.0692736458129461, 'C02': 1.0719061777555439, 'C03': 1.0698186646485832, 'C04': 1.0755065917205915, 'C05': 1.0697223373796645, 'C06': 1.0696511819243888, 'C07': 1.0618560986812993, 'C10': 1.0788291305881819, 'C11': 1.0833586774001116, 'C12': 1.0952379465349606, 'C13': 1.0914619568425337, 'C14': 1.0958556943098245, 'C15': 1.0984705558827348, 'C16': 1.0973459162125423, 'C17': 1.096906929928102} Noise: {'C00': 6.153475805831076, 'C01': 5.973331468670271, 'C02': 5.699858878211565, 'C03': 5.61691772662747, 'C04': 5.635055718329871, 'C05': 6.079625027880279, 'C06': 5.569378782470496, 'C07': 6.159928134084513, 'C10': 5.875580728681517, 'C11': 6.0344819056114, 'C12': 5.772831136821642, 'C13': 5.899530865591045, 'C14': 6.069236296852714, 'C15': 6.724938160329195, 'C16': 6.67640273657015, 'C17': 5.950307038961287} Using FULLCOVARIANCE : /project/shared/BOT/rerun/cslage/PTC_LSSTCAM_New_12606/rerun/plazas/ DM-23159 /2021JAN02/FULLCOVARIANCE/calibrations/ptc/ptcDataset-det094.fits Gain: {'C00': 1.0425531053098844, 'C01': 1.0530460028237154, 'C02': 1.0540841995693877, 'C03': 1.0386628419980093, 'C04': 1.061489431439738, 'C05': 1.0556060905559954, 'C06': 1.0434438508943065, 'C07': 1.0356673273507941, 'C10': 1.0582499068756281, 'C11': 1.0645580775004337, 'C12': 1.081944596613428, 'C13': 1.0939098534071015, 'C14': 1.084900009252566, 'C15': 1.0783756852548805, 'C16': 1.0788008311368777, 'C17': 1.0809517945824252} Noise: {'C00': 5.033844945221069, 'C01': 5.16586296071402, 'C02': 4.767118832864183, 'C03': 4.076103010864511, 'C04': 4.853851376347369, 'C05': 5.324508253437716, 'C06': 4.403355871485114, 'C07': 5.684202672212702, 'C10': 4.859855629246495, 'C11': 5.235422775870591, 'C12': 5.925012945621021, 'C13': 6.718464542041558, 'C14': 5.5733037153519325, 'C15': 6.025369660453583, 'C16': 5.901638560696901, 'C17': 5.429647760406786}
            Hide
            plazas Andrés Alejandro Plazas Malagón added a comment -

            Comparing the results to the verions of cp_pipe and ip_isr currently in master (w_2020_52):

            ptcFitType: EXPAPPROXIMATION: /project/shared/BOT/rerun/cslage/PTC_LSSTCAM_New_12606/rerun/plazas/DM-23159/2021JAN02/EXPAPPROX-MASTER/calibrations/ptc/ptcDataset-det094.fits

            /home/plazas/lsst_devel/LSST/cp_pipe/bin/measurePhotonTransferCurve.py /project/shared/BOT/rerun/cslage/PTC_LSSTCAM_New_12606 --rerun plazas/DM-23159/2021JAN02/EXPAPPROX-MASTER --id detector=94 expId=3020100800155^3020100800156^3020100800158^3020100800159^3020100800185^3020100800186^3020100800161^3020100800162^3020100800188^3020100800189^3020100800164^3020100800165^3020100800191^3020100800192^3020100800167^3020100800168^3020100800194^3020100800195^3020100800170^3020100800171^3020100800197^3020100800198^3020100800173^3020100800174^3020100800200^3020100800201^3020100800176^3020100800177^3020100800203^3020100800204^3020100800179^3020100800180^3020100800206^3020100800207^3020100800182^3020100800183^3020100800209^3020100800210^3020100800212^3020100800213^3020100800215^3020100800216 -c ptcFitType=EXPAPPROXIMATION doPhotodiode=False --clobber-config --clobber-version -j 1
            

            Gain:

            {'C00':  1.0623665682015864
            'C01':  1.0692736458129461
            'C02':  1.0719061777555439
            'C03':  1.0698186646485832
            'C04':  1.0755065917205915
            'C05':  1.0697223373796645
            'C06':  1.0696511819243888
            'C07':  1.0618560986812993
            'C10':  1.0788291305881819
            'C11':  1.0833586774001116
            'C12':  1.0952379465349606
            'C13':  1.0914619568425337
            'C14':  1.0958556943098245
            'C15':  1.0984705558827348
            'C16':  1.0973459162125423
            'C17':  1.096906929928102}
            

            Noise:

            'C00':  6.153475805831076
            'C01':  5.973331468670271
            'C02':  5.699858878211565
            'C03':  5.61691772662747
            'C04':  5.635055718329871
            'C05':  6.079625027880279
            'C06':  5.569378782470496
            'C07':  6.159928134084513
            'C10':  5.875580728681517
            'C11':  6.0344819056114
            'C12':  5.772831136821642
            'C13':  5.899530865591045
            'C14':  6.069236296852714
            'C15':  6.724938160329195
            'C16':  6.67640273657015
            'C17':  5.950307038961287
            

            ptcFitType: FULLCOVARIANCE:

            /project/shared/BOT/rerun/cslage/PTC_LSSTCAM_New_12606/rerun/plazas/DM-23159/2021JAN02/EXPAPPROX-FULLCOVARIANCE/calibrations/ptc/ptcDataset-det094.fits

            measurePhotonTransferCurve.py /project/shared/BOT/rerun/cslage/PTC_LSSTCAM_New_12606 --rerun plazas/DM-23159/2021JAN02/EXPAPPROX-FULLCOVARIANCE --id detector=94 expId=3020100800155^3020100800156^3020100800158^3020100800159^3020100800185^3020100800186^3020100800161^3020100800162^3020100800188^3020100800189^3020100800164^3020100800165^3020100800191^3020100800192^3020100800167^3020100800168^3020100800194^3020100800195^3020100800170^3020100800171^3020100800197^3020100800198^3020100800173^3020100800174^3020100800200^3020100800201^3020100800176^3020100800177^3020100800203^3020100800204^3020100800179^3020100800180^3020100800206^3020100800207^3020100800182^3020100800183^3020100800209^3020100800210^3020100800212^3020100800213^3020100800215^3020100800216 -c ptcFitType=FULLCOVARIANCE doPhotodiode=False --clobber-config --clobber-version -j 1
            

            Gain:

            'C00':  1.0425556887637621
            'C01':  1.0530442420728823
            'C02':  1.0540860068259987
            'C03':  1.0386621624006385
            'C04':  1.0614891977940373
            'C05':  1.0556061121455635
            'C06':  1.0434440018245594
            'C07':  1.035667225196643
            'C10':  1.058252331030148
            'C11':  1.0645580285748109
            'C12':  1.0819436197242243
            'C13':  1.0939079018240114
            'C14':  1.0849000245608877
            'C15':  1.07837551226726
            'C16':  1.078800800817564
            'C17':  1.0809518420287751
            

            Noise:

            'C00':  5.033950113017649
            'C01':  5.165794683782765
            'C02':  4.767193068143683
            'C03':  4.076072653277696
            'C04':  4.853841940191721
            'C05':  5.3245091460021134
            'C06':  4.403362155458655
            'C07':  5.684199300487573
            'C10':  4.859952630497146
            'C11':  5.235420893116862
            'C12':  5.9249790705589565
            'C13':  6.7183983328087855
            'C14':  5.573304230848151
            'C15':  6.0253641613470155
            'C16':  5.901637552758507
            'C17':  5.429649457878611
            

            Show
            plazas Andrés Alejandro Plazas Malagón added a comment - Comparing the results to the verions of cp_pipe and ip_isr currently in master ( w_2020_52 ): ptcFitType: EXPAPPROXIMATION : /project/shared/BOT/rerun/cslage/PTC_LSSTCAM_New_12606/rerun/plazas/ DM-23159 /2021JAN02/EXPAPPROX-MASTER/calibrations/ptc/ptcDataset-det094.fits /home/plazas/lsst_devel/LSST/cp_pipe/bin/measurePhotonTransferCurve.py /project/shared/BOT/rerun/cslage/PTC_LSSTCAM_New_12606 --rerun plazas/DM-23159/2021JAN02/EXPAPPROX-MASTER --id detector=94 expId=3020100800155^3020100800156^3020100800158^3020100800159^3020100800185^3020100800186^3020100800161^3020100800162^3020100800188^3020100800189^3020100800164^3020100800165^3020100800191^3020100800192^3020100800167^3020100800168^3020100800194^3020100800195^3020100800170^3020100800171^3020100800197^3020100800198^3020100800173^3020100800174^3020100800200^3020100800201^3020100800176^3020100800177^3020100800203^3020100800204^3020100800179^3020100800180^3020100800206^3020100800207^3020100800182^3020100800183^3020100800209^3020100800210^3020100800212^3020100800213^3020100800215^3020100800216 -c ptcFitType=EXPAPPROXIMATION doPhotodiode=False --clobber-config --clobber-version -j 1 Gain: {'C00': 1.0623665682015864 'C01': 1.0692736458129461 'C02': 1.0719061777555439 'C03': 1.0698186646485832 'C04': 1.0755065917205915 'C05': 1.0697223373796645 'C06': 1.0696511819243888 'C07': 1.0618560986812993 'C10': 1.0788291305881819 'C11': 1.0833586774001116 'C12': 1.0952379465349606 'C13': 1.0914619568425337 'C14': 1.0958556943098245 'C15': 1.0984705558827348 'C16': 1.0973459162125423 'C17': 1.096906929928102} Noise: 'C00': 6.153475805831076 'C01': 5.973331468670271 'C02': 5.699858878211565 'C03': 5.61691772662747 'C04': 5.635055718329871 'C05': 6.079625027880279 'C06': 5.569378782470496 'C07': 6.159928134084513 'C10': 5.875580728681517 'C11': 6.0344819056114 'C12': 5.772831136821642 'C13': 5.899530865591045 'C14': 6.069236296852714 'C15': 6.724938160329195 'C16': 6.67640273657015 'C17': 5.950307038961287 ptcFitType: FULLCOVARIANCE : /project/shared/BOT/rerun/cslage/PTC_LSSTCAM_New_12606/rerun/plazas/ DM-23159 /2021JAN02/EXPAPPROX-FULLCOVARIANCE/calibrations/ptc/ptcDataset-det094.fits measurePhotonTransferCurve.py /project/shared/BOT/rerun/cslage/PTC_LSSTCAM_New_12606 --rerun plazas/DM-23159/2021JAN02/EXPAPPROX-FULLCOVARIANCE --id detector=94 expId=3020100800155^3020100800156^3020100800158^3020100800159^3020100800185^3020100800186^3020100800161^3020100800162^3020100800188^3020100800189^3020100800164^3020100800165^3020100800191^3020100800192^3020100800167^3020100800168^3020100800194^3020100800195^3020100800170^3020100800171^3020100800197^3020100800198^3020100800173^3020100800174^3020100800200^3020100800201^3020100800176^3020100800177^3020100800203^3020100800204^3020100800179^3020100800180^3020100800206^3020100800207^3020100800182^3020100800183^3020100800209^3020100800210^3020100800212^3020100800213^3020100800215^3020100800216 -c ptcFitType=FULLCOVARIANCE doPhotodiode=False --clobber-config --clobber-version -j 1 Gain: 'C00': 1.0425556887637621 'C01': 1.0530442420728823 'C02': 1.0540860068259987 'C03': 1.0386621624006385 'C04': 1.0614891977940373 'C05': 1.0556061121455635 'C06': 1.0434440018245594 'C07': 1.035667225196643 'C10': 1.058252331030148 'C11': 1.0645580285748109 'C12': 1.0819436197242243 'C13': 1.0939079018240114 'C14': 1.0849000245608877 'C15': 1.07837551226726 'C16': 1.078800800817564 'C17': 1.0809518420287751 Noise: 'C00': 5.033950113017649 'C01': 5.165794683782765 'C02': 4.767193068143683 'C03': 4.076072653277696 'C04': 4.853841940191721 'C05': 5.3245091460021134 'C06': 4.403362155458655 'C07': 5.684199300487573 'C10': 4.859952630497146 'C11': 5.235420893116862 'C12': 5.9249790705589565 'C13': 6.7183983328087855 'C14': 5.573304230848151 'C15': 6.0253641613470155 'C16': 5.901637552758507 'C17': 5.429649457878611
            Hide
            plazas Andrés Alejandro Plazas Malagón added a comment - - edited

            Plotting routine:

            The gen2 plotting code currently in master has been moved to cp_pipepython/lsst/cp/pipe/plotPtcGen2.py to maintain backward compatibility with gen2; most of that code remains in cp_pipe/python/lsst/cp/pipe/plotPtc.py and has been wrapped around a script in cp_pipe/python/lsst/cp/pipe/script/plot_ptc_dataset.py.

            The plotting code can now be called as follows:

            plotPhotonTransferCurve.py /project/plazas/WORK/DM-23159/TEST_BOT/ptcTestResultado.06/20201231T020616Z/ptcDataset/ptcDataset_94_LSSTCam_ptcTestResultado_06_20201231T020616Z.fits --detNum=94 --outDir=/project/plazas/WORK/DM-23159/TEST_BOT/ptcTestResultado.06/PDF_PLOTS
            

            PDF (EXPAPPROXIMATION with gen3 code): PTC_det94_EXPAPPROX_DM-23159.pdf

            plotPhotonTransferCurve.py /project/plazas/WORK/DM-23159/TEST_BOT/ptcTestResultado.07/20201231T024110Z/ptcDataset/ptcDataset_94_LSSTCam_ptcTestResultado_07_20201231T024110Z.fits --detNum=94 --outDir=/project/plazas/WORK/DM-23159/TEST_BOT/ptcTestResultado.06/PDF_PLOTS_FULLCOVARIANCE
            

            PDF (FULLCOVARIANCE with gen3 code): PTC_det94_FULLCOVARIANCE_DM-23159.pdf

            Show
            plazas Andrés Alejandro Plazas Malagón added a comment - - edited Plotting routine: The gen2 plotting code currently in master has been moved to cp_pipepython/lsst/cp/pipe/plotPtcGen2.py to maintain backward compatibility with gen2; most of that code remains in cp_pipe/python/lsst/cp/pipe/plotPtc.py and has been wrapped around a script in cp_pipe/python/lsst/cp/pipe/script/plot_ptc_dataset.py . The plotting code can now be called as follows: plotPhotonTransferCurve.py /project/plazas/WORK/DM-23159/TEST_BOT/ptcTestResultado.06/20201231T020616Z/ptcDataset/ptcDataset_94_LSSTCam_ptcTestResultado_06_20201231T020616Z.fits --detNum=94 --outDir=/project/plazas/WORK/DM-23159/TEST_BOT/ptcTestResultado.06/PDF_PLOTS PDF ( EXPAPPROXIMATION with gen3 code): PTC_det94_EXPAPPROX_DM-23159.pdf plotPhotonTransferCurve.py /project/plazas/WORK/DM-23159/TEST_BOT/ptcTestResultado.07/20201231T024110Z/ptcDataset/ptcDataset_94_LSSTCam_ptcTestResultado_07_20201231T024110Z.fits --detNum=94 --outDir=/project/plazas/WORK/DM-23159/TEST_BOT/ptcTestResultado.06/PDF_PLOTS_FULLCOVARIANCE PDF ( FULLCOVARIANCE with gen3 code): PTC_det94_FULLCOVARIANCE_DM-23159.pdf
            Hide
            plazas Andrés Alejandro Plazas Malagón added a comment - - edited

            A new test was added to ci_cpp_gen3 running basically the following command:

            pipetask run -d "detector=0 and exposure IN (2020012800028, 2020012800029, 2020012800030, 2020012800031, 2020012800032, 2020012800033, 2020012800034, 2020012800035, 2020012800036, 2020012800037, 2020012800038, 2020012800039, 2020012800040, 2020012800041, 2020012800042, 2020012800043, 2020012800044, 2020012800045, 2020012800046, 2020012800047, 2020012800048, 2020012800049, 2020012800050, 2020012800051, 2020012800052, 2020012800053, 2020012800054, 2020012800055, 2020012800056, 2020012800057, 2020012800058, 2020012800059, 2020012800060, 2020012800061, 2020012800062, 2020012800063, 2020012800064, 2020012800065, 2020012800066, 2020012800067, 2020012800068, 2020012800069, 2020012800070, 2020012800071, 2020012800072, 2020012800073, 2020012800074, 2020012800075, 2020012800076, 2020012800077, 2020012800078, 2020012800079, 2020012800080, 2020012800081, 2020012800082, 2020012800083, 2020012800084, 2020012800085, 2020012800086, 2020012800087, 2020012800088, 2020012800089, 2020012800090, 2020012800091, 2020012800092, 2020012800093, 2020012800094, 2020012800095, 2020012800096, 2020012800097, 2020012800098, 2020012800099, 2020012800100, 2020012800101, 2020012800102, 2020012800103, 2020012800104, 2020012800105, 2020012800106, 2020012800107) and instrument = 'LATISS'" -b ./DATA/butler.yaml -i LATISS/raw/all,LATISS/calib,calib/v00 -p /home/plazas/lsst_devel/LSST/cp_pipe/pipelines/measurePhotonTransferCurve.yaml -c ptcSolve:ptcFitType=EXPAPPROXIMATION -o ptcTestResultado.204 --register-dataset-types
            

            Show
            plazas Andrés Alejandro Plazas Malagón added a comment - - edited A new test was added to ci_cpp_gen3 running basically the following command: pipetask run -d "detector=0 and exposure IN (2020012800028, 2020012800029, 2020012800030, 2020012800031, 2020012800032, 2020012800033, 2020012800034, 2020012800035, 2020012800036, 2020012800037, 2020012800038, 2020012800039, 2020012800040, 2020012800041, 2020012800042, 2020012800043, 2020012800044, 2020012800045, 2020012800046, 2020012800047, 2020012800048, 2020012800049, 2020012800050, 2020012800051, 2020012800052, 2020012800053, 2020012800054, 2020012800055, 2020012800056, 2020012800057, 2020012800058, 2020012800059, 2020012800060, 2020012800061, 2020012800062, 2020012800063, 2020012800064, 2020012800065, 2020012800066, 2020012800067, 2020012800068, 2020012800069, 2020012800070, 2020012800071, 2020012800072, 2020012800073, 2020012800074, 2020012800075, 2020012800076, 2020012800077, 2020012800078, 2020012800079, 2020012800080, 2020012800081, 2020012800082, 2020012800083, 2020012800084, 2020012800085, 2020012800086, 2020012800087, 2020012800088, 2020012800089, 2020012800090, 2020012800091, 2020012800092, 2020012800093, 2020012800094, 2020012800095, 2020012800096, 2020012800097, 2020012800098, 2020012800099, 2020012800100, 2020012800101, 2020012800102, 2020012800103, 2020012800104, 2020012800105, 2020012800106, 2020012800107) and instrument = 'LATISS'" -b ./DATA/butler.yaml -i LATISS/raw/all,LATISS/calib,calib/v00 -p /home/plazas/lsst_devel/LSST/cp_pipe/pipelines/measurePhotonTransferCurve.yaml -c ptcSolve:ptcFitType=EXPAPPROXIMATION -o ptcTestResultado.204 --register-dataset-types
            Hide
            plazas Andrés Alejandro Plazas Malagón added a comment -

            Scons has built successfully on ci_cpp_gen3: ci_cpp_gen3_DM-23159.txt

            Show
            plazas Andrés Alejandro Plazas Malagón added a comment - Scons has built successfully on ci_cpp_gen3 : ci_cpp_gen3_DM-23159.txt
            Show
            plazas Andrés Alejandro Plazas Malagón added a comment - Jenkins: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/33335/pipeline
            Hide
            nlust Nate Lust added a comment -

            Comments are up in cp_pipe

            Show
            nlust Nate Lust added a comment - Comments are up in cp_pipe
            Hide
            nlust Nate Lust added a comment -

            Thanks for the changes. Overall looks fine, I get why you would want to keep code similar to what you ported from for now. A few minor comments. I will ask that you file a ticket describing what work you want to do next to clean things up or reorganize and what the plan to do that will be, even if that does not occur until you get rid of gen2 support. This will help make sure nothing falls through the cracks.

            Show
            nlust Nate Lust added a comment - Thanks for the changes. Overall looks fine, I get why you would want to keep code similar to what you ported from for now. A few minor comments. I will ask that you file a ticket describing what work you want to do next to clean things up or reorganize and what the plan to do that will be, even if that does not occur until you get rid of gen2 support. This will help make sure nothing falls through the cracks.
            Hide
            nlust Nate Lust added a comment -

            I took myself off reviewer list

            Show
            nlust Nate Lust added a comment - I took myself off reviewer list
            Hide
            plazas Andrés Alejandro Plazas Malagón added a comment -

            We had to run Jenkins with only ci_cpp_gen3 due to a ci_cpp_gen2 problem with the mac build: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/33396/

            But, I was able to successfully build ci_cpp_gen2: ci_cpp_gen2_output_2021JAN20.txt

            Show
            plazas Andrés Alejandro Plazas Malagón added a comment - We had to run Jenkins with only ci_cpp_gen3 due to a ci_cpp_gen2 problem with the mac build: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/33396/ But, I was able to successfully build ci_cpp_gen2 : ci_cpp_gen2_output_2021JAN20.txt
            Hide
            plazas Andrés Alejandro Plazas Malagón added a comment -

            Future PTC work: DM-28588

            Show
            plazas Andrés Alejandro Plazas Malagón added a comment - Future PTC work: DM-28588

              People

              Assignee:
              plazas Andrés Alejandro Plazas Malagón
              Reporter:
              plazas Andrés Alejandro Plazas Malagón
              Reviewers:
              Christopher Waters
              Watchers:
              Andrés Alejandro Plazas Malagón, Christopher Waters, Nate Lust
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins Builds

                  No builds found.