Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-31938

Have PTC dataset unpad results

    XMLWordPrintable

Details

    • Improvement
    • Status: Done
    • Resolution: Done
    • None
    • ip_isr
    • None

    Description

      The PTC code currently pads finalVars, finalMeans, and covariances (and other?) with NaN values to ensure that all amplifiers have the same length for FITS compatibility.  It might be useful to have the read methods unpad these values, so the returned data is whatever length was populated for that amplifier.

      Attachments

        Activity

          pipetask run -j 32 -d "detector IN (55) AND instrument='LSSTCam' AND exposure IN (3021111500030..3021111500150) " -b /repo/main -i LSSTCam/raw/all,LSSTCam/calib,LSSTCam/calib/u/cslage/20211117A -p ${CP_PIPE_DIR}/pipelines/cpPtc.yaml -c isr:doFlat=False -c isr:doCrosstalk=False -c ptcSolve:ptcFitType=FULLCOVARIANCE -o u/plazas/DM-31938.test.10 --register-dataset-types
          

              import lsst.daf.butler as dB
              import numpy as np
             genCollection = "u/plazas/DM-31938.test.10"
             butler = dB.Butler("/repo/main", collections=genCollection)
             ptcDataset = butler.get("ptc", instrument="LSSTCam", detector=55, exposure=3021111500150)
          

          In [5]: ptcDataset.finalMeans['C15']
          Out[5]: 
          [2131.914311622597,
           2599.4489882208527,
           3176.2206420720277,
           3874.6031636729003,
           4730.795394857724,
           5778.363401045608,
           54817.668863291736,
           56470.07757644713,
           63628.38805759857,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan,
           nan]
          

          After:

          In [2]: ptcDataset.finalMeans['C15']
          Out[2]: 
          [2131.914311622597,
           2599.4489882208527,
           3176.2206420720277,
           3874.6031636729003,
           4730.795394857724,
           5778.363401045608,
           54817.668863291736,
           56470.07757644713,
           63628.38805759857]
          

          plazas Andrés Alejandro Plazas Malagón added a comment - - edited pipetask run -j 32 -d "detector IN (55) AND instrument='LSSTCam' AND exposure IN (3021111500030..3021111500150) " -b /repo/main -i LSSTCam/raw/all,LSSTCam/calib,LSSTCam/calib/u/cslage/20211117A -p ${CP_PIPE_DIR}/pipelines/cpPtc.yaml -c isr:doFlat=False -c isr:doCrosstalk=False -c ptcSolve:ptcFitType=FULLCOVARIANCE -o u/plazas/DM-31938.test.10 --register-dataset-types import lsst.daf.butler as dB import numpy as np genCollection = "u/plazas/DM-31938.test.10" butler = dB.Butler( "/repo/main" , collections = genCollection) ptcDataset = butler.get( "ptc" , instrument = "LSSTCam" , detector = 55 , exposure = 3021111500150 ) In [ 5 ]: ptcDataset.finalMeans[ 'C15' ] Out[ 5 ]: [ 2131.914311622597 , 2599.4489882208527 , 3176.2206420720277 , 3874.6031636729003 , 4730.795394857724 , 5778.363401045608 , 54817.668863291736 , 56470.07757644713 , 63628.38805759857 , nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan] After: In [ 2 ]: ptcDataset.finalMeans[ 'C15' ] Out[ 2 ]: [ 2131.914311622597 , 2599.4489882208527 , 3176.2206420720277 , 3874.6031636729003 , 4730.795394857724 , 5778.363401045608 , 54817.668863291736 , 56470.07757644713 , 63628.38805759857 ]

          Looks good to me.

          czw Christopher Waters added a comment - Looks good to me.
          plazas Andrés Alejandro Plazas Malagón added a comment - Jenkins: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/36811/pipeline/
          plazas Andrés Alejandro Plazas Malagón added a comment - Jenkins: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/36842/pipeline/

          People

            plazas Andrés Alejandro Plazas Malagón
            czw Christopher Waters
            Christopher Waters
            Andrés Alejandro Plazas Malagón, Christopher Waters
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Jenkins

                No builds found.