Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-35316

Add 'detector' information (and any other missing info) to PTC dataset metadata

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Currently, the PTC dataset metadata is missing information, such as the detector ID. We should be able to pass a detector object to updateMetadata and have that information populated.

        Attachments

          Activity

          Ascending order - Click to sort in descending order
          Hide
          Permalink
          plazas Andrés Alejandro Plazas Malagón added a comment - 

          pipetask run -j 32 -d "detector IN (0,2,3) AND instrument='LSSTComCam' AND exposure IN (2022021800078..2022021800157) AND exposure.observation_type='flat'" -b /repo/main -i LSSTComCam/raw/all,LSSTComCam/calib,u/cslage/comcam/calib_20220218 -o u/plazas/2022JUL15-ptc_20220505.comcam.5 -p $CP_PIPE_DIR/pipelines/cpPtc.yaml -c isr:doFlat=False -c isr:doCrosstalk=False -c isr:doLinearize=False --register-dataset-types


          In [4]:  ptcDataset = butler.get("ptc", instrument="LSSTComCam", detector=2)
          		 
           
          		 
          In [5]: ptcDataset.getMetadata().toDict()
          		 
          Out[5]: 
          {'OBSTYPE': 'PTC',
          		 
           'PTC_SCHEMA': 'Gen3 Photon Transfer Curve',
          		 
           'PTC_VERSION': 1.1,
          		 
           'INSTRUME': 'LSSTComCam',
          		 
           'RAFTNAME': 'R22',
          		 
           'SLOTNAME': 'S02',
          		 
           'DETECTOR': 2,
          		 
           'DET_NAME': 'R22_S02',
          		 
           'DET_SER': 'ITL-3800C-215',
          		 
           'FILTER': None,
          		 
           'CALIB_ID': None,
          		 
           'CALIBCLS': 'lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset',
          		 
           'PTC_FIT_TYPE': 'EXPAPPROXIMATION',
          		 
           'CALIBDATE': '2022-07-15T15:16:43.523467',
          		 
           'CALIB_CREATION_DATE': '2022-07-15',
          		 
           'CALIB_CREATION_TIME': '15:16:43.523467'}
          		 
           
          		 
          In [6]:  ptcDataset = butler.get("ptc", instrument="LSSTComCam", detector=3)
          		 
           
          		 
          In [7]: ptcDataset.getMetadata().toDict()
          		 
          Out[7]: 
          {'OBSTYPE': 'PTC',
          		 
           'PTC_SCHEMA': 'Gen3 Photon Transfer Curve',
          		 
           'PTC_VERSION': 1.1,
          		 
           'INSTRUME': 'LSSTComCam',
          		 
           'RAFTNAME': 'R22',
          		 
           'SLOTNAME': 'S10',
          		 
           'DETECTOR': 3,
          		 
           'DET_NAME': 'R22_S10',
          		 
           'DET_SER': 'ITL-3800C-326',
          		 
           'FILTER': None,
          		 
           'CALIB_ID': None,
          		 
           'CALIBCLS': 'lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset',
          		 
           'PTC_FIT_TYPE': 'EXPAPPROXIMATION',
          		 
           'CALIBDATE': '2022-07-15T15:16:52.273906',
          		 
           'CALIB_CREATION_DATE': '2022-07-15',
          		 
           'CALIB_CREATION_TIME': '15:16:52.273906'}

          Show
          plazas Andrés Alejandro Plazas Malagón added a comment - pipetask run -j 32 -d "detector IN (0,2,3) AND instrument='LSSTComCam' AND exposure IN (2022021800078..2022021800157) AND exposure.observation_type='flat'" -b /repo/main -i LSSTComCam/raw/all,LSSTComCam/calib,u/cslage/comcam/calib_20220218 -o u/plazas/2022JUL15-ptc_20220505.comcam.5 -p $CP_PIPE_DIR/pipelines/cpPtc.yaml -c isr:doFlat=False -c isr:doCrosstalk=False -c isr:doLinearize=False --register-dataset-types In [ 4 ]: ptcDataset = butler.get( "ptc" , instrument = "LSSTComCam" , detector = 2 )   In [ 5 ]: ptcDataset.getMetadata().toDict() Out[ 5 ]: { 'OBSTYPE' : 'PTC' , 'PTC_SCHEMA' : 'Gen3 Photon Transfer Curve' , 'PTC_VERSION' : 1.1 , 'INSTRUME' : 'LSSTComCam' , 'RAFTNAME' : 'R22' , 'SLOTNAME' : 'S02' , 'DETECTOR' : 2 , 'DET_NAME' : 'R22_S02' , 'DET_SER' : 'ITL-3800C-215' , 'FILTER' : None , 'CALIB_ID' : None , 'CALIBCLS' : 'lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset' , 'PTC_FIT_TYPE' : 'EXPAPPROXIMATION' , 'CALIBDATE' : '2022-07-15T15:16:43.523467' , 'CALIB_CREATION_DATE' : '2022-07-15' , 'CALIB_CREATION_TIME' : '15:16:43.523467' }   In [ 6 ]: ptcDataset = butler.get( "ptc" , instrument = "LSSTComCam" , detector = 3 )   In [ 7 ]: ptcDataset.getMetadata().toDict() Out[ 7 ]: { 'OBSTYPE' : 'PTC' , 'PTC_SCHEMA' : 'Gen3 Photon Transfer Curve' , 'PTC_VERSION' : 1.1 , 'INSTRUME' : 'LSSTComCam' , 'RAFTNAME' : 'R22' , 'SLOTNAME' : 'S10' , 'DETECTOR' : 3 , 'DET_NAME' : 'R22_S10' , 'DET_SER' : 'ITL-3800C-326' , 'FILTER' : None , 'CALIB_ID' : None , 'CALIBCLS' : 'lsst.ip.isr.ptcDataset.PhotonTransferCurveDataset' , 'PTC_FIT_TYPE' : 'EXPAPPROXIMATION' , 'CALIBDATE' : '2022-07-15T15:16:52.273906' , 'CALIB_CREATION_DATE' : '2022-07-15' , 'CALIB_CREATION_TIME' : '15:16:52.273906' }
          Hide
          Permalink
          plazas Andrés Alejandro Plazas Malagón added a comment -

          https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/36940/pipeline

          Show
          plazas Andrés Alejandro Plazas Malagón added a comment - https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/36940/pipeline
          Hide
          Permalink
          czw Christopher Waters added a comment -

          Some concern about inputExpList not being used, with comments on the PR.

          Show
          czw Christopher Waters added a comment - Some concern about inputExpList not being used, with comments on the PR.
          Hide
          Permalink
          plazas Andrés Alejandro Plazas Malagón added a comment -

          https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/36957/pipeline

          Show
          plazas Andrés Alejandro Plazas Malagón added a comment - https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/36957/pipeline

            People

            Assignee:
            plazas Andrés Alejandro Plazas Malagón
            Reporter:
            plazas Andrés Alejandro Plazas Malagón
            Reviewers:
            Christopher Waters
            Watchers:
            Andrés Alejandro Plazas Malagón, Christopher Waters
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Jenkins

                No builds found.