Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-23538

Bug of Data Butler to Get the Raw Data

    Details

    • Urgent?:
      No

      Description

      It looks like there is the bug for butler to get the raw exposure data in lsst_distrib: w_2020_06. This bug did not happen in w_2020_04. The following is the test data: input.zip

      I use the followings to try to get the raw data:

       
      from lsst.daf.persistence import Butler
       butler = Butler(inputs="input/")
       dataId = dict(expId=20, raftName="R00", detectorName="S22")
       raw = butler.get("{color:#ff2600}raw{color}", dataId=dataId)
      

      The following is the error message (You can also see details in errorMsg.txt):

       {color:#0747a6}NoResults: No locations for get: datasetType:raw_amp dataId:DataId(initialdata=\{'expId': 20, 'raftName': 'R00', 'detectorName': 'S22', 'run': '20', 'detector': 197, 'channel': 1}, tag=set()){color}
      

      It looks like the dataset type becomes "raw_amp" even though I use "raw" in the get().
      The followins is the data ID of "raw" if I use butler.getKeys("raw"):

      {'run': str,
        'raftName': str,
        'expId': int,
        'detectorName': str,
        'detector': int}
      

      There should be no 'channel' because it is not "raw_amp".

        Attachments

          Activity

          ttsai Te-Wei Tsai created issue -
          krughoff Simon Krughoff made changes -
          Field Original Value New Value
          Assignee Simon Krughoff [ krughoff ]
          tjenness Tim Jenness made changes -
          Component/s obs_lsst [ 16504 ]
          Component/s Science Pipelines [ 10706 ]
          tjenness Tim Jenness made changes -
          Labels SciencePipelines
          tjenness Tim Jenness made changes -
          Watchers Simon Krughoff, Te-Wei Tsai [ Simon Krughoff, Te-Wei Tsai ] Kian-Tat Lim, Simon Krughoff, Te-Wei Tsai [ Kian-Tat Lim, Simon Krughoff, Te-Wei Tsai ]
          tjenness Tim Jenness made changes -
          Resolution Done [ 10000 ]
          Status To Do [ 10001 ] Invalid [ 11005 ]
          tjenness Tim Jenness made changes -
          Description It looks like there is the bug for butler to get the raw exposure data in lsst_distrib: w_2020_06. This bug did not happen in w_2020_04. The following is the test data: [^input.zip]

          I use the followings to try to get the raw data:
           -----------------------------------------------------------------
           from lsst.daf.persistence import Butler
           butler = Butler(inputs="input/")
           dataId = dict(expId=20, raftName="R00", detectorName="S22")
           raw = butler.get("{color:#ff2600}raw{color}", dataId=dataId)
           -----------------------------------------------------------------

          The following is the error message (You can also see details in [^errorMsg.txt]):

          ------------------------------
           {color:#0747a6}NoResults: No locations for get: datasetType:raw_amp dataId:DataId(initialdata=\{'expId': 20, 'raftName': 'R00', 'detectorName': 'S22', 'run': '20', 'detector': 197, 'channel': 1}, tag=set()){color}
           ------------------------------

          It looks like the dataset type becomes "raw_amp" even though I use "raw" in the get().
           The followins is the data ID of "raw" if I use butler.getKeys("raw"):

          \{'run': str,
            'raftName': str,
            'expId': int,
            'detectorName': str,
            'detector': int}

          There should be no 'channel' because it is not "raw_amp".
          It looks like there is the bug for butler to get the raw exposure data in lsst_distrib: w_2020_06. This bug did not happen in w_2020_04. The following is the test data: [^input.zip]

          I use the followings to try to get the raw data:

          {code}
          from lsst.daf.persistence import Butler
           butler = Butler(inputs="input/")
           dataId = dict(expId=20, raftName="R00", detectorName="S22")
           raw = butler.get("{color:#ff2600}raw{color}", dataId=dataId)
          {code}


          The following is the error message (You can also see details in [^errorMsg.txt]):

          {code}
           {color:#0747a6}NoResults: No locations for get: datasetType:raw_amp dataId:DataId(initialdata=\{'expId': 20, 'raftName': 'R00', 'detectorName': 'S22', 'run': '20', 'detector': 197, 'channel': 1}, tag=set()){color}
          {code}

          It looks like the dataset type becomes "raw_amp" even though I use "raw" in the get().
           The followins is the data ID of "raw" if I use butler.getKeys("raw"):

          {code}
          {'run': str,
            'raftName': str,
            'expId': int,
            'detectorName': str,
            'detector': int}
          {code}
          There should be no 'channel' because it is not "raw_amp".

            People

            • Assignee:
              Unassigned
              Reporter:
              ttsai Te-Wei Tsai
              Watchers:
              Kian-Tat Lim, Simon Krughoff, Te-Wei Tsai, Tim Jenness
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Summary Panel