Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-35349

Fix brighter fatter kernel backwards compatibility

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: ip_isr
    • Labels:
      None
    • Story Points:
      1
    • Epic Link:
    • Team:
      Data Release Production

      Description

      DM-31938 introduced an error that breaks backwards compatibility for previously written brighter fatter kernels.  This ticket will find and fix that error.

        Attachments

          Activity

          Ascending order - Click to sort in descending order
          Hide
          Permalink
          czw Christopher Waters added a comment -

          On main, the reported error is reproduced with the following:

          >>> import lsst.ip.isr as ipIsr
          		 
          >>> ff = '/repo/dc2/2.2i/calib/DM-30694/curated/19700101T000000Z/bfk/bfk_LSSTCam-imSim_R33_S02_2_2i_calib_DM-30694_curated_19700101T000000Z.fits'
          		 
          >>> cc = ipIsr.BrighterFatterKernel.readFits(ff)
          		 
          Traceback (most recent call last):
          		 
            File "<stdin>", line 1, in <module>
          		 
            File "/software/lsstsw/stack_20220525/stack/miniconda3-py38_4.9.2-4.0.0/Linux64/ip_isr/gec061abbff+76058f227a/python/lsst/ip/isr/calibType.py", line 470, in readFits
          		 
              return calibClass.fromTable(tableList, **kwargs)
          		 
            File "/software/lsstsw/stack_20220525/stack/miniconda3-py38_4.9.2-4.0.0/Linux64/ip_isr/gec061abbff+76058f227a/python/lsst/ip/isr/brighterFatterKernel.py", line 346, in fromTable
          		 
              expIdMaskList = ampTable['EXP_ID_MASK']
          		 
            File "/software/lsstsw/stack_20220525/conda/miniconda3-py38_4.9.2/envs/lsst-scipipe/lib/python3.10/site-packages/astropy/table/table.py", line 1882, in __getitem__
          		 
              return self.columns[item]
          		 
            File "/software/lsstsw/stack_20220525/conda/miniconda3-py38_4.9.2/envs/lsst-scipipe/lib/python3.10/site-packages/astropy/table/table.py", line 247, in __getitem__
          		 
              return OrderedDict.__getitem__(self, item)
          		 
          KeyError: 'EXP_ID_MASK'

          With the fix on this ticket (which ensures the table information is added to the intermediate dictionary with the expected fields):

          >>> import lsst.ip.isr as ipIsr
          		 
          >>> ff = '/repo/dc2/2.2i/calib/DM-30694/curated/19700101T000000Z/bfk/bfk_LSSTCam-imSim_R33_S02_2_2i_calib_DM-30694_curated_19700101T000000Z.fits'
          		 
          >>> cc = ipIsr.BrighterFatterKernel.readFits(ff)
          		 
          Old Version of brightter-fatter kernel found. Current version: 1.1. The new attribute 'expIdMask' will be populated with 'True' values, and the new attributes 'rawMeans'and 'rawVariances' will be populated with the masked 'means'.and 'variances' values.

          Show
          czw Christopher Waters added a comment - On main, the reported error is reproduced with the following: >>> import lsst.ip.isr as ipIsr >>> ff = '/repo/dc2/2.2i/calib/DM-30694/curated/19700101T000000Z/bfk/bfk_LSSTCam-imSim_R33_S02_2_2i_calib_DM-30694_curated_19700101T000000Z.fits' >>> cc = ipIsr.BrighterFatterKernel.readFits(ff) Traceback (most recent call last): File "<stdin>" , line 1 , in <module> File "/software/lsstsw/stack_20220525/stack/miniconda3-py38_4.9.2-4.0.0/Linux64/ip_isr/gec061abbff+76058f227a/python/lsst/ip/isr/calibType.py" , line 470 , in readFits return calibClass.fromTable(tableList, **kwargs) File "/software/lsstsw/stack_20220525/stack/miniconda3-py38_4.9.2-4.0.0/Linux64/ip_isr/gec061abbff+76058f227a/python/lsst/ip/isr/brighterFatterKernel.py" , line 346 , in fromTable expIdMaskList = ampTable[ 'EXP_ID_MASK' ] File "/software/lsstsw/stack_20220525/conda/miniconda3-py38_4.9.2/envs/lsst-scipipe/lib/python3.10/site-packages/astropy/table/table.py" , line 1882 , in __getitem__ return self.columns[item] File "/software/lsstsw/stack_20220525/conda/miniconda3-py38_4.9.2/envs/lsst-scipipe/lib/python3.10/site-packages/astropy/table/table.py" , line 247 , in __getitem__ return OrderedDict.__getitem__(self, item) KeyError: 'EXP_ID_MASK' With the fix on this ticket (which ensures the table information is added to the intermediate dictionary with the expected fields): >>> import lsst.ip.isr as ipIsr >>> ff = '/repo/dc2/2.2i/calib/DM-30694/curated/19700101T000000Z/bfk/bfk_LSSTCam-imSim_R33_S02_2_2i_calib_DM-30694_curated_19700101T000000Z.fits' >>> cc = ipIsr.BrighterFatterKernel.readFits(ff) Old Version of brightter-fatter kernel found. Current version: 1.1 . The new attribute 'expIdMask' will be populated with 'True' values, and the new attributes 'rawMeans' and 'rawVariances' will be populated with the masked 'means' .and 'variances' values.
          Hide
          Permalink
          tjenness Tim Jenness added a comment -

          Thanks for the quick fix.

          Show
          tjenness Tim Jenness added a comment - Thanks for the quick fix.

            People

            Assignee:
            czw Christopher Waters
            Reporter:
            czw Christopher Waters
            Reviewers:
            Tim Jenness
            Watchers:
            Andrés Alejandro Plazas Malagón, Christopher Waters, Tim Jenness
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Jenkins

                No builds found.