[DM-35349] Fix brighter fatter kernel backwards compatibility - Jira

XML

Word

Printable

Details

Type: Improvement
Status: Done
Resolution: Done
Fix Version/s: None
Component/s: ip_isr
Labels:
None

Story Points:
1
Epic Link:
f22a-drp-isr
Team:
Data Release Production

Description

~~DM-31938~~ introduced an error that breaks backwards compatibility for previously written brighter fatter kernels. This ticket will find and fix that error.

Attachments

Activity

Ascending order - Click to sort in descending order

No builds found.

Christopher Waters created issue - 28/Jun/22 5:02 PM

Christopher Waters made changes - 28/Jun/22 5:26 PM

Field	Original Value	New Value
Status	To Do [ 10001 ]	In Progress [ 3 ]

Hide

Permalink

Christopher Waters added a comment - 28/Jun/22 5:29 PM

On main, the reported error is reproduced with the following:

>>> import lsst.ip.isr as ipIsr

>>> ff = '/repo/dc2/2.2i/calib/DM-30694/curated/19700101T000000Z/bfk/bfk_LSSTCam-imSim_R33_S02_2_2i_calib_DM-30694_curated_19700101T000000Z.fits'

>>> cc = ipIsr.BrighterFatterKernel.readFits(ff)

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

  File "/software/lsstsw/stack_20220525/stack/miniconda3-py38_4.9.2-4.0.0/Linux64/ip_isr/gec061abbff+76058f227a/python/lsst/ip/isr/calibType.py", line 470, in readFits

    return calibClass.fromTable(tableList, **kwargs)

  File "/software/lsstsw/stack_20220525/stack/miniconda3-py38_4.9.2-4.0.0/Linux64/ip_isr/gec061abbff+76058f227a/python/lsst/ip/isr/brighterFatterKernel.py", line 346, in fromTable

    expIdMaskList = ampTable['EXP_ID_MASK']

  File "/software/lsstsw/stack_20220525/conda/miniconda3-py38_4.9.2/envs/lsst-scipipe/lib/python3.10/site-packages/astropy/table/table.py", line 1882, in __getitem__

    return self.columns[item]

  File "/software/lsstsw/stack_20220525/conda/miniconda3-py38_4.9.2/envs/lsst-scipipe/lib/python3.10/site-packages/astropy/table/table.py", line 247, in __getitem__

    return OrderedDict.__getitem__(self, item)

KeyError: 'EXP_ID_MASK'

With the fix on this ticket (which ensures the table information is added to the intermediate dictionary with the expected fields):

>>> import lsst.ip.isr as ipIsr

>>> ff = '/repo/dc2/2.2i/calib/DM-30694/curated/19700101T000000Z/bfk/bfk_LSSTCam-imSim_R33_S02_2_2i_calib_DM-30694_curated_19700101T000000Z.fits'

>>> cc = ipIsr.BrighterFatterKernel.readFits(ff)

Old Version of brightter-fatter kernel found. Current version: 1.1. The new attribute 'expIdMask' will be populated with 'True' values, and the new attributes 'rawMeans'and 'rawVariances' will be populated with the masked 'means'.and 'variances' values.

Show

Christopher Waters added a comment - 28/Jun/22 5:29 PM On main, the reported error is reproduced with the following: >>> import lsst.ip.isr as ipIsr >>> ff = '/repo/dc2/2.2i/calib/DM-30694/curated/19700101T000000Z/bfk/bfk_LSSTCam-imSim_R33_S02_2_2i_calib_DM-30694_curated_19700101T000000Z.fits' >>> cc = ipIsr.BrighterFatterKernel.readFits(ff) Traceback (most recent call last): File "<stdin>" , line 1 , in <module> File "/software/lsstsw/stack_20220525/stack/miniconda3-py38_4.9.2-4.0.0/Linux64/ip_isr/gec061abbff+76058f227a/python/lsst/ip/isr/calibType.py" , line 470 , in readFits return calibClass.fromTable(tableList, **kwargs) File "/software/lsstsw/stack_20220525/stack/miniconda3-py38_4.9.2-4.0.0/Linux64/ip_isr/gec061abbff+76058f227a/python/lsst/ip/isr/brighterFatterKernel.py" , line 346 , in fromTable expIdMaskList = ampTable[ 'EXP_ID_MASK' ] File "/software/lsstsw/stack_20220525/conda/miniconda3-py38_4.9.2/envs/lsst-scipipe/lib/python3.10/site-packages/astropy/table/table.py" , line 1882 , in __getitem__ return self.columns[item] File "/software/lsstsw/stack_20220525/conda/miniconda3-py38_4.9.2/envs/lsst-scipipe/lib/python3.10/site-packages/astropy/table/table.py" , line 247 , in __getitem__ return OrderedDict.__getitem__(self, item) KeyError: 'EXP_ID_MASK' With the fix on this ticket (which ensures the table information is added to the intermediate dictionary with the expected fields): >>> import lsst.ip.isr as ipIsr >>> ff = '/repo/dc2/2.2i/calib/DM-30694/curated/19700101T000000Z/bfk/bfk_LSSTCam-imSim_R33_S02_2_2i_calib_DM-30694_curated_19700101T000000Z.fits' >>> cc = ipIsr.BrighterFatterKernel.readFits(ff) Old Version of brightter-fatter kernel found. Current version: 1.1 . The new attribute 'expIdMask' will be populated with 'True' values, and the new attributes 'rawMeans' and 'rawVariances' will be populated with the masked 'means' .and 'variances' values.

Christopher Waters made changes - 28/Jun/22 8:16 PM

Reviewers		Tim Jenness [ tjenness ]
Status	In Progress [ 3 ]	In Review [ 10004 ]

Hide

Permalink

Tim Jenness added a comment - 28/Jun/22 8:22 PM

Thanks for the quick fix.

Show

Tim Jenness added a comment - 28/Jun/22 8:22 PM Thanks for the quick fix.

Tim Jenness made changes - 28/Jun/22 8:22 PM

Status

In Review [ 10004 ]

Reviewed [ 10101 ]

Christopher Waters made changes - 29/Jun/22 5:08 AM

Resolution		Done [ 10000 ]
Status	Reviewed [ 10101 ]	Done [ 10002 ]

Christopher Waters made changes - 29/Jun/22 5:08 AM

Story Points

Yusra AlSayyad made changes - 29/Jul/22 4:39 PM

Epic Link

~~DM-30554~~ [ 511220 ]

~~DM-32156~~ [ 779836 ]

People

Assignee:: Christopher Waters

Reporter:: Christopher Waters

Reviewers:: Tim Jenness

Watchers:: Andrés Alejandro Plazas Malagón, Christopher Waters, Tim Jenness

Votes:: 0 Vote for this issue

Watchers:: 3 Start watching this issue

Dates

Created:: 28/Jun/22 5:02 PM

Updated:: 29/Jul/22 4:39 PM

Resolved:: 29/Jun/22 5:08 AM

Jenkins