# Create task to add local PhotoCalib and WCS value to source table rows

XMLWordPrintable

## Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Story Points:
8
• Team:

## Description

As part of the ScienceDataModel we need to have calibration products available for use in the Functors in the SDM code. One solution to making these products available is to compute their local values and store them per row in the afw tables output by various SciencePipelines tasks. This ticket will implement a task to perform this operation.

## Activity

Hide
Chris Morrison added a comment -

One Jenkins job crashed on the error below, which looks to be a file not found. All other jobs passed. Gonna merge as this looks like a Jenkins error not a code error.

https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/30514/pipeline

 // meas_base: tickets.DM-21171-g6504b5248a ......................................................................................::::: [2019-09-23T22:50:06.787874Z] ========================= 240 passed in 119.91 seconds ========================= ::::: [2019-09-23T22:50:06.794642Z] Coverage.py warning: No data was collected. (no-data-collected) ::::: [2019-09-23T22:50:06.914227Z] Global pytest run completed successfully ::::: [2019-09-23T22:50:07.047914Z] scons: done building targets. ::::: [2019-09-23T22:50:07.161181Z] + '[' -d tests/.tests ']' ::::: [2019-09-23T22:50:07.162090Z] ++ wc -l ::::: [2019-09-23T22:50:07.162265Z] ++ ls 'tests/.tests/*.failed' ::::: [2019-09-23T22:50:07.210820Z] + '[' 0 -ne 0 ']' ::::: [2019-09-23T22:50:07.210870Z] + eupspkg PRODUCT=meas_base VERSION=tickets.DM-21171-g6504b5248a FLAVOR=generic install ::::: [2019-09-23T22:50:07.211230Z] /j/ws/stack-os-matrix/centos-7.devtoolset-6.py3/lsstsw/build/meas_base/_build.sh: line 91: /j/ws/stack-os-matrix/centos-7.devtoolset-6.py3/lsstsw/eups/2.1.5/bin/eupspkg: No such file or directory ERROR (229 sec). 

Show
Chris Morrison added a comment - One Jenkins job crashed on the error below, which looks to be a file not found. All other jobs passed. Gonna merge as this looks like a Jenkins error not a code error.  https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/30514/pipeline // meas_base: tickets.DM-21171-g6504b5248a ......................................................................................::::: [2019-09-23T22:50:06.787874Z] ========================= 240 passed in 119.91 seconds ========================= ::::: [ 2019 - 09 -23T22: 50 : 06 .794642Z] Coverage.py warning: No data was collected. (no-data-collected) ::::: [ 2019 - 09 -23T22: 50 : 06 .914227Z] Global pytest run completed successfully ::::: [ 2019 - 09 -23T22: 50 : 07 .047914Z] scons: done building targets. ::::: [ 2019 - 09 -23T22: 50 : 07 .161181Z] + '[' -d tests/.tests ']' ::::: [ 2019 - 09 -23T22: 50 : 07 .162090Z] ++ wc -l ::::: [ 2019 - 09 -23T22: 50 : 07 .162265Z] ++ ls 'tests/.tests/*.failed' ::::: [ 2019 - 09 -23T22: 50 : 07 .210820Z] + '[' 0 -ne 0 ']' ::::: [ 2019 - 09 -23T22: 50 : 07 .210870Z] + eupspkg PRODUCT=meas_base VERSION=tickets.DM- 21171 -g6504b5248a FLAVOR=generic install ::::: [ 2019 - 09 -23T22: 50 : 07 .211230Z] /j/ws/stack-os-matrix/centos- 7 .devtoolset- 6 .py3/lsstsw/build/meas_base/_build.sh: line 91 : /j/ws/stack-os-matrix/centos- 7 .devtoolset- 6 .py3/lsstsw/eups/ 2.1 . 5 /bin/eupspkg: No such file or directory ERROR ( 229 sec).
Hide

We did chat and Jim convinced me that this particular task shouldn't load any other datasets. So galactic extinction will get applied later.

This task is just taking advantage of the fact that the WCS, PhotoCalib, PSF, VisitInfo etc... is already in memory and extracting what we need from it so that we don't have to fetch it all from disk again to do the transform. I CAN envision these as plugins, but I think this your task will do the trick as-is.

Looks good. Remember that the config doEvalLocCalibration IS the part that people will write a lot and usually from memory. I have to run, and will think about, what abbreviation I would type to doEvaluateLocalCalib

Show
Yusra AlSayyad added a comment - We did chat and Jim convinced me that this particular task shouldn't load any other datasets. So galactic extinction will get applied later. This task is just taking advantage of the fact that the WCS, PhotoCalib, PSF, VisitInfo etc... is already in memory and extracting what we need from it so that we don't have to fetch it all from disk again to do the transform. I CAN envision these as plugins, but I think this your task will do the trick as-is. Looks good. Remember that the config doEvalLocCalibration IS the part that people will write a lot and usually from memory. I have to run, and will think about, what abbreviation I would type to doEvaluateLocalCalib
Hide
Chris Morrison added a comment -

EvaluateLocalCalibration sounds good to me. I'll fix the comments and merge. Jim and yourself where talking about using this to store local extinctions as well? In that case would you want this task to evolve into something more like a measurement tasks with plugins or is it fine as is for the future and we just add new methods?

Show
Chris Morrison added a comment - EvaluateLocalCalibration sounds good to me. I'll fix the comments and merge. Jim and yourself where talking about using this to store local extinctions as well? In that case would you want this task to evolve into something more like a measurement tasks with plugins or is it fine as is for the future and we just add new methods?
Hide

Pleasure to read. Let's change the name, before merging. I already like EvaluteLocalCalibration better than "StoreExposureCalibration." When the discussion on #dm-namingthings settles down, pick what you like from there.

Show
Yusra AlSayyad added a comment - Pleasure to read. Let's change the name, before merging. I already like EvaluteLocalCalibration better than "StoreExposureCalibration." When the discussion on #dm-namingthings settles down, pick what you like from there.
Hide
Chris Morrison added a comment -

After review is complete I will make a ticket to vectorize/C++ the code if needed.

Show
Chris Morrison added a comment - Jenkin: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/30492/pipeline After review is complete I will make a ticket to vectorize/C++ the code if needed.

## People

• Assignee:
Chris Morrison
Reporter:
Chris Morrison
Reviewers: