Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: None
-
Story Points:8
-
Epic Link:
-
Team:Data Release Production
-
Urgent?:No
Description
DP0.2 is the main driver of this ticket.
step 1 was run with v23.0.0 which was before DM-34019, which means that we need to
a) run a task that produces src and/or source afwTables with corrected local_photoCalibs and local_Wcs's.
b) Add this task to step6 along with transformSourceTable and consolidateSourceTable.
Something like this was needed for HSC PDR3. Because it was an incremental data release (we wanted to reuse the src's that were already run in the previous data release AND the updated FGCM which took sourceTable_visits) we needed a way to add base_localPhotoCalib's and base_localWcs's after the fact. This functionality didn't get ported over to Gen3 because it wasn't needed YET. What's unique about this is that we need to be able to REPLACE the localPhotoCalib and localWcs columns, in addition to being able to add them.
Attachments
Issue Links
- mentioned in
-
Page Loading...
Activity
This is also ready for review:
There's a v23.0.x jenkins run going here: https://ci.lsst.codes/job/stack-os-matrix/36507/display/redirect (note ci_hsc_gen2 has to pass for v23!)
There's a main jenkins run going here: https://ci.lsst.codes/job/stack-os-matrix/36508/display/redirect
Most of the application of FGCM/jointcal is boilerplate. See e.g.
makeWarpTask : https://github.com/lsst/pipe_tasks/blob/7adadf9be28dc46bd79bbe9064fda191da55a9fb/python/lsst/pipe/tasks/makeCoaddTempExp.py#L718
- forcedPhotCcd: https://github.com/lsst/meas_base/blob/main/python/lsst/meas/base/forcedPhotCcd.py#L246
Yes, it is super messy, but it's more important that it's the same application code everywhere to make refactoring easier later.
It is OK for the application the obs_lsst/ci_imsim PRs and the code (the pipe_tasks PRs) to be reviewed by different people.
Requesting reviews from Ken on the pipe_tasks PR and colin on the obs_lsst/ci_imsim application.
Application side looks good, taking my name off the list. Ken can hit the reviewed button when he's finished.
Colin Slater
added a comment - Application side looks good, taking my name off the list. Ken can hit the reviewed button when he's finished. pipe_tasks review complete; see comments.
Kenneth Herner
added a comment - pipe_tasks review complete; see comments. Kenneth Herner https://github.com/lsst/pipe_tasks/pull/676 is still showing "changes requested." Would you take a look and see if the changes satisfy your requests.
Also for posterity, relevant convo on slack decision to make exposure an input:
They do; I changed the PR state to approved.
Kenneth Herner
added a comment - They do; I changed the PR state to approved.
This ticket has:
1) the gen3 backport of the code path that extracts local calib columns from calibs attached to the exposure.
2) the application of this code to recalibrate source tables for DP0.2
What this ticked DOES NOT HAVE: application of this code to recalibrate source tables after jointcal and FGCM. That is going on
DM-33959. I tested it with both and even ran jointcal on DC2 so I'd have data where the tracts overlap.DM-33959will involve naming decisions and new defaults in jointcal/fgcm so I'm going to get that reviewed separately.Some decisions and why I made them:
1) Why did I move the addCalibColumns functionality to a subclass called WriteRecalibratedSourceTableTask? Because it now needs skymap in the dimensions. I want to keep a basic WriteSourceTableTask that doesn't require a skymap.
2) Why does it need a skymap? Jointcal will write out full visit solutions per tract. For detectors that overlap 2 tracts, this task now needs to choose its favorite: based on which tract center is closest.
3) delegates will be interacting with a table called calibratedSourceTable_visit if you want this called something different, yell now!!