Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: None
-
Labels:None
-
Story Points:2
-
Epic Link:
-
Sprint:AP S22-5 (April)
-
Team:Alert Production
-
Urgent?:No
Description
When Karlo Mrakovcic was trying to build HSC goodSeeing coadds for use as AP templates, he ran into a problem with no skyCorr dataset. This ticket is to update the HSC-specific ApTemplate pipeline to fix this, and test it to make sure nothin awful happened. (While I'm at it, I should probably fix the comment in the subset definition area that errantly refers to DECam... oy).
Options include: (1) adding the skyCorr step or (2) turning skyCorr off in makeWarp. If (1) is chosen, it's important to keep in mind that skyCorr does not play well with patch and tract dataIds, per comments in the DRP-RC2 pipeline. So unless something changes in that regard and/or we desperately need skyCorr, it would be simplest to just explicitly turn it off in ApTemplate.
Relevant line to potentially add to pipeline:
skyCorr: # This comes from the obs_subaru DRP.yaml. Alternative is `doApplySkyCorr: False` in makeWarp.
|
class: lsst.pipe.drivers.skyCorrection.SkyCorrectionTask |
Attachments
Activity
A wonderful question that I do not know the answer to!
Krzysztof Findeisen, currently we only have sky frames constructed for HSC data. The sky dataset types which live in HSC/calib/DM-32378 are one of the two inputs required (alongside raw's) to produce the skyCorr dataset type using the SkyCorrectionTask. As such, full-focal-plane sky correction is only supported for HSC data reductions at present.
With that said, we may want to generalize this task in the future to work with both LSST data, and potentially other survey data such as DECam. From discussions I've had with other survey team members, it seems like this form of sky correction is used elsewhere such as in the Community Pipeline for DECam, and so might be worth considering for the science pipelines as well.
Thanks very much for the additional context, Lee!
For now, I think it makes sense to just add the skyCorrectionTask step to HSC's ApTemplate.yaml. This is similar to how only the DRP-RC2.yaml has this step, vanilla DRP.yaml does not.
I updated the pipeline and verified that this successfully begins to process data on lsst-devl:
pipetask run -j 4 -b /repo/main -i HSC/RC2/defaults -o u/mrawls/DM-33692-take1 -p $AP_PIPE_DIR/pipelines/HyperSuprimeCam/ApTemplate.yaml -d "skymap='hsc_rings_v1' AND exposure IN (30482, 30484, 30486)" |
Karlo Mrakovcic, would you be willing to review this ticket, since you originally noticed the problem? In this situation, I think a review consists of (1) looking at the pipeline/code changes on the ticket branch and agreeing it makes sense, and (2) trying steps similar to what you tried before to make sure the problem is resolved. (More info on our code review process: https://developer.lsst.io/work/flow.html?#workflow-code-review). I am not running Jenkins for this small change, because Jenkins does not run or test pipelines in the scope of its CI.
I used this command
pipetask run -j 4 -b /repo/main -d "instrument='HSC' AND skymap='hsc_rings_v1' AND tract=9813 AND band='g' AND patch=50" -i HSC/RC2/defaults -o u/kmrakovcic/DM-33692-take2 -p $AP_PIPE_DIR/pipelines/HyperSuprimeCam/ApTemplate.yaml |
The process successfully ended. Thank you for your patience.
Karlo Mrakovcic
added a comment - - edited I used this command
pipetask run -j 4 -b /repo/main -d "instrument='HSC' AND skymap='hsc_rings_v1' AND tract=9813 AND band='g' AND patch=50" -i HSC/RC2/defaults -o u/kmrakovcic/DM- 33692 -take2 -p $AP_PIPE_DIR/pipelines/HyperSuprimeCam/ApTemplate.yaml
The process successfully ended. Thank you for your patience. Thanks for the review! You errantly assigned yourself to the ticket (you're the reviewer, not the assignee) and put it all the way to Done instead of just Reviewed - no problem. I'm supposed to move it to Done once I actually merge the ticket, which I will do now. Glad your test run ran through to completion - awesome!
Just to be clear, is skyCorr something that's only needed for HSC, or should it be part of template generation in general (i.e., everything except DECam)?