Having spent a while getting to grips with this, I want to summarize where things stand and get a better sense of the intended scope of this issue.
The coaddition code in hscPipe is built around stack.py. This provides StackTask as an an "all-in-one" solution for:
- Selecting images to process (BaseSelectImagesTask, WcsSelectImagesTask);
- Warping them to create coaddTempExp s (MakeCoaddTempExpTask);
- Build a background reference for background-matched coaddition (BackgroundReferenceTask);
- Assembling coadds, including optional background matching (SimpleAssembleCoaddTask);
- Performing detection and measurement on coadds (ProcessCoaddTask).
The following significant differences exist in these tasks as opposed to those on LSST:
- There is no equivalent of BackgroundReferenceTask on LSST;
- SimpleAssembleCoaddTask is a significant restructuring of LSST's AssembleCoaddTask, primarily to support a new way of performing background matching;
- ProcessCoaddTask has been completely rewritten for hscPipe to use DetectCoaddSourcesTask. It no longer depends on ProcessImageTask;
- There is no equivalent of the stack.py "all-in-one" task on LSST.
In addition, there are many more minor structural differences, bug fixes and new features which have been introduced to code throughout the hierarchy touched by the above.
To date, I have:
- Agreed with Paul Price and Jim Bosch that it's not currently appropriate to port the HSC background matching code to LSST. Rather, we will preserve this for posterity on the HSC side, and use it a basis for future thinking about how to approach this on LSST.
- Performed a comparison of code touched by the coaddition process to the end of (Simple)AssembleCoaddTask on HSC and LSST, accounted for all differences between them, and either cherry-picked those differences to an appropriate branch for this ticket (where minor), filed issues to bring over other, more significant, changes (eg
DM-3243, DM-3258), or ensured that these changes are already covered by scheduled work elsewhere.
- Performed some restructuring of LSST's AssembleCoaddTask so that it better corresponds to the SimpleAssembleCoaddTask on HSC, but without changing the background matching code (see above) or the "safe coadd clipping" (
None of these changes involve significant changes to interface or functionality which seem worth of RFCing.
My naive reading of this issue and its reference to "coaddition code" was that it referred to code which performed coaddition, and not to the subsequent processing of those coadds. However, given the mention of an RFC, and given that the changes here so far seem relatively modest, I'm wondering if the intention was also to bring across the redesigned ProcessCoaddTask.
It looks as though bringing across the complete StackTask would be blocked by
DM-2983 (it derives from BatchPoolTask for example). I've not investigated how easy it would be to create a purely serial version, but it doesn't seem worthwhile putting the effort into now just to re-port it to the HSC parallelization framework once that makes the transition.
Jim Bosch, since you wrote the original description of this story, could you please clarify exactly what the success criteria here are, either with reference to my summary above or (if necessary!) by telling me what I've missed? Thanks!