Fix Version/s: None
Component/s: Design Documents
Write more detailed descriptions and possibly draw a rough diagram for the single-frame processing and simultaneous calibration components of Data Release Production.
Does not necessarily involve turning this section into prose.
DM-6260 Cleanup and standardize DRP imchar/jointcal diagrams
- Won't Fix
DM-6265 Audit DRP LDM-151 for correct handling of chromaticity
- Won't Fix
DM-6271 Audit DRP LDM-151 for correct handling of crowded fields
- Won't Fix
- is blocked by
DM-6248 DRP Top-Level Diagram and Descriptions, Draft 1
At several places, you refer to things which are "TBD". In some cases, I'm not sure whether they will be determined by you (or Robert, etc) based on some hard thinking as part of putting this document together, or whether they will be the result of research or experimentation during construction. Where it's the latter, we should call them out.
I believe all of these are research projects (or at least "thinking projects" that we'll need to plan for, not complete prior to planning). I'll update the text to clarify.
Related to the above, where there are decisions to be made during construction, it would be helpful to indicate how that decision should be made: is it driven by external factors (e.g. availability of Gaia catalogues), by research time that needs to be scheduled, etc?
I'll try to add annotations to this effect.
How robust is this design against algorithmic components that don't come up to scratch? Would failure to deliver some component operating at some level of performance cause major changes to plan?
I'm afraid it's not robust at all, but I think it generally gets more robust the deeper we get into the production. Essentially, we have a lot of iterative procedures that re hard to bootstrap, and that tends to put a much bigger burden on whichever of the components we choose to start with. If one of those components doesn't work out, the best response may be to restructure the pipelines to put some other component first.
Related to the above, we should attempt to capture performance & fidelity requirements for algorithms wherever we can.
I agree this is important (and I'm sorry I keep forgetting about it), but I'm hoping we can mostly defer figuring out success criteria until the designs themselves are better described (just because this would be enough extra work for me that I don't think I can get it all done by Zeljko Ivezic's deadline).
By the way, in the spirit of Zeljko's wide-then-deep plan, I don't think going back to add annotations as above should necessarily be your top priority: I think from my point of view of planning it's useful to sketch out the later stages of the system then return to refine this later.
(Similarly, I'll add some more minor comments/request for clarification to the PR, but I don't think they need to be addressed urgently.)
Sounds good. I've just clarified all the places I actually wrote "TBD" in the text, but I'll save the other issues for later (including looking for places where I implied TBD without actually writing it).
I've merged the ticket branch to the draft branch, since I'm not planning any more work here soon. I'm leaving the ticket open for now so I don't forget to return to John Swinbank's comments at some point, but I'd be happy to take another approach to capturing those if he has a recommendation.
Jim Bosch – thanks for this document. I have a few questions about the details of the procedures you're proposing (and will probably want to walk through them with you when we're doing detailed planning in a week or two) but I think this presentation is exceptionally clear and useful.
For this section of the document, I think the level of detail you're capturing is quite appropriate for planning, especially bearing in mind that a lot (not all, of course) of the hard work (where we'll really need to invest time) will be in figuring out the details of the algorithmic components that make these high-level pipelines possible.
A few points which are worth bearing in mind: