Decreasing story point to reflect change in scope.

John Swinbank, could you take a quick look at this to see how the level of detail feels vis a vis planning needs (I'll make the GitHub PR shortly)?

I strongly suspect the BootstrapImChar section is more detail than you need; I was worried about unknown unknowns in that area and wanted to go into extra detail to make sure we wouldn't have any big surprises there. I'm hoping the later sections are more at the level we'll need for planning.

Colin Slater: some notes for you on diagrams:

• I haven't done any draft diagrams for this section; I found it easier to express the tricky stuff as a pseudocode block. Diagrams might be easier for users to interpret, if you want to take a look at what I've written and cook one up (though I think we may not want to capture the level of detail in the pseudocode in a diagram).
• I've determined that the content of the RefineJointCal and BootstrapJointCal pipelines is identical, so I've merged them in the text with the new name StandardJointCal. Diagrams should be updated accordingly (I've not yet updated the draft overview diagram I had already put in the document, as I know you're already working on its replacement).

Thanks for the ping; on my list for this afternoon.

First comment – the travis build is failing, so this work is not showing up at https://ldm-151.lsst.io/v/DM-6255/LDM-151.pdf.

! LaTeX Error: File `xcolor.sty' not found.

Type X to quit or <RETURN> to proceed,

or enter new name. (Default extension: sty)

Enter file name: 

No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.

The build has been terminated

Bleh. I thought that was a pretty standard latex package. I'll find another package that's more common.

Great. I had a fix to avoid xcolor by defining a color manually, but I like yours better.

Robert Lupton this is probably ready for you to start taking a look at; see the GitHub PR linked on the side.

Jim Bosch – thanks for this document. I have a few questions about the details of the procedures you're proposing (and will probably want to walk through them with you when we're doing detailed planning in a week or two) but I think this presentation is exceptionally clear and useful.

For this section of the document, I think the level of detail you're capturing is quite appropriate for planning, especially bearing in mind that a lot (not all, of course) of the hard work (where we'll really need to invest time) will be in figuring out the details of the algorithmic components that make these high-level pipelines possible.

A few points which are worth bearing in mind:

• At several places, you refer to things which are "TBD". In some cases, I'm not sure whether they will be determined by you (or Robert, etc) based on some hard thinking as part of putting this document together, or whether they will be the result of research or experimentation during construction. Where it's the latter, we should call them out.
• Related to the above, where there are decisions to be made during construction, it would be helpful to indicate how that decision should be made: is it driven by external factors (e.g. availability of Gaia catalogues), by research time that needs to be scheduled, etc?
• How robust is this design against algorithmic components that don't come up to scratch? Would failure to deliver some component operating at some level of performance cause major changes to plan?
• Related to the above, we should attempt to capture performance & fidelity requirements for algorithms wherever we can. I understand – and we've discussed before – that this is not necessarily possible in general (often, it's a matter of assembling the system an then hammering on the tall poles), but the more definitive we can be about this in advance, the better. (Plus, we must have a justification for believing that the system as designed will satisfy the science requirements.)

At several places, you refer to things which are "TBD". In some cases, I'm not sure whether they will be determined by you (or Robert, etc) based on some hard thinking as part of putting this document together, or whether they will be the result of research or experimentation during construction. Where it's the latter, we should call them out.

I believe all of these are research projects (or at least "thinking projects" that we'll need to plan for, not complete prior to planning). I'll update the text to clarify.

Related to the above, where there are decisions to be made during construction, it would be helpful to indicate how that decision should be made: is it driven by external factors (e.g. availability of Gaia catalogues), by research time that needs to be scheduled, etc?

I'll try to add annotations to this effect.

How robust is this design against algorithmic components that don't come up to scratch? Would failure to deliver some component operating at some level of performance cause major changes to plan?

I'm afraid it's not robust at all, but I think it generally gets more robust the deeper we get into the production. Essentially, we have a lot of iterative procedures that re hard to bootstrap, and that tends to put a much bigger burden on whichever of the components we choose to start with. If one of those components doesn't work out, the best response may be to restructure the pipelines to put some other component first.

Related to the above, we should attempt to capture performance & fidelity requirements for algorithms wherever we can.

I agree this is important (and I'm sorry I keep forgetting about it), but I'm hoping we can mostly defer figuring out success criteria until the designs themselves are better described (just because this would be enough extra work for me that I don't think I can get it all done by Zeljko Ivezic's deadline).

By the way, in the spirit of Zeljko's wide-then-deep plan, I don't think going back to add annotations as above should necessarily be your top priority: I think from my point of view of planning it's useful to sketch out the later stages of the system then return to refine this later.

(Similarly, I'll add some more minor comments/request for clarification to the PR, but I don't think they need to be addressed urgently.)

Sounds good. I've just clarified all the places I actually wrote "TBD" in the text, but I'll save the other issues for later (including looking for places where I implied TBD without actually writing it).

I've merged the ticket branch to the draft branch, since I'm not planning any more work here soon. I'm leaving the ticket open for now so I don't forget to return to John Swinbank's comments at some point, but I'd be happy to take another approach to capturing those if he has a recommendation.