Details
Description
One of the key DM components for the LSST2015 meting in Bremerton is to attempt to have an end-to-end demonstration running for data release production and alert production. In addition we will attempt to calculate key performance metrics and compare them to the summer 15 cycle requirements.
I propose to have a meeting in the 11th August slot at 12:30pm Pacific to ensure that we will be prepared for the meeting. This will be an open discussion rather than a presentation.
Attached are two draft diagrams. One is an outline of the processing flow for DRP from raw data to SUI visualization. The other describes the hardware resources required at NCSA for the test.
Attachments
Activity
Jacek Becla: I would expect that next week's work will be hard-pressed to even get to TB. I don't expect you to sacrifice any lsst-dbdevNN machines beyond next week. That usage should move to the OpenStack or a dedicated integration machine (or machines) soon thereafter.
Jim Bosch: Thanks much for your diagram; it would be nice to incorporate it into the overall picture. It looks to me like you can draw everything you used in Gliffy: https://confluence.lsstcorp.org/display/~ktl/Gliffy+Test
Kian-Tat Lim, thanks, and sorry I didn't figure that out for myself - when I tried gliffy before I got frustrated with not being able to have multiple fonts in one text box and gave up without really trying to find a workaround. I'll convert my diagram to that sometime after today's meeting.
Meeting took place.
I've just attached a flowchart of the current data flow and parallelization for the science pipelines part of DRP (i.e. I made no attempt to include anything after the pixel-level processing). It's a bit of a mess in places because it tries to include both the state of the HSC pipeline now and what I expect the LSST pipeline to look like when the merge is complete (they're not quite the same). So there's a lot there that needs explanation; I need to find a good way to annotate it without cluttering up what's there further (I just did this in OpenOffice, since I couldn't find a free flowchart tool that gave me the kind of advanced formatting I've used to convey a lot of the information here).
I've also started a bit on the much more complex diagram for what I'm imagining in the future, but I very much doubt I'll do any more work on that before Bremerton, and I'm sort of hoping we'll identify a better tool/format by then.