Based on the slurm statistics collected during the processing or anything post-mortem, summarize the computing resource used in DM-10404.
Transform HSC Reprocessing Analysis into Test Report form
S17B full HSC reprocessing with the SSP PDR1 data
analyze the storage usage of the output butler repos from S17B reprocessing
Wiki Page Loading...
As discussed with Greg Daues, the Slurm accounting may not have worked completely likely due to 1) our simple configuration,and 2) use of GPFS rather than local file systems. Figuring out what need to improve is something for this cycle. In this ticket we try to make the most out of what we collected in DM-10404.
Started working on this with Greg Daues earlier this week.
Greg Daues I've finished editing the confluence page for this ticket and have copied most of our discussions over. For now I left out the "experiments" and investigation ideas for further understanding the software performance. Is there anything you would like to add before we close this ticket?
Michelle Gower I'd appreciate if you may read our brief summary beginning here to the end of the page.
Yes I think the extensive report on the Confluence page gives a great, wide-ranging summary and is a strong basis for future campaign reporting/summaries. And so I think the ticket can now be closed.
The report looks great. I think the policy now is for reports like these to be either DM Test Reports or DM Tech Notes rather than confluence pages. Wil O'Mullane can say for sure.
Yes this should be a test report in DocuShare. Ideally using the TeX templates provided. We should provide documents for the status review by Aug 22 so could we get a DMTR version of this for Docushare bu Aug 21?
The report will be DMTR-31.
I've created a Story (DM-11559) for this, currently in an emergent work epic under the 01.02C.07.01 WBS. Is this formatting the responsibility of the data producer, or is it handled by documentation specialists in the project? If the former, could you provide a pointer to guidance on Test Report content and formatting requirements? If the latter, please move the Story to the appropriate Epic & WBS.
This is not really emergent work - this is documenting your work and this your responsibility. For this report simply depositing a PDF which looks like an LSST document (i.e. use the tex templates https://github.com/lsst/lsst-texmf/tree/master/examples, there are some word templates in Docushare)