Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-10649

Analyze and summarize computing resource used in S17B full HSC reprocessing

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Based on the slurm statistics collected during the processing or anything post-mortem, summarize the computing resource used in DM-10404.

        Attachments

          Issue Links

            Activity

            hchiang2 Hsin-Fang Chiang created issue -
            hchiang2 Hsin-Fang Chiang made changes -
            Field Original Value New Value
            Link This issue relates to DM-10404 [ DM-10404 ]
            hchiang2 Hsin-Fang Chiang made changes -
            Epic Link DM-10702 [ 32655 ]
            hchiang2 Hsin-Fang Chiang made changes -
            Remote Link This issue links to "Page (Confluence)" [ 15222 ]
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            As discussed with Greg Daues, the Slurm accounting may not have worked completely likely due to 1) our simple configuration,and 2) use of GPFS rather than local file systems. Figuring out what need to improve is something for this cycle. In this ticket we try to make the most out of what we collected in DM-10404.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - As discussed with Greg Daues , the Slurm accounting may not have worked completely likely due to 1) our simple configuration,and 2) use of GPFS rather than local file systems. Figuring out what need to improve is something for this cycle. In this ticket we try to make the most out of what we collected in DM-10404 .
            hchiang2 Hsin-Fang Chiang made changes -
            Story Points 5
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            Started working on this with Greg Daues earlier this week.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - Started working on this with Greg Daues earlier this week.
            hchiang2 Hsin-Fang Chiang made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            hchiang2 Hsin-Fang Chiang made changes -
            Link This issue relates to DM-10904 [ DM-10904 ]
            hchiang2 Hsin-Fang Chiang made changes -
            Remote Link This issue links to "Page (Confluence)" [ 15222 ]
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            Greg Daues I've finished editing the confluence page for this ticket and have copied most of our discussions over. For now I left out the "experiments" and investigation ideas for further understanding the software performance. Is there anything you would like to add before we close this ticket?

            Michelle Gower I'd appreciate if you may read our brief summary beginning here to the end of the page.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - Greg Daues I've finished editing the confluence page for this ticket and have copied most of our discussions over. For now I left out the "experiments" and investigation ideas for further understanding the software performance. Is there anything you would like to add before we close this ticket? Michelle Gower I'd appreciate if you may read our brief summary beginning here to the end of the page.
            hchiang2 Hsin-Fang Chiang made changes -
            Reviewers Michelle Gower [ mgower ]
            Status In Progress [ 3 ] In Review [ 10004 ]
            Hide
            gdaues Greg Daues added a comment -

            Yes I think the extensive report on the Confluence page gives a great, wide-ranging summary and is a strong basis for future campaign reporting/summaries. And so I think the ticket can now be closed.

            Show
            gdaues Greg Daues added a comment - Yes I think the extensive report on the Confluence page gives a great, wide-ranging summary and is a strong basis for future campaign reporting/summaries. And so I think the ticket can now be closed.
            Hide
            tjenness Tim Jenness added a comment -

            The report looks great. I think the policy now is for reports like these to be either DM Test Reports or DM Tech Notes rather than confluence pages. Wil O'Mullane can say for sure.

            Show
            tjenness Tim Jenness added a comment - The report looks great. I think the policy now is for reports like these to be either DM Test Reports or DM Tech Notes rather than confluence pages. Wil O'Mullane can say for sure.
            Hide
            womullan Wil O'Mullane added a comment -

            Yes this should be a test report in DocuShare. Ideally using the TeX templates provided. We should provide documents for the status review by Aug 22 so could we get a DMTR version of this for Docushare bu Aug 21?

            Show
            womullan Wil O'Mullane added a comment - Yes this should be a test report in DocuShare. Ideally using the TeX templates provided. We should provide documents for the status review by Aug 22 so could we get a DMTR version of this for Docushare bu Aug 21?
            Hide
            tjenness Tim Jenness added a comment -

            The report will be DMTR-31.

            Show
            tjenness Tim Jenness added a comment - The report will be DMTR-31.
            plutchak Joel Plutchak (Inactive) made changes -
            Priority Undefined [ 10000 ] Major [ 3 ]
            plutchak Joel Plutchak (Inactive) made changes -
            Resolution Done [ 10000 ]
            Status In Review [ 10004 ] Done [ 10002 ]
            Hide
            plutchak Joel Plutchak (Inactive) added a comment -

            I've created a Story (DM-11559) for this, currently in an emergent work epic under the 01.02C.07.01 WBS. Is this formatting the responsibility of the data producer, or is it handled by documentation specialists in the project? If the former, could you provide a pointer to guidance on Test Report content and formatting requirements? If the latter, please move the Story to the appropriate Epic & WBS.

            Show
            plutchak Joel Plutchak (Inactive) added a comment - I've created a Story ( DM-11559 ) for this, currently in an emergent work epic under the 01.02C.07.01 WBS. Is this formatting the responsibility of the data producer, or is it handled by documentation specialists in the project? If the former, could you provide a pointer to guidance on Test Report content and formatting requirements? If the latter, please move the Story to the appropriate Epic & WBS.
            tjenness Tim Jenness made changes -
            Link This issue is triggering DM-11559 [ DM-11559 ]
            Hide
            womullan Wil O'Mullane added a comment -

            This is not really emergent work - this is documenting your work and this your responsibility. For this report simply depositing a PDF which looks like an LSST document (i.e. use the tex templates https://github.com/lsst/lsst-texmf/tree/master/examples, there are some word templates in Docushare)

            Show
            womullan Wil O'Mullane added a comment - This is not really emergent work - this is documenting your work and this your responsibility. For this report simply depositing a PDF which looks like an LSST document (i.e. use the tex templates https://github.com/lsst/lsst-texmf/tree/master/examples , there are some word templates in Docushare)
            gpdf Gregory Dubois-Felsmann made changes -
            Remote Link This issue links to "Wiki Page (Confluence)" [ 15340 ]
            frossie Frossie Economou made changes -
            Team Process Middleware [ 10206 ] Data Facility [ 12219 ]

              People

              Assignee:
              hchiang2 Hsin-Fang Chiang
              Reporter:
              hchiang2 Hsin-Fang Chiang
              Reviewers:
              Michelle Gower
              Watchers:
              Greg Daues, Hsin-Fang Chiang, Joel Plutchak (Inactive), John Swinbank, Michelle Gower, Tim Jenness, Wil O'Mullane
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: