Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-13547

Plot the node utilization of the w_2018_03 RC1 reprocessing

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Plot the node utilization versus time of the HSC RC1 reprocessing in DM-13463.
      Use Mikolaj Kowalik's existing code and improve it as needed/desired (e.g. do the plotting in python instead). Push the code to https://github.com/lsst-dm/ldf_ops_tools

      Example plots attached. 

      A future application of this is to plot the node utilization of a PDR1 run given the slurm job IDs, like this https://github.com/lsst-dm/DMTR-31/blob/master/figures/nodeUtilization.png

        Attachments

        1. time_vs_nnodes.png
          time_vs_nnodes.png
          39 kB
        2. usage-w_2017_26.png
          usage-w_2017_26.png
          43 kB
        3. usage-w_2017_28.png
          usage-w_2017_28.png
          44 kB
        4. usage-w_2017_30.png
          usage-w_2017_30.png
          45 kB
        5. usage-w_2017_32.png
          usage-w_2017_32.png
          45 kB

          Issue Links

            Activity

            Hide
            sthrush Samantha Thrush added a comment -


            The plot is finally up to my standards, however, the code is still a bit sloppy. I will keep working on it and will hopefully be done with it tomorrow, at which point I will push the codes to github.

            Show
            sthrush Samantha Thrush added a comment - The plot is finally up to my standards, however, the code is still a bit sloppy. I will keep working on it and will hopefully be done with it tomorrow, at which point I will push the codes to github.
            Hide
            sthrush Samantha Thrush added a comment -

            The codes have been pushed to github, I have asked Mikolaj if he would be up for reviewing the code, and a pull request has been logged. 

            Show
            sthrush Samantha Thrush added a comment - The codes have been pushed to github, I have asked Mikolaj if he would be up for reviewing the code, and a pull request has been logged. 
            Hide
            sthrush Samantha Thrush added a comment - - edited

            I have addressed all of Mikolaj's comments.  However, I seem to be having a difficult time making it so that the original README is included in the "Add Mikolaj's Code" (git won't let me fixup any commits that have the old README to the "Add Mikolaj's Code" commit for some reason).  I currently have the modified README in the directory, but I'm still working on rebasing things so that the old README will be associated with the "Add Mikolaj's Code" commit.

            Show
            sthrush Samantha Thrush added a comment - - edited I have addressed all of Mikolaj's comments.  However, I seem to be having a difficult time making it so that the original README is included in the "Add Mikolaj's Code" (git won't let me fixup any commits that have the old README to the "Add Mikolaj's Code" commit for some reason).  I currently have the modified README in the directory, but I'm still working on rebasing things so that the old README will be associated with the "Add Mikolaj's Code" commit.
            Hide
            sthrush Samantha Thrush added a comment -

            I've just added Mikolaj's original README and then my modifications to it into the directory so that it will be clear which modifications I made to the document. 

            Show
            sthrush Samantha Thrush added a comment - I've just added Mikolaj's original README and then my modifications to it into the directory so that it will be clear which modifications I made to the document. 
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            Please be sure to clean up the commit history before merging. 

             

            The Dev Guide suggests 

            During this rebase, we recommend squashing any fixup commits into the main commit implementing that feature. Git commit history should not record the iterative improvements from code review.

            and other details therein.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - Please be sure to clean up the commit history before merging.    The Dev Guide suggests  During this rebase, we recommend squashing any fixup commits into the main commit implementing that feature. Git commit history should not record the iterative improvements from code review. and other details therein.
            Hide
            sthrush Samantha Thrush added a comment -

            I've cleaned the commit history and the ticket has been merged. 

            Show
            sthrush Samantha Thrush added a comment - I've cleaned the commit history and the ticket has been merged. 

              People

              Assignee:
              sthrush Samantha Thrush
              Reporter:
              hchiang2 Hsin-Fang Chiang
              Reviewers:
              Mikolaj Kowalik
              Watchers:
              Hsin-Fang Chiang, Mikolaj Kowalik, Samantha Thrush
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.