Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-34623 AP Performance sprint
  3. DM-34825

Perform initial profiling run on test dataset

    XMLWordPrintable

    Details

    • Type: Technical task
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Sprint:
      AP S22-6 (May)
    • Team:
      Alert Production

      Description

      Using the dataset defined in DM-34824, perform an initial profiling run. Document the steps taken on this ticket and attach results.

        Attachments

        1. detector_164.png
          detector_164.png
          1.06 MB
        2. detector_174.png
          detector_174.png
          1.01 MB
        3. prof_detector_164.out
          1.22 MB
        4. prof_detector_174.out
          1.22 MB
        5. profile_script-DM-34825.sh
          1 kB

          Issue Links

          relates to

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-32838 AP pipeline performance characterization

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-39491 Run ap_verify on DC2 with profiling before rubinenv 7

          • Done

            Activity

            Ascending order - Click to sort in descending order
            No builds found.
            ebellm Eric Bellm created issue -
            ebellm Eric Bellm made changes -
            Field Original Value New Value
            Link This issue relates to DM-32838 [ DM-32838 ]
            Parejkoj John Parejko made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            Parejkoj John Parejko made changes -
            Attachment prof_detector_164.out [ 59742 ]
            Attachment prof_detector_174.out [ 59743 ]
            Parejkoj John Parejko made changes -
            Attachment profile_script-DM-34825.sh [ 59744 ]
            Parejkoj John Parejko made changes -
            Attachment profile_script-DM-34825.sh [ 59744 ]
            Parejkoj John Parejko made changes -
            Attachment profile_script-DM-34825.sh [ 59745 ]
            Hide
            Permalink
            Parejkoj John Parejko added a comment -

            I've attached the profiling script I ran on lsst-devl02, and the results for both of the selected detectors. Not a big difference between the two, but this gives us a useful starting point.

            These profiles open in snakeviz fine, which is probably as good a way as any to start digging into them.

            Show
            Parejkoj John Parejko added a comment - I've attached the profiling script I ran on lsst-devl02, and the results for both of the selected detectors. Not a big difference between the two, but this gives us a useful starting point. These profiles open in snakeviz fine, which is probably as good a way as any to start digging into them.
            Hide
            Permalink
            Parejkoj John Parejko added a comment -

            Take a look at the script I attached here, and let me know if it makes sense to you. This is what I'm using to do the profiling, I'll just modify it for each appropriate ticket.

            Show
            Parejkoj John Parejko added a comment - Take a look at the script I attached here, and let me know if it makes sense to you. This is what I'm using to do the profiling, I'll just modify it for each appropriate ticket.
            Parejkoj John Parejko made changes -
            Reviewers Meredith Rawls [ mrawls ]
            Status In Progress [ 3 ] In Review [ 10004 ]
            tjenness Tim Jenness made changes -
            Attachment detector_174.png [ 59746 ]
            tjenness Tim Jenness made changes -
            Attachment detector_164.png [ 59747 ]
            Hide
            Permalink
            tjenness Tim Jenness added a comment -

            I've attached pngs of the call graph in case that's easier for a quick look before delving into snakeviz.

            Show
            tjenness Tim Jenness added a comment - I've attached pngs of the call graph in case that's easier for a quick look before delving into snakeviz.
            Hide
            Permalink
            mrawls Meredith Rawls added a comment -

            I'm not sure how to open the .out files, so I appreciate Tim making nice little pngs!

            The profiling script looks like you ran all the intended things, and we poked at the outputs together earlier today. Maybe sometime later we should document how to run (and interpret) profiling since I had no idea prior to now? Otherwise all set!

            Show
            mrawls Meredith Rawls added a comment - I'm not sure how to open the .out files, so I appreciate Tim making nice little pngs! The profiling script looks like you ran all the intended things, and we poked at the outputs together earlier today. Maybe sometime later we should document how to run (and interpret) profiling since I had no idea prior to now? Otherwise all set!
            mrawls Meredith Rawls made changes -
            Status In Review [ 10004 ] Reviewed [ 10101 ]
            Hide
            Permalink
            tjenness Tim Jenness added a comment -

            I made the PNGs with something like: gprof2dot -f pstats profile.pstat | dot -Tpdf -o profile.pdf.

            Show
            tjenness Tim Jenness added a comment - I made the PNGs with something like: gprof2dot -f pstats profile.pstat | dot -Tpdf -o profile.pdf .
            Parejkoj John Parejko made changes -
            Resolution Done [ 10000 ]
            Status Reviewed [ 10101 ] Done [ 10002 ]
            Parejkoj John Parejko made changes -
            Link This issue relates to DM-39491 [ DM-39491 ]

              People

              Assignee:
              Parejkoj John Parejko
              Reporter:
              ebellm Eric Bellm
              Reviewers:
              Meredith Rawls
              Watchers:
              Eric Bellm, John Parejko, Meredith Rawls, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.