Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-34623 AP Performance sprint
  3. DM-34825

Perform initial profiling run on test dataset

    XMLWordPrintable

    Details

    • Type: Technical task
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Sprint:
      AP S22-6 (May)
    • Team:
      Alert Production

      Description

      Using the dataset defined in DM-34824, perform an initial profiling run. Document the steps taken on this ticket and attach results.

        Attachments

        1. profile_script-DM-34825.sh
          1 kB
        2. prof_detector_174.out
          1.22 MB
        3. prof_detector_164.out
          1.22 MB
        4. detector_174.png
          detector_174.png
          1.01 MB
        5. detector_164.png
          detector_164.png
          1.06 MB

          Issue Links

          relates to

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-32838 AP pipeline performance characterization

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-39491 Run ap_verify on DC2 with profiling before rubinenv 7

          • Done

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            Parejkoj John Parejko added a comment -

            I've attached the profiling script I ran on lsst-devl02, and the results for both of the selected detectors. Not a big difference between the two, but this gives us a useful starting point.

            These profiles open in snakeviz fine, which is probably as good a way as any to start digging into them.

            Show
            Parejkoj John Parejko added a comment - I've attached the profiling script I ran on lsst-devl02, and the results for both of the selected detectors. Not a big difference between the two, but this gives us a useful starting point. These profiles open in snakeviz fine, which is probably as good a way as any to start digging into them.
            Hide
            Permalink
            Parejkoj John Parejko added a comment -

            Take a look at the script I attached here, and let me know if it makes sense to you. This is what I'm using to do the profiling, I'll just modify it for each appropriate ticket.

            Show
            Parejkoj John Parejko added a comment - Take a look at the script I attached here, and let me know if it makes sense to you. This is what I'm using to do the profiling, I'll just modify it for each appropriate ticket.
            Hide
            Permalink
            tjenness Tim Jenness added a comment -

            I've attached pngs of the call graph in case that's easier for a quick look before delving into snakeviz.

            Show
            tjenness Tim Jenness added a comment - I've attached pngs of the call graph in case that's easier for a quick look before delving into snakeviz.
            Hide
            Permalink
            mrawls Meredith Rawls added a comment -

            I'm not sure how to open the .out files, so I appreciate Tim making nice little pngs!

            The profiling script looks like you ran all the intended things, and we poked at the outputs together earlier today. Maybe sometime later we should document how to run (and interpret) profiling since I had no idea prior to now? Otherwise all set!

            Show
            mrawls Meredith Rawls added a comment - I'm not sure how to open the .out files, so I appreciate Tim making nice little pngs! The profiling script looks like you ran all the intended things, and we poked at the outputs together earlier today. Maybe sometime later we should document how to run (and interpret) profiling since I had no idea prior to now? Otherwise all set!
            Hide
            Permalink
            tjenness Tim Jenness added a comment -

            I made the PNGs with something like: gprof2dot -f pstats profile.pstat | dot -Tpdf -o profile.pdf.

            Show
            tjenness Tim Jenness added a comment - I made the PNGs with something like: gprof2dot -f pstats profile.pstat | dot -Tpdf -o profile.pdf .

              People

              Assignee:
              Parejkoj John Parejko
              Reporter:
              ebellm Eric Bellm
              Reviewers:
              Meredith Rawls
              Watchers:
              Eric Bellm, John Parejko, Meredith Rawls, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.