[DM-34623] AP Performance sprint - Jira

XML

Word

Printable

Details

Type: Story
Status: Done
Resolution: Done
Fix Version/s: None
Component/s: None
Labels:
None

Story Points:
20
Epic Link:
s22b-a&p-preops
Sprint:
AP S22-6 (May)
Team:
Alert Production
Urgent?:
No

Description

This is an umbrella ticket for technical tasks related to the AP performance tuning sprint in May 2022.

Goals include:

update profiling and identify future areas for performance tuning
make AP-specific pipeline & configurations
think about how to project timing scenarios more accurately in steady state operations.

We will test on DC2 data to capture scaling with pixels and image depth most accurately.

Attachments

Issue Links

is triggering

DM-34993 Investigate failed detector-visit in DC2 4 patch run

To Do

relates to

DM-35158 Investigate scaling of DIAForcedSource

To Do

DM-31653 Reprocess a subset of DC2 with profiling

Done

DM-34845 Create ap_verify dataset for DC2

Done

DM-34976 Prepare presentation on the performance tuning sprint

Done

DM-34993 Investigate failed detector-visit in DC2 4 patch run

To Do

DM-32838 AP pipeline performance characterization

Done

DM-34865 Script for looking at ap_verify metrics

Done

DM-34896 Collate timing metrics from run on 4-patch >200 visit DC2 dataset

Done

mentioned in: Page Loading...; Page Loading...; Page Loading...

(4 relates to, 3 mentioned in)

Options

Sub-Tasks

1.	Identify two DC2 detector-visits for profiling	Done	Meredith Rawls
2.	Perform initial profiling run on test dataset	Done	John Parejko
3.	Identify unnecessary configs and Tasks and test removal from ap_pipe	Done	John Parejko
4.	Identify larger DC2 dataset for timing tests	Done	Meredith Rawls
5.	Process large-scale processing run and persist timing metrics	Done	Meredith Rawls
6.	Create stack with timer decorators removed	Done	John Parejko
7.	Try profiling ap_verify with pyinstrument	Done	John Parejko
8.	Make plots of timing distributions	Done	Meredith Rawls
9.	Investigate imageDifference failures	Done	Meredith Rawls
10.	Compare timing profiles with PCA, PSFex, and PIFF	Done	John Parejko

Activity

Hide

Permalink

John Parejko added a comment - 04/Jun/22 12:26 AM

Confluence summary page: https://confluence.lsstcorp.org/display/DM/May+2022+Performance+Sprint+Summary

Show

John Parejko added a comment - 04/Jun/22 12:26 AM Confluence summary page: https://confluence.lsstcorp.org/display/DM/May+2022+Performance+Sprint+Summary

People

Assignee:: Eric Bellm

Reporter:: Ian Sullivan

Watchers:: Ian Sullivan, John Parejko, Meredith Rawls

Votes:: 0 Vote for this issue

Watchers:: 3 Start watching this issue

Dates

Created:: 02/May/22 7:44 PM

Updated:: 08/Jun/22 11:25 PM

Resolved:: 31/May/22 5:47 PM

Jenkins

No builds found.

Data Management