Details
-
Type:
Epic
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: None
-
Labels:
-
Epic Name:Dataset Reprocessing Campaigns (FY18b-1)
-
Story Points:80
-
WBS:02C.07.06.01
-
Team:Data Facility
-
Cycle:Fall 2018
Description
Execute data reprocessing campaigns on test data. Processing campaigns are specified by a goal/purpose, pipeline(s) to be run, configurations, and input and output dataset(s). Pre-camera data campaigns support pipeline development and evolution of the batch production service. Work includes setup, testing, execution, debugging, verification of results, handling of data products, and liaison with the Pipelines teams. Campaigns anticipated for FY18b include:
1) "Weeklies," which are run ~biweekly on a pre-defined subset of HSC data (RC2) using the latest tagged version of the LSST Stack.
2) "PDR1 Reprocessing," which is run over the entire HSC PDR dataset using a stable version of the LSST Stack. PDR1 reprocessing campaigns are scheduled with the DRP group.
Deliverables include data products delivered to the developer-accessible file system.
Time-bounded epic. Resulting datasets and logs are in subdirectories of /datasets/hsc/repo/rerun/RC/. Results and processes are described at https://confluence.lsstcorp.org/display/DM/Data+Processing+End+to+End+Testing