Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-12831

Periodic dataset reprocessing - FY18a

    Details

    • Epic Name:
      Periodic dataset reprocessing = FY18a
    • Story Points:
      40
    • WBS:
      02C.07.06.01
    • Team:
      Data Facility
    • Cycle:
      Spring 2018

      Description

      Bucket epic. Continue dataset reprocessing to support stack release and science pipeline testing.

        Attachments

        Stories in Epic (Custom Issue Matrix)

        Key Summary Story Points Assignee Status
         
        DM-13078

        Add HSC y-band stray light calibration data to /datasets/hsc

        1 Hsin-Fang Chiang Done
         
        DM-13256

        Reprocess RC with w_2018_04

        4 Hsin-Fang Chiang Done
         
        DM-13066

        Reprocess RC1 with w_2018_02

        4 Hsin-Fang Chiang Done
         
        DM-12982

        Reprocess RC with w_2017_52

        4 Samantha Thrush Done
         
        DM-13894

        Run validate_drp with w_2018_12 HSC RC2 outputs

        1 Hsin-Fang Chiang Done
         
        DM-13815

        Find elapsed code times from usage.py/usageplot.py

        2 Samantha Thrush Done
         
        DM-13804

        Run validate_drp with w_2018_10 HSC RC2 outputs

        1 Hsin-Fang Chiang Done
         
        DM-13795

        Reprocess RC2 with w_2018_12

        3 Hsin-Fang Chiang Done
         
        DM-13789

        Incorrect task names in usage.py output

        1 Samantha Thrush Done
         
        DM-13783

        Make color-coded node-usage plot for S17B HSC PDR1 reprocessing and find total node-hours

        1 Samantha Thrush Done
         
        DM-13761

        Run pipe_analysis scripts with the RC2 data of w_2018_10

        1 Hsin-Fang Chiang Done
         
        DM-13893

        Run pipe_analysis scripts with the RC2 data of w_2018_12

        1 Hsin-Fang Chiang Done
         
        DM-13463

        Reprocess RC1 with w_2018_03

        4 Samantha Thrush Done
         
        DM-13547

        Plot the node utilization of the w_2018_03 RC1 reprocessing

        2 Samantha Thrush Done
         
        DM-13533

        Sky frame calibration data not available for early 2014

        0.2 Hsin-Fang Chiang Done
         
        DM-13532

        Reprocess RC with w_2018_08

        4 Hsin-Fang Chiang Done
         
        DM-13523

        Run pipe_analysis scripts with the RC2 data of w_2018_06

        1 Hsin-Fang Chiang Done
         
        DM-13497

        Remake the transmission curve data for the NB filter fix

        0.2 Hsin-Fang Chiang Done
         
        DM-13502

        Reprocess the 2 WIDE tracts of RC2 with w_2018_03

        1 Samantha Thrush Done
         
        DM-13436

        Run pipe_analysis scripts with the RC2 data of w_2018_04

        1 Hsin-Fang Chiang Done
         
        DM-13435

        Reprocess RC with w_2018_06

        4 Hsin-Fang Chiang Done
         
        DM-13647

        Reprocess RC2 with w_2018_10

        3 Hsin-Fang Chiang Done
         
        DM-13646

        Run pipe_analysis scripts with the RC2 data of w_2018_08

        1 Hsin-Fang Chiang Done
         
        DM-13622

        Make usageplot.py output step plots

        0.5 Samantha Thrush Done
         
        DM-13619

        Modify usage.py for NODE_FAIL/COMPLETED failure case

        1 Samantha Thrush Done
         
        DM-13618

        Find node-hour usage from usage codes

        0.5 Samantha Thrush Done
         
        DM-13699

        Modify usage.py and usageplot.py to allow for color-coded plots

        6 Samantha Thrush Done
         
        DM-13697

        Move the w_2017_52 HSC-RC processed repo to /scratch

        0.1 Samantha Thrush Done
         
        DM-13578

        Plot the node utilization for RC1 Reprocessed Jobs

        2 Samantha Thrush Done

          Activity

          Hide
          plutchak Joel Plutchak (Inactive) added a comment -

          Dataset reprocessing for this epic has completed. Output datasets and logs have been placed in the /projects partition and documented at https://confluence.lsstcorp.org/display/DM/Reprocessing+of+the+HSC+RC+dataset and https://confluence.lsstcorp.org/display/DM/Reprocessing+of+the+HSC+RC2+dataset .

          Show
          plutchak Joel Plutchak (Inactive) added a comment - Dataset reprocessing for this epic has completed. Output datasets and logs have been placed in the /projects partition and documented at https://confluence.lsstcorp.org/display/DM/Reprocessing+of+the+HSC+RC+dataset and https://confluence.lsstcorp.org/display/DM/Reprocessing+of+the+HSC+RC2+dataset .

            People

            • Assignee:
              hchiang2 Hsin-Fang Chiang
              Reporter:
              plutchak Joel Plutchak (Inactive)
              Watchers:
              Joel Plutchak (Inactive)
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Start date:
                End date:

                Summary Panel