Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-16011

Reprocess RC2 with w_2018_41

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Reprocess the HSC RC2 dataset, as defined in DM-11345, using Stack w_2018_41. Steps include makeSkyMap.py, singleFrameDriver.py, mosaic.py, skyCorrection.py, coaddDriver.py, and multiBandDriver.py

        Attachments

          Issue Links

            Activity

            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            This run is postponed because the installation of w_2018_40 on the shared stack at /software failed during last weekend and is not available so far. It was caused by afw test failures, as a dependency on Numpy 1.14 was introduced to the stack, despite the baseline was 1.13 and the declared dependencies weren't updated.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - This run is postponed because the installation of w_2018_40 on the shared stack at /software failed during last weekend and is not available so far. It was caused by afw test failures, as a dependency on Numpy 1.14 was introduced to the stack, despite the baseline was 1.13 and the declared dependencies weren't updated.
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            As w_2018_40 wasn't installed (see DM-16129 for more details) and now w_2018_41 is available in the new stack, I'm changing this ticket to use w_2018_41 instead.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - As w_2018_40 wasn't installed (see DM-16129 for more details) and now w_2018_41 is available in the new stack, I'm changing this ticket to use w_2018_41 instead.
            Hide
            hchiang2 Hsin-Fang Chiang added a comment - - edited

            w_2018_41 from DM-16129's new stack was used for running singleFrameDriver and other pipelines, unless specified otherwise in the following comments. 

            In singleFrameDriver, among the 432 visits of the RC2 dataset, 10 CCDs failed, same data IDs as w_2018_32/DM-15184.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - - edited w_2018_41 from DM-16129 's new stack was used for running singleFrameDriver and other pipelines, unless specified otherwise in the following comments.  In singleFrameDriver, among the 432 visits of the RC2 dataset, 10 CCDs failed, same data IDs as w_2018_32/ DM-15184 .
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            meas_mosaic is broken in w_2018_41 and DM-16170 has been filed.

            To workaround the bug, w_2018_39 from the old stack (/software/lsstsw/stack3_20171023/) was used to run mosaic.py for this ticket.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - meas_mosaic is broken in w_2018_41 and DM-16170 has been filed. To workaround the bug, w_2018_39 from the old stack ( /software/lsstsw/stack3_20171023/ ) was used to run mosaic.py for this ticket.
            Hide
            hchiang2 Hsin-Fang Chiang added a comment - - edited

            w_2018_41 coaddDriver.py was broken by the API change of DM-15663; DM-16183 was filed.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - - edited w_2018_41 coaddDriver.py was broken by the API change of DM-15663 ; DM-16183 was filed.
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            pipe_drivers branch tickets/DM-16183 was used to run coaddDriver.py and multiBandDriver.py.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - pipe_drivers branch tickets/ DM-16183 was used to run coaddDriver.py and multiBandDriver.py .
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            coadd/multiband: the number of data products and empty patches are the same as w_2018_15/DM-14123 (this comment)

            Show
            hchiang2 Hsin-Fang Chiang added a comment - coadd/multiband: the number of data products and empty patches are the same as w_2018_15/ DM-14123 ( this comment )
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            The output repo is at /datasets/hsc/repo/rerun/RC/w_2018_41/DM-16011/
            The logs have been copied to /datasets/hsc/repo/rerun/RC/w_2018_41/DM-16011/logs
            The pipe_analysis and validate_drp runs are at DM-16055; pipe_analysis has been broken (DM-15869).
            The jointcal run is at DM-16097.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - The output repo is at /datasets/hsc/repo/rerun/RC/w_2018_41/ DM-16011 / The logs have been copied to /datasets/hsc/repo/rerun/RC/w_2018_41/ DM-16011 /logs The pipe_analysis and validate_drp runs are at DM-16055 ; pipe_analysis has been broken ( DM-15869 ). The jointcal run is at DM-16097 .

              People

              • Assignee:
                hchiang2 Hsin-Fang Chiang
                Reporter:
                hchiang2 Hsin-Fang Chiang
                Watchers:
                Hsin-Fang Chiang
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel