Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-11165

Reprocess RC with w_2017_26

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Reprocess the HSC RC dataset using w_2017_26 following the same steps as in DM-10129/DM-11020

        Attachments

          Issue Links

          mentioned in

          Page Loading...

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            hchiang2 Hsin-Fang Chiang added a comment -

            Some job attempts failed due to IHS-341

            Show
            hchiang2 Hsin-Fang Chiang added a comment - Some job attempts failed due to IHS-341
            Hide
            Permalink
            hchiang2 Hsin-Fang Chiang added a comment -

            After I removed some files in /project, IHS-341 is no longer blocking for now.

            In the single frame processing, there were 81 reproducible fatal failures, 62 from COSMOS and 19 from WIDE, same as in DM-11020. The failed data IDs are the same as in DM-11020.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - After I removed some files in /project , IHS-341 is no longer blocking for now. In the single frame processing, there were 81 reproducible fatal failures, 62 from COSMOS and 19 from WIDE, same as in DM-11020 . The failed data IDs are the same as in DM-11020 .
            Hide
            Permalink
            hchiang2 Hsin-Fang Chiang added a comment -
            mosaic

            Current master of meas_mosaic (f4b19d1) built with w_2017_26 is used. All 3 tracts completed.

            coadd

            In COSMOS tract 9813, the coadd products have 75 patches in HSC-G, 74 in HSC-R, 79 in HSC-I, 79 in HSC-Y, 79 in HSC-Z, and 76 in NB0921. In WIDE, tract=8766 misses HSC-R in two patches; tract=8767 misses HSC-R in 5 patches. Same as in DM-11020.

            The empty patches of the 3 tracts are:

            • tract 9813 patch 0,0: HSC-G,HSC-I,HSC-R,HSC-Y,HSC-Z,NB0921
            • tract 9813 patch 0,8: HSC-G,HSC-I,HSC-R,HSC-Y,HSC-Z,NB0921
            • tract 9813 patch 0,1: HSC-G, HSC-R
            • tract 9813 patch 0,7: HSC-G, HSC-R, NB0921
            • tract 9813 patch 1,8: HSC-R, NB0921
            • tract 9813 patch 3,8: HSC-G
            • tract 9813 patch 8,0: HSC-G, HSC-R
            • tract 9813 patch 8,8: HSC-R, NB0921
            • tract 8766 patch 0,8: HSC-R
            • tract 8766 patch 7,8: HSC-R
            • tract 8767 patch 0,8: HSC-R
            • tract 8767 patch 1,8: HSC-R
            • tract 8767 patch 2,8: HSC-R
            • tract 8767 patch 8,7: HSC-R
            • tract 8767 patch 8,8: HSC-R
            Show
            hchiang2 Hsin-Fang Chiang added a comment - mosaic Current master of meas_mosaic ( f4b19d1 ) built with w_2017_26 is used. All 3 tracts completed. coadd In COSMOS tract 9813, the coadd products have 75 patches in HSC-G, 74 in HSC-R, 79 in HSC-I, 79 in HSC-Y, 79 in HSC-Z, and 76 in NB0921. In WIDE, tract=8766 misses HSC-R in two patches; tract=8767 misses HSC-R in 5 patches. Same as in DM-11020 . The empty patches of the 3 tracts are: tract 9813 patch 0,0: HSC-G,HSC-I,HSC-R,HSC-Y,HSC-Z,NB0921 tract 9813 patch 0,8: HSC-G,HSC-I,HSC-R,HSC-Y,HSC-Z,NB0921 tract 9813 patch 0,1: HSC-G, HSC-R tract 9813 patch 0,7: HSC-G, HSC-R, NB0921 tract 9813 patch 1,8: HSC-R, NB0921 tract 9813 patch 3,8: HSC-G tract 9813 patch 8,0: HSC-G, HSC-R tract 9813 patch 8,8: HSC-R, NB0921 tract 8766 patch 0,8: HSC-R tract 8766 patch 7,8: HSC-R tract 8767 patch 0,8: HSC-R tract 8767 patch 1,8: HSC-R tract 8767 patch 2,8: HSC-R tract 8767 patch 8,7: HSC-R tract 8767 patch 8,8: HSC-R
            Hide
            Permalink
            hchiang2 Hsin-Fang Chiang added a comment -
            MultiBand

            DM-11091 has fixed the multiband problem seen in DM-11020, so all patches now have multiband products: 81 patches in WIDE tracts 8766 and 8767, 79 patches in COSMOS tract 9813.

            Butler repo path

            Results are at repo /project/hsc_rc/w_2017_26/DM-11165
            Logs are at /project/hsc_rc/w_2017_26/DM-11165/logs

            Show
            hchiang2 Hsin-Fang Chiang added a comment - MultiBand DM-11091 has fixed the multiband problem seen in DM-11020 , so all patches now have multiband products: 81 patches in WIDE tracts 8766 and 8767, 79 patches in COSMOS tract 9813. Butler repo path Results are at repo /project/hsc_rc/w_2017_26/ DM-11165 Logs are at /project/hsc_rc/w_2017_26/ DM-11165 /logs
            Hide
            Permalink
            hchiang2 Hsin-Fang Chiang added a comment -

            With RFC-362 the nominal butler path is now /datasets/hsc/repo/rerun/RC/w_2017_26/DM-11165/

            With w_2017_26 the mosaic jobs had many warnings about the missed matches, e.g.

            afw.table WARN: Persisted match record with ID 111331498155159102 not found in catalog 1.

            This has been reported in DM-11356.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - With RFC-362 the nominal butler path is now /datasets/hsc/repo/rerun/RC/w_2017_26/ DM-11165 / With w_2017_26 the mosaic jobs had many warnings about the missed matches, e.g. afw.table WARN: Persisted match record with ID 111331498155159102 not found in catalog 1 . This has been reported in DM-11356 .

              People

              Assignee:
              hchiang2 Hsin-Fang Chiang
              Reporter:
              hchiang2 Hsin-Fang Chiang
              Watchers:
              Hsin-Fang Chiang
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: