Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-11165

Reprocess RC with w_2017_26

    XMLWordPrintable

Details

    • Story
    • Status: Done
    • Resolution: Done
    • None
    • None
    • None

    Description

      Reprocess the HSC RC dataset using w_2017_26 following the same steps as in DM-10129/DM-11020

      Attachments

        Issue Links

          Activity

            Some job attempts failed due to IHS-341

            hchiang2 Hsin-Fang Chiang added a comment - Some job attempts failed due to IHS-341

            After I removed some files in /project, IHS-341 is no longer blocking for now.

            In the single frame processing, there were 81 reproducible fatal failures, 62 from COSMOS and 19 from WIDE, same as in DM-11020. The failed data IDs are the same as in DM-11020.

            hchiang2 Hsin-Fang Chiang added a comment - After I removed some files in /project , IHS-341 is no longer blocking for now. In the single frame processing, there were 81 reproducible fatal failures, 62 from COSMOS and 19 from WIDE, same as in DM-11020 . The failed data IDs are the same as in DM-11020 .
            mosaic

            Current master of meas_mosaic (f4b19d1) built with w_2017_26 is used. All 3 tracts completed.

            coadd

            In COSMOS tract 9813, the coadd products have 75 patches in HSC-G, 74 in HSC-R, 79 in HSC-I, 79 in HSC-Y, 79 in HSC-Z, and 76 in NB0921. In WIDE, tract=8766 misses HSC-R in two patches; tract=8767 misses HSC-R in 5 patches. Same as in DM-11020.

            The empty patches of the 3 tracts are:

            • tract 9813 patch 0,0: HSC-G,HSC-I,HSC-R,HSC-Y,HSC-Z,NB0921
            • tract 9813 patch 0,8: HSC-G,HSC-I,HSC-R,HSC-Y,HSC-Z,NB0921
            • tract 9813 patch 0,1: HSC-G, HSC-R
            • tract 9813 patch 0,7: HSC-G, HSC-R, NB0921
            • tract 9813 patch 1,8: HSC-R, NB0921
            • tract 9813 patch 3,8: HSC-G
            • tract 9813 patch 8,0: HSC-G, HSC-R
            • tract 9813 patch 8,8: HSC-R, NB0921
            • tract 8766 patch 0,8: HSC-R
            • tract 8766 patch 7,8: HSC-R
            • tract 8767 patch 0,8: HSC-R
            • tract 8767 patch 1,8: HSC-R
            • tract 8767 patch 2,8: HSC-R
            • tract 8767 patch 8,7: HSC-R
            • tract 8767 patch 8,8: HSC-R
            hchiang2 Hsin-Fang Chiang added a comment - mosaic Current master of meas_mosaic ( f4b19d1 ) built with w_2017_26 is used. All 3 tracts completed. coadd In COSMOS tract 9813, the coadd products have 75 patches in HSC-G, 74 in HSC-R, 79 in HSC-I, 79 in HSC-Y, 79 in HSC-Z, and 76 in NB0921. In WIDE, tract=8766 misses HSC-R in two patches; tract=8767 misses HSC-R in 5 patches. Same as in DM-11020 . The empty patches of the 3 tracts are: tract 9813 patch 0,0: HSC-G,HSC-I,HSC-R,HSC-Y,HSC-Z,NB0921 tract 9813 patch 0,8: HSC-G,HSC-I,HSC-R,HSC-Y,HSC-Z,NB0921 tract 9813 patch 0,1: HSC-G, HSC-R tract 9813 patch 0,7: HSC-G, HSC-R, NB0921 tract 9813 patch 1,8: HSC-R, NB0921 tract 9813 patch 3,8: HSC-G tract 9813 patch 8,0: HSC-G, HSC-R tract 9813 patch 8,8: HSC-R, NB0921 tract 8766 patch 0,8: HSC-R tract 8766 patch 7,8: HSC-R tract 8767 patch 0,8: HSC-R tract 8767 patch 1,8: HSC-R tract 8767 patch 2,8: HSC-R tract 8767 patch 8,7: HSC-R tract 8767 patch 8,8: HSC-R
            MultiBand

            DM-11091 has fixed the multiband problem seen in DM-11020, so all patches now have multiband products: 81 patches in WIDE tracts 8766 and 8767, 79 patches in COSMOS tract 9813.

            Butler repo path

            Results are at repo /project/hsc_rc/w_2017_26/DM-11165
            Logs are at /project/hsc_rc/w_2017_26/DM-11165/logs

            hchiang2 Hsin-Fang Chiang added a comment - MultiBand DM-11091 has fixed the multiband problem seen in DM-11020 , so all patches now have multiband products: 81 patches in WIDE tracts 8766 and 8767, 79 patches in COSMOS tract 9813. Butler repo path Results are at repo /project/hsc_rc/w_2017_26/ DM-11165 Logs are at /project/hsc_rc/w_2017_26/ DM-11165 /logs

            With RFC-362 the nominal butler path is now /datasets/hsc/repo/rerun/RC/w_2017_26/DM-11165/

            With w_2017_26 the mosaic jobs had many warnings about the missed matches, e.g.

            afw.table WARN: Persisted match record with ID 111331498155159102 not found in catalog 1.
            

            This has been reported in DM-11356.

            hchiang2 Hsin-Fang Chiang added a comment - With RFC-362 the nominal butler path is now /datasets/hsc/repo/rerun/RC/w_2017_26/ DM-11165 / With w_2017_26 the mosaic jobs had many warnings about the missed matches, e.g. afw.table WARN: Persisted match record with ID 111331498155159102 not found in catalog 1 . This has been reported in DM-11356 .

            People

              hchiang2 Hsin-Fang Chiang
              hchiang2 Hsin-Fang Chiang
              Hsin-Fang Chiang
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Jenkins

                  No builds found.