Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-14988

Reprocess RC2 with w_2018_28

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Reprocess the HSC RC2 dataset, as defined in DM-11345, using Stack w_2018_28. Steps include makeSkyMap.py, singleFrameDriver.py, mosaic.py, skyCorrection.py, coaddDriver.py, and multiBandDriver.py. 

        Attachments

          Issue Links

          relates to

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-15117 processCcd passed in w_2018_26 but failed in w_2018_27 for HSC visit=36234 ccd=24

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-15119 Run pipe_analysis and validate_drp with w_2018_28 HSC RC2 outputs

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-15171 Silence warnings about aperture correction of undeblended convolved measurements

          • Done

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            hchiang2 Hsin-Fang Chiang added a comment -

            For the 432 visits of the RC2 dataset, 8 CCDs failed processCcd/singleFrameDriver, including:

            • 1 failed with "Unable to match sources" 

              --id visit=34412 ccd=78

            • 6 failed with "No matches to use for photocal" 

              --id visit=35892 ccd=12
              		 
              --id visit=36216 ccd=6
              		 
              --id visit=27068 ccd=96
              		 
              --id visit=19712 ccd=33
              		 
              --id visit=17934 ccd=1
              		 
              --id visit=23596 ccd=6

            • 1 failed with "Unable to measure aperture correction for required algorithm 'modelfit_CModel': only 1 sources, but require at least 2." 

              --id visit=36234 ccd=24

            A log including the above 8 traceback in attched in retry-failed-sfm-log-141019.txt

            Compared with w_2018_26/DM-14689, visit=36234 ccd=24 is new; it passed with w_2018_26 but failed with w_2018_27; DM-15117 has bee filed.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - For the 432 visits of the RC2 dataset, 8 CCDs failed processCcd/singleFrameDriver, including: 1 failed with "Unable to match sources" --id visit=34412 ccd=78 6 failed with "No matches to use for photocal" --id visit=35892 ccd=12 --id visit=36216 ccd=6 --id visit=27068 ccd=96 --id visit=19712 ccd=33 --id visit=17934 ccd=1 --id visit=23596 ccd=6 1 failed with "Unable to measure aperture correction for required algorithm 'modelfit_CModel': only 1 sources, but require at least 2." --id visit=36234 ccd=24 A log including the above 8 traceback in attched in retry-failed-sfm-log-141019.txt Compared with w_2018_26 / DM-14689 , visit=36234 ccd=24 is new; it passed with w_2018_26 but failed with w_2018_27; DM-15117 has bee filed.
            Hide
            Permalink
            hchiang2 Hsin-Fang Chiang added a comment -

            Also filed DM-15121, a WARN-level error that has been appearing since w_2018_18 for processCcd-ing visit=38950 ccd=64.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - Also filed DM-15121 , a WARN-level error that has been appearing since w_2018_18 for processCcd-ing visit=38950 ccd=64 .
            Hide
            Permalink
            hchiang2 Hsin-Fang Chiang added a comment -

            coadd/multiband: the number of data products and empty patches are the same as w_2018_15/DM-14123 (this comment)

            Show
            hchiang2 Hsin-Fang Chiang added a comment - coadd/multiband: the number of data products and empty patches are the same as w_2018_15/ DM-14123 ( this comment )
            Hide
            Permalink
            hchiang2 Hsin-Fang Chiang added a comment -

            The output repo is at /datasets/hsc/repo/rerun/RC/w_2018_28/DM-14988/
            The logs have been copied to /datasets/hsc/repo/rerun/RC/w_2018_28/DM-14988/logs
            The pipe_analysis and validate_drp runs are at DM-15119

            Show
            hchiang2 Hsin-Fang Chiang added a comment - The output repo is at /datasets/hsc/repo/rerun/RC/w_2018_28/ DM-14988 / The logs have been copied to /datasets/hsc/repo/rerun/RC/w_2018_28/ DM-14988 /logs The pipe_analysis and validate_drp runs are at DM-15119
            Hide
            Permalink
            hchiang2 Hsin-Fang Chiang added a comment -

            In multiband.forcedPhotCoadd, lots of new WARNs appeared in the logs, such as

            multiBandDriver.forcedPhotCoadd.applyApCorr: Cannot aperture correct undeblended_ext_convolved_ConvolvedFlux_3_kron because could not find undeblended_ext_convolved_ConvolvedFlux_3_kron_flux or undeblended_ext_convolved_ConvolvedFlux_3_kron_fluxSigma in apCorrMap
            		 
             
            		 
            multiBandDriver.forcedPhotCoadd.applyApCorr: Cannot aperture correct undeblended_ext_convolved_ConvolvedFlux_0_kron because could not find undeblended_ext_convolved_ConvolvedFlux_0_kron_flux or undeblended_ext_convolved_ConvolvedFlux_0_kron_fluxSigma in apCorrMap
            		 
             
            		 
            multiBandDriver.forcedPhotCoadd.applyApCorr: Cannot aperture correct undeblended_ext_convolved_ConvolvedFlux_1_6_0 because could not find undeblended_ext_convolved_ConvolvedFlux_1_6_0_flux or undeblended_ext_convolved_ConvolvedFlux_1_6_0_fluxSigma in apCorrMap
            		 
             
            		 
            ...

            Lauren MacArthur pointed out the newly merged DM-15045 activated these. Paul Price may know if the warnings imply problems.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - In multiband.forcedPhotCoadd , lots of new WARNs appeared in the logs, such as multiBandDriver.forcedPhotCoadd.applyApCorr: Cannot aperture correct undeblended_ext_convolved_ConvolvedFlux_3_kron because could not find undeblended_ext_convolved_ConvolvedFlux_3_kron_flux or undeblended_ext_convolved_ConvolvedFlux_3_kron_fluxSigma in apCorrMap   multiBandDriver.forcedPhotCoadd.applyApCorr: Cannot aperture correct undeblended_ext_convolved_ConvolvedFlux_0_kron because could not find undeblended_ext_convolved_ConvolvedFlux_0_kron_flux or undeblended_ext_convolved_ConvolvedFlux_0_kron_fluxSigma in apCorrMap   multiBandDriver.forcedPhotCoadd.applyApCorr: Cannot aperture correct undeblended_ext_convolved_ConvolvedFlux_1_6_0 because could not find undeblended_ext_convolved_ConvolvedFlux_1_6_0_flux or undeblended_ext_convolved_ConvolvedFlux_1_6_0_fluxSigma in apCorrMap   ... Lauren MacArthur pointed out the newly merged DM-15045 activated these. Paul Price may know if the warnings imply problems.
            Hide
            Permalink
            hchiang2 Hsin-Fang Chiang added a comment -

            DM-15171 is filed for the new WARNs in multiband.forcedPhotCoadd

            Show
            hchiang2 Hsin-Fang Chiang added a comment - DM-15171 is filed for the new WARNs in multiband.forcedPhotCoadd

              People

              Assignee:
              hchiang2 Hsin-Fang Chiang
              Reporter:
              hchiang2 Hsin-Fang Chiang
              Watchers:
              Hsin-Fang Chiang
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.