# coaddDriver of the HSC PDR1 dataset

XMLWordPrintable

#### Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s: None
• Labels:
None
• Story Points:
4
• Team:
Data Facility

#### Description

coaddDriver.py of the entire HSC PDR1 dataset with w_2018_15

#### Activity

No builds found.
Hsin-Fang Chiang created issue -
Field Original Value New Value
Epic Link DM-13926 [ 39699 ]
 Risk Score 0
 Status To Do [ 10001 ] In Progress [ 3 ]
Hide
Hsin-Fang Chiang added a comment -

The execution unit of coaddDriver was the same as mosaic (DM-14200). It was run for each tract x filter combination, using all visits overlapping that tract in that filter. Each layer was done separately. There were 69 slurm jobs in UDEEP, 218 in DEEP, and 455 in WIDE (same as in DM-14200).

Show
Hsin-Fang Chiang added a comment - The execution unit of coaddDriver was the same as mosaic ( DM-14200 ). It was run for each tract x filter combination, using all visits overlapping that tract in that filter. Each layer was done separately. There were 69 slurm jobs in UDEEP, 218 in DEEP, and 455 in WIDE (same as in DM-14200 ).
 Link This issue relates to DM-14282 [ DM-14282 ]
 Link This issue relates to DM-14286 [ DM-14286 ]
 Link This issue relates to DM-14303 [ DM-14303 ]
Hide
Hsin-Fang Chiang added a comment -

The 1484 log files from the 742 jobs have been copied to

 /datasets/hsc/repo/rerun/DM-13666/UDEEP/logs/coadd/ /datasets/hsc/repo/rerun/DM-13666/DEEP/logs/coadd/ /datasets/hsc/repo/rerun/DM-13666/WIDE/logs/coadd/ 

There are no FATAL- or ERROR-level log records found in the log files. Many WARN-level records appeared. I've filed 3 new tickets for some new non-fatal errors I noticed and the Science Pipeline team might wish to investigate more and fix. I think all cases are edge patches with little coverage, but did not check all images one by one. The three tickets are:

• DM-14282: IndexError: cannot do a non-empty take from an empty axes. at detectCoaddSources.runDetection in 8 patches, 2 in UDEEP, 5 in DEEP, and 1 in WIDE.
• DM-14286: RuntimeError: No PsfMatched warps were found to build the template coadd which is required to run CompareWarpAssembleCoaddTask. while there are 0 good pixels for psfMatchedWarp so no psfMatchedWarp files were written. These happened in 5 patches, 4 in DEEP and 1 in WIDE.
• DM-14303: RuntimeError: Variance rescaling factor (20.491061) exceeds configured limit (10.000000) in one edge patch in WIDE.
Show
Hsin-Fang Chiang added a comment - The 1484 log files from the 742 jobs have been copied to /datasets/hsc/repo/rerun/DM-13666/UDEEP/logs/coadd/ /datasets/hsc/repo/rerun/DM-13666/DEEP/logs/coadd/ /datasets/hsc/repo/rerun/DM-13666/WIDE/logs/coadd/ There are no FATAL- or ERROR-level log records found in the log files. Many WARN-level records appeared. I've filed 3 new tickets for some new non-fatal errors I noticed and the Science Pipeline team might wish to investigate more and fix. I think all cases are edge patches with little coverage, but did not check all images one by one. The three tickets are: DM-14282 : IndexError: cannot do a non-empty take from an empty axes. at detectCoaddSources.runDetection in 8 patches, 2 in UDEEP, 5 in DEEP, and 1 in WIDE. DM-14286 : RuntimeError: No PsfMatched warps were found to build the template coadd which is required to run CompareWarpAssembleCoaddTask. while there are 0 good pixels for psfMatchedWarp so no psfMatchedWarp files were written. These happened in 5 patches, 4 in DEEP and 1 in WIDE. DM-14303 : RuntimeError: Variance rescaling factor (20.491061) exceeds configured limit (10.000000) in one edge patch in WIDE.
Hide
Hsin-Fang Chiang added a comment -

The data IDs to reproduce the above three cases can be found in the tickets.

Show
Hsin-Fang Chiang added a comment - The data IDs to reproduce the above three cases can be found in the tickets.
 Story Points 4
 Resolution Done [ 10000 ] Status In Progress [ 3 ] Done [ 10002 ]

#### People

Assignee:
Hsin-Fang Chiang
Reporter:
Hsin-Fang Chiang
Watchers:
Hsin-Fang Chiang