Details
-
Type:
Bug
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: jenkins
-
Labels:None
-
Story Points:1
-
Epic Link:
-
Sprint:AP S21-4 (March)
-
Team:Alert Production
-
Urgent?:No
Description
Following the resolution of DM-29004, gen3_to_job.py runs in ap_verify Gen 3 runs, but no Job files get detected and uploaded. This could be either because gen3_to_job.py does not generate any files, or because they do not get picked up by the upload selector.
The command line invoked was:
gen3_to_job.py '/j/ws/scipipe/ap_verify/cosmos_pdr2-master^gen3^ap_verify-installed/run/CI-CosmosPDR2/repo' ap_verify-output --dataset_name CI-CosmosPDR2
|
I've confirmed on lsst-dev that this syntax (absolute path, single quotes, no trailing slash, etc.) should work, producing .verify.json files in the working directory. I've also confirmed that the repository location was cosmos_pdr2-master^gen3^ap_verify-installed/run/CI-CosmosPDR2/repo/, so the absolute version of the path is probably correct.
Attachments
Issue Links
- relates to
-
- Done
Activity
Kian-Tat Lim, could you review the (merged) jenkins-dm-jobs#770 a posteriori? Happy to make any needed changes on a "b" branch.
This should the last review I ask you for, since I've confirmed that we now have Gen 3 metrics in SQuaSH. 
Krzysztof Findeisen
added a comment - Kian-Tat Lim , could you review the (merged) jenkins-dm-jobs#770 a posteriori? Happy to make any needed changes on a "b" branch.
This should the last review I ask you for, since I've confirmed that we now have Gen 3 metrics in SQuaSH. Looks fine, no objections.
I think I found the problem: the files were created in /j/ws/scipipe/ap_verify (a.k.a. jobDir in the script), but the search for the files was relative to /j/ws/scipipe/ap_verify/cosmos_pdr2-master^gen3^ap_verify-installed/run/ (a.k.a. runDir). I need to be calling the script from runDir as well.
I think this means I need to modify util.groovy:runGen3ToJob to take separate arguments for the working directory and the Gen 3 repo. It's pointlessly redundant, but it's what's consistent with other functions (e.g., runApVerify, runDispatchVerify), and the only other way to handle the PWD internally is to let the output go into the repo directory.