Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: None
-
Labels:None
-
Story Points:18
-
Epic Link:
-
Team:Data Release Production
-
Urgent?:No
Description
Ran RC2 with `w_2022_40` with the default and with FitAffineWcs on. Here we compare the astrometry metrics.
Attachments
Issue Links
- relates to
-
- Done
Activity
After obtaining the faroall results, I made modifications to a script originally written by Clare. The aim was to compare metrics, focusing on those crucial for astrometry, such as AM1, AM2, AD1, AD2, AF1, and AF2. I have attached the customized script and the comparison plots for tracts = [9615, 9697, 9813] with jointcal, which is the default setting in drp_pipe.
Runs without jointcal
We need to set up the environment first.
:$ SCIPIPE=/scratch/gpfs/HSC/LSST/stack/loadLSST.sh
|
:$ LSST_CONDA_ENV_NAME=lsst-scipipe-5.1.0 source "$SCIPIPE"
|
:$ setup lsst_distrib -t w_2023_09
|
lsst_distrib gdf42428520+072f704fbe w_2023_09 setup
|
:$ REPO=/projects/HSC/repo/main
|
:$ GPFSDIR=/scratch/gpfs/$USER
|
:$ LOGDIR=/home/en7908/logs
|
To rerun previous tests without using jointcal, we should navigate to the local repository for drp_pipe and modify this file:
pipelines/_ingredients/HSC/DRP.yaml
|
by replacing
$FARO_DIR/pipelines/metrics_pipeline_jointcal_fgcm.yaml
|
with
$FARO_DIR/pipelines/metrics_pipeline.yaml
|
Next, we proceeded to set up drp_pipe:
:$ setup -j -r .
|
lsst_distrib gdf42428520+6e28b0b2df w_2023_25 current w_latest setup
|
drp_pipe LOCAL:/home/en7908/repos/drp_pipe setup
|
It is worth noting that there was no need to perform a git checkout w.2022.40 within drp_pipe. Since drp_pipe doesn't contain any code, there won't be any algorithmic changes. Instead, we specifically referred to each task we intended to run, and they will utilize the w_2023_09 lsst_distrib settings that we've set up.
From here, we can proceed with our bps runs.
1. Baseline RC2 metrics run
REPO=/projects/HSC/repo/main
|
GPFSDIR=/scratch/gpfs/$USER
|
LOGDIR=/home/en7908/logs
|
|
|
cd $GPFSDIR; \
|
LOGFILE=$LOGDIR/rc2_main_metrics/rc2_main_metrics_faroall_no_jointcal.log; \
|
rm $LOGFILE; \
|
export OMP_NUM_THREADS=1; \
|
export NUMEXPR_MAX_THREADS=1; \
|
date | tee -a $LOGFILE; \
|
$(which time) -f "Total runtime: %E" \
|
bps submit /projects/HSC/LSST/bps/bps_tiger.yaml \
|
--compute-site tiger_1h_4n_10c \
|
-b $REPO \
|
-i HSC/runs/RC2/w_2022_40/DM-36151 \
|
-o u/en7908/HSC/runs/RC2/w_2022_40/DM-36151-metrics-no-jointcal \
|
-p $DRP_PIPE_DIR/pipelines/HSC/DRP-RC2.yaml#matchCatalogsTractMag17to21p5,AM1,AM2,AD1_design,AD2_design,AF1_design,AF2_design \
|
2>&1 | tee -a $LOGFILE; \
|
date | tee -a $LOGFILE
|
This time, I did not run a complete faroall job. Instead, I only specified the necessary tables that were required. The reason matchCatalogsTractMag17to21p5 is included is because it was a pre-requisite.
Using Lee's task_times, it turns out that there are some incomplete quanta:
en7908@tiger2-sumire:en7908$ ~lkelvin/software/task_times.py /scratch/gpfs/en7908/submit/u/en7908/HSC/runs/RC2/w_2022_40/DM-36151-metrics-no-jointcal/20230508T012359Z
|
|
|
Concatenating BPS log files... done!
|
|
|
QuantumGraph contains 1211 quanta for 7 tasks.
|
|
|
task time (s) pass fail skip
|
--------------------------------------------------------------------------------
|
matchCatalogsTractMag17to21p5 71195 (~68%, ~431/q) 165 7 0
|
AM2 7967 (~8%, ~50/q) 158 6 0
|
AF2_design 7686 (~7%, ~49/q) 158 6 0
|
AD2_design 7317 (~7%, ~47/q) 157 6 0
|
AM1 3594 (~3%, ~23/q) 159 6 0
|
AD1_design 3579 (~3%, ~23/q) 159 6 0
|
AF1_design 3566 (~3%, ~22/q) 159 6 0
|
--------------------------------------------------------------------------------
|
total 104904 1115 43 0
|
|
|
Executed 1115 quanta out of a total of 1211 quanta (~92%).
|
My BPS afterburners were not successful in completing those quanta. A head-node afterburner was needed:
LOGFILE=$LOGS/erfan1.log; \
|
rm $LOGFILE; \
|
DATAQUERY="band!='N921' AND skymap='hsc_rings_v1' AND tract IN (9615,9697,9813)"; \
|
pipetask --long-log --log-file $LOGFILE \
|
run -j 8 \
|
-b $REPO \
|
-i HSC/runs/RC2/w_2022_40/DM-36151,u/en7908/HSC/runs/RC2/w_2022_40/DM-36151-metrics-no-jointcal \
|
-o u/lkelvin/scratch/erfan1 \
|
-p $DRP_PIPE_DIR/pipelines/HSC/DRP-RC2.yaml#matchCatalogsTractMag17to21p5,AM1,AM2,AD1_design,AD2_design,AF1_design,AF2_design \
|
-d "instrument='HSC' AND $DATAQUERY" \
|
--skip-existing-in u/en7908/HSC/runs/RC2/w_2022_40/DM-36151-metrics-no-jointcal
|
It ended with:
Executed 11 quanta successfully, 0 failed and 0 remain out of total 11 quanta.
|
1. FitAffineWcs RC2 metrics run
REPO=/projects/HSC/repo/main
|
GPFSDIR=/scratch/gpfs/$USER
|
LOGDIR=/home/en7908/logs
|
|
|
cd $GPFSDIR; \
|
LOGFILE=$LOGDIR/rc2_affine_metrics/rc2_affine_metrics_faroall_no_jointcal.log; \
|
rm $LOGFILE; \
|
export OMP_NUM_THREADS=1; \
|
export NUMEXPR_MAX_THREADS=1; \
|
date | tee -a $LOGFILE; \
|
$(which time) -f "Total runtime: %E" \
|
bps submit /projects/HSC/LSST/bps/bps_tiger.yaml \
|
--compute-site tiger_1h_4n_10c \
|
-b $REPO \
|
-i u/en7908/HSC/runs/RC2/w_2022_40/DM-35492 \
|
-o u/en7908/HSC/runs/RC2/w_2022_40/DM-35492-metrics-no-jointcal \
|
-p $DRP_PIPE_DIR/pipelines/HSC/DRP-RC2.yaml#matchCatalogsTractMag17to21p5,AM1,AM2,AD1_design,AD2_design,AF1_design,AF2_design \
|
2>&1 | tee -a $LOGFILE; \
|
date | tee -a $LOGFILE
|
Report from task_times:
en7908@tiger2-sumire:en7908$ ~lkelvin/software/task_times.py /scratch/gpfs/en7908/submit/u/en7908/HSC/runs/RC2/w_2022_40/DM-35492-metrics-no-jointcal/20230508T013649Z
|
|
|
Concatenating BPS log files... done!
|
|
|
QuantumGraph contains 1211 quanta for 7 tasks.
|
|
|
task time (s) pass fail skip
|
--------------------------------------------------------------------------------
|
matchCatalogsTractMag17to21p5 73904 (~70%, ~453/q) 163 7 0
|
AM2 7640 (~7%, ~49/q) 157 6 0
|
AF2_design 7400 (~7%, ~47/q) 157 6 0
|
AD2_design 6620 (~6%, ~43/q) 155 6 0
|
AM1 3484 (~3%, ~22/q) 157 6 0
|
AD1_design 3341 (~3%, ~21/q) 156 6 0
|
AF1_design 3306 (~3%, ~21/q) 156 6 0
|
--------------------------------------------------------------------------------
|
total 105694 1101 43 0
|
|
|
Executed 1101 quanta out of a total of 1211 quanta (~91%).
|
A quick head-node afterburner to finish the remaining quanta:
LOGFILE=$LOGS/erfan2.log; \
|
rm $LOGFILE; \
|
DATAQUERY="band!='N921' AND skymap='hsc_rings_v1' AND tract IN (9615,9697,9813)"; \
|
pipetask --long-log --log-file $LOGFILE \
|
run -j 8 \
|
-b $REPO \
|
-i u/en7908/HSC/runs/RC2/w_2022_40/DM-35492,u/en7908/HSC/runs/RC2/w_2022_40/DM-35492-metrics-no-jointcal \
|
-o u/lkelvin/scratch/erfan2 \
|
-p $DRP_PIPE_DIR/pipelines/HSC/DRP-RC2.yaml#matchCatalogsTractMag17to21p5,AM1,AM2,AD1_design,AD2_design,AF1_design,AF2_design \
|
-d "instrument='HSC' AND $DATAQUERY" \
|
--skip-existing-in u/en7908/HSC/runs/RC2/w_2022_40/DM-35492-metrics-no-jointcal
|
It ended with:
Executed 25 quanta successfully, 0 failed and 0 remain out of total 25 quanta.
|
After obtaining the faroall results, I compared resultant astrometry metrics: AM1, AM2, AD1, AD2, AF1, and AF2. I have attached the comparison plots for tracts = [9615, 9697, 9813] without jointcal.
In the post-jointcal runs, the disparity between the Baseline and FitAffineWcs is not noticeable. However, our primary focus lies on the metrics obtained pre-jointcal, where we noticed a substantial distinction between the two.
Thank you for the repeatability plots for the pre-jointcal astrometric fits. That completes this investigation.
Although our RC2 runs are based on w_2022_40, we are utilizing w_2023_09 to obtain the metric results. I had Lee Kelvin run some of the afterburners for me simultaneously using his account on Tiger.
DM-36151: the ticket for the Baseline RC2 runDM-35492: the ticket for the FitAffineWcs RC2 runRuns with joint call
1. Baseline RC2 metrics run
:$ SCIPIPE=/scratch/gpfs/HSC/LSST/stack/loadLSST.sh:$ LSST_CONDA_ENV_NAME=lsst-scipipe-5.1.0 source "$SCIPIPE":$ setup lsst_distrib -t w_2023_09lsst_distrib gdf42428520+072f704fbe w_2023_09 setup:$ REPO=/projects/HSC/repo/main:$ GPFSDIR=/scratch/gpfs/$USER:$ LOGDIR=/home/en7908/logs:$ cd $GPFSDIR; \LOGFILE=$LOGDIR/rc2_main_metrics/rc2_main_metrics_faroall.log; \rm $LOGFILE; \export OMP_NUM_THREADS=1; \export NUMEXPR_MAX_THREADS=1; \date | tee -a $LOGFILE; \$(which time) -f "Total runtime: %E" \bps submit /projects/HSC/LSST/bps/bps_tiger.yaml \--compute-site tiger_24h_4n_40c \-b $REPO \-i HSC/runs/RC2/w_2022_40/DM-36151 \-o u/en7908/HSC/runs/RC2/w_2022_40/DM-36151-metrics \-p $DRP_PIPE_DIR/pipelines/HSC/DRP-RC2.yaml#faro_all \2>&1 | tee -a $LOGFILE; \date | tee -a $LOGFILEUsing Lee's task_times:
:$ ~lkelvin/software/task_times.py /scratch/gpfs/en7908/submit/u/en7908/HSC/runs/RC2/w_2022_40/DM-36151-metrics/20230419T205244ZConcatenating BPS log files... done!QuantumGraph contains 4306 quanta for 38 tasks.task time (s) pass fail skip--------------------------------------------------------------------------------matchCatalogsPatch 46378 (~18%, ~39/q) 1193 0 0matchCatalogsPatchMultiBand 38118 (~15%, ~159/q) 240 0 0matchCatalogsTractGxsSNR5to80 24796 (~9%, ~1653/q) 15 0 0nsrcMeasVisit 17854 (~7%, ~44/q) 404 0 0matchCatalogsTractStarsSNR5to80 17461 (~7%, ~1164/q) 15 0 0matchCatalogsTract 16770 (~6%, ~1118/q) 15 0 0TE4 15918 (~6%, ~39/q) 404 0 0matchCatalogsTractMag17to21p5 15701 (~6%, ~1047/q) 15 0 0TE3 13487 (~5%, ~33/q) 404 0 0TE1 13412 (~5%, ~894/q) 15 0 0TE2 11559 (~4%, ~771/q) 15 0 0AB1 5738 (~2%, ~5/q) 1193 0 0wPerp 4704 (~2%, ~1568/q) 3 0 0AF2_design 3236 (~1%, ~216/q) 15 0 0AD2_design 3184 (~1%, ~212/q) 15 0 0AM2 3014 (~1%, ~201/q) 15 0 0modelPhotRepGal1 1232 (~0%, ~82/q) 15 0 0AF1_design 1231 (~0%, ~82/q) 15 0 0AD1_design 1226 (~0%, ~82/q) 15 0 0AM1 1185 (~0%, ~79/q) 15 0 0modelPhotRepGal4 1044 (~0%, ~70/q) 15 0 0modelPhotRepGal3 910 (~0%, ~61/q) 15 0 0modelPhotRepGal2 896 (~0%, ~60/q) 15 0 0psfPhotRepStar1 294 (~0%, ~20/q) 15 0 0modelPhotRepStar1 261 (~0%, ~17/q) 15 0 0psfPhotRepStar2 250 (~0%, ~17/q) 15 0 0psfPhotRepStar4 236 (~0%, ~16/q) 15 0 0modelPhotRepStar3 233 (~0%, ~16/q) 15 0 0modelPhotRepStar2 219 (~0%, ~15/q) 15 0 0modelPhotRepStar4 198 (~0%, ~13/q) 15 0 0psfPhotRepStar3 186 (~0%, ~12/q) 15 0 0AD3_design 182 (~0%, ~12/q) 15 0 0AM3 179 (~0%, ~12/q) 15 0 0AF3_design 175 (~0%, ~12/q) 15 0 0PA1 91 (~0%, ~6/q) 15 0 0PF1_design 73 (~0%, ~5/q) 15 0 0skyObjectStd 23 (~0%, ~2/q) 15 0 0skyObjectMean 18 (~0%, ~1/q) 15 0 0--------------------------------------------------------------------------------total 261672 4306 0 0Executed 4306 quanta out of a total of 4306 quanta (100%).2. FitAffineWcs RC2 metrics run
:$ SCIPIPE=/scratch/gpfs/HSC/LSST/stack/loadLSST.sh:$ LSST_CONDA_ENV_NAME=lsst-scipipe-5.1.0 source "$SCIPIPE":$ setup lsst_distrib -t w_2023_09lsst_distrib gdf42428520+072f704fbe w_2023_09 setup:$ REPO=/projects/HSC/repo/main:$ GPFSDIR=/scratch/gpfs/$USER:$ LOGDIR=/home/en7908/logs:$ cd $GPFSDIR; \LOGFILE=$LOGDIR/rc2_affine_metrics/rc2_affine_metrics_faroall.log; \rm $LOGFILE; \export OMP_NUM_THREADS=1; \export NUMEXPR_MAX_THREADS=1; \date | tee -a $LOGFILE; \$(which time) -f "Total runtime: %E" \bps submit /projects/HSC/LSST/bps/bps_tiger.yaml \--compute-site tiger_24h_4n_40c \-b $REPO \-i u/en7908/HSC/runs/RC2/w_2022_40/DM-35492 \-o u/en7908/HSC/runs/RC2/w_2022_40/DM-35492-metrics \-p $DRP_PIPE_DIR/pipelines/HSC/DRP-RC2.yaml#faro_all \2>&1 | tee -a $LOGFILE; \date | tee -a $LOGFILEI ran task_times on my affine collection, it did not finish with 100% and it complained about failing in matchCatalogsTractGxsSNR5to80.
en7908@tiger2-sumire:en7908$ ~lkelvin/software/task_times.py /scratch/gpfs/en7908/submit/u/en7908/HSC/runs/RC2/w_2022_40/DM-35492-metrics/20230420T211844ZConcatenating BPS log files... done!QuantumGraph contains 4222 quanta for 38 tasks.task time (s) pass fail skip--------------------------------------------------------------------------------matchCatalogsPatchMultiBand 470792 (~35%, ~2140/q) 220 0 0matchCatalogsPatch 334855 (~25%, ~281/q) 1193 0 0nsrcMeasVisit 127741 (~9%, ~340/q) 376 0 0TE3 127346 (~9%, ~339/q) 376 0 0TE4 109457 (~8%, ~291/q) 376 0 0TE1 45595 (~3%, ~3507/q) 13 0 0TE2 38367 (~3%, ~2740/q) 14 0 0matchCatalogsTractGxsSNR5to80 29628 (~2%, ~3703/q) 8 1 0matchCatalogsTractStarsSNR5to80 16570 (~1%, ~2367/q) 7 0 0wPerp 14661 (~1%, ~4887/q) 3 0 0matchCatalogsTract 13109 (~1%, ~2622/q) 5 0 0AB1 9629 (~1%, ~9/q) 1084 0 0matchCatalogsTractMag17to21p5 5540 (~0%, ~2770/q) 2 0 0modelPhotRepGal2 599 (~0%, ~75/q) 8 0 0modelPhotRepGal1 512 (~0%, ~64/q) 8 0 0modelPhotRepGal4 487 (~0%, ~61/q) 8 0 0modelPhotRepGal3 450 (~0%, ~56/q) 8 0 0AD2_design 346 (~0%, ~173/q) 2 0 0AM2 335 (~0%, ~168/q) 2 0 0AF2_design 331 (~0%, ~166/q) 2 0 0AF1_design 163 (~0%, ~82/q) 2 0 0skyObjectStd 150 (~0%, ~10/q) 15 0 0AD1_design 147 (~0%, ~73/q) 2 0 0AM1 138 (~0%, ~69/q) 2 0 0psfPhotRepStar1 119 (~0%, ~17/q) 7 0 0modelPhotRepStar1 110 (~0%, ~16/q) 7 0 0modelPhotRepStar2 107 (~0%, ~15/q) 7 0 0psfPhotRepStar2 106 (~0%, ~15/q) 7 0 0skyObjectMean 96 (~0%, ~6/q) 15 0 0modelPhotRepStar3 93 (~0%, ~13/q) 7 0 0psfPhotRepStar3 92 (~0%, ~13/q) 7 0 0modelPhotRepStar4 91 (~0%, ~13/q) 7 0 0psfPhotRepStar4 89 (~0%, ~13/q) 7 0 0PF1_design 31 (~0%, ~6/q) 5 0 0AF3_design 29 (~0%, ~14/q) 2 0 0PA1 29 (~0%, ~6/q) 5 0 0AM3 27 (~0%, ~14/q) 2 0 0AD3_design 26 (~0%, ~13/q) 2 0 0--------------------------------------------------------------------------------total 1347989 3823 1 0Executed 3823 quanta out of a total of 4222 quanta (~91%).Afterburner using BPS:
REPO=/projects/HSC/repo/mainGPFSDIR=/scratch/gpfs/$USERLOGDIR=$LOGSLSST_CONDA_ENV_NAME=lsst-scipipe-5.1.0source /scratch/gpfs/HSC/LSST/stack/loadLSST.shsetup -t w_2023_09 lsst_distribcd $GPFSDIR; \LOGFILE=$LOGDIR/faro_all.log; \rm $LOGFILE; \export OMP_NUM_THREADS=1; \export NUMEXPR_MAX_THREADS=1; \date | tee -a $LOGFILE; \$(which time) -f "Total runtime: %E" \bps submit /projects/HSC/LSST/bps/bps_tiger.yaml \--compute-site tiger_1h_4n_10c \-b $REPO \-i u/en7908/HSC/runs/RC2/w_2022_40/DM-35492,u/en7908/HSC/runs/RC2/w_2022_40/DM-35492-metrics \-o u/lskelvin/en7908/faro_all \-p $DRP_PIPE_DIR/pipelines/HSC/DRP-RC2.yaml#faro_all \--extra-qgraph-options "--skip-existing-in u/en7908/HSC/runs/RC2/w_2022_40/DM-35492-metrics --clobber-outputs" \2>&1 | tee -a $LOGFILE; \date | tee -a $LOGFILEAll but 6 quanta succeeded:
en7908@tiger2-sumire:en7908$ /home/lkelvin/software/task_check.py $REPO u/lskelvin/en7908/faro_all/20230427T143610Z -q u_lskelvin_en7908_faro_all_20230427T143610Z.qgraphtasks expected completed missing------------------------------- -------- ------------ -------matchCatalogsTract 10 10 (100.0%) 0matchCatalogsTractMag17to21p5 13 13 (100.0%) 0matchCatalogsTractGxsSNR5to80 7 6 (~85.7%) 1matchCatalogsPatchMultiBand 20 20 (100.0%) 0matchCatalogsTractStarsSNR5to80 8 8 (100.0%) 0TE1 2 2 (100.0%) 0TE2 1 1 (100.0%) 0PF1_design 10 10 (100.0%) 0PA1 10 10 (100.0%) 0AF3_design 13 13 (100.0%) 0AD1_design 13 13 (100.0%) 0AF1_design 13 13 (100.0%) 0AD2_design 13 13 (100.0%) 0AM2 13 12 (~92.3%) 1AD3_design 13 13 (100.0%) 0AF2_design 13 13 (100.0%) 0AM3 13 13 (100.0%) 0AM1 13 13 (100.0%) 0modelPhotRepGal1 7 6 (~85.7%) 1modelPhotRepGal3 7 6 (~85.7%) 1modelPhotRepGal4 7 6 (~85.7%) 1modelPhotRepGal2 7 6 (~85.7%) 1AB1 109 109 (100.0%) 0modelPhotRepStar2 8 8 (100.0%) 0psfPhotRepStar3 8 8 (100.0%) 0modelPhotRepStar4 8 8 (100.0%) 0modelPhotRepStar1 8 8 (100.0%) 0psfPhotRepStar2 8 8 (100.0%) 0psfPhotRepStar4 8 8 (100.0%) 0modelPhotRepStar3 8 8 (100.0%) 0psfPhotRepStar1 8 8 (100.0%) 0TOTAL 399 393 (~98.5%) 6A quick follow-up head-node run to run those final 6 failing quanta:
pipetask --long-log --log-file $LOGDIR/faro_all_afterburner.log run \-b $REPO \-i u/en7908/HSC/runs/RC2/w_2022_40/DM-35492,u/en7908/HSC/runs/RC2/w_2022_40/DM-35492-metrics \-o u/lskelvin/en7908/faro_all \-p $DRP_PIPE_DIR/pipelines/HSC/DRP-RC2.yaml#faro_all \--skip-existing-in u/en7908/HSC/runs/RC2/w_2022_40/DM-35492-metrics \--skip-existing-in u/lskelvin/en7908/faro_all \--clobber-outputsIt ended with:
INFO 2023-04-27T17:33:57.005-04:00 lsst.ctrl.mpexec.mpGraphExecutor ()(mpGraphExecutor.py:508) - Executed 6 quanta successfully, 0 failed and 0 remain out of total 6 quanta.