Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: None
-
Labels:None
-
Story Points:1
-
Epic Link:
-
Team:Data Facility
Description
Run pipe_analysis scripts
- visitAnalysis
- coaddAnalysis
- colorAnalysis
- compareVisitAnalysis
- compareCoaddAnalysis
and validate_drp
- matchedVisitMetrics.py
- validateDrp.py
- reportPerformance.py
Upload the validate_drp metrics (json files) to SQuaSH:
- dispatch_verify.py
with the w_2018_30 RC2 outputs from DM-14988
Attachments
Issue Links
- relates to
-
- Done
Activity
Like before, filter=NB0921 is ignored for validateDrp.py due to DM-15127.
Hsin-Fang Chiang
added a comment - Like before, filter=NB0921 is ignored for validateDrp.py due to DM-15127 . pipe_analysis logs are at
/datasets/hsc/repo/rerun/RC/w_2018_30/DM-15120/qaLogs/
validate_drp results and logs are at
/datasets/hsc/repo/rerun/RC/w_2018_30/DM-15120/validateDrp/
They can also be accessed via
https://lsst-web.ncsa.illinois.edu/~hchiang2/RC2_w_2018_30/
Hsin-Fang Chiang
added a comment - pipe_analysis logs are at
/datasets/hsc/repo/rerun/RC/w_2018_30/ DM-15120 /qaLogs/
validate_drp results and logs are at
/datasets/hsc/repo/rerun/RC/w_2018_30/ DM-15120 /validateDrp/
They can also be accessed via
https://lsst-web.ncsa.illinois.edu/~hchiang2/RC2_w_2018_30/
validate_drp results of all 6 filters in the COSMOS tract were uploaded to SQuaSH (dataset = HSC RC2) at https://squash.lsst.codes/ on Aug 3.
To do so using dispatch_verify.py, user environment variables need to be set (e.g. notes on https://confluence.lsstcorp.org/display/~hchiang2/validateDrp#validateDrp-Sendverify.JobJSONfilestoSQuaSH )
I mistakenly used the old and wrong setup (VERSION_TAG=w_2018_28, RUN_ID, RUN_ID_URL), rather than the w_2018_30 ticket. However, they don't seem to matter as displayed on https://squash.lsst.codes/dash/code_changes/ (those detailed information seem not shown at this moment) so I'm not re-uploading for now.
Hsin-Fang Chiang
added a comment - validate_drp results of all 6 filters in the COSMOS tract were uploaded to SQuaSH (dataset = HSC RC2) at https://squash.lsst.codes/ on Aug 3.
To do so using dispatch_verify.py , user environment variables need to be set (e.g. notes on https://confluence.lsstcorp.org/display/~hchiang2/validateDrp#validateDrp-Sendverify.JobJSONfilestoSQuaSH )
I mistakenly used the old and wrong setup (VERSION_TAG=w_2018_28, RUN_ID, RUN_ID_URL), rather than the w_2018_30 ticket. However, they don't seem to matter as displayed on https://squash.lsst.codes/dash/code_changes/ (those detailed information seem not shown at this moment) so I'm not re-uploading for now. From Slack conversations, Angelo Fausti pointed out that the information is stored in the SQuaSH database as environment metadata.
Currently, unlike Angelo Fausti, I do not have the admin access to the SQuaSH database for correcting the records. When the time comes we will discuss database access and related topics.
Hsin-Fang Chiang
added a comment - From Slack conversations, Angelo Fausti pointed out that the information is stored in the SQuaSH database as environment metadata.
Currently, unlike Angelo Fausti , I do not have the admin access to the SQuaSH database for correcting the records. When the time comes we will discuss database access and related topics. Angelo Fausti has kindly corrected the records in the SQuaSH database in DM-15407.
Hsin-Fang Chiang
added a comment - Angelo Fausti has kindly corrected the records in the SQuaSH database in DM-15407 . 
The version of pipe_analysis was master at the time of the w_2018_30 release, at commit 2f72a4e.
The comparison scripts were run against RC/w_2018_28/
DM-14988