Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: QA
-
Labels:None
-
Story Points:1
-
Epic Link:
-
Team:SQuaRE
Description
From Dominique Boutigny
I have installed some simple stack validation tools working on CFHT data in /lsst8/boutigny/valid_cfht
Here is the content of the README file :
------------------------------------------------------------------------------------------------------------------------
This directory contains a set of utilities to validate a stack release with CFHT data
At the moment, only validation plots for the astrometry are produced
Directories :
-------------
rawDownload : contain raw CFHT images (flat, dark, bias, fringe,... corrected)
reference_plots : contain reference plots corresponding to the best results obtain so far.
Files :
-------
setup.cfht : stack environment setup
valid_cfht.sh : run processCcd taks on the cfht images
valid_cfht.sh init : create the input/ouput directories, ingest raw images and run processCcd
valid_cfht.sh : without the "init" argument, runs processCcd assuming that the directory structure exists and that the raw images have been ingested.
valid_cfht.py : run some analysis on the output data produced by valid_cfht.sh
processConfig.py : configuration parameters for processCcd
run.list : list of vistits / ccd to be processed by processCcd
Requirements :
--------------
obs_cfht : tickets/DM-1593
astrometry_net_data : SDSS_DR9 reference catalog corresponding for CFHT Deep Field #3
------------------------------------------------------------------------------------------------------------------------
Basically it produces a set of plots stored in a png image that can be compared to a reference plot corresponding to the best results obtained so far with stack_v10_0
I hope that this is useful. Just be careful that I wrote these scripts with my own "fat hand full of fingers" and that it is just basic code from a non expert. If it is useful, I can certainly add more plots to validate the psf determination, photometry, etc.
Comments, suggestions and criticisms are very welcome.
Attachments
Issue Links
- is blocked by
-
- Done
-
- Done
-
- Done
-
- Done
Activity
This is the output of the valid_cfht.py script:
CameraMapper: Loading registry registry from /home/lsstsw/tmp/valid_cfht/output/_parent/registry.sqlite3
|
CameraMapper WARNING: Unable to locate calibRegistry registry in root: /home/lsstsw/tmp/valid_cfht/output/calibRegistry.sqlite3
|
CameraMapper WARNING: Unable to locate calibRegistry registry in current dir: ./calibRegistry.sqlite3
|
CameraMapper WARNING: No registry loaded; proceeding without one
|
1067 sources in ccd : 12
|
1082 sources in ccd : 13
|
1046 sources in ccd : 14
|
1100 sources in ccd : 21
|
1107 sources in ccd : 22
|
1120 sources in ccd : 23
|
6522 Sources in reference visit : 849375
|
1594 sources in ccd : 12
|
3268 sources in ccd : 13
|
5020 sources in ccd : 14
|
6733 sources in ccd : 21
|
8541 sources in ccd : 22
|
10341 sources in ccd : 23
|
Visit : 850587 5609 matches found
|
Median value of the astrometric scatter - all magnitudes: 106.116602143 mas
|
Astrometric scatter (median) - mag < 21 : 36.4968012258 mas
|
As well as plots which are used for manual inspection. I assume we can just take the final two numbers as a measure for how well we are doing (and put them on a graphite display in the SQuaRE corridor).
I have written a simple test for obs_cfht based on what I did to check the astrometry task. It is at NCSA in /lsst8/boutigny/valid_cfht. There is one single script : run_test.sh which is setting up the directory structure, ingest raw CFHT images, run processCcd and run a validation python script. It exits with error if the astrometric scatter is too large or if the number of matched sources is too small.
It runs in 1.5 minutes at NCSA. I tested it against b1545 stack release and the current obs_cfht from the current master branch.
Dominique Boutigny
added a comment - I have written a simple test for obs_cfht based on what I did to check the astrometry task. It is at NCSA in /lsst8/boutigny/valid_cfht. There is one single script : run_test.sh which is setting up the directory structure, ingest raw CFHT images, run processCcd and run a validation python script. It exits with error if the astrometric scatter is too large or if the number of matched sources is too small.
It runs in 1.5 minutes at NCSA. I tested it against b1545 stack release and the current obs_cfht from the current master branch. https://github.com/lsst/validate_drp
now provides the basis for satisfying this ticket. The key last step is including `validate_drp` in "the sandbox build."
I think we need to plan out
1. What "the sandbox build" look like.
- Right now you can just following the instructions in the README.md at https://github.com/lsst/validate_drp and you will get some output including an astrometric median scatter and a plot.
- How do we connect this to a test harness?
Michael Wood-Vasey
added a comment - https://github.com/lsst/validate_drp
now provides the basis for satisfying this ticket. The key last step is including `validate_drp` in "the sandbox build."
I think we need to plan out
1. What "the sandbox build" look like.
Right now you can just following the instructions in the README.md at https://github.com/lsst/validate_drp and you will get some output including an astrometric median scatter and a plot.
How do we connect this to a test harness?
1. validate_drp exists and contains an example test on CFHT data that does this astrometry test in this Story and also additional metrics from the LSST Science Requirements Document.
2. lsst_ci now runs obs_cfht with its testdata_cfht.
3. The gist of this story is complete. Related followup will happen under lsst_qa under DM-5381
Michael Wood-Vasey
added a comment - 1. validate_drp exists and contains an example test on CFHT data that does this astrometry test in this Story and also additional metrics from the LSST Science Requirements Document.
2. lsst_ci now runs obs_cfht with its testdata_cfht .
3. The gist of this story is complete. Related followup will happen under lsst_qa under DM-5381 
Josh, can you give this a try in a sandbox? (We don't have to run it automatically at this point, just understand the prereqs)