# Add a CFHT-based post-build integration test to the sandbox build

XMLWordPrintable

#### Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Story Points:
1
• Team:
SQuaRE

#### Description

I have installed some simple stack validation tools working on CFHT data in /lsst8/boutigny/valid_cfht

Here is the content of the README file :

------------------------------------------------------------------------------------------------------------------------
This directory contains a set of utilities to validate a stack release with CFHT data

At the moment, only validation plots for the astrometry are produced

Directories :
-------------
reference_plots : contain reference plots corresponding to the best results obtain so far.

Files :
-------
setup.cfht : stack environment setup
valid_cfht.sh : run processCcd taks on the cfht images
valid_cfht.sh init : create the input/ouput directories, ingest raw images and run processCcd
valid_cfht.sh : without the "init" argument, runs processCcd assuming that the directory structure exists and that the raw images have been ingested.
valid_cfht.py : run some analysis on the output data produced by valid_cfht.sh
processConfig.py : configuration parameters for processCcd
run.list : list of vistits / ccd to be processed by processCcd

Requirements :
--------------
obs_cfht : tickets/DM-1593
astrometry_net_data : SDSS_DR9 reference catalog corresponding for CFHT Deep Field #3
------------------------------------------------------------------------------------------------------------------------

Basically it produces a set of plots stored in a png image that can be compared to a reference plot corresponding to the best results obtained so far with stack_v10_0

I hope that this is useful. Just be careful that I wrote these scripts with my own "fat hand full of fingers" and that it is just basic code from a non expert. If it is useful, I can certainly add more plots to validate the psf determination, photometry, etc.

Comments, suggestions and criticisms are very welcome.

#### Activity

Hide
Frossie Economou added a comment -

Josh, can you give this a try in a sandbox? (We don't have to run it automatically at this point, just understand the prereqs)

Show
Frossie Economou added a comment - Josh, can you give this a try in a sandbox? (We don't have to run it automatically at this point, just understand the prereqs)
Hide
Tim Jenness added a comment -

This is the output of the valid_cfht.py script:

 CameraMapper: Loading registry registry from /home/lsstsw/tmp/valid_cfht/output/_parent/registry.sqlite3 CameraMapper WARNING: Unable to locate calibRegistry registry in root: /home/lsstsw/tmp/valid_cfht/output/calibRegistry.sqlite3 CameraMapper WARNING: Unable to locate calibRegistry registry in current dir: ./calibRegistry.sqlite3 CameraMapper WARNING: No registry loaded; proceeding without one 1067 sources in ccd : 12 1082 sources in ccd : 13 1046 sources in ccd : 14 1100 sources in ccd : 21 1107 sources in ccd : 22 1120 sources in ccd : 23 6522 Sources in reference visit : 849375 1594 sources in ccd : 12 3268 sources in ccd : 13 5020 sources in ccd : 14 6733 sources in ccd : 21 8541 sources in ccd : 22 10341 sources in ccd : 23 Visit : 850587 5609 matches found Median value of the astrometric scatter - all magnitudes: 106.116602143 mas Astrometric scatter (median) - mag < 21 : 36.4968012258 mas

As well as plots which are used for manual inspection. I assume we can just take the final two numbers as a measure for how well we are doing (and put them on a graphite display in the SQuaRE corridor).

Show
Tim Jenness added a comment - This is the output of the valid_cfht.py script: CameraMapper: Loading registry registry from /home/lsstsw/tmp/valid_cfht/output/_parent/registry.sqlite3 CameraMapper WARNING: Unable to locate calibRegistry registry in root: /home/lsstsw/tmp/valid_cfht/output/calibRegistry.sqlite3 CameraMapper WARNING: Unable to locate calibRegistry registry in current dir: ./calibRegistry.sqlite3 CameraMapper WARNING: No registry loaded; proceeding without one 1067 sources in ccd : 12 1082 sources in ccd : 13 1046 sources in ccd : 14 1100 sources in ccd : 21 1107 sources in ccd : 22 1120 sources in ccd : 23 6522 Sources in reference visit : 849375 1594 sources in ccd : 12 3268 sources in ccd : 13 5020 sources in ccd : 14 6733 sources in ccd : 21 8541 sources in ccd : 22 10341 sources in ccd : 23 Visit : 850587 5609 matches found Median value of the astrometric scatter - all magnitudes: 106.116602143 mas Astrometric scatter (median) - mag < 21 : 36.4968012258 mas As well as plots which are used for manual inspection. I assume we can just take the final two numbers as a measure for how well we are doing (and put them on a graphite display in the SQuaRE corridor).
Hide
Dominique Boutigny added a comment -

I have written a simple test for obs_cfht based on what I did to check the astrometry task. It is at NCSA in /lsst8/boutigny/valid_cfht. There is one single script : run_test.sh which is setting up the directory structure, ingest raw CFHT images, run processCcd and run a validation python script. It exits with error if the astrometric scatter is too large or if the number of matched sources is too small.
It runs in 1.5 minutes at NCSA. I tested it against b1545 stack release and the current obs_cfht from the current master branch.

Show
Dominique Boutigny added a comment - I have written a simple test for obs_cfht based on what I did to check the astrometry task. It is at NCSA in /lsst8/boutigny/valid_cfht. There is one single script : run_test.sh which is setting up the directory structure, ingest raw CFHT images, run processCcd and run a validation python script. It exits with error if the astrometric scatter is too large or if the number of matched sources is too small. It runs in 1.5 minutes at NCSA. I tested it against b1545 stack release and the current obs_cfht from the current master branch.
Hide
Michael Wood-Vasey added a comment -

https://github.com/lsst/validate_drp

now provides the basis for satisfying this ticket. The key last step is including validate_drp in "the sandbox build."

I think we need to plan out
1. What "the sandbox build" look like.

• Right now you can just following the instructions in the README.md at https://github.com/lsst/validate_drp and you will get some output including an astrometric median scatter and a plot.
• How do we connect this to a test harness?
Show
Michael Wood-Vasey added a comment - https://github.com/lsst/validate_drp now provides the basis for satisfying this ticket. The key last step is including validate_drp in "the sandbox build." I think we need to plan out 1. What "the sandbox build" look like. Right now you can just following the instructions in the README.md at https://github.com/lsst/validate_drp and you will get some output including an astrometric median scatter and a plot. How do we connect this to a test harness?
Hide
Michael Wood-Vasey added a comment -

1. validate_drp exists and contains an example test on CFHT data that does this astrometry test in this Story and also additional metrics from the LSST Science Requirements Document.

2. lsst_ci now runs obs_cfht with its testdata_cfht.

3. The gist of this story is complete. Related followup will happen under lsst_qa under DM-5381

Show
Michael Wood-Vasey added a comment - 1. validate_drp exists and contains an example test on CFHT data that does this astrometry test in this Story and also additional metrics from the LSST Science Requirements Document. 2. lsst_ci now runs obs_cfht with its testdata_cfht . 3. The gist of this story is complete. Related followup will happen under lsst_qa under DM-5381

#### People

Assignee:
Michael Wood-Vasey
Reporter:
Frossie Economou
Watchers:
David Nidever [X] (Inactive), Dominique Boutigny, Frossie Economou, Michael Wood-Vasey, Tim Jenness