Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: Validation
-
Labels:None
-
Story Points:3
-
Epic Link:
-
Team:SQuaRE
Description
Re-interpreted ticket:
1. Provide already-initialized repositories in the `validation_data_cfht`, `validation_data_decam`, and `validation_data_hsc` packages alongside the raw data. The goal is to allow both easy quick-start analyses as well as comparisons of output steps from processCcd.py and friends at each step of the processing. 
2. Add (Cfht,Decam,HSC).list files to provide for easy processing of the available dataIds in the example data.
3. Update README files to explain available data.
[Original request:]
In validation_drp when I run examples/runXTest.sh I find that any data I had saved in CFHT or DECam is lost, even if I have carefully renamed it. This is very dangerous and I lost a lot of work due to it. At a bare minimum please do NOT touch any directories not named "input" or "output".
Lower priority requests that I hope you will consider:
- Have the the input repo be entirely contained in the validation_data_X packages, ready to use "as is". That would simplify the use of those packages by other code. It would also simplify validate_drp, and it would just leave the output repo to generate (which already has a link back to the input repo).
- Have runXTest.sh accept a single argument: the path to the output. (The path to the input is not necessary if you implement the first suggestion).
Attachments
Activity
lsst_qa will be the night/weekly/monthly package and is under development as DM-5381. It will exercise each of these validation_data_* packages through ci_hsc and related scripts to be developed for cfht and decam.
And yes, not automatically, but on a regular, and curated, basis, the output repos in these validation_data_* repos will be updated based on the output of lsst_qa.
Michael Wood-Vasey
added a comment - lsst_qa will be the night/weekly/monthly package and is under development as DM-5381 . It will exercise each of these validation_data_* packages through ci_hsc and related scripts to be developed for cfht and decam .
And yes, not automatically, but on a regular, and curated, basis, the output repos in these validation_data_* repos will be updated based on the output of lsst_qa . This all sounds good to me, and the structure looks fine, with the caveat that the procedure for updating these be worked out ahead of time, and documented. Preferably in a way that a human will not have to have too much input. Hopefully this will not be a chore for anyone.
Recreating/updating instructions are in the respective README.md files under the Recreating the repository section. It's not meant to completely trivial as my vision still involves a human thinking about any updates to the results and making sure they are understood. It's possible that creating an updating script might reduce the chances of mistakes when doing such updates – future experience will inform this.
Michael Wood-Vasey
added a comment - Recreating/updating instructions are in the respective README.md files under the Recreating the repository section. It's not meant to completely trivial as my vision still involves a human thinking about any updates to the results and making sure they are understood. It's possible that creating an updating script might reduce the chances of mistakes when doing such updates – future experience will inform this. Merged to master.
Michael Wood-Vasey
added a comment - Merged to master. 
if there will be a nightly/weekly/monthly build running (ci_hsc descendent) Would it make sense to have that get piped into these to be checked out?
And no there was no indication that there was an issue with the astrometry, I just didn't have a great way to check for the decam and cfht data sets, and wanted to make sure that it was thought about, and that it was the minimum amount necessary for the repository.