Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-2518

Add a CFHT-based post-build integration test to the sandbox build

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: QA
    • Labels:
      None

      Description

      From Dominique Boutigny

      I have installed some simple stack validation tools working on CFHT data in /lsst8/boutigny/valid_cfht

      Here is the content of the README file :

      ------------------------------------------------------------------------------------------------------------------------
      This directory contains a set of utilities to validate a stack release with CFHT data

      At the moment, only validation plots for the astrometry are produced

      Directories :
      -------------
      rawDownload : contain raw CFHT images (flat, dark, bias, fringe,... corrected)
      reference_plots : contain reference plots corresponding to the best results obtain so far.

      Files :
      -------
      setup.cfht : stack environment setup
      valid_cfht.sh : run processCcd taks on the cfht images
      valid_cfht.sh init : create the input/ouput directories, ingest raw images and run processCcd
      valid_cfht.sh : without the "init" argument, runs processCcd assuming that the directory structure exists and that the raw images have been ingested.
      valid_cfht.py : run some analysis on the output data produced by valid_cfht.sh
      processConfig.py : configuration parameters for processCcd
      run.list : list of vistits / ccd to be processed by processCcd

      Requirements :
      --------------
      obs_cfht : tickets/DM-1593
      astrometry_net_data : SDSS_DR9 reference catalog corresponding for CFHT Deep Field #3
      ------------------------------------------------------------------------------------------------------------------------

      Basically it produces a set of plots stored in a png image that can be compared to a reference plot corresponding to the best results obtained so far with stack_v10_0

      I hope that this is useful. Just be careful that I wrote these scripts with my own "fat hand full of fingers" and that it is just basic code from a non expert. If it is useful, I can certainly add more plots to validate the psf determination, photometry, etc.

      Comments, suggestions and criticisms are very welcome.

        Attachments

          Issue Links

            Activity

            Hide
            frossie Frossie Economou added a comment -

            Josh, can you give this a try in a sandbox? (We don't have to run it automatically at this point, just understand the prereqs)

            Show
            frossie Frossie Economou added a comment - Josh, can you give this a try in a sandbox? (We don't have to run it automatically at this point, just understand the prereqs)
            Hide
            tjenness Tim Jenness added a comment -

            This is the output of the valid_cfht.py script:

            CameraMapper: Loading registry registry from /home/lsstsw/tmp/valid_cfht/output/_parent/registry.sqlite3
            CameraMapper WARNING: Unable to locate calibRegistry registry in root: /home/lsstsw/tmp/valid_cfht/output/calibRegistry.sqlite3
            CameraMapper WARNING: Unable to locate calibRegistry registry in current dir: ./calibRegistry.sqlite3
            CameraMapper WARNING: No registry loaded; proceeding without one
            1067 sources in ccd : 12
            1082 sources in ccd : 13
            1046 sources in ccd : 14
            1100 sources in ccd : 21
            1107 sources in ccd : 22
            1120 sources in ccd : 23
            6522 Sources in reference visit : 849375
            1594 sources in ccd :  12
            3268 sources in ccd :  13
            5020 sources in ccd :  14
            6733 sources in ccd :  21
            8541 sources in ccd :  22
            10341 sources in ccd :  23
            Visit : 850587 5609 matches found
            Median value of the astrometric scatter - all magnitudes: 106.116602143 mas
            Astrometric scatter (median) - mag < 21 : 36.4968012258 mas

            As well as plots which are used for manual inspection. I assume we can just take the final two numbers as a measure for how well we are doing (and put them on a graphite display in the SQuaRE corridor).

            Show
            tjenness Tim Jenness added a comment - This is the output of the valid_cfht.py script: CameraMapper: Loading registry registry from /home/lsstsw/tmp/valid_cfht/output/_parent/registry.sqlite3 CameraMapper WARNING: Unable to locate calibRegistry registry in root: /home/lsstsw/tmp/valid_cfht/output/calibRegistry.sqlite3 CameraMapper WARNING: Unable to locate calibRegistry registry in current dir: ./calibRegistry.sqlite3 CameraMapper WARNING: No registry loaded; proceeding without one 1067 sources in ccd : 12 1082 sources in ccd : 13 1046 sources in ccd : 14 1100 sources in ccd : 21 1107 sources in ccd : 22 1120 sources in ccd : 23 6522 Sources in reference visit : 849375 1594 sources in ccd : 12 3268 sources in ccd : 13 5020 sources in ccd : 14 6733 sources in ccd : 21 8541 sources in ccd : 22 10341 sources in ccd : 23 Visit : 850587 5609 matches found Median value of the astrometric scatter - all magnitudes: 106.116602143 mas Astrometric scatter (median) - mag < 21 : 36.4968012258 mas As well as plots which are used for manual inspection. I assume we can just take the final two numbers as a measure for how well we are doing (and put them on a graphite display in the SQuaRE corridor).
            Hide
            boutigny Dominique Boutigny added a comment -

            I have written a simple test for obs_cfht based on what I did to check the astrometry task. It is at NCSA in /lsst8/boutigny/valid_cfht. There is one single script : run_test.sh which is setting up the directory structure, ingest raw CFHT images, run processCcd and run a validation python script. It exits with error if the astrometric scatter is too large or if the number of matched sources is too small.
            It runs in 1.5 minutes at NCSA. I tested it against b1545 stack release and the current obs_cfht from the current master branch.

            Show
            boutigny Dominique Boutigny added a comment - I have written a simple test for obs_cfht based on what I did to check the astrometry task. It is at NCSA in /lsst8/boutigny/valid_cfht. There is one single script : run_test.sh which is setting up the directory structure, ingest raw CFHT images, run processCcd and run a validation python script. It exits with error if the astrometric scatter is too large or if the number of matched sources is too small. It runs in 1.5 minutes at NCSA. I tested it against b1545 stack release and the current obs_cfht from the current master branch.
            Hide
            wmwood-vasey Michael Wood-Vasey added a comment -

            https://github.com/lsst/validate_drp

            now provides the basis for satisfying this ticket. The key last step is including `validate_drp` in "the sandbox build."

            I think we need to plan out
            1. What "the sandbox build" look like.

            • Right now you can just following the instructions in the README.md at https://github.com/lsst/validate_drp and you will get some output including an astrometric median scatter and a plot.
            • How do we connect this to a test harness?
            Show
            wmwood-vasey Michael Wood-Vasey added a comment - https://github.com/lsst/validate_drp now provides the basis for satisfying this ticket. The key last step is including `validate_drp` in "the sandbox build." I think we need to plan out 1. What "the sandbox build" look like. Right now you can just following the instructions in the README.md at https://github.com/lsst/validate_drp and you will get some output including an astrometric median scatter and a plot. How do we connect this to a test harness?
            Hide
            wmwood-vasey Michael Wood-Vasey added a comment -

            1. validate_drp exists and contains an example test on CFHT data that does this astrometry test in this Story and also additional metrics from the LSST Science Requirements Document.

            2. lsst_ci now runs obs_cfht with its testdata_cfht.

            3. The gist of this story is complete. Related followup will happen under lsst_qa under DM-5381

            Show
            wmwood-vasey Michael Wood-Vasey added a comment - 1. validate_drp exists and contains an example test on CFHT data that does this astrometry test in this Story and also additional metrics from the LSST Science Requirements Document. 2. lsst_ci now runs obs_cfht with its testdata_cfht . 3. The gist of this story is complete. Related followup will happen under lsst_qa under DM-5381

              People

              • Assignee:
                wmwood-vasey Michael Wood-Vasey
                Reporter:
                frossie Frossie Economou
                Watchers:
                David Nidever [X] (Inactive), Dominique Boutigny, Frossie Economou, Michael Wood-Vasey, Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel