Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-24625

Evaluate the lsst.verify.tasks framework for performance metrics

    Details

    • Type: Story
    • Status: To Do
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: Validation, Verification
    • Labels:
      None

      Description

      Contribute to the evaluation of the lsst.verify.task framework for scientific performance metrics.

        Attachments

          Activity

          Hide
          krughoff Simon Krughoff added a comment -

          Over the last couple of weeks there has been significant effort to look at how `PipelineTask` classes can be used to execute the broad design goals presented in DMTN-141. This has required implementing three different tasks:
          1) A task to create matched catalogs that can be used by other measurement tasks
          2) The actual measurement task
          3) And a task to aggregate the individual measurements into a single summary number

          My contribution has been in the task to produce matched catalogs and in the very simple pipeline description.

          The current pipeline runs in about 1 minute on the r-band visits in the ci_hsc_gen3 repository on a single core. This can be parallelized and has been shown to run in about 20 seconds on eight cores.

          The quantum graph builder identifies 25 patches as quanta for the tasks (the measurements are at the patch level for this prototype). Though only 9 patches have data since the quantum building is conservative in terms of a boundary around in put datasets for which to select patches.

          The code resides in this repository. If a build ci_hsc_gen3 repository is set up, the pipeline can be run with a call similar to the following:

          pipetask run -j 1 -b "$CI_HSC_GEN3_DIR"/DATA/butler.yaml --register-dataset-types -p pipelines/metrics_pipeline.yaml -d "abstract_filter = 'r'" -o pipeTest -i shared/ci_hsc_output
          

          Note that the tasks directory of the above github repository must be put on your PYTHONPATH by hand for things to work.

          Show
          krughoff Simon Krughoff added a comment - Over the last couple of weeks there has been significant effort to look at how `PipelineTask` classes can be used to execute the broad design goals presented in DMTN-141 . This has required implementing three different tasks: 1) A task to create matched catalogs that can be used by other measurement tasks 2) The actual measurement task 3) And a task to aggregate the individual measurements into a single summary number My contribution has been in the task to produce matched catalogs and in the very simple pipeline description. The current pipeline runs in about 1 minute on the r-band visits in the ci_hsc_gen3 repository on a single core. This can be parallelized and has been shown to run in about 20 seconds on eight cores. The quantum graph builder identifies 25 patches as quanta for the tasks (the measurements are at the patch level for this prototype). Though only 9 patches have data since the quantum building is conservative in terms of a boundary around in put datasets for which to select patches. The code resides in this repository . If a build ci_hsc_gen3 repository is set up, the pipeline can be run with a call similar to the following: pipetask run -j 1 -b "$CI_HSC_GEN3_DIR"/DATA/butler.yaml --register-dataset-types -p pipelines/metrics_pipeline.yaml -d "abstract_filter = 'r'" -o pipeTest -i shared/ci_hsc_output Note that the tasks directory of the above github repository must be put on your PYTHONPATH by hand for things to work.

            People

            • Assignee:
              krughoff Simon Krughoff
              Reporter:
              lguy Leanne Guy
              Watchers:
              Leanne Guy, Simon Krughoff
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Due:
                Created:
                Updated:
                Start date:

                Summary Panel