Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-24625

Evaluate the lsst.verify.tasks framework for performance metrics

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: Validation, Verification
    • Labels:
      None

      Description

      Contribute to the evaluation of the lsst.verify.task framework for scientific performance metrics.

        Attachments

          Activity

          Hide
          krughoff Simon Krughoff added a comment -

          Over the last couple of weeks there has been significant effort to look at how `PipelineTask` classes can be used to execute the broad design goals presented in DMTN-141. This has required implementing three different tasks:
          1) A task to create matched catalogs that can be used by other measurement tasks
          2) The actual measurement task
          3) And a task to aggregate the individual measurements into a single summary number

          My contribution has been in the task to produce matched catalogs and in the very simple pipeline description.

          The current pipeline runs in about 1 minute on the r-band visits in the ci_hsc_gen3 repository on a single core. This can be parallelized and has been shown to run in about 20 seconds on eight cores.

          The quantum graph builder identifies 25 patches as quanta for the tasks (the measurements are at the patch level for this prototype). Though only 9 patches have data since the quantum building is conservative in terms of a boundary around in put datasets for which to select patches.

          The code resides in this repository. If a build ci_hsc_gen3 repository is set up, the pipeline can be run with a call similar to the following:

          pipetask run -j 1 -b "$CI_HSC_GEN3_DIR"/DATA/butler.yaml --register-dataset-types -p pipelines/metrics_pipeline.yaml -d "abstract_filter = 'r'" -o pipeTest -i shared/ci_hsc_output
          

          Note that the tasks directory of the above github repository must be put on your PYTHONPATH by hand for things to work.

          Show
          krughoff Simon Krughoff added a comment - Over the last couple of weeks there has been significant effort to look at how `PipelineTask` classes can be used to execute the broad design goals presented in DMTN-141 . This has required implementing three different tasks: 1) A task to create matched catalogs that can be used by other measurement tasks 2) The actual measurement task 3) And a task to aggregate the individual measurements into a single summary number My contribution has been in the task to produce matched catalogs and in the very simple pipeline description. The current pipeline runs in about 1 minute on the r-band visits in the ci_hsc_gen3 repository on a single core. This can be parallelized and has been shown to run in about 20 seconds on eight cores. The quantum graph builder identifies 25 patches as quanta for the tasks (the measurements are at the patch level for this prototype). Though only 9 patches have data since the quantum building is conservative in terms of a boundary around in put datasets for which to select patches. The code resides in this repository . If a build ci_hsc_gen3 repository is set up, the pipeline can be run with a call similar to the following: pipetask run -j 1 -b "$CI_HSC_GEN3_DIR"/DATA/butler.yaml --register-dataset-types -p pipelines/metrics_pipeline.yaml -d "abstract_filter = 'r'" -o pipeTest -i shared/ci_hsc_output Note that the tasks directory of the above github repository must be put on your PYTHONPATH by hand for things to work.
          Hide
          krughoff Simon Krughoff added a comment -

          Leanne Guy this seems to basically be done. Should I just mark it as such, or do you want more action to be taken?

          Show
          krughoff Simon Krughoff added a comment - Leanne Guy this seems to basically be done. Should I just mark it as such, or do you want more action to be taken?
          Hide
          lguy Leanne Guy added a comment -

          Looks good. To summarize, the evaluation of lsst.verify.tasks has shown it to be a suitable framework for this task and work is progressing.

          Show
          lguy Leanne Guy added a comment - Looks good. To summarize, the evaluation of lsst.verify.tasks has shown it to be a suitable framework for this task and work is progressing.

            People

            Assignee:
            krughoff Simon Krughoff
            Reporter:
            lguy Leanne Guy
            Watchers:
            Leanne Guy, Simon Krughoff
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Due:
              Created:
              Updated:
              Resolved:
              Start date:

                Jenkins

                No builds found.