Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-21912

Implement PpdbMetricTask.runQuantum

    XMLWordPrintable

    Details

      Description

      lsst.verify.tasks.PpdbMetricTask must take a single database as input, but may compute a more fine-grained metric. Therefore, its run method takes the output data ID as a keyword argument. Create a custom runQuantum method to support this argument; it should be identical to PipelineTask.runQuantum except for the extra keyword.

        Attachments

          Issue Links

            Activity

            Hide
            krzys Krzysztof Findeisen added a comment -

            Possibly blocked by DM-21877. The issue is being able to test the new runQuantum; for this I need some kind of Gen 3 dataset, but not necessarily the production one (e.g., the Gen 2 PpdbMetricTask uses a fictitious dataset that exists only in the test suite).

            Maybe Jim Bosch can comment on exactly how much environment is needed to run runQuantum?

            Show
            krzys Krzysztof Findeisen added a comment - Possibly blocked by DM-21877 . The issue is being able to test the new runQuantum ; for this I need some kind of Gen 3 dataset, but not necessarily the production one (e.g., the Gen 2 PpdbMetricTask uses a fictitious dataset that exists only in the test suite). Maybe Jim Bosch can comment on exactly how much environment is needed to run runQuantum ?
            Hide
            jbosch Jim Bosch added a comment - - edited

            To call runQuantum from test code you'll want to start by making a butler repo in a temporary directory with a SQLite database (possibly an in-memory one), which we do frequently in daf_butler tests (e.g. https://github.com/lsst/daf_butler/blob/master/tests/test_butler.py#L122). You'll generally then need to insert some dimension records to match whatever data IDs you want to use (https://github.com/lsst/daf_butler/blob/master/tests/test_butler.py#L169), and register your dataset types (https://github.com/lsst/daf_butler/blob/master/tests/test_butler.py#L105).  From there you can call put and get.

             

            Show
            jbosch Jim Bosch added a comment - - edited To call runQuantum from test code you'll want to start by making a butler repo in a temporary directory with a SQLite database (possibly an in-memory one), which we do frequently in daf_butler tests (e.g. https://github.com/lsst/daf_butler/blob/master/tests/test_butler.py#L122 ). You'll generally then need to insert some dimension records to match whatever data IDs you want to use ( https://github.com/lsst/daf_butler/blob/master/tests/test_butler.py#L169 ), and register your dataset types ( https://github.com/lsst/daf_butler/blob/master/tests/test_butler.py#L105 ).  From there you can call put and get .  
            Hide
            krzys Krzysztof Findeisen added a comment - - edited

            Hi Ian Sullivan, could you review this? It's 230 lines, mostly test code. (Said test code should be more general than lsst.verify, but generalizing/moving it is for DM-22599.)

            Show
            krzys Krzysztof Findeisen added a comment - - edited Hi Ian Sullivan , could you review this? It's 230 lines, mostly test code. (Said test code should be more general than lsst.verify , but generalizing/moving it is for DM-22599 .)
            Hide
            sullivan Ian Sullivan added a comment -

            Looks good. Just a few minor suggestions on the pull request about formatting and documentation.

            Show
            sullivan Ian Sullivan added a comment - Looks good. Just a few minor suggestions on the pull request about formatting and documentation.

              People

              Assignee:
              krzys Krzysztof Findeisen
              Reporter:
              krzys Krzysztof Findeisen
              Reviewers:
              Ian Sullivan
              Watchers:
              Ian Sullivan, Jim Bosch, Krzysztof Findeisen
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: