Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-26690

Add command-line tool for Registry.queryDataIds

    XMLWordPrintable

    Details

    • Story Points:
      8
    • Sprint:
      DB_F20_09
    • Team:
      Data Access and Database
    • Urgent?:
      No

      Description

      Add a command-line interface for Registry.queryDataIds. Like queryDatasets (DM-26685), this one has a lot of optional keyword arguments that may make it more difficult to wrap well, and just like there, I think the dataId argument is at least one that can be ignored. It doesn't have as many "downstream" methods whose scripts will need to call it as well, but I think it will eventually have a few.

      I'm calling this a deprecation blocker because it's probably the most important tool for debugging problems in QG generation without resorting to direct SQL.

        Attachments

          Issue Links

            Activity

            Hide
            npease Nate Pease added a comment -

            There's a lot of duplicated test code between the test for query-data-ids and the query-datasets code. At some point I think it would make sense to factor this into a small toolkit for creating a butler repo that can be used by these (and other) tests. Maybe we are there already, or maybe sometime after the Nov 4 deadline. Any thoughts/guidance appreciated.

            Show
            npease Nate Pease added a comment - There's a lot of duplicated test code between the test for query-data-ids and the query-datasets code. At some point I think it would make sense to factor this into a small toolkit for creating a butler repo that can be used by these (and other) tests. Maybe we are there already, or maybe sometime after the Nov 4 deadline. Any thoughts/guidance appreciated.
            Hide
            npease Nate Pease added a comment -

            Tim Jenness can you review?

            Show
            npease Nate Pease added a comment - Tim Jenness can you review?
            Hide
            tjenness Tim Jenness added a comment -

            In addition to the code comments, I'm not entirely sure I completely understand the output of the command so it would be great if there was some extra help text somewhere (I'm not sure how easy it is to add more information in click vs adding extra text to the sphinx page. Mentioning DimensionGraph in the help string may be confusing.

            A few things jumped out:

            • I was able to run with a dimensions argument and no --collections option so the docstring was confusing at best. It wasn't at all clear what I would do with the dimension argument and it was a pleasant surprise when for a whim I asked for "patch" on a calexp dataset and was rewarded by it telling me the tract/patch for each calexp. Even better with htm7. That's great but not obvious.
            • Sometimes you get identical rows repeated and it's not clear why. It might be because there are repeated options with dimensions that we do not see but it doesn't help anybody to include the duplicates. Filtering them out should be straightforward.
            • Table sorting should be used as is now done in query-datasets.
            Show
            tjenness Tim Jenness added a comment - In addition to the code comments, I'm not entirely sure I completely understand the output of the command so it would be great if there was some extra help text somewhere (I'm not sure how easy it is to add more information in click vs adding extra text to the sphinx page. Mentioning DimensionGraph in the help string may be confusing. A few things jumped out: I was able to run with a dimensions argument and no --collections option so the docstring was confusing at best. It wasn't at all clear what I would do with the dimension argument and it was a pleasant surprise when for a whim I asked for "patch" on a calexp dataset and was rewarded by it telling me the tract/patch for each calexp. Even better with htm7. That's great but not obvious. Sometimes you get identical rows repeated and it's not clear why. It might be because there are repeated options with dimensions that we do not see but it doesn't help anybody to include the duplicates. Filtering them out should be straightforward. Table sorting should be used as is now done in query-datasets.
            Hide
            npease Nate Pease added a comment -

            Tim Jenness I think this is ready for re-review.

            Show
            npease Nate Pease added a comment - Tim Jenness I think this is ready for re-review.
            Hide
            tjenness Tim Jenness added a comment -

            Looks good now. Thanks.

            I think if you switch your new astropy tables equal function into a class and use multiple inheritance in your tests then it will look a lot more like a test assert.

            Show
            tjenness Tim Jenness added a comment - Looks good now. Thanks. I think if you switch your new astropy tables equal function into a class and use multiple inheritance in your tests then it will look a lot more like a test assert.

              People

              Assignee:
              npease Nate Pease
              Reporter:
              jbosch Jim Bosch
              Reviewers:
              Tim Jenness
              Watchers:
              Jim Bosch, Nate Pease, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  CI Builds

                  No builds found.