Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-26686

Add command-line tools for Registry.decertify.

    XMLWordPrintable

    Details

    • Team:
      Ops Middleware
    • Urgent?:
      No

      Description

      Add command-line tools for Registry.certify and Registry.decertify.

      These methods are being added on DM-24432, so that's a blocker for this ticket.

      These scripts will need to start with a call to Registry.queryDatasets, and so will build on the script for that; I'll add DM-26685 as a blocker, too.

      I'm calling this a blocker for Gen2 deprecation because I think it's an important part of the CPP workflow. But it's possible Christopher Waters or myself will write command-line tools that call these Registry methods as well as doing other things, and that would at least reduce the priority of this ticket.

        Attachments

          Issue Links

            Activity

            Hide
            jbosch Jim Bosch added a comment -

            butler certify now exists.  But we still need a command-line tool for decertify, so it's worth keeping this ticket around for that.  Before we start working on it, I think we need:

            • Jim Bosch to address the limitation that we can't use queryDatasets on CALIBRATION collections (probably part of DM-27660), as we'll definitely want to do that here.
            • Get input from the CPP team and others on what use cases they want this to support, so we have a sense for what kinds of inputs it should take.  I think it's probably likely we'll want to have some options on butler certify that can decertify existing calibrations in order to make room for new ones.

             

            Show
            jbosch Jim Bosch added a comment - butler certify now exists.  But we still need a command-line tool for decertify , so it's worth keeping this ticket around for that.  Before we start working on it, I think we need: Jim Bosch to address the limitation that we can't use  queryDatasets on CALIBRATION collections (probably part of DM-27660 ), as we'll definitely want to do that here. Get input from the CPP team and others on what use cases they want this to support, so we have a sense for what kinds of inputs it should take.  I think it's probably likely we'll want to have some options on butler certify that can decertify existing calibrations in order to make room for new ones.  
            Hide
            czw Christopher Waters added a comment -

            Thinking through this problem after reviewing DM-26971, the use cases that make sense to me right now are:

            • Decertify an arbitrary calibration.  From the review, it seems like the set of {instrument, datasetType, CALIBRATION collection, filter, validStart} are all necessary for this.  Unfortunately, this also seems like a lot of information that is not obviously visible from the butler.  If `queryDatasets` on CALIBRATION collections will yield all this, that would be ideal, otherwise a `queryCalibrations` may be needed.
            • Decertify a calibration based on a new calibration.  This case is for when a replacement calib exists, and should be swapped in for one that already exists.  This likely still requires all the information for the arbitrary calibration, but I'm not sure if this "swap" mode is something that the middleware will support.
            • Update the validEnd date of a calibration.  When calibrations have a validation path, being able to update the validEnd based on the metrics that generates will be necessary.  I include this here even though this may be a different operation.
            • Decertifying an entire CALIBRATION collection.  I expect this is more of a rename operation (moving an INSTRUMENT/calib/x20201216 to INSTRUMENT/calib/x20201216.deprecated), but is still something I image as a way to remove now-unwanted calibrations. 
            Show
            czw Christopher Waters added a comment - Thinking through this problem after reviewing DM-26971 , the use cases that make sense to me right now are: Decertify an arbitrary calibration.  From the review, it seems like the set of {instrument, datasetType, CALIBRATION collection, filter, validStart} are all necessary for this.  Unfortunately, this also seems like a lot of information that is not obviously visible from the butler.  If `queryDatasets` on CALIBRATION collections will yield all this, that would be ideal, otherwise a `queryCalibrations` may be needed. Decertify a calibration based on a new calibration.  This case is for when a replacement calib exists, and should be swapped in for one that already exists.  This likely still requires all the information for the arbitrary calibration, but I'm not sure if this "swap" mode is something that the middleware will support. Update the validEnd date of a calibration.  When calibrations have a validation path, being able to update the validEnd based on the metrics that generates will be necessary.  I include this here even though this may be a different operation. Decertifying an entire CALIBRATION collection.  I expect this is more of a rename operation (moving an INSTRUMENT/calib/x20201216 to INSTRUMENT/calib/x20201216.deprecated), but is still something I image as a way to remove now-unwanted calibrations. 
            Hide
            npease Nate Pease [X] (Inactive) added a comment -

            Jim Boschis there anything worth doing on this ticket before DM-27760 is done? (since it seems queryDatasets won't work on calibration collections, and that's the only use case for this ticket)

            Show
            npease Nate Pease [X] (Inactive) added a comment - Jim Bosch is there anything worth doing on this ticket before DM-27760 is done? (since it seems queryDatasets won't work on calibration collections, and that's the only use case for this ticket)
            Hide
            jbosch Jim Bosch added a comment -

            Probably nothing to do yet (and I need to give you a new ticket to track what used to be DM-27760).

            Show
            jbosch Jim Bosch added a comment - Probably nothing to do yet (and I need to give you a new ticket to track what used to be DM-27760 ).

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              jbosch Jim Bosch
              Watchers:
              Christopher Waters, Jim Bosch
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:

                  Jenkins

                  No builds found.