Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-25180

Add ingest time to registry

    XMLWordPrintable

    Details

    • Story Points:
      1
    • Team:
      Architecture
    • Urgent?:
      No

      Description

      It would be very helpful for the OODS if butler recorded ingest time as well as observation time. Storing ingest time and allowing queries such as "give me all datasets in this collection that were stored more than N days ago" would significantly streamline OODS data expiry. The returned refs could immediately be passed to pruneDatasets. Currently OODS works by going behind gen2 butler's back and deleting the files directly from datastore without involving butler.

        Attachments

          Issue Links

          is triggering

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-27374 Add ingest time support to queryDatasets

          • Done
          relates to

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-27033 Integration of pre-middleware-release dimensions changes

          • Done

            Activity

            Ascending order - Click to sort in descending order
            5 older comments
            Hide
            Permalink
            tjenness Tim Jenness added a comment -

            I think this trivial change does everything we need. I could also add the querying side of things but I'd need some pointers as to where in the code we could possibly handle it. I think the main use case is in a queryDatasets WHERE expression.

            Show
            tjenness Tim Jenness added a comment - I think this trivial change does everything we need. I could also add the querying side of things but I'd need some pointers as to where in the code we could possibly handle it. I think the main use case is in a queryDatasets WHERE expression.
            Hide
            Permalink
            tjenness Tim Jenness added a comment - - edited

            I assume my PR should go against DM-27033

            Show
            tjenness Tim Jenness added a comment - - edited I assume my PR should go against DM-27033
            Hide
            Permalink
            jbosch Jim Bosch added a comment -

            Yes. I think we may have lost the previous merge commits in the last rebase, so it may be worth fixing that before you merge if we want to bother fixing it at all. Linking all of the tickets that get merged on Jira may be sufficient.

            Show
            jbosch Jim Bosch added a comment - Yes. I think we may have lost the previous merge commits in the last rebase, so it may be worth fixing that before you merge if we want to bother fixing it at all. Linking all of the tickets that get merged on Jira may be sufficient.
            Hide
            Permalink
            tjenness Tim Jenness added a comment -

            Jim Bosch I've rebased this ticket onto the DM-27033 integration branch (and added back the other merge commits).

            Show
            tjenness Tim Jenness added a comment - Jim Bosch I've rebased this ticket onto the DM-27033 integration branch (and added back the other merge commits).
            Hide
            Permalink
            jbosch Jim Bosch added a comment -

            Looks good! Sorry it took me so long to get to reviewing such a tiny change.

            Show
            jbosch Jim Bosch added a comment - Looks good! Sorry it took me so long to get to reviewing such a tiny change.

              People

              Assignee:
              tjenness Tim Jenness
              Reporter:
              tjenness Tim Jenness
              Reviewers:
              Jim Bosch
              Watchers:
              Andy Salnikov, Jim Bosch, Steve Pietrowicz, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.