Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-29203

Add option to flatten CHAINED collections when defining them

    XMLWordPrintable

    Details

    • Story Points:
      1
    • Epic Link:
    • Team:
      Data Release Production
    • Urgent?:
      No

      Description

      Some of the time we want CHAINED collections to point to other CHAINED collections, so the definition of the child one can change and the parent's content will be updated automatically. But when we define the CHAINED collection that combines a processing run and its inputs, it'd be better to flatten out any nested chains instead, because our goal is to capture the inputs to the output collection as of the time it was produced. We can't fully guarantee that without using real provenance, but a flattened chain is much better (it at least correctly handles the case of someone updating the recommended calibration collection).

      On this ticket, add an option to Registry.setCollectionChain to do that automatically, and update ctrl_mpexec and the Gen2 repo conversion code in obs_base to use it.

        Attachments

          Activity

          Hide
          jbosch Jim Bosch added a comment -

          This should be a small and hopefully straightforward review, but with changes in three packages:

          Show
          jbosch Jim Bosch added a comment - This should be a small and hopefully straightforward review, but with changes in three packages: https://github.com/lsst/daf_butler/pull/488 https://github.com/lsst/obs_base/pull/361 https://github.com/lsst/ctrl_mpexec/pull/117
          Hide
          salnikov Andy Salnikov added a comment -

          Looks good, couple of minor comments on PRs.

          Show
          salnikov Andy Salnikov added a comment - Looks good, couple of minor comments on PRs.

            People

            Assignee:
            jbosch Jim Bosch
            Reporter:
            jbosch Jim Bosch
            Reviewers:
            Andy Salnikov
            Watchers:
            Andy Salnikov, Jim Bosch
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Jenkins

                No builds found.