Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-27941

Update the import system for Pipelines

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: pipe_base
    • Labels:
      None
    • Story Points:
      4
    • Epic Link:
    • Sprint:
      DRP F20-5 (Oct)
    • Team:
      Data Release Production
    • Urgent?:
      No

      Description

      This ticket will be a tweak to how the import system works.

      This will include changing the top level section from 'inherits' to 'imports'. The later term seemed to be more confusing as it carried a lot of OO meaning in it. The old 'inherits' key will continue to work but will raise a warning that it will be removed in 6 months.

      The other change is to the way instruments are handled. Instruments will now carry though in an import. This means if an imported `Pipeline` defines an instrument, the `Pipeline` importing it will then have the instrument set to that value if it is not defined. There can only be one instrument will defined for a `Pipeline` so if any conflicting instruments are defined an error will be generated.

      When importing a `Pipeline` there is now an optional instrument key, that allows an the instrument for the `Pipeline` being imported set to None during import. This may be important for instance when testing and do not want instrument overrides to be loaded.

      It will not be an error to define the same instrument more than once.

        Attachments

          Issue Links

            Activity

            No builds found.
            nlust Nate Lust created issue -
            nlust Nate Lust made changes -
            Field Original Value New Value
            Epic Link DM-26786 [ 439736 ]
            yusra Yusra AlSayyad made changes -
            Epic Link DM-26786 [ 439736 ] DM-27956 [ 442730 ]
            nlust Nate Lust made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            nlust Nate Lust made changes -
            Reviewers Christopher Waters [ cwaters ]
            Status In Progress [ 3 ] In Review [ 10004 ]
            Hide
            czw Christopher Waters added a comment -

            Fine other than the missing newline and the request for one more unit test.

            More documentation on pipeline yaml syntax would be helpful, even if just a louder note that each section corresponds to a pipelineIR.py "sectionIR" which describes the allowable keys.  If this has been explained before, it didn't click with me until this review made me review the logic more fully.

            Show
            czw Christopher Waters added a comment - Fine other than the missing newline and the request for one more unit test. More documentation on pipeline yaml syntax would be helpful, even if just a louder note that each section corresponds to a pipelineIR.py "sectionIR" which describes the allowable keys.  If this has been explained before, it didn't click with me until this review made me review the logic more fully.
            czw Christopher Waters made changes -
            Status In Review [ 10004 ] Reviewed [ 10101 ]
            nlust Nate Lust made changes -
            Resolution Done [ 10000 ]
            Status Reviewed [ 10101 ] Done [ 10002 ]
            lguy Leanne Guy made changes -
            Link This issue relates to DM-28353 [ DM-28353 ]

              People

              Assignee:
              nlust Nate Lust
              Reporter:
              nlust Nate Lust
              Reviewers:
              Christopher Waters
              Watchers:
              Christopher Waters, Nate Lust
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.