Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-21871

Convert pipe_analysis tasks to PipelineTasks

    XMLWordPrintable

    Details

    • Story Points:
      0
    • Team:
      Data Release Production

      Description

      This is an umbrella ticket for making everything in pipe_analysis usable with the Gen3 middleware one way or another.

       

      I expect we'll eventually zero out the story points and add multiple linked issues for the implementation, but in the meantime (for middleware planning purposes), it's nice to have one ticket with a (rough!) estimate of how much work there is.

       

      Lauren MacArthur and Yusra AlSayyad, please link any tickets that already exist for this (I didn't spot any), and feel free to chime in with thoughts about the SP estimate.

        Attachments

          Issue Links

          blocks

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-21872 Final middleware consistency check for DRP pipelines

          • Invalid
          is blocked by

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-22062 Add parquet support to Gen3 Butler

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-22073 Add matplotlib (output) support to Gen3 butler

          • Done
          mentioned in

          Page Loading...

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            tjenness Tim Jenness added a comment -

            Is this ticket relevant any longer?

            Show
            tjenness Tim Jenness added a comment - Is this ticket relevant any longer?
            Hide
            Permalink
            tjenness Tim Jenness added a comment -

            Is this ticket still relevant?

            Show
            tjenness Tim Jenness added a comment - Is this ticket still relevant?
            Hide
            Permalink
            jbosch Jim Bosch added a comment -

            I think we can just close it as Done with zero story points since I think this has effectively happened, though I'd like to get Lauren MacArthur 's opinion on that.

            Show
            jbosch Jim Bosch added a comment - I think we can just close it as Done with zero story points since I think this has effectively happened, though I'd like to get Lauren MacArthur 's opinion on that.
            Hide
            Permalink
            lauren Lauren MacArthur added a comment -

            While it doesn't feel quite right to mark this as "Done" since there is certainly still functionality that pipe_analysis had that has not yet made it to gen3/analysis_tools, the main (and possibly only) middleware-specific functionality I am missing most – the ability to directly compare the outputs of two butler repos having processed the same dataset – may never be possible, so I'm fine with Jim's suggestion.

            Show
            lauren Lauren MacArthur added a comment - While it doesn't feel quite right to mark this as "Done" since there is certainly still functionality that pipe_analysis had that has not yet made it to gen3/ analysis_tools , the main (and possibly only) middleware-specific functionality I am missing most – the ability to directly compare the outputs of two butler repos having processed the same dataset – may never be possible, so I'm fine with Jim's suggestion.

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              jbosch Jim Bosch
              Watchers:
              Hsin-Fang Chiang, Jim Bosch, Lauren MacArthur, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.