Uploaded image for project: 'Request For Comments'
  1. Request For Comments
  2. RFC-505

Include initial Gen3 Butler and PipelineTask in lsst_distrib

    Details

    • Type: RFC
    • Status: Implemented
    • Resolution: Done
    • Component/s: DM
    • Labels:
      None

      Description

      The Gen3 middleware team is hoping to merge our development branch ("gen3-middleware") with master and include the new daf_butler package in lsst_distrib this Friday, making those components more accessible for ongoing integration work.

      This will include adding the new PipelineTask base class (née SuperTask) to pipe_base.  That new name was unanimously regarded as an improvement on the old one on #dm-naming-things in Slack, and after this RFC is accepted we'll all try to stick to the new nomenclature (that will be hardest for those of us who have been working with it longest, I'm sure).

      Neither the Gen3 Butler nor PipelineTask are ready to be used by DM developers at large yet, but enough of the team is now working on integrating these with existing code (rather than developing the new components themselves) that it makes sense to cease having a separate development branch.

      Beyond daf_butler and PipelineTask, the main changes that most developers will notice will be small, not-imported-by-default additions to obs_* packages and additional tests in ci_hsc to exercise Gen3 Butler functionality.

        Attachments

          Issue Links

            Activity

            Hide
            Parejkoj John Parejko added a comment -

            +1 Sounds great!

            Show
            Parejkoj John Parejko added a comment - +1 Sounds great!
            Hide
            rowen Russell Owen added a comment -

            +1 for the name PipelineTask; I think it is more descriptive (if less fun) than SuperTask
            +1 for moving PipelineTask to pipe_base
            I don't have enough details to comment on the rest of it – e.g. what belongs in the new package daf_butler and where does the PipelineTask execution support code go. But overall I think it's a good idea to make it available.

            Show
            rowen Russell Owen added a comment - +1 for the name PipelineTask ; I think it is more descriptive (if less fun) than SuperTask +1 for moving PipelineTask to pipe_base I don't have enough details to comment on the rest of it – e.g. what belongs in the new package daf_butler and where does the PipelineTask execution support code go. But overall I think it's a good idea to make it available.
            Hide
            tjenness Tim Jenness added a comment -

            daf_butler will be pulled in as a dependency of pipe_base when PipelineTask is added. daf_butler will not be added separately.

            Show
            tjenness Tim Jenness added a comment - daf_butler will be pulled in as a dependency of pipe_base when PipelineTask is added. daf_butler will not be added separately.
            Hide
            ktl Kian-Tat Lim added a comment - - edited

            OK from me.  I assume the remainder of pipe_supertask will get a new name as well at some point; as I understand it, that will not be part of the dependency tree at this time.

            Show
            ktl Kian-Tat Lim added a comment - - edited OK from me.  I assume the remainder of  pipe_supertask will get a new name as well at some point; as I understand it, that will not be part of the dependency tree at this time.
            Hide
            jbosch Jim Bosch added a comment -

            I assume the remainder of pipe_supertask will get a new name as well at some point; as I understand it, that will not be part of the dependency tree at this time.

            Correct.  The remaining code is both a bit less mature and and we don't have a direct need for it downstream yet.  The new name for the package has not yet been decided.

            Show
            jbosch Jim Bosch added a comment - I assume the remainder of  pipe_supertask  will get a new name as well at some point; as I understand it, that will  not  be part of the dependency tree at this time. Correct.  The remaining code is both a bit less mature and and we don't have a direct need for it downstream yet.  The new name for the package has not yet been decided.
            Hide
            jbosch Jim Bosch added a comment -

            Adopted.

            Andy Salnikov, I assume you'll take care of DM-15220 after testing lsst_distrib Jenkins with that ticket branch (which should just update pipe_base, adding daf_butler as a dependency of lsst_distrib).

            I'll do DM-15268 after also testing ci_hsc Jenkins with the gen3-middleware branch (that will merge the gen3-middleware branches of obs_subaru, skymap, and ci_hsc to master).

             

            Show
            jbosch Jim Bosch added a comment - Adopted. Andy Salnikov , I assume you'll take care of DM-15220 after testing lsst_distrib Jenkins with that ticket branch (which should just update pipe_base, adding daf_butler as a dependency of lsst_distrib). I'll do DM-15268 after also testing ci_hsc Jenkins with the gen3-middleware branch (that will merge the gen3-middleware branches of obs_subaru, skymap, and ci_hsc to master).  

              People

              • Assignee:
                jbosch Jim Bosch
                Reporter:
                jbosch Jim Bosch
                Watchers:
                Jim Bosch, John Parejko, Kian-Tat Lim, Russell Owen, Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:
                  Planned End:

                  Summary Panel