Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-35670

Remove gen2 support from pipe_tasks

    XMLWordPrintable

    Details

      Description

      Remove all gen2 code from pipe_tasks.

        Attachments

          Issue Links

            Activity

            No builds found.
            erykoff Eli Rykoff created issue -
            erykoff Eli Rykoff made changes -
            Field Original Value New Value
            Description Remove all gen2 code from pipe_tasks.
            tjenness Tim Jenness made changes -
            Labels gen2-removal SciencePipelines gen2-removal
            tjenness Tim Jenness made changes -
            Link This issue blocks DM-35724 [ DM-35724 ]
            erykoff Eli Rykoff made changes -
            Assignee Eli Rykoff [ erykoff ]
            erykoff Eli Rykoff made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            tjenness Tim Jenness made changes -
            Link This issue blocks DM-19374 [ DM-19374 ]
            tjenness Tim Jenness made changes -
            Link This issue blocks DM-31047 [ DM-31047 ]
            Hide
            erykoff Eli Rykoff added a comment -

            There are a few PRs here.

            The big one is pipe_tasks: https://github.com/lsst/pipe_tasks/pull/712

            This deprecates the old location of MakeWarpTask and so I've updated to the new location:
            https://github.com/lsst/ci_hsc_gen3/pull/85
            https://github.com/lsst/drp_pipe/pull/37
            https://github.com/lsst/obs_subaru/pull/431

            And this update broke obs_cfht because (a) there was still a gen2 test there (oops) and the tests were relying on some side-effects to load detector tests; this makes the detector testing work correctly without side effects: https://github.com/lsst/obs_cfht/pull/105

            Show
            erykoff Eli Rykoff added a comment - There are a few PRs here. The big one is pipe_tasks: https://github.com/lsst/pipe_tasks/pull/712 This deprecates the old location of MakeWarpTask and so I've updated to the new location: https://github.com/lsst/ci_hsc_gen3/pull/85 https://github.com/lsst/drp_pipe/pull/37 https://github.com/lsst/obs_subaru/pull/431 And this update broke obs_cfht because (a) there was still a gen2 test there (oops) and the tests were relying on some side-effects to load detector tests; this makes the detector testing work correctly without side effects: https://github.com/lsst/obs_cfht/pull/105
            erykoff Eli Rykoff made changes -
            Watchers Eli Rykoff [ Eli Rykoff ] Eli Rykoff, Kenneth Herner, Tim Jenness [ Eli Rykoff, Kenneth Herner, Tim Jenness ]
            Hide
            erykoff Eli Rykoff added a comment -

            I have added initial reviewers. I will add more as I find volunteers.

            Show
            erykoff Eli Rykoff added a comment - I have added initial reviewers. I will add more as I find volunteers.
            erykoff Eli Rykoff made changes -
            Reviewers Kenneth Herner, Tim Jenness [ kherner, tjenness ]
            Status In Progress [ 3 ] In Review [ 10004 ]
            Hide
            kherner Kenneth Herner added a comment - - edited

            I have reviewed the PR for pipe_tasks.

             

            EDIT: I have also done the PR for ci_hsc_gen3 and drp_pipe.

            Show
            kherner Kenneth Herner added a comment - - edited I have reviewed the PR for pipe_tasks.   EDIT: I have also done the PR for ci_hsc_gen3 and drp_pipe.
            Hide
            kherner Kenneth Herner added a comment - - edited

            Eli Rykoff The APTemplate.yaml in ap_pipe assumes the old location of the makeWarp task, so that should be updated as well (I will pass it along to rest of the AP folks):

            https://github.com/lsst/ap_pipe/blob/main/pipelines/ApTemplate.yaml

            Show
            kherner Kenneth Herner added a comment - - edited Eli Rykoff  The APTemplate.yaml in ap_pipe assumes the old location of the makeWarp task, so that should be updated as well (I will pass it along to rest of the AP folks): https://github.com/lsst/ap_pipe/blob/main/pipelines/ApTemplate.yaml
            Hide
            erykoff Eli Rykoff added a comment -

            Good catch! I had done this but failed to make the PR, which is now here: https://github.com/lsst/ap_pipe/pull/123

            Show
            erykoff Eli Rykoff added a comment - Good catch! I had done this but failed to make the PR, which is now here: https://github.com/lsst/ap_pipe/pull/123
            Hide
            tjenness Tim Jenness added a comment -

            I looked at the obs_cfht PR and think the fix is wrong and needs to go into obs_base. We also have some cruft processCcd.py and singleFrameDriver.py configs lying around in obs_cfht and obs_subaru (I have removed those for obs_lsst in another ticket).

            Show
            tjenness Tim Jenness added a comment - I looked at the obs_cfht PR and think the fix is wrong and needs to go into obs_base. We also have some cruft processCcd.py and singleFrameDriver.py configs lying around in obs_cfht and obs_subaru (I have removed those for obs_lsst in another ticket).
            Parejkoj John Parejko made changes -
            Link This issue is triggering DM-35871 [ DM-35871 ]
            Hide
            erykoff Eli Rykoff added a comment -

            I think everything is taken care of after Nate Lust clarified the supplementary data methods. I have a final jenkins here: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/37169/pipeline (it all ran fine yesterday but I made more than just documentation changes in assembleCoadd and dcrAssembleCoadd in the meantime). Kenneth Herner, Tim Jenness thanks for the review! Please somebody click the "reviewed" button (or does one of you have to take your name off first?). Fingers crossed this will be in the new weekly! 

            Show
            erykoff Eli Rykoff added a comment - I think everything is taken care of after Nate Lust  clarified the supplementary data methods. I have a final jenkins here: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/37169/pipeline (it all ran fine yesterday but I made more than just documentation changes in assembleCoadd and dcrAssembleCoadd in the meantime). Kenneth Herner , Tim Jenness thanks for the review! Please somebody click the "reviewed" button (or does one of you have to take your name off first?). Fingers crossed this will be in the new weekly! 
            Hide
            tjenness Tim Jenness added a comment -

            Review completed by multiple people.

            Show
            tjenness Tim Jenness added a comment - Review completed by multiple people.
            tjenness Tim Jenness made changes -
            Status In Review [ 10004 ] Reviewed [ 10101 ]
            erykoff Eli Rykoff made changes -
            Resolution Done [ 10000 ]
            Status Reviewed [ 10101 ] Done [ 10002 ]
            Parejkoj John Parejko made changes -
            Link This issue relates to DM-35940 [ DM-35940 ]
            Parejkoj John Parejko made changes -
            Link This issue relates to DM-35939 [ DM-35939 ]
            yusra Yusra AlSayyad made changes -
            Epic Link DM-35761 [ 1880231 ]
            Story Points 8
            Team Data Release Production [ 10301 ]

              People

              Assignee:
              erykoff Eli Rykoff
              Reporter:
              erykoff Eli Rykoff
              Reviewers:
              Kenneth Herner, Tim Jenness
              Watchers:
              Eli Rykoff, Kenneth Herner, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.