Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-20572

Remove "Generation 2" Middleware

    XMLWordPrintable

    Details

    • Team:
      Architecture
    • Urgent?:
      No

      Description

      The "Generation 2" Middleware must be removed... sometime in the future. Tim Jenness will provide schedule information as it becomes available.

        Attachments

          Issue Links

          blocks

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-31698 Move ConvertReferenceCatalog classes out of ingestIndex file

          • Done
          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. DM-27843 add anyFilterMapsToThis support to ReferenceObjectLoader

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-26173 Write plan for formally beginning Gen2 deprecation

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-28863 Add metrics output and tests to gen3 jointcal

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-29008 Make gen3 jointcal configs the default

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-29437 Do Release 23.0.0 of Science Pipelines

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-29497 Add gen2 deprecation message

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-29535 Create gen3 cfht testdata_jointcal repos

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-29542 Move jointcal cfht tests to gen3

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-29543 Gen3 refcat converter

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-29857 Create pure Gen 3 dataset management scripts for ap_verify datasets

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-32601 Move remaining jointcal tests to gen3

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-32615 Move jointcal cfht_minimal tests to gen3

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-35697 Move profile context manager out of pipe_base.cmdLineTask

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-28991 Fix gen3 jointcal refcat area

          • Won't Fix

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-29659 convert astrometrySourceSelector to gen3 or remove it

          • Won't Fix

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-32584 Create gen3 decam testdata_jointcal repo

          • Won't Fix

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-29186 Gen3 refloader incorrectly adds filter_flux fields to refcat schema

          • Invalid
          relates to

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-31704 Design and document reference catalog dataset policy

          • To Do

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-33416 Write up a sketch of how to use pytest coverage to assist in gen2 removal

          • Done

          Improvement - An improvement or enhancement to an existing feature or task. DM-4740 Audit and document obs_subaru scripts

          • To Do

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-33150 Remove Gen 2 support from ap_verify

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-35655 Remove gen2 jointcal code and tests

          • Done
          mentioned in

          Page Loading...

          Page Loading...

          Page Loading...

          Page Loading...

          Page Loading...

          Page Loading...

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            swinbank John Swinbank added a comment -

            I think we need to go through a complete release cycle with operational gen 2 and gen 3 middleware before we can remove gen 2 altogether. It follows that this issue can't block release 20. Moving the blocker to 21.

            Show
            swinbank John Swinbank added a comment - I think we need to go through a complete release cycle with operational gen 2 and gen 3 middleware before we can remove gen 2 altogether. It follows that this issue can't block release 20. Moving the blocker to 21.
            Hide
            Permalink
            swinbank John Swinbank added a comment -

            Pushing this to block v22 for now, since it's clearly not being removed in v21.

            Show
            swinbank John Swinbank added a comment - Pushing this to block v22 for now, since it's clearly not being removed in v21.
            Hide
            Permalink
            tjenness Tim Jenness added a comment -

            I would not be surprised if daf_persistence is still being imported by code in science pipelines at the end of 2021.

            Show
            tjenness Tim Jenness added a comment - I would not be surprised if daf_persistence is still being imported by code in science pipelines at the end of 2021.
            Hide
            Permalink
            yusra Yusra AlSayyad added a comment -

            Tim for the win. This is definitely not happening for v22 or v23. I'll push it out to v24, but this this is going to be a lot of work. If pipelines removes gen2 from all the tasks in the background of their regular development, this will probably span a 6+ month process.

            Show
            yusra Yusra AlSayyad added a comment - Tim for the win. This is definitely not happening for v22 or v23. I'll push it out to v24, but this this is going to be a lot of work. If pipelines removes gen2 from all the tasks in the background of their regular development, this will probably span a 6+ month process.
            Hide
            Permalink
            tjenness Tim Jenness added a comment -

            What I really want to know is when I can add the deprecation warning to Mapper and CmdLineTask ?

            Show
            tjenness Tim Jenness added a comment - What I really want to know is when I can add the deprecation warning to Mapper and CmdLineTask ?
            Hide
            Permalink
            tjenness Tim Jenness added a comment -

            And I'm more than happy to take a chainsaw to it all on my own time because obs_base has too much cruft in it at the moment.

            Show
            tjenness Tim Jenness added a comment - And I'm more than happy to take a chainsaw to it all on my own time because obs_base has too much cruft in it at the moment.
            Hide
            Permalink
            tjenness Tim Jenness added a comment -

            With the release of v23.0 DM-29437 gen2 can be removed as soon as the pipelines team declare that they have completed the gen2/gen3 parity tests.

            Show
            tjenness Tim Jenness added a comment - With the release of v23.0 DM-29437 gen2 can be removed as soon as the pipelines team declare that they have completed the gen2/gen3 parity tests.
            Hide
            Permalink
            tjenness Tim Jenness added a comment -

            All gen2 code has been removed from lsst_distrib (pipe_drivers, ctrl_pool, and daf_persistence still exist but nothing uses them – pipe_drivers is being retained for a while because of the gen3 class that was being used there until recently DM-29835)

            Show
            tjenness Tim Jenness added a comment - All gen2 code has been removed from lsst_distrib (pipe_drivers, ctrl_pool, and daf_persistence still exist but nothing uses them – pipe_drivers is being retained for a while because of the gen3 class that was being used there until recently DM-29835 )

              People

              Assignee:
              tjenness Tim Jenness
              Reporter:
              gcomoretto Gabriele Comoretto [X] (Inactive)
              Watchers:
              Gabriele Comoretto [X] (Inactive), Tim Jenness, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.