Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-14003

Create minimal ShimButler for v2 on top of v3

    Details

    • Type: Story
    • Status: Won't Fix
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
    • Story Points:
      12
    • Sprint:
      BG3_S18_04, BG3_S18_05, BG3_F18_06, BG3_F18_07
    • Team:
      Data Release Production

      Description

      Create a ShimButler that emulates a minimal subset of gen2 Butler on top of gen3 Butler.
      It should support all calls needed for coadd processing in ci_hsc.

        Attachments

          Issue Links

            Activity

            pschella Pim Schellart [X] (Inactive) created issue -
            Hide
            jbosch Jim Bosch added a comment -

            First thing this will need is a DataRef object that combines a Butler and a Data ID, so you can call get, put, and datasetExists with different DatasetType strings without providing a Data ID.

            We'll also need to make sure that the shim doesn't complain about extra data ID keys, as long as it has the ones it needs for a unique lookup for the given DatasetType.  We frequently pass in extra keys in Gen2 usage so we can use the same Data ID dict or DataRef for different closely-related DatasetTypes.

            Show
            jbosch Jim Bosch added a comment - First thing this will need is a DataRef object that combines a Butler and a Data ID, so you can call get , put , and datasetExists with different DatasetType  strings without providing a Data ID. We'll also need to make sure that the shim doesn't complain about extra data ID keys, as long as it has the ones it needs for a unique lookup for the given DatasetType.  We frequently pass in extra keys in Gen2 usage so we can use the same Data ID dict or DataRef for different closely-related DatasetTypes.
            jbosch Jim Bosch made changes -
            Field Original Value New Value
            Sprint BG3_S18_04 [ 704 ]
            jbosch Jim Bosch made changes -
            Rank Ranked higher
            pschella Pim Schellart [X] (Inactive) made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            pschella Pim Schellart [X] (Inactive) made changes -
            Risk Score 0
            jbosch Jim Bosch made changes -
            Link This issue is duplicated by DM-13991 [ DM-13991 ]
            vaikunth Vaikunth Thukral made changes -
            Sprint BG3_S18_04 [ 704 ] BG3_S18_04, BG3_S18_05 [ 704, 728 ]
            pschella Pim Schellart [X] (Inactive) made changes -
            Story Points 8 12
            fritzm Fritz Mueller made changes -
            Sprint BG3_S18_04, BG3_S18_05 [ 704, 728 ] BG3_S18_04, BG3_S18_05, BG3_F18_06 [ 704, 728, 745 ]
            yusra Yusra AlSayyad made changes -
            Epic Link DM-12738 [ 36337 ]
            swinbank John Swinbank made changes -
            Epic Link DM-12738 [ 36337 ] DM-14448 [ 80390 ]
            vaikunth Vaikunth Thukral made changes -
            Sprint BG3_S18_04, BG3_S18_05, BG3_F18_06 [ 704, 728, 745 ] BG3_S18_04, BG3_S18_05, BG3_F18_06, BG3_F18_07 [ 704, 728, 745, 759 ]
            pschella Pim Schellart [X] (Inactive) made changes -
            Assignee Pim Schellart [ pschella ]
            vaikunth Vaikunth Thukral made changes -
            Sprint BG3_S18_04, BG3_S18_05, BG3_F18_06, BG3_F18_07 [ 704, 728, 745, 759 ] BG3_S18_04, BG3_S18_05, BG3_F18_06, BG3_F18_07, BG3_F18_08 [ 704, 728, 745, 759, 772 ]
            jbosch Jim Bosch made changes -
            Sprint BG3_S18_04, BG3_S18_05, BG3_F18_06, BG3_F18_07, BG3_F18_08 [ 704, 728, 745, 759, 772 ] BG3_S18_04, BG3_S18_05, BG3_F18_06, BG3_F18_07 [ 704, 728, 745, 759 ]
            Hide
            jbosch Jim Bosch added a comment -

            We have decided to take another approach to Gen3 integration - PipelineTasks and CmdLineTasks will coexist, the former will use Gen3, while the latter use Gen2.

            We won't proceed with making Gen3 work exactly like Gen2 unless something goes very wrong.

            Show
            jbosch Jim Bosch added a comment - We have decided to take another approach to Gen3 integration - PipelineTasks and CmdLineTasks will coexist, the former will use Gen3, while the latter use Gen2. We won't proceed with making Gen3 work exactly like Gen2 unless something goes very wrong.
            jbosch Jim Bosch made changes -
            Resolution Done [ 10000 ]
            Status In Progress [ 3 ] Won't Fix [ 10405 ]

              People

              • Assignee:
                Unassigned
                Reporter:
                pschella Pim Schellart [X] (Inactive)
                Watchers:
                Jim Bosch, Pim Schellart [X] (Inactive)
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel