Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-21939

Create Gen 3 AP Pipeline

    Details

    • Story Points:
      4
    • Sprint:
      AP F19-6 (November), AP S20-6 (May), AP F20-1 (June)
    • Team:
      Alert Production

      Description

      This is a mostly-umbrella ticket for getting all current Gen2 ap_pipe functionality working in Gen3. The goal is being able to run AP on HiTS 2015 using the CmdLineActivator. It includes:

      • creating scripts that ingest HiTS test data into a Gen3 repository (DM-21862)
      • conversion of all remaining ApPipeTask subtasks to Gen 3 (DM-21874, DM-21886)
      • creation of a YAML file configuring the pipeline (1 SP on this ticket)

      ApPipeTask has a lot of code for handling the case of calexp difference imaging templates. By general agreement within the AP and Middleware groups, we will not be porting this functionality to Gen 3 (see also DM-21874), and will reimplement it from scratch should we need it after migration.

        Attachments

          Issue Links

            Activity

            krzys Krzysztof Findeisen created issue -
            krzys Krzysztof Findeisen made changes -
            Field Original Value New Value
            Link This issue blocks DM-21888 [ DM-21888 ]
            krzys Krzysztof Findeisen made changes -
            Link This issue blocks DM-21919 [ DM-21919 ]
            krzys Krzysztof Findeisen made changes -
            Link This issue is blocked by DM-21862 [ DM-21862 ]
            krzys Krzysztof Findeisen made changes -
            Link This issue is blocked by DM-21886 [ DM-21886 ]
            krzys Krzysztof Findeisen made changes -
            Link This issue is blocked by DM-21874 [ DM-21874 ]
            krzys Krzysztof Findeisen made changes -
            Link This issue relates to DM-21915 [ DM-21915 ]
            krzys Krzysztof Findeisen made changes -
            Labels gen2-deprecation-blocker gen3-middleware
            krzys Krzysztof Findeisen made changes -
            Link This issue blocks DM-21888 [ DM-21888 ]
            krzys Krzysztof Findeisen made changes -
            Link This issue blocks DM-21888 [ DM-21888 ]
            krzys Krzysztof Findeisen made changes -
            Description This is a mostly-umbrella ticket for getting all current Gen2 ap_pipe functionality working in Gen3. The goal is being able to run AP on HiTS 2015 using the "laptop" executor. It includes:
            * creating scripts that ingest HiTS test data into a Gen3 repository (DM-21862)
            * conversion of all remaining {{ApPipeTask}} subtasks to Gen 3 (DM-21874, DM-21886)
            * creation of a YAML file configuring the pipeline (1 SP on this ticket)

            {{ApPipeTask}} has a lot of code for handling the case of calexp difference imaging templates. By general agreement within the AP and Middleware groups, we will *not* be porting this functionality to Gen 3 (see also DM-21874), and will reimplement it from scratch should we need it after migration.
            This is a mostly-umbrella ticket for getting all current Gen2 ap_pipe functionality working in Gen3. The goal is being able to run AP on HiTS 2015 using the {{CmdLineActivator}}. It includes:
            * creating scripts that ingest HiTS test data into a Gen3 repository (DM-21862)
            * conversion of all remaining {{ApPipeTask}} subtasks to Gen 3 (DM-21874, DM-21886)
            * creation of a YAML file configuring the pipeline (1 SP on this ticket)

            {{ApPipeTask}} has a lot of code for handling the case of calexp difference imaging templates. By general agreement within the AP and Middleware groups, we will *not* be porting this functionality to Gen 3 (see also DM-21874), and will reimplement it from scratch should we need it after migration.
            ebellm Eric Bellm made changes -
            Assignee Meredith Rawls [ mrawls ]
            ebellm Eric Bellm made changes -
            Epic Link DM-21442 [ 423049 ]
            ebellm Eric Bellm made changes -
            Sprint AP F19-6 (November) [ 958 ]
            ebellm Eric Bellm made changes -
            Rank Ranked higher
            swinbank John Swinbank made changes -
            Sprint AP F19-6 (November) [ 958 ] AP F19-6 (November), AP S20-1 (December) [ 958, 981 ]
            krzys Krzysztof Findeisen made changes -
            Link This issue relates to DM-22599 [ DM-22599 ]
            swinbank John Swinbank made changes -
            Epic Link DM-21442 [ 423049 ] DM-22633 [ 427742 ]
            krzys Krzysztof Findeisen made changes -
            Link This issue relates to DM-22663 [ DM-22663 ]
            cmorrison Chris Morrison made changes -
            Link This issue is blocked by DM-22741 [ DM-22741 ]
            swinbank John Swinbank made changes -
            Sprint AP F19-6 (November), AP S20-1 (December) [ 958, 981 ] AP F19-6 (November) [ 958 ]
            swinbank John Swinbank made changes -
            Link This issue is blocked by DM-24260 [ DM-24260 ]
            swinbank John Swinbank made changes -
            Assignee Meredith Rawls [ mrawls ] Krzysztof Findeisen [ krzys ]
            swinbank John Swinbank made changes -
            Sprint AP F19-6 (November) [ 958 ] AP F19-6 (November), AP S20-5 (April) [ 958, 986 ]
            swinbank John Swinbank made changes -
            Story Points 1 4
            swinbank John Swinbank made changes -
            Sprint AP F19-6 (November), AP S20-5 (April) [ 958, 986 ] AP F19-6 (November), AP S20-6 (May) [ 958, 987 ]
            swinbank John Swinbank made changes -
            Rank Ranked lower
            swinbank John Swinbank made changes -
            Epic Link DM-22633 [ 427742 ] DM-24341 [ 433028 ]
            krzys Krzysztof Findeisen made changes -
            Link This issue relates to DM-12549 [ DM-12549 ]
            krzys Krzysztof Findeisen made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            krzys Krzysztof Findeisen made changes -
            Link This issue is blocked by DM-25014 [ DM-25014 ]
            krzys Krzysztof Findeisen made changes -
            Link This issue is blocked by DM-23992 [ DM-23992 ]
            krzys Krzysztof Findeisen made changes -
            Link This issue is blocked by DM-25040 [ DM-25040 ]
            krzys Krzysztof Findeisen made changes -
            Link This issue blocks DM-23983 [ DM-23983 ]
            krzys Krzysztof Findeisen made changes -
            Link This issue blocks DM-23983 [ DM-23983 ]
            krzys Krzysztof Findeisen made changes -
            Link This issue is blocked by DM-23983 [ DM-23983 ]
            krzys Krzysztof Findeisen made changes -
            Link This issue is blocked by DM-23983 [ DM-23983 ]
            swinbank John Swinbank made changes -
            Epic Link DM-24341 [ 433028 ] DM-25145 [ 435263 ]
            swinbank John Swinbank made changes -
            Sprint AP F19-6 (November), AP S20-6 (May) [ 958, 987 ] AP F19-6 (November), AP S20-6 (May), AP F20-1 (June) [ 958, 987, 1019 ]
            krzys Krzysztof Findeisen made changes -
            Attachment convertRepo_standalone.py [ 44305 ]
            krzys Krzysztof Findeisen made changes -
            Attachment ap_pipe_docs.tar.gz [ 44307 ]
            krzys Krzysztof Findeisen made changes -
            Reviewers Chris Morrison, Gabor Kovacs, Meredith Rawls [ cmorrison, gkovacs, mrawls ]
            Status In Progress [ 3 ] In Review [ 10004 ]
            mrawls Meredith Rawls made changes -
            Status In Review [ 10004 ] Reviewed [ 10101 ]
            krzys Krzysztof Findeisen made changes -
            Resolution Done [ 10000 ]
            Status Reviewed [ 10101 ] Done [ 10002 ]

              People

              • Assignee:
                krzys Krzysztof Findeisen
                Reporter:
                krzys Krzysztof Findeisen
                Reviewers:
                Chris Morrison, Gabor Kovacs, Meredith Rawls
                Watchers:
                Chris Morrison, Gabor Kovacs, John Swinbank, Krzysztof Findeisen, Meredith Rawls, Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel