Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-23737

DRP efforts toward Gen 3 middleware S20B: PipelineTask conversion

    Details

    • Type: Epic
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Epic Name:
      s20b-drp-gen3
    • Story Points:
      87
    • WBS:
      02C.04.01
    • Team:
      Data Release Production
    • Cycle:
      Spring 2020

      Description

      Registry refactorization for stabilization and speed
      Basic quantum graph generation for FGCM and jointcal
      Start of conversion of pipe_analysis, postprocessing, and coordination with AP

        Attachments

        Stories in Epic (Custom Issue Matrix)

        Key Summary Story Points Assignee Status
         
        DM-24940

        Move NamedKeyDict and NamedValueSet out of utils and clean up typing

        1 Jim Bosch Done
         
        DM-25080

        Finish static typing in daf.butler.registry

        1 Jim Bosch Done
         
        DM-25014

        Coadds converted to Gen 3 without abstract_filter

        3 Jim Bosch Done
         
        DM-25000

        Using components in PipelineTaskConnections no longer seems to work.

        2 Jim Bosch Done
         
        DM-24892

        Fix bug in gen2to3 when only special dataset types are being converted

        1 Jim Bosch Done
         
        DM-24780

        Initial mypy configuration for daf_butler

        1 Jim Bosch Done
         
        DM-24734

        Revive and profile RC2 QuantumGraph generation

        2 Jim Bosch Done
         
        DM-24698

        Clean up (at least) doc bug introduced in DM-21764

        1 Jim Bosch Done
         
        DM-24664

        Resurrect HSC RC2 Gen3 repo bootstrap, part 2

        3 Jim Bosch Done
         
        DM-24614

        Move dataset_location tables into manager/storage hierarchy

        4 Jim Bosch Done
         
        DM-24612

        Add indexes to dataset_collection tables

        1 Jim Bosch Done
         
        DM-24515

        Refactor gen3 butler.prune

        2 Jim Bosch Done
         
        DM-24435

        Freezing a config locks the registry(ies) of other instances of that config

        8 Nate Lust Done
         
        DM-24244

        w_2020_13 makeButlerRepo.py missing left parens error with Oracle

        1 Jim Bosch Done
         
        DM-24018

        Failure to flatten or filter chained collections in queries

        0.5 Jim Bosch Done
         
        DM-24013

        Fix bug in --output-run handling introduced in DM-21849

        0.5 Jim Bosch Done
         
        DM-22643

        convert visualizeVisit to gen3

        2 Christopher Waters Done
         
        DM-21849

        Make runs a type of collection

        12 Jim Bosch Done
         
        DM-21766

        Add per-dataset-type tables to Registry

        0 Jim Bosch Done
         
        DM-21764

        Better encapsulate dataset storage in Registry

        4 Jim Bosch Done
         
        DM-19807

        Avoid savepoints in Gen3 ingest and bulk conversions

        0 Jim Bosch Done
         
        DM-15536

        Add level of indirection in defining Visits from Exposures

        8 Jim Bosch Done
         
        DM-21850

        Investigate using ranges for skypix IDs in database

        4 Jim Bosch Won't Fix
         
        DM-16876

        Revisit Butler ctor keyword argument names

        1 Jim Bosch Won't Fix

          Activity

          There are no comments yet on this issue.

            People

            • Assignee:
              yusra Yusra AlSayyad
              Reporter:
              yusra Yusra AlSayyad
              Watchers:
              Jim Bosch, Nate Lust, Yusra AlSayyad
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Summary Panel