Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-12738

Butler development during S18

    Details

    • Type: Epic
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
    • Templates:
    • Epic Name:
      s18-drp-butler
    • Story Points:
      140
    • WBS:
      02C.04.01
    • Team:
      Data Release Production
    • Cycle:
      Spring 2018

      Description

      This represents the DRP team contribution to the cross-subsystem "third generation butler" project. This project will be managed during S18 by Fritz Mueller.

        Attachments

        Stories in Epic (Custom Issue Matrix)

        Key Summary Story Points Assignee Status
         
        DM-14639

        Make Butler optionally transactional

        2 Pim Schellart [X] (Inactive) Done
         
        DM-14638

        Make Datastore optionally transactional

        3 Pim Schellart [X] (Inactive) Done
         
        DM-14545

        Add test for composite calexp to Butler

        0.5 Pim Schellart [X] (Inactive) Done
         
        DM-14544

        Add pickle support to Butler

        0.5 Pim Schellart [X] (Inactive) Done
         
        DM-14543

        Fix DatasetType registration

        0.5 Pim Schellart [X] (Inactive) Done
         
        DM-14520

        Re-enable Dataset->DataUnit foreign keys

        1 Jim Bosch Done
         
        DM-14467

        Make SqlRegistry more transactional

        6 Pim Schellart [X] (Inactive) Done
         
        DM-14321

        Make StorageClass pickleable (by making instances instead of subclasses)

        0.5 Pim Schellart [X] (Inactive) Done
         
        DM-14225

        Make PosixDatastore's internal records persistent

        4 Jim Bosch Done
         
        DM-12640

        Implement Initial Butler Registry Hierarchy

        6 Pim Schellart [X] (Inactive) Done
         
        DM-12635

        Write initial Gen2->Gen3 conversion script

        10 Jim Bosch Done
         
        DM-12627

        Implement Butler DatasetType

        4 Pim Schellart [X] (Inactive) Done
         
        DM-12624

        Implement Butler DataUnits

        Pim Schellart [X] (Inactive) Done
         
        DM-12620

        Write Initial Butler Metadata Schema Proposal

        12 Jim Bosch Done
         
        DM-12618

        Define Storage Classes for Butler

        2 Jim Bosch Done
         
        DM-13787

        afw::table::SchemaMapper::addMapping calls wrong overload with string literal

        0.1 Pim Schellart [X] (Inactive) Done
         
        DM-13700

        Discuss composites

        1 Pim Schellart [X] (Inactive) Done
         
        DM-13689

        Write up design proposal for Butler composites

        4 Jim Bosch Done
         
        DM-13688

        Write up design proposal for Registry/Datastore boundary

        2 Jim Bosch Done
         
        DM-13627

        Discuss Registry / Datastore boundary

        0.5 Pim Schellart [X] (Inactive) Done
         
        DM-12921

        Implement makePath in butler prototype

        1 Pim Schellart [X] (Inactive) Done
         
        DM-12920

        Prototype Butler configuration

        6 Pim Schellart [X] (Inactive) Done
         
        DM-13246

        Synchronize DMTN-056 with Butler prototype

        1.5 Pim Schellart [X] (Inactive) Done
         
        DM-13197

        Explore design issues with DatasetRef hierarchy and PreFlight (J)

        4 Jim Bosch Done
         
        DM-13196

        Explore design issues with DatasetRef hierarchy and PreFlight (P)

        4 Pim Schellart [X] (Inactive) Done
         
        DM-13372

        Attend Butler/SuperTask Meeting at Princeton

        8 Pim Schellart [X] (Inactive) Done
         
        DM-13399

        Use cls for first argument to Python metaclass methods

        0.5 Pim Schellart [X] (Inactive) Done
         
        DM-14139

        Implement Registry.ensureRun on Gen3 Butler

        0.5 Pim Schellart [X] (Inactive) Done
         
        DM-14114

        Add interface for camera specialization to Gen3 Butler

        2 Pim Schellart [X] (Inactive) Done
         
        DM-13869

        Implement basic Registry.find and related functionality

        3 Pim Schellart [X] (Inactive) Done
         
        DM-13842

        Implement access to DatasetStorage table

        2 Pim Schellart [X] (Inactive) Done
         
        DM-13840

        Prepare Butler for composite work to begin

        8 Pim Schellart [X] (Inactive) Done
         
        DM-13496

        Prototype Butler Registry schema definition in YAML

        4 Pim Schellart [X] (Inactive) Done
         
        DM-13378

        Attend Gen3 Middleware Design/Hack Week

        8 Jim Bosch Done
         
        DM-14377

        calexp_camera is an ImageF

        0.1 Pim Schellart [X] (Inactive) Done
         
        DM-14012

        Add helper container class supporting topologically sorted iteration over elements

        1 Pim Schellart [X] (Inactive) Done
         
        DM-13895

        Implement Butler provenance primitives

        4 Pim Schellart [X] (Inactive) Done
         
        DM-13963

        Enable foreign_key integrity checking on Butler SqlRegistry

        1 Pim Schellart [X] (Inactive) Done
         
        DM-12633

        Implement Direct SQL Butler Schema Creation

        2 Pim Schellart [X] (Inactive) Won't Fix
         
        DM-12632

        Implement Butler DataUnit Packing or Hashing

        10 Unassigned Won't Fix
         
        DM-12625

        Investigate machine generating/verifying Butler DataUnits

        4 Unassigned Won't Fix
         
        DM-12613

        Implement minimal butler Registry prototype with sqlalchemy orm

        6 Pim Schellart [X] (Inactive) Won't Fix
         
        DM-13063

        Find solution for ids in butler prototype

        2 Pim Schellart [X] (Inactive) Won't Fix
         
        DM-13171

        Convert butler Registry test data to use sqlalchemy

        1 Pim Schellart [X] (Inactive) Won't Fix
         
        DM-13287

        Redesign DataUnit class (and children) in Butler prototype

        4 Pim Schellart [X] (Inactive) Won't Fix
         
        DM-12647

        Implement Basic Collection Management in Registries

        2 Pim Schellart [X] (Inactive) Invalid
         
        DM-12646

        Implement Provenance Methods in Registries

        6 Pim Schellart [X] (Inactive) Invalid
         
        DM-12645

        Implement addDataset in Registries

        6 Pim Schellart [X] (Inactive) Invalid
         
        DM-12644

        Implement registerDatasetType in Registries

        2 Pim Schellart [X] (Inactive) Invalid
         
        DM-12630

        Implement Butler Quantum

        2 Pim Schellart [X] (Inactive) Invalid
         
        DM-12629

        Implement Butler Run

        2 Pim Schellart [X] (Inactive) Invalid
         
        DM-12628

        Implement Butler DatasetRef

        6 Pim Schellart [X] (Inactive) Invalid
         
        DM-12626

        Implement Butler DataUnitSet

        4 Pim Schellart [X] (Inactive) Invalid
         
        DM-13033

        Implement support for composite datasets in butler Registry

        6 Pim Schellart [X] (Inactive) Invalid
         
        DM-13820

        Factor out DataUnit description from Schema object

        2 Pim Schellart [X] (Inactive) Invalid
         
        DM-13991

        Add Gen3 Butler support to CmdLineTask

        4 Jim Bosch Invalid

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              swinbank John Swinbank
              Watchers:
              John Swinbank, Pim Schellart [X] (Inactive), Yusra AlSayyad
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Summary Panel