Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-12738

Butler development during S18

    XMLWordPrintable

    Details

    • Type: Epic
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
    • Epic Name:
      s18-drp-butler
    • Story Points:
      140
    • WBS:
      02C.04.01
    • Team:
      Data Release Production
    • Cycle:
      Spring 2018

      Description

      This represents the DRP team contribution to the cross-subsystem "third generation butler" project. This project will be managed during S18 by Fritz Mueller.

        Attachments

        Stories in Epic (Custom Issue Matrix)

        Key Summary Story Points Assignee Status
         
        		DM-14639

        Make Butler optionally transactional

        		 2 Pim Schellart [X] (Inactive) Done
         
        		DM-14638

        Make Datastore optionally transactional

        		 3 Pim Schellart [X] (Inactive) Done
         
        		DM-14545

        Add test for composite calexp to Butler

        		 0.5 Pim Schellart [X] (Inactive) Done
         
        		DM-14544

        Add pickle support to Butler

        		 0.5 Pim Schellart [X] (Inactive) Done
         
        		DM-14543

        Fix DatasetType registration

        		 0.5 Pim Schellart [X] (Inactive) Done
         
        		DM-14377

        calexp_camera is an ImageF

        		 0.1 Pim Schellart [X] (Inactive) Done
         
        		DM-14321

        Make StorageClass pickleable (by making instances instead of subclasses)

        		 0.5 Pim Schellart [X] (Inactive) Done
         
        		DM-14520

        Re-enable Dataset->DataUnit foreign keys

        		 1 Jim Bosch Done
         
        		DM-13496

        Prototype Butler Registry schema definition in YAML

        		 4 Pim Schellart [X] (Inactive) Done
         
        		DM-12640

        Implement Initial Butler Registry Hierarchy

        		 6 Pim Schellart [X] (Inactive) Done
         
        		DM-12635

        Write initial Gen2->Gen3 conversion script

        		 10 Jim Bosch Done
         
        		DM-12627

        Implement Butler DatasetType

        		 4 Pim Schellart [X] (Inactive) Done
         
        		DM-12624

        Implement Butler DataUnits

        		 Pim Schellart [X] (Inactive) Done
         
        		DM-12620

        Write Initial Butler Metadata Schema Proposal

        		 12 Jim Bosch Done
         
        		DM-12618

        Define Storage Classes for Butler

        		 2 Jim Bosch Done
         
        		DM-12921

        Implement makePath in butler prototype

        		 1 Pim Schellart [X] (Inactive) Done
         
        		DM-12920

        Prototype Butler configuration

        		 6 Pim Schellart [X] (Inactive) Done
         
        		DM-13399

        Use cls for first argument to Python metaclass methods

        		 0.5 Pim Schellart [X] (Inactive) Done
         
        		DM-13378

        Attend Gen3 Middleware Design/Hack Week

        		 8 Jim Bosch Done
         
        		DM-13372

        Attend Butler/SuperTask Meeting at Princeton

        		 8 Pim Schellart [X] (Inactive) Done
         
        		DM-13246

        Synchronize DMTN-056 with Butler prototype

        		 1.5 Pim Schellart [X] (Inactive) Done
         
        		DM-14225

        Make PosixDatastore's internal records persistent

        		 4 Jim Bosch Done
         
        		DM-14139

        Implement Registry.ensureRun on Gen3 Butler

        		 0.5 Pim Schellart [X] (Inactive) Done
         
        		DM-14114

        Add interface for camera specialization to Gen3 Butler

        		 2 Pim Schellart [X] (Inactive) Done
         
        		DM-14467

        Make SqlRegistry more transactional

        		 6 Pim Schellart [X] (Inactive) Done
         
        		DM-13700

        Discuss composites

        		 1 Pim Schellart [X] (Inactive) Done
         
        		DM-13689

        Write up design proposal for Butler composites

        		 4 Jim Bosch Done
         
        		DM-13688

        Write up design proposal for Registry/Datastore boundary

        		 2 Jim Bosch Done
         
        		DM-13627

        Discuss Registry / Datastore boundary

        		 0.5 Pim Schellart [X] (Inactive) Done
         
        		DM-13895

        Implement Butler provenance primitives

        		 4 Pim Schellart [X] (Inactive) Done
         
        		DM-13869

        Implement basic Registry.find and related functionality

        		 3 Pim Schellart [X] (Inactive) Done
         
        		DM-13787

        afw::table::SchemaMapper::addMapping calls wrong overload with string literal

        		 0.1 Pim Schellart [X] (Inactive) Done
         
        		DM-13842

        Implement access to DatasetStorage table

        		 2 Pim Schellart [X] (Inactive) Done
         
        		DM-13840

        Prepare Butler for composite work to begin

        		 8 Pim Schellart [X] (Inactive) Done
         
        		DM-13963

        Enable foreign_key integrity checking on Butler SqlRegistry

        		 1 Pim Schellart [X] (Inactive) Done
         
        		DM-14012

        Add helper container class supporting topologically sorted iteration over elements

        		 1 Pim Schellart [X] (Inactive) Done
         
        		DM-13197

        Explore design issues with DatasetRef hierarchy and PreFlight (J)

        		 4 Jim Bosch Done
         
        		DM-13196

        Explore design issues with DatasetRef hierarchy and PreFlight (P)

        		 4 Pim Schellart [X] (Inactive) Done
         
        		DM-12633

        Implement Direct SQL Butler Schema Creation

        		 2 Pim Schellart [X] (Inactive) Won't Fix
         
        		DM-12632

        Implement Butler DataUnit Packing or Hashing

        		 10 Unassigned Won't Fix
         
        		DM-12625

        Investigate machine generating/verifying Butler DataUnits

        		 4 Unassigned Won't Fix
         
        		DM-12613

        Implement minimal butler Registry prototype with sqlalchemy orm

        		 6 Pim Schellart [X] (Inactive) Won't Fix
         
        		DM-13287

        Redesign DataUnit class (and children) in Butler prototype

        		 4 Pim Schellart [X] (Inactive) Won't Fix
         
        		DM-13063

        Find solution for ids in butler prototype

        		 2 Pim Schellart [X] (Inactive) Won't Fix
         
        		DM-13171

        Convert butler Registry test data to use sqlalchemy

        		 1 Pim Schellart [X] (Inactive) Won't Fix
         
        		DM-12647

        Implement Basic Collection Management in Registries

        		 2 Pim Schellart [X] (Inactive) Invalid
         
        		DM-12646

        Implement Provenance Methods in Registries

        		 6 Pim Schellart [X] (Inactive) Invalid
         
        		DM-12645

        Implement addDataset in Registries

        		 6 Pim Schellart [X] (Inactive) Invalid
         
        		DM-12644

        Implement registerDatasetType in Registries

        		 2 Pim Schellart [X] (Inactive) Invalid
         
        		DM-12630

        Implement Butler Quantum

        		 2 Pim Schellart [X] (Inactive) Invalid
         
        		DM-12629

        Implement Butler Run

        		 2 Pim Schellart [X] (Inactive) Invalid
         
        		DM-12628

        Implement Butler DatasetRef

        		 6 Pim Schellart [X] (Inactive) Invalid
         
        		DM-12626

        Implement Butler DataUnitSet

        		 4 Pim Schellart [X] (Inactive) Invalid
         
        		DM-13820

        Factor out DataUnit description from Schema object

        		 2 Pim Schellart [X] (Inactive) Invalid
         
        		DM-13991

        Add Gen3 Butler support to CmdLineTask

        		 4 Jim Bosch Invalid
         
        		DM-13033

        Implement support for composite datasets in butler Registry

        		 6 Pim Schellart [X] (Inactive) Invalid

          Activity

          There are no comments yet on this issue.

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            swinbank John Swinbank
            Watchers:
            John Swinbank, Pim Schellart [X] (Inactive), Yusra AlSayyad
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Jenkins

                No builds found.