A good way to make this minimal and yet functional might be to only have it support a single StorageClass (or a very limited set of StorageClasses), perhaps limited to some with no slicing or composites. Adding support for more complex StorageClasses could then be separate tickets.

Yes, that sounds like a good plan.

Ready for initial review of the general structure. There is virtually unlimited scope for improvement and cleanup/robustification, but the basic structure is there and all required methods to support a prototype Butler are implemented.

Please be brutal in this review, but keep in mind that this is a prototype implementation meant to verify the design and not (yet) production code.

Things that should still be done, but probably on separate small tickets are:

• Improve configuration
• Investigate adding a buffering layer to Formatter (write to buffer -> write buffer to file)
• Add more default storage classes
• Add basic building blocks for chaining (caching, multiple output formats for a single put, etc.) (mostly trivial separate implementations of the Datastore interface that just forward calls based on some further configuration)
• Add support for composites

To clarify, there are three levels of plugability/reuseability in the current design:

• Different implementations of the Datastore interface support different storage types (POSIX/HTTP/Object Store), these can then reuse:
• Different implementations of the Formatter interface that read and write files of a particular StorageClass (which can be configured and swapped out dynamically based on a config file).
• Different Datastore instances (of different type) can be chained to support things like a Proxy, storing to multiple file formats, etc.

Reviewed. I hope it's helpful.

Thanks for the review!