While the Gen3 Butler provides some intrinsic structure to its data repositories, considerably more is left to convention (often encoded in higher-level packages, like obs_base). This RFC is a proposal for how to organize data repositories in detail, focusing on collection naming conventions, filesystem locations, and developer workflows. The immediate focus is the environment at NCSA, but it is hoped that much of this will hold for the IDF and USDF as well.
The content of the proposal is on the tickets/
DM-27147 branch of DMTN-167, which you can find:
To improve threading (there are a lot of disparate topics here), I'd prefer to have most discussion on the PR. If you want to make a big-picture comment, just find a semi-relevant line. Please also watch the PR if you'd like to keep up with updates. I do plan to update the technote in response to uncontroversial comments throughout the RFC.
There is one topic that I would like to keep here on Jira: the "meta" discussion of where these policies should live after they are adopted. I assume the answer is mostly the DM Developer Guide, but that may not be appropriate for everything.
There are also code branches for
DM-27147, which include the changes to the codebase necessary to implement these policies. I'm hoping to get these reviewed and merged before the RFC completes, as I'm confident they'll bring us much closer to whatever the agreed-upon policy ends up being, and I strongly suspect we'll want to start trying to actually stand up a shared repository based on this proposal before we've given it enough time to gather comments.