Details
-
Type:
Story
-
Status: To Do
-
Resolution: Unresolved
-
Fix Version/s: None
-
Component/s: Documentation
-
Labels:
-
Urgent?:No
Description
The Common Dataset Policy document has a refcat section that is currently gen2 specific: all gen2 refcats live in one directory tree, which is where we put the readme files that describe them. We need to update this document to reflect how we plan to manage gen3 refcats and their associated documentation (currently in various readme files in the gen2 /datasets/ref_cats/ directory tree).
For the /repo/main gen3 repo on lsst-devl, we can continue to keep refcats that directory, and ingest them with --transfer=direct so that they stay together in the same path. For S3 or other bucket-based storage, we'll have to come up with something other policy.
That whole document is very gen2-specific, but I thought it served a very useful purpose for managing large ingests; coming up with the equivalent gen3 policies and updating it for gen3 overall would be good, I think.