Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: None
-
Labels:
-
Story Points:7
-
Team:Data Facility
Description
Migration of scientific datasets to butler repository structure per RFC-95.
Attachments
Issue Links
- is parent task of
-
DM-8116 /datasets/hsc/raw/newhorizons/ (source: /lsst8/newhorizons)
- Done
-
DM-8079 /datasets/hsc/raw/commissioning/ (source: /lsst3/HSC/)
- Done
-
DM-8117 /datasets/sdss/preprocessed/dr9/ (source: /lsst7/sdss/dr9/)
- Done
-
DM-8118 /datasets/refcats/gaia_DR1_v1 (source: /gpfs/fs0/home/ctslater/gaia_refcat/*)
- Done
-
DM-8121 /datasets/sdss/preprocessed/dr7/ (source: /datasets/stripe82/dr7)
- Done
-
DM-8123 /datasets/refcats/astrometry_net_data/ (source: /lsst7/astrometry_net_data/*)
- Done
-
DM-8124 /dataset/decam/ (source: /lsst8/decam)
- Done
Using this format:
/datasets/<camera>/repo/ (This is where Butler root locates. It has _mapper, registry.sqlite3, links to actual data, etc)
/datasets/<camera>/repo/rerun/ (later the processed results go here.)
/datasets/<camera>/preprocessed/ (For example /datasets/sdss/preprocessed/dr9/, /datasets/decam/preprocessed/cp/, etc)
/datasets/<camera>/raw/<survey-name>/<actual raw data> (where actual files live, uningested. They don't change with Butler template changes.)
Final format may change pending discussions.
/datasets/hsc/raw/commissioning/ (source: /lsst3/HSC/, owner: price )
/datasets/hsc/raw/newhorizons/ (source: /lsst8/newhorizons, owner: ctslater)
/datasets/sdss/preprocessed/dr9/ (source: /lsst7/sdss/dr9/, owner: mjuric )
/datasets/sdss/preprocessed/dr7/ (source: /datasets/stripe82/dr7 owner: daues)
/datasets/all-sky (source: /lsst/all-sky-ASIVA, owner: Mike Fitzgerald)
/datasets/all-sky-ASIVA (source: /lsst/all-sky-ASIVA, owner: Jacques Sebag)
/datasets/refcats/gaia_DR1_v1 (source: /gpfs/fs0/home/ctslater/gaia_refcat/*, owner: ctslater)
/datasets/refcats/astrometry_net_data/ (source: /lsst7/astrometry_net_data/*, owner: price)
/dataset/decam/data/ (source: /lsst8/decam, owner: mwv)
/datasets/lsstSim (source: external)
/datasets/decam/preprocessed/hits (source: TBD)