Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: daf_butler
-
Labels:
-
Story Points:10
-
Epic Link:
-
Sprint:BG3_S18_01, BG3_S18_02, BG3_S18_03, BG3_S18_04
-
Team:Data Release Production
Description
(This ticket has evolved considerably from its initial content; I'm rewriting the description rather late in the implementation to make it more accurate and provide more reasonable stopping criteria).
Put together a set of scripts for populating Gen3 Registry databases from Gen2 data repositories, with a focus on being able to generate a Registry test dataset from ci_hsc.
This will have a few limitations relative to the more full-featured script we'll need for converting more important Gen2 repositories:
- No Datastore/StorageInfo content will be written yet (just Registry itself).
- Only HSC data will be supported.
- We'll assume a 1-1 relationship between Exposures and Visits.
- We will not try to convert calibration repositories.
- No spatial join tables will be populated.
- All entries in parent repos will be included in the Collections created for their children, rather than trying to limit this to just the relevant DataUnits/DatasetTypes.
Attachments
Issue Links
Activity
This is not done, but I'm putting it out to "review" to give Pim Schellart [X] a chance to take a look at what's there, ask questions (PR should be good for this), and take it over if it becomes a blocker for other Butler work while I'm away next week. It may also be most efficient for the same person to work on both this and the Registry DataUnit interfaces at the same time.
Only about half of this is working, but it should be safe to merge to master so the remainder of the work can be done on another ticket. I also don't have any problem with just continuing to work on this branch.
More comments how to approach the code forthcoming on the PR.
Pim Schellart [X], this is now ready for "real" review. I have no plans to do further work on this ticket.
Thanks! I'll have a look.
Pim Schellart [X] (Inactive)
added a comment - Thanks! I'll have a look. See comments on PR.
Pim Schellart [X] (Inactive)
added a comment - See comments on PR. 
Stealing this from Pim Schellart [X]. I'll start by making a script to dump some simple-coadd processing datasets (outputs of mulitBandDriver.py) and their associated data units from a Gen2 repo. I may just transfer the results of that (rather than the script itself) to a commit on another branch so others can use them to write unit tests while I make the script more general.