Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: daf_butler, obs_subaru
-
Labels:
-
Story Points:2
-
Epic Link:
-
Team:Data Release Production
Description
DM-16467 added some Python and bash scripts that add brighter-fatter kernel and defect datasets to a Gen3 repo. These use direct SQL calls instead of the Registry API, and have been broken by changes on DM-16227. This ticket will move that work into the daf.butler.Instrument class and its obs_* package specializations, allowing it to be done automatically by the gen2convert tool (in addition to fixing the breakage, of course).
Coming up with a new, consistent-across-obs-packages way to manage human-curated source for these datasets is definitely out of scope right now, but this might provide an example of how the Butler side of that might look to those who do design that.
Attachments
Activity
Looks generally very good. My only concern is that the ticket talks about guiding functional code into daf_butler, but that doesn't appear to have happened. The code seems to have been migrated into ci_hsc instead. I'm sure there's a reason, but it was not obvious to me from the review why that discrepancy exists.
My only concern is that the ticket talks about guiding functional code into daf_butler, but that doesn't appear to have happened. The code seems to have been migrated into ci_hsc instead
When writing up the ticket, I had forgotten that the gen2convert code in daf_butler doesn't really have a good way to obtain the right Instrument class. We'll probably want to address that eventually if we can cook up a good way to run gen2convert generically from the command-line, but at present it's really more of a set of high-level tools that can be used to put together a convert for a particular repo (which is what we've done in ci_hsc).
OK. Sounds fine to me.
Simon Krughoff, assigning this to you in the hopes it might be a good way to spread some knowledge in a good direction. PRs are:
Nate Lust, if it'd be helpful for your ongoing work to get this on master sooner, feel free to steal the review; I'll be around off-and-on over the weekend to follow up if that's the case.