Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: daf_butler, obs_base
-
Labels:
-
Story Points:3
-
Epic Link:
-
Team:Data Release Production
-
Urgent?:No
Description
We bumped the LATISS visit region padding on DM-30869, but have no way to update dimension records already in the database (other than dangerous direct-SQL usage).
On this ticket:
- add Registry.replaceDimensionData (should be able to delegate essentially everything to Database.replace);
- add config options to RawIngestTask and DefineVisitsTask that switch them to use this method instead of syncDimensionData;
- run DefineVisitsTask for LATISS in /repo/main to update those regions.
All of these new interfaces need to come with warnings about them being advanced/admin operations that should be only used to fix bad data after careful consideration - e.g. when the visit padding is increased, or we fix something in the metadata translation.
Attachments
Activity
Looks good. Thanks.
I've added one more commit to the daf_butler branch to add support for testing for NULL in query expressions, which was missing functionality I discovered while trying to identify the LATISS exposures that needed padding.
I think this is ready for review. I'll actually re-run DefineVisitsTask on LATISS after the review is complete. Somebody should probably also run it on the OODS repos, though I have no idea what the current state of visits is there.
Instead of adding Registry.replaceDimensionData, I added an update flag to syncDimensionData and a replace flag to insertDimensionData; as with those methods more generally, the sync version is more flexible and returns more information but can't be vectorized.
PRs are:
First Jenkins run almost succeeded (and the ci_hscs did complete) but I accidentally included ci_cpp_gen2, which seems to no longer be maintained. So I'm running another one without ci_hsc here: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/34522/pipeline