Fix Version/s: None
Component/s: daf_butler, obs_base
We bumped the LATISS visit region padding on
DM-30869, but have no way to update dimension records already in the database (other than dangerous direct-SQL usage).
On this ticket:
- add Registry.replaceDimensionData (should be able to delegate essentially everything to Database.replace);
- add config options to RawIngestTask and DefineVisitsTask that switch them to use this method instead of syncDimensionData;
- run DefineVisitsTask for LATISS in /repo/main to update those regions.
All of these new interfaces need to come with warnings about them being advanced/admin operations that should be only used to fix bad data after careful consideration - e.g. when the visit padding is increased, or we fix something in the metadata translation.
Jim Bosch added a comment -
I've added one more commit to the daf_butler branch to add support for testing for NULL in query expressions, which was missing functionality I discovered while trying to identify the LATISS exposures that needed padding.
Jim Bosch added a comment - I've added one more commit to the daf_butler branch to add support for testing for NULL in query expressions, which was missing functionality I discovered while trying to identify the LATISS exposures that needed padding.
I think this is ready for review. I'll actually re-run DefineVisitsTask on LATISS after the review is complete. Somebody should probably also run it on the OODS repos, though I have no idea what the current state of visits is there.
Instead of adding Registry.replaceDimensionData, I added an update flag to syncDimensionData and a replace flag to insertDimensionData; as with those methods more generally, the sync version is more flexible and returns more information but can't be vectorized.
First Jenkins run almost succeeded (and the ci_hscs did complete) but I accidentally included ci_cpp_gen2, which seems to no longer be maintained. So I'm running another one without ci_hsc here: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/34522/pipeline