Fix Version/s: None
The registry refactor to delegate to polymorphic helper classes should allow us to make many different kinds of changes to the Registry implementation that affect the database schema without breaking our ability to read old repos with new code. Actually making that happen requires a number of smaller changes, and there are many more that would further improve our backwards compatibility situation:
copying (and effectively freezing) the Registry config subtrees for dimensions and manager classes into per-repo configs, instead of always pulling defaults from daf_butler;we've already moved the dimension config into the database, which is even better; and check that the manager classes are consistent. Here, I'll just write the manager classes into the per-repo config to decouple them from the subject-to-change daf_butler defaults. starting to increment the version number in the dimensions config; recording the dimensions version in the database somewhere, so we can easily check for consistency and raise more helpful exceptions when the config is inconsistent with the database; recording the manager classes used in the database (for the same reason), and perhaps adding version numbers for these as well; adding dimension version numbers or some other form of stability to the byte-encoded form of DimensionGraph recording the names of dynamic tables in the static tables that manage them to reduce reliance on any particular name format making it so Manager classes that add foreign key fields can use compound keys, further encapsulating DDL schemas inside the managers;rejected as a bad idea (see first comment).
- start recording a version number in YAML export files.
Note that this ticket is about reducing the need for schema migration (or at least giving us more flexibility about when any particular repo is migrated). We'll still need to have some way to do actual schema migrations to upgrade old repos when they can't just be thrown away.
Jim Bosch added a comment - Most of this is already done (see annotations and strikethroughs in the original ticket description), and what's left is pretty easy and still worth doing. I'm moving the current tickets/ DM-24660 code to u/jbosch/ DM-24660 /compound-keys to make room.
Jim Bosch added a comment - I realize this is two review requests at once, but this is a very small one, and (with DM-27397 , In Review to Andy and Nate) the last schema-change one I've got on my milestone radar. PR is here .
I started working on this bullet:
on DM-24613 (where I thought it might save work overall, with
DM-21794on the horizon). After that attempt, I think it's not actually helpful for schema stability and is a distraction from more pressing work. I've put what I've done so far on the tickets/ DM-24660branch, and I'll revisit what to do with it when I actually get to this ticket.