It's becoming increasingly clear that supporting schema migrations of butler registries is only part of the migration work that is necessary.

The missing part is migrating all the export.yaml files that are being used to the current schema. In DM-33942 I modified import to understand the old and new layouts of visit definitions but that is only part of the story.

For registry migrations we add additional logic to fill in missing record metadata (and allow command lines to specify some values) but butler import can not do any of this. This means that we end up in situations where importing a yaml drops the new metadata fields but migrating a registry fills in gaps. This happened today with one of the AP packages that uses export yaml for exposure records and then ingests raws (which tries to recreate the exposure records). Telling them to force overwrite of the records might be an option (but there is no command line) but I think it's warning us of a bigger issue.

We need to work out a way to be able to migrate export yaml files. We now store the universe version/namespace in files as well as the export format itself so this is tractable. Then we could make it so that butler import only has to support the newest variant and people can run butler migrate on their YAML files to bring them up to date.