Fix Version/s: None
Sprint:AP F22-1 (June)
Our ap_verify datasets currently use version 1.0.1 of the registry schema. However, when we try to copy the dataset to a sandbox repository, it gets created with version 1.0.2, which causes a conflict between the imported exposure records and those created in raw ingest. For now, hand-patch the export.yaml file to match the 1.0.2 records that would have been created.
(This problem will also affect our test data for the Google Cloud prototype, but fixing that is out of scope for this issue.)
- relates to
DM-35069 Devise a scheme for butler export YAML migration
- To Do
|Field||Original Value||New Value|
|Status||To Do [ 10001 ]||In Progress [ 3 ]|
Krzysztof Findeisen added a comment -
Hi Tim Jenness, can you take a look, since you understand the implications of overwriting the records?
Krzysztof Findeisen added a comment - Hi Tim Jenness , can you take a look, since you understand the implications of overwriting the records?
|Reviewers||Tim Jenness [ tjenness ]|
|Status||In Progress [ 3 ]||In Review [ 10004 ]|
Tim Jenness made changes -
|Status||In Review [ 10004 ]||Reviewed [ 10101 ]|
|Resolution||Done [ 10000 ]|
|Status||Reviewed [ 10101 ]||Done [ 10002 ]|
Following discussion with Tim Jenness, I propose instead modifying DatasetIngestTask to overwrite the exposure records on raw ingest. We only have preexisting exposure records in order to support the visitSsoObjects dataset, and these records are supposed to be based on the same raws. The risk that updated records will disagree in a way that breaks e.g. solar system processing is small.