Details
-
Type:
Bug
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: ap_verify
-
Labels:None
-
Story Points:1
-
Epic Link:
-
Sprint:AP F22-1 (June)
-
Team:Alert Production
-
Urgent?:No
Description
Our ap_verify datasets currently use version 1.0.1 of the registry schema. However, when we try to copy the dataset to a sandbox repository, it gets created with version 1.0.2, which causes a conflict between the imported exposure records and those created in raw ingest. For now, hand-patch the export.yaml file to match the 1.0.2 records that would have been created.
(This problem will also affect our test data for the Google Cloud prototype, but fixing that is out of scope for this issue.)
Attachments
Issue Links
- relates to
-
DM-35069 Devise a scheme for butler export YAML migration
- To Do
Activity
Hi Tim Jenness, can you take a look, since you understand the implications of overwriting the records?
Krzysztof Findeisen
added a comment - Hi Tim Jenness , can you take a look, since you understand the implications of overwriting the records? Looks like a good pragmatic fix to me.
Following discussion with Tim Jenness, I propose instead modifying DatasetIngestTask to overwrite the exposure records on raw ingest. We only have preexisting exposure records in order to support the visitSsoObjects dataset, and these records are supposed to be based on the same raws. The risk that updated records will disagree in a way that breaks e.g. solar system processing is small.