Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-35071

ap_verify can't import from old ap_verify dataset

    XMLWordPrintable

Details

    • Bug
    • Status: Done
    • Resolution: Done
    • None
    • ap_verify
    • None
    • 1
    • AP F22-1 (June)
    • Alert Production
    • No

    Description

      Our ap_verify datasets currently use version 1.0.1 of the registry schema. However, when we try to copy the dataset to a sandbox repository, it gets created with version 1.0.2, which causes a conflict between the imported exposure records and those created in raw ingest. For now, hand-patch the export.yaml file to match the 1.0.2 records that would have been created.

      (This problem will also affect our test data for the Google Cloud prototype, but fixing that is out of scope for this issue.)

      Attachments

        Issue Links

          Activity

            Following discussion with tjenness, I propose instead modifying DatasetIngestTask to overwrite the exposure records on raw ingest. We only have preexisting exposure records in order to support the visitSsoObjects dataset, and these records are supposed to be based on the same raws. The risk that updated records will disagree in a way that breaks e.g. solar system processing is small.

            krzys Krzysztof Findeisen added a comment - Following discussion with tjenness , I propose instead modifying DatasetIngestTask to overwrite the exposure records on raw ingest. We only have preexisting exposure records in order to support the visitSsoObjects dataset, and these records are supposed to be based on the same raws. The risk that updated records will disagree in a way that breaks e.g. solar system processing is small.

            Hi tjenness, can you take a look, since you understand the implications of overwriting the records?

            krzys Krzysztof Findeisen added a comment - Hi tjenness , can you take a look, since you understand the implications of overwriting the records?
            tjenness Tim Jenness added a comment -

            Looks like a good pragmatic fix to me.

            tjenness Tim Jenness added a comment - Looks like a good pragmatic fix to me.

            People

              krzys Krzysztof Findeisen
              krzys Krzysztof Findeisen
              Tim Jenness
              Eric Bellm, Ian Sullivan, Krzysztof Findeisen, Meredith Rawls, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Jenkins

                  No builds found.