Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-39876

Investigate pydantic 2 breakage of middleware

    XMLWordPrintable

    Details

      Description

      Pydantic 2 came out last Friday. This breaks daf_butler. Investigate whether we can support both pydantic versions for now, else pin to <2.

        Attachments

          Activity

          Hide
          tjenness Tim Jenness added a comment -

          A quick look at the migration guide (https://docs.pydantic.dev/dev-v2/migration/) and the bump-pydantic script suggests that supporting v2 and v1 at the same time is a bit of a non-starter.

          I propose we do the following:

          • Pin all the rubin-env <6 to pydantic < 2.
          • Change daf_butler et al to first try to import from pydantic.v1 (which is still available in v2) before importing from pydantic. This will let us work with v1 and v2 in the short term.
          • Allow v7 to use pydantic but because we learn from our mistakes pin it to <3.
          • At some later date after v7 is released we migrate the models over to v2.
          Show
          tjenness Tim Jenness added a comment - A quick look at the migration guide ( https://docs.pydantic.dev/dev-v2/migration/ ) and the bump-pydantic script suggests that supporting v2 and v1 at the same time is a bit of a non-starter. I propose we do the following: Pin all the rubin-env <6 to pydantic < 2. Change daf_butler et al to first try to import from pydantic.v1 (which is still available in v2) before importing from pydantic. This will let us work with v1 and v2 in the short term. Allow v7 to use pydantic but because we learn from our mistakes pin it to <3. At some later date after v7 is released we migrate the models over to v2.
          Hide
          tjenness Tim Jenness added a comment -

          Andy Salnikov I've tested this with pydantic v2 and ci_middleware builds fine so it looks like it will be good enough.

          Show
          tjenness Tim Jenness added a comment - Andy Salnikov I've tested this with pydantic v2 and ci_middleware builds fine so it looks like it will be good enough.
          Hide
          salnikov Andy Salnikov added a comment -

          I checked daf_butler PR, but I see there are also branches in ctrl_mpexec and ci_middleware without corresponding PRs. Do you want to make PRs for those and have me look at them?

          Show
          salnikov Andy Salnikov added a comment - I checked daf_butler PR, but I see there are also branches in ctrl_mpexec and ci_middleware without corresponding PRs. Do you want to make PRs for those and have me look at them?
          Hide
          salnikov Andy Salnikov added a comment -

          pipe_base branches do not even register in JIRA, maybe we need to clean all outdated branches?

          Show
          salnikov Andy Salnikov added a comment - pipe_base branches do not even register in JIRA, maybe we need to clean all outdated branches?
          Hide
          tjenness Tim Jenness added a comment -

          I do clean outdated branches in pipe_base and I'm at a bit of a loss as to why it never shows up. Sorry about the missing PRs – I forgot I hadn't made them yet.

          Show
          tjenness Tim Jenness added a comment - I do clean outdated branches in pipe_base and I'm at a bit of a loss as to why it never shows up. Sorry about the missing PRs – I forgot I hadn't made them yet.
          Hide
          salnikov Andy Salnikov added a comment -

          Tim Jenness, do you want to make a PR for ci_middleware or should I mark the ticket as reviewed?

          Show
          salnikov Andy Salnikov added a comment - Tim Jenness , do you want to make a PR for ci_middleware or should I mark the ticket as reviewed?
          Hide
          salnikov Andy Salnikov added a comment -

          Looks good.

          Show
          salnikov Andy Salnikov added a comment - Looks good.

            People

            Assignee:
            tjenness Tim Jenness
            Reporter:
            tjenness Tim Jenness
            Reviewers:
            Andy Salnikov
            Watchers:
            Andy Salnikov, Jim Bosch, Kian-Tat Lim, Lee Kelvin, Matthias Wittgen, Tim Jenness
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Jenkins

                No builds found.