Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-39876

Investigate pydantic 2 breakage of middleware

    XMLWordPrintable

Details

    Description

      Pydantic 2 came out last Friday. This breaks daf_butler. Investigate whether we can support both pydantic versions for now, else pin to <2.

      Attachments

        Activity

          No builds found.
          tjenness Tim Jenness created issue -
          tjenness Tim Jenness made changes -
          Field Original Value New Value
          Watchers Andy Salnikov, Jim Bosch, Kian-Tat Lim, Lee Kelvin, Tim Jenness [ Andy Salnikov, Jim Bosch, Kian-Tat Lim, Lee Kelvin, Tim Jenness ] Andy Salnikov, Jim Bosch, Kian-Tat Lim, Lee Kelvin, Matthias Wittgen, Tim Jenness [ Andy Salnikov, Jim Bosch, Kian-Tat Lim, Lee Kelvin, Matthias Wittgen, Tim Jenness ]
          tjenness Tim Jenness added a comment -

          A quick look at the migration guide (https://docs.pydantic.dev/dev-v2/migration/) and the bump-pydantic script suggests that supporting v2 and v1 at the same time is a bit of a non-starter.

          I propose we do the following:

          • Pin all the rubin-env <6 to pydantic < 2.
          • Change daf_butler et al to first try to import from pydantic.v1 (which is still available in v2) before importing from pydantic. This will let us work with v1 and v2 in the short term.
          • Allow v7 to use pydantic but because we learn from our mistakes pin it to <3.
          • At some later date after v7 is released we migrate the models over to v2.
          tjenness Tim Jenness added a comment - A quick look at the migration guide ( https://docs.pydantic.dev/dev-v2/migration/ ) and the bump-pydantic script suggests that supporting v2 and v1 at the same time is a bit of a non-starter. I propose we do the following: Pin all the rubin-env <6 to pydantic < 2. Change daf_butler et al to first try to import from pydantic.v1 (which is still available in v2) before importing from pydantic. This will let us work with v1 and v2 in the short term. Allow v7 to use pydantic but because we learn from our mistakes pin it to <3. At some later date after v7 is released we migrate the models over to v2.
          tjenness Tim Jenness made changes -
          Status To Do [ 10001 ] In Progress [ 3 ]
          tjenness Tim Jenness made changes -
          Component/s ci_middleware [ 20728 ]
          Component/s ctrl_mpexec [ 16800 ]
          Component/s pipe_base [ 10727 ]
          tjenness Tim Jenness added a comment -

          salnikov I've tested this with pydantic v2 and ci_middleware builds fine so it looks like it will be good enough.

          tjenness Tim Jenness added a comment - salnikov I've tested this with pydantic v2 and ci_middleware builds fine so it looks like it will be good enough.
          tjenness Tim Jenness made changes -
          Reviewers Andy Salnikov [ salnikov ]
          Status In Progress [ 3 ] In Review [ 10004 ]

          I checked daf_butler PR, but I see there are also branches in ctrl_mpexec and ci_middleware without corresponding PRs. Do you want to make PRs for those and have me look at them?

          salnikov Andy Salnikov added a comment - I checked daf_butler PR, but I see there are also branches in ctrl_mpexec and ci_middleware without corresponding PRs. Do you want to make PRs for those and have me look at them?

          pipe_base branches do not even register in JIRA, maybe we need to clean all outdated branches?

          salnikov Andy Salnikov added a comment - pipe_base branches do not even register in JIRA, maybe we need to clean all outdated branches?
          tjenness Tim Jenness added a comment -

          I do clean outdated branches in pipe_base and I'm at a bit of a loss as to why it never shows up. Sorry about the missing PRs – I forgot I hadn't made them yet.

          tjenness Tim Jenness added a comment - I do clean outdated branches in pipe_base and I'm at a bit of a loss as to why it never shows up. Sorry about the missing PRs – I forgot I hadn't made them yet.

          tjenness, do you want to make a PR for ci_middleware or should I mark the ticket as reviewed?

          salnikov Andy Salnikov added a comment - tjenness , do you want to make a PR for ci_middleware or should I mark the ticket as reviewed?

          Looks good.

          salnikov Andy Salnikov added a comment - Looks good.
          salnikov Andy Salnikov made changes -
          Status In Review [ 10004 ] Reviewed [ 10101 ]
          tjenness Tim Jenness made changes -
          Resolution Done [ 10000 ]
          Status Reviewed [ 10101 ] Done [ 10002 ]

          People

            tjenness Tim Jenness
            tjenness Tim Jenness
            Andy Salnikov
            Andy Salnikov, Jim Bosch, Kian-Tat Lim, Lee Kelvin, Matthias Wittgen, Tim Jenness
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Jenkins

                No builds found.