Uploaded image for project: 'Request For Comments'
  1. Request For Comments
  2. RFC-770

Allowed to use pydantic in pipelines code (add pydantic to scipipe env)

    XMLWordPrintable

Details

    • RFC
    • Status: Implemented
    • Resolution: Done
    • DM, TCT
    • None

    Description

      pydantic https://pydantic-docs.helpmanual.io is a python package that allows you to define classes using python type annotations, using this information to simplify serialization and support parameter validation.

      I have experimented with pydantic in DM-28678 in daf_butler to define classes for the butler client/server interface and it seemed to work well. Pydantic underpins the fastapi python web server https://fastapi.tiangolo.com that is the recommended python server to use to implement the Butler web service. It would be very helpful if I was allowed to use pydantic directly in daf_butler rather than trying to define the pydantic models solely in the server code.

      Can we please add pydantic to the conda scipipe environment? Bringing in pydantic is trivial and does not pull in any additional packages.

      Attachments

        Issue Links

          Activity

            In SQuaRE we've been using pydantic for some time and have found it very useful for defining configuration files. I would tend to still use dataclasses in most situations, but pydantic has much more powerful validation features, better serialization support, and support for annotating fields with documentation and examples, which makes it a better choice for self-documenting APIs (this is why FastAPI uses it) and configuration files.

            My work is outside the scipipe environment, so not directly relevant, but given my experiences I think this is a good idea.

            rra Russ Allbery added a comment - In SQuaRE we've been using pydantic for some time and have found it very useful for defining configuration files. I would tend to still use dataclasses in most situations, but pydantic has much more powerful validation features, better serialization support, and support for annotating fields with documentation and examples, which makes it a better choice for self-documenting APIs (this is why FastAPI uses it) and configuration files. My work is outside the scipipe environment, so not directly relevant, but given my experiences I think this is a good idea.
            rowen Russell Owen added a comment -

            I'm in favor. It would allow butler service to use FastAPI, which seems to be a good choice and the direction SQuaRE is heading.

            rowen Russell Owen added a comment - I'm in favor. It would allow butler service to use FastAPI, which seems to be a good choice and the direction SQuaRE is heading.
            jbosch Jim Bosch added a comment -

            I'm in favor, too. And I think this isn't just useful for daf_butler; I think it's more broadly the kind of way we should be interacting with YAML and JSON most of the time, and it's worth educating more developers about.

            jbosch Jim Bosch added a comment - I'm in favor, too. And I think this isn't just useful for daf_butler ; I think it's more broadly the kind of way we should be interacting with YAML and JSON most of the time, and it's worth educating more developers about.
            ktl Kian-Tat Lim added a comment -

            I would like to have this be the preferred way of validating YAML/JSON, over any other packages we may still have that accomplish the same function.

            ktl Kian-Tat Lim added a comment - I would like to have this be the preferred way of validating YAML/JSON, over any other packages we may still have that accomplish the same function.

            People

              tjenness Tim Jenness
              tjenness Tim Jenness
              Jim Bosch, Kian-Tat Lim, Russ Allbery, Russell Owen, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:
                Planned End:

                Jenkins

                  No builds found.