This RFC proposes to use pybind11, instead of swig, to generate Python wrappers for C++ code in the LSST stack.
This was originally motivated by the desire to have more Pythonic interfaces for C++ code (see
In addition the Swig wrapper code is generally perceived to be complex and thus difficult to understand, extend, debug and maintain.
To investigate alternative wrapper solutions
DM-5470 created an example codebase with some tricky details likely to be encountered in the stack.
This codebase was wrapped with CFFI (
DM-5677), Cython ( DM-5471) and pybind11 ( DM-5676) with the latter found to be best by far (see http://dmtn-014.lsst.io/en/latest/ for more information).
It was decided, in
RFC-182, to attempt a trial conversion of the LSST stack (up to afw) which was done in DM-6168.
The current status as well as the pros and cons of a potential switch from swig to pybind11 were presented on community.
The main arguments for pybind11 are:
- wrappers are explicit, what you see is what you get (no magic);
- wrappers are just C++11 with no intermediate language so developers only need to know C++ (and Python);
- easier to support more Pythonic interfaces, for instance with attributes and keyword arguments;
- natural integration of docstrings describing the interface (as exposed to Python) into the wrapper file itself (if desired);
- pybind11 is small and header only, which makes it relatively easy to maintain, debug and extend if necessary (which would probably be considerably harder with swig).
Arguments against are:
- significant conversion effort;
- wrappers are verbose and require maintenance when the C++ interfaces change;
- pybind11 is a younger and smaller project (then swig) and the risk of abandonment by its maintainers is therefore probably higher.
This RFC intends to build consensus for a decision ultimately to be made by the System Architect.