This is worth doing (we don't want to be stuck on an old Eigen), but I worry that it will be sufficiently difficult that we should find some other way to get that determinant calculation. Here's the full story (copying and pasting from a Slack conversation with John Parejko):
- The first bit of bad news is that ndarray::EigenView has always been built on top of Eigen interfaces that were not documented and not considered to be public by the Eigen team, so it's not shocking that they've changed underneath us. But using those private interfaces was and continues to be the only way to write something like ndarray::EigenView: it's an object with Eigen-compatible operators but ndarray/numpy-compatible reference-counted memory management.
- The good news is that we may not need ndarray::EigenView anymore; its primary purpose was to allow ndarray to convert Eigen objects to Python, and to manage that memory safely on the C++ side. But pybind11 now has its own bindings for Eigen that are probably as good as ndarray's, though they have a different philosophy for memory management.
- The next piece of bad news is that we would need to convert all of our code at once to use pybind11's Eigen bindings; a package-by-package change would not work, because the two approaches define the same template specialization different ways, and using them both in shared libraries that can see each other's symbols would be a violation of the One Definition Rule. Building our pybind11 wrappers with symbols hidden by default would probably allow us to sidestep that issue, and is probably worth doing for its own sake (it may, for instance, help improve build times, build sizes, and module import times), but it will probably also take some effort to get working.
- The last piece of bad news is that we do occasionally use ndarray::EigenView in regular C++ code, especially in the bowels of the shapelet and meas_modelfit packages, and if we wanted to avoid continuing to rely on private Eigen interfaces, we'd need to modify that code to be able to work on regular Eigen::Map objects. That would require a lot of care and probably some refactoring, because it isn't a drop-in replacement; Eigen::Map doesn't manage its own memory, and can't be used safely (e.g.) as a data member without a lot of care.
So, the hopefully simple way to upgrade Eigen is to keep ndarray::EigenView and just fix whatever broke; "hopeful" because I don't know what changed in the private interfaces it relies on, and there's a chance that fixing it will essentially require reimplementing it on a new set of private Eigen interfaces. The more principled fix is to drop it, but that's both a lot of effort overall and the kind of effort that may require a lot of input from me, since I'm the big offender in terms of ndarray::EigenView usage in the current codebase (mea culpa).
This is worth doing (we don't want to be stuck on an old Eigen), but I worry that it will be sufficiently difficult that we should find some other way to get that determinant calculation. Here's the full story (copying and pasting from a Slack conversation with John Parejko):
So, the hopefully simple way to upgrade Eigen is to keep ndarray::EigenView and just fix whatever broke; "hopeful" because I don't know what changed in the private interfaces it relies on, and there's a chance that fixing it will essentially require reimplementing it on a new set of private Eigen interfaces. The more principled fix is to drop it, but that's both a lot of effort overall and the kind of effort that may require a lot of input from me, since I'm the big offender in terms of ndarray::EigenView usage in the current codebase (mea culpa).