Details
-
Type:
Bug
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: afw
-
Labels:None
-
Story Points:6
-
Epic Link:
-
Sprint:AP F19-5 (October)
-
Team:Alert Production
Description
DM-19582 revealed that a Python subclass of lsst.afw.typehandling.Storable may be garbage collected while C++ code still has a pointer to it (for example, if the Python object is constructed as a "temporary" that is never bound to a name). When this happens, the C++ Storable does not segfault, but instead behaves as if none of Storable's methods were overridden.
This is a known bug in pybind11, but two years of discussion have failed to converge to a solution. There are several workarounds posted on GitHub; I briefly tried the one listed in pybind11#1546 (which most closely matches our case) but wasn't able to fix the problem.
Investigate this issue more systematically and determine if there is any workaround that works in our case. This problem is unlikely to be fixed upstream, and being unable to use Python subclasses of Storable from C++ will significantly impair our persistence plans.
Attachments
Issue Links
- relates to
-
DM-19582 Add cross-language GenericMap unit tests
- Done
Activity
Progress report, by workaround:
- pybind11#1546: no effect; can't figure out why.
- pybind11#1389, modified "trampoline" class: cannot be done in our code because our trampoline needs to support C++ subclasses of Storable.
- pybind11#1389, custom shared pointer: works for Python classes that inherit directly from Storable. Inheriting from C++ subclasses of Storable not yet tested.
- pybind11#1333: not yet tested (I have only a rough description to go by).
- pybind11#1145: cannot be done in our code because most Storable classes need multiple inheritance on the C++ side.
Krzysztof Findeisen
added a comment - - edited Progress report, by workaround:
pybind11#1546 : no effect; can't figure out why.
pybind11#1389 , modified "trampoline" class: cannot be done in our code because our trampoline needs to support C++ subclasses of Storable .
pybind11#1389 , custom shared pointer: works for Python classes that inherit directly from Storable . Inheriting from C++ subclasses of Storable not yet tested .
pybind11#1333 : not yet tested (I have only a rough description to go by).
pybind11#1145 : cannot be done in our code because most Storable classes need multiple inheritance on the C++ side.
Hi Jim Bosch, sorry for the timing, but would you be willing to review this? It's 340 lines, but requires an in-depth understanding of pybind11.
Krzysztof Findeisen
added a comment - Hi Jim Bosch , sorry for the timing, but would you be willing to review this? It's 340 lines, but requires an in-depth understanding of pybind11. 
The unit test test_Storable.py:PythonStorableTestSuite.testGarbageCollection reproduces this bug reliably, at least on my computer.