Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: meas_base
-
Labels:None
-
Story Points:3
-
Team:Architecture
-
Urgent?:No
Description
One mitigation for the Ventura linking problem (DM-37301) could be to reduce the number of shared libraries we are linking against. Import of meas_base can trigger the linking problem in Ventura and would be a good place to experiment since there are currently 21 separate pybind11 interface files being loaded (each of those linking against many shared libraries).
This ticket is to combine them into a single shared library, similar to what was done with sphgeom in DM-25877. This should have the side effect of making the build faster and also improving import time in general.
Attachments
Issue Links
- is cloned by
-
- Done
Activity
https://github.com/lsst/testdata_jointcal/blob/main/hsc/repo/config/singleFrameDriver.py
imports a bunch of meas.base submodules.
One could adapt Eli's implementation of adding additional python files to not mess with the
namespaces.
Matthias Wittgen
added a comment - - edited https://github.com/lsst/testdata_jointcal/blob/main/hsc/repo/config/singleFrameDriver.py
imports a bunch of meas.base submodules.
One could adapt Eli's implementation of adding additional python files to not mess with the
namespaces. That is a problem with configs trying to record everything that has been imported. This is a long standing problem of pex_config not being able to tell what it needs to import versus everything that is currently imported when the config is saved.
See DM-7002 and DM-19831 (which are effectively describing the same problem). Fixing that would solve the real problem but not solve the problem of being able to read persisted configs. Kian-Tat Lim I assume this means that all these modules are part of the public API solely because of the config persistence?
This specific config is not a problem because singleFrameDriver no longer exists.
Jim Bosch suggests you include stubs that issue deprecation warnings – they should only be loaded by older configs. Eventually the configs will no longer list them and in a year we can delete the files.
I think it's always going to be a problem reading old config files with newer Science Pipelines versions. We can't guarantee that the config parameters will remain the same, even if the modules remained the same.
Matthias Wittgen can you explain what's blocking this ticket at the moment? Are you needing to RFC whether you can remove the sdssShape import from meas_base? Are you needing to optimize the code some more?