Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: afw
-
Labels:
-
Story Points:1
-
Epic Link:
-
Team:Data Release Production
-
Urgent?:No
Description
Timing tests in meas_extension_trailedSources in DM-35008 show that there is a hotspot from iterating through all the records in a catalog. This is caused by python deciding to call getitem repeatedly rather than using a custom iterator.
Eli Rykoff has shown that with a simple addition of an __iter__ method for rec in catalog can run 7 times faster. We speculate that adding a pybind11 iter implementation would be even quicker (since the python is calling the _getitem_ pybind11 implementation which includes an instance check for string).
With the new code, the tests in meas_extensions_trailedSources run about 5% faster than on main. (that's a "real world" number, not an iteration bench mark).
Using the HEAD^ commit seems to get a minor speed up of 3 seconds out of 413 (main takes 431 seconds) but that might just be normal variation.