Thanks Chris — nice work!
However, I have left a bunch of comments on GitHub, of a variety of levels of pickiness.
I guess the big issue I don't understand is why we convert everything from a Pandas DataFrame to dicts, then convert them all back again at the end of the calculations. This seems to be less than ideal for a few reasons:
- That conversion presumably isn't particularly fast;
- I'm guessing (without testing) that it would be more efficient for the calculations to be done column-wise on the DataFrame rather than by iterating over each DIAObject individually;
- The DataFrame has a well-structured data model (somewhat equivalent to the afw::table schema) which I think would actually be useful here (to avoid plugins clobbering each other's results, for example).
I should add that I'm a total Pandas neophyte, so it might be that I've missed something obvious. Can you fill me in on the thinking?
I think the answer to the above has a bearing on a bunch of my comments on GitHub — some of my comments there will become more or less important when I understand how Pandas fits in to the big picture — so it's probably not worth your while spending a lot of time on them until we've converged here.