Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-9786

Investigate pybind11 build size

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Story Points:
      4
    • Sprint:
      DRP S17-4, DRP S17-5
    • Team:
      Data Release Production

      Description

      Pybind11 wrapped modules seem to be roughly twice the size of their Swig equivalents.
      Investigate the cause of this and suggest strategy to reduce the build size.

        Attachments

          Issue Links

            Activity

            Hide
            pschella Pim Schellart [X] (Inactive) added a comment -
            • The build size increase is due to wrapper overhead for each module.
            • The proposed strategy to fix it is to combine the wrappers into fewer modules.
            • A suggested implementation is attached to the ticket.
            • The change can to a large extent (in principle fully but that would make scripting needlessly complex) automated and the attached solution was largely generated such (need a good place to attach my quite ugly script for future reference).
            • There is some debate about the precise structure of the solution which should be resolved in an RFC (that is needed anyway because of required changes to the dev-guide, but that is outside of this tickets scope).
            Show
            pschella Pim Schellart [X] (Inactive) added a comment - The build size increase is due to wrapper overhead for each module. The proposed strategy to fix it is to combine the wrappers into fewer modules. A suggested implementation is attached to the ticket. The change can to a large extent (in principle fully but that would make scripting needlessly complex) automated and the attached solution was largely generated such (need a good place to attach my quite ugly script for future reference). There is some debate about the precise structure of the solution which should be resolved in an RFC (that is needed anyway because of required changes to the dev-guide, but that is outside of this tickets scope).
            Hide
            jbosch Jim Bosch added a comment -

            I'm happy to sign off on this. I don't think I'll have time to contribute as much to the design of the new desired code layout, but I'm happy to leave that to Pim Schellart [X]'s judgement.

            Show
            jbosch Jim Bosch added a comment - I'm happy to sign off on this. I don't think I'll have time to contribute as much to the design of the new desired code layout, but I'm happy to leave that to Pim Schellart [X] 's judgement.
            Hide
            swinbank John Swinbank added a comment -

            Pim Schellart [X], Jim Bosch and I discussed this at our meeting of 2017-04-26.

            My understanding is that, despite the above discussion, on further reflection they've reached the conclusion that the best approach to take here is actually that suggestion by Serge Monkewitz.

            We don't think that getting this done is an immediate priority, and we'd like to make sure we set aside enough time to do it properly. We're therefore going to schedule this in F17. Ticket will be DM-10384.

            Pim Schellart [X], can you sanity check my statement above, and, if you don't disagree, mark this ticket as "done"? Thanks!

            Show
            swinbank John Swinbank added a comment - Pim Schellart [X] , Jim Bosch and I discussed this at our meeting of 2017-04-26. My understanding is that, despite the above discussion, on further reflection they've reached the conclusion that the best approach to take here is actually that suggestion by Serge Monkewitz . We don't think that getting this done is an immediate priority, and we'd like to make sure we set aside enough time to do it properly. We're therefore going to schedule this in F17. Ticket will be DM-10384 . Pim Schellart [X] , can you sanity check my statement above, and, if you don't disagree, mark this ticket as "done"? Thanks!
            Hide
            jbosch Jim Bosch added a comment -

            I do think we may want to tweak Serge Monkewitz's approach quite a bit, actually, to reduce boilerplate, and part of why I want to take our time is to get those modifications figured out properly. But the core concept in his design of declaring all classes in a module before declaring any of their methods is something I think we need to adopt.

            Show
            jbosch Jim Bosch added a comment - I do think we may want to tweak Serge Monkewitz 's approach quite a bit, actually, to reduce boilerplate, and part of why I want to take our time is to get those modifications figured out properly. But the core concept in his design of declaring all classes in a module before declaring any of their methods is something I think we need to adopt.
            Hide
            pschella Pim Schellart [X] (Inactive) added a comment -

            Agreed

            Show
            pschella Pim Schellart [X] (Inactive) added a comment - Agreed

              People

              • Assignee:
                pschella Pim Schellart [X] (Inactive)
                Reporter:
                pschella Pim Schellart [X] (Inactive)
                Reviewers:
                Jim Bosch
                Watchers:
                Jim Bosch, John Parejko, John Swinbank, Krzysztof Findeisen, Pim Schellart [X] (Inactive), Russell Owen, Serge Monkewitz, Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel