Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-13534

Upgrade ndarray to upstream 1.4.2

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: ndarray
    • Labels:
      None
    • Story Points:
      1
    • Sprint:
      DRP S18-3, DRP S18-4
    • Team:
      Data Release Production

      Description

      If all goes well, this should allow us to start removing our dependence on the NumPy C API (left for another ticket).

      This will probably require modifying our pybind11 build to install its CMake config files (unless we're doing that already, which I doubt), since ndarray now uses those to find pybind11.

        Attachments

          Issue Links

            Activity

            Hide
            jbosch Jim Bosch added a comment -

            Pim Schellart [X], here's that upgrade to the upstream ndarray version with no NumPy C API dependency that I've been promising for a while.  So far I've just tested that ndarray builds through Jenkins, but I've just kicked off the bigger run to make sure everything else still builds against it (with no modifications).

            I'm not super proud of the very platform-dependent patch I've added to the ndarray build, but it works on LSST's supported platforms and we can drop it when we go to pybind11 2.2.x.

             

            Show
            jbosch Jim Bosch added a comment - Pim Schellart [X] , here's that upgrade to the upstream ndarray version with no NumPy C API dependency that I've been promising for a while.  So far I've just tested that ndarray builds through Jenkins, but I've just kicked off the bigger run to make sure everything else still builds against it (with no modifications). I'm not super proud of the very platform-dependent patch I've added to the ndarray build, but it works on LSST's supported platforms and we can drop it when we go to pybind11 2.2.x.  
            Hide
            jbosch Jim Bosch added a comment -

            Upstream 1.4.0 generates a ton of new signed-unsigned comparison warnings (didn't see them earlier because CMake is less verbose than SCons by default).  I'll fix that and make a point release.

            Jenkins also reveald some code in afw that explicitly dependended on the old NumpyTraits stuff that will have to be rewritten.

            Pulling this out of review for now.

            Show
            jbosch Jim Bosch added a comment - Upstream 1.4.0 generates a ton of new signed-unsigned comparison warnings (didn't see them earlier because CMake is less verbose than SCons by default).  I'll fix that and make a point release. Jenkins also reveald some code in afw that explicitly dependended on the old NumpyTraits stuff that will have to be rewritten. Pulling this out of review for now.
            Hide
            jbosch Jim Bosch added a comment -

            Fixed the problems (mostly upstream, but a bit in afw), so this is ready for review again.

            I've also have a branch for the dev guide to update the documentation for how to use the ndarray/pybind11 wrappers.  Kian-Tat Lim, could you review that while Pim Schellart [X] looks at pybind11, ndarray, and afw?

             

            Show
            jbosch Jim Bosch added a comment - Fixed the problems (mostly upstream, but a bit in afw), so this is ready for review again. I've also have a branch for the dev guide to update the documentation for how to use the ndarray/pybind11 wrappers.  Kian-Tat Lim , could you review that while Pim Schellart [X] looks at pybind11, ndarray, and afw?  
            Hide
            pschella Pim Schellart [X] (Inactive) added a comment -

            Looks fine.

            Show
            pschella Pim Schellart [X] (Inactive) added a comment - Looks fine.
            Hide
            jbosch Jim Bosch added a comment -

            I have merged the code branches in case anyone wants to do cleanup work in this area at the DM AHM.  Leaving the ticket open until the dev guide changes have been reviewed and merged.

            Show
            jbosch Jim Bosch added a comment - I have merged the code branches in case anyone wants to do cleanup work in this area at the DM AHM.  Leaving the ticket open until the dev guide changes have been reviewed and merged.
            Hide
            ktl Kian-Tat Lim added a comment -

            Sorry I hadn't noticed this before. Text for the dev guide looks OK, modulo the one missing word mentioned in the PR (and I don't think this section needs formal approval).

            Show
            ktl Kian-Tat Lim added a comment - Sorry I hadn't noticed this before. Text for the dev guide looks OK, modulo the one missing word mentioned in the PR (and I don't think this section needs formal approval).

              People

              • Assignee:
                jbosch Jim Bosch
                Reporter:
                jbosch Jim Bosch
                Reviewers:
                Kian-Tat Lim, Pim Schellart [X] (Inactive)
                Watchers:
                Jim Bosch, Kian-Tat Lim, Pim Schellart [X] (Inactive)
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel