Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-14253

Modernize python in meas_base and meas_algorithms

    XMLWordPrintable

Details

    • 0.25
    • AP S18-5
    • Alert Production

    Description

      Modernize how ndarray is called, remove python 2 support and automatic pep8 checking

      Attachments

        Activity

          rowen Russell Owen added a comment -

          lauren do you have time to look at this? It's a set of small changes on two packages. It may be a bit easier to review them commit by commit (since so much of it is automated and trivial), but please do it however you like.

          rowen Russell Owen added a comment - lauren do you have time to look at this? It's a set of small changes on two packages. It may be a bit easier to review them commit by commit (since so much of it is automated and trivial), but please do it however you like.

          I'm happy to review, but I know all of nothing about pybind11 and proper "modern" usage of ndarray and I have yet to dip a toe into the world of "Travis".  Could you either point me to LSST documentation and/or coding standards that will allow me to asses the changes (or, perhaps more efficiently, point this to a reviewer who does know what is required!)?

          lauren Lauren MacArthur added a comment - I'm happy to review, but I know all of nothing about pybind11 and proper "modern" usage of ndarray and I have yet to dip a toe into the world of "Travis".  Could you either point me to LSST documentation and/or coding standards that will allow me to asses the changes (or, perhaps more efficiently, point this to a reviewer who does know what is required!)?
          rowen Russell Owen added a comment -

          Modernizing the use of ndarray in pybind11 wrappers is trivial:

          1) Remove #include "numpy/arrayobject.h" (and any other numpy header files)
          2) Remove the following:

              if (_import_array() < 0) {
                  PyErr_SetString(PyExc_ImportError, "numpy.core.multiarray failed to import");
                  return nullptr;
              }
          

          I don't know where it's written down (jbosch might know). However, it works, it makes warnings about using an outdated numpy API go away, and Jenkins is happy. I've done it to many packages now.

          For travis, we have a standard .travis.yml file we add to a repo. It is described here:
          https://developer.lsst.io/stack/adding-a-new-package.html?highlight=travis#configuring-github-repositories
          Fortunately it needs to changes per repo.

          The other step in enabling automatic pep8 checking with flake8 is to add a setup.cfg file. That one needs a few special exclusions if the package contains tasks (as meas_algorithms does), since tasks have those Doxygen ## blocks that flake8 hates. I'm not sure where the official docs are.

          rowen Russell Owen added a comment - Modernizing the use of ndarray in pybind11 wrappers is trivial: 1) Remove #include "numpy/arrayobject.h" (and any other numpy header files) 2) Remove the following: if (_import_array() < 0) { PyErr_SetString(PyExc_ImportError, "numpy.core.multiarray failed to import"); return nullptr; } I don't know where it's written down ( jbosch might know). However, it works, it makes warnings about using an outdated numpy API go away, and Jenkins is happy. I've done it to many packages now. For travis, we have a standard .travis.yml file we add to a repo. It is described here: https://developer.lsst.io/stack/adding-a-new-package.html?highlight=travis#configuring-github-repositories Fortunately it needs to changes per repo. The other step in enabling automatic pep8 checking with flake8 is to add a setup.cfg file. That one needs a few special exclusions if the package contains tasks (as meas_algorithms does), since tasks have those Doxygen ## blocks that flake8 hates. I'm not sure where the official docs are.
          jbosch Jim Bosch added a comment -

          Modernizing ndarray usage is not documented per se, but modern ndarray usage is:

          https://developer.lsst.io/pybind11/how-to.html#ndarray

          jbosch Jim Bosch added a comment - Modernizing ndarray usage is not documented per se, but modern ndarray usage is: https://developer.lsst.io/pybind11/how-to.html#ndarray
          Parejkoj John Parejko added a comment -

          See comments on the PR. Thanks for doing this: you cleaned up more things that I'd started to.

          Parejkoj John Parejko added a comment - See comments on the PR. Thanks for doing this: you cleaned up more things that I'd started to.

          People

            rowen Russell Owen
            rowen Russell Owen
            John Parejko
            Jim Bosch, John Parejko, Lauren MacArthur, Russell Owen
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Jenkins

                No builds found.