Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-7717

Full-stack port to pybind11

    XMLWordPrintable

    Details

    • Type: Epic
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Epic Name:
      s17-drp-pybind11
    • Story Points:
      100
    • WBS:
      02C.04.01
    • Team:
      Data Release Production
    • Cycle:
      Spring 2017

      Description

      Complete the work started in F16 to convert the Science Pipelines stack (lsst_distrib, plus associated meas_extensions) to expose C++ based functionality to Python using pybind11. Remove all Swig from the stack.

        Attachments

          Issue Links

          blocks

          Bug - A problem which impairs or prevents the functions of the product. DM-6718 afw table and record should have useful str() and repr()

          • Done

          Bug - A problem which impairs or prevents the functions of the product. DM-8015 VisitInfo repr() and str() should print a useful summary of contents

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-8467 Wrap lsst_distrib with pybind11

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-9071 Experiment with ProcessPoolExecutor for reading/writing jointcal results

          • Invalid
          is triggered by

          RFC - An issue whose primary purpose is to advertise an intent and allow stakeholders to comment. RFC-216 Use pybind11 instead of swig to generate Python wrappers for C++ code

          • Implemented

          Stories in Epic (Custom Issue Matrix)

          Key Summary Story Points Assignee Status
           
          		DM-9157

          Write release notes for pybind11 port

          		 3 Pim Schellart [X] (Inactive) Done
           
          		DM-9156

          Wrap ci_hsc with pybind11

          		 4 Pim Schellart [X] (Inactive) Done
           
          		DM-9112

          Determine strategy for dealing with butler proxies in pybind11

          		 6 Pim Schellart [X] (Inactive) Done
           
          		DM-9100

          Merge pybind11 branches to master

          		 0.5 Pim Schellart [X] (Inactive) Done
           
          		DM-9099

          Begin cleanup of pybind11 branch

          		 0 Jim Bosch Done
           
          		DM-9096

          Wrap geom with pybind11

          		 0 Jim Bosch Done
           
          		DM-9089

          Move pybind11 how-to technote to dev guide

          		 5 Pim Schellart [X] (Inactive) Done
           
          		DM-9063

          Audit pybind11 port for outstanding technical debt

          		 3 Jim Bosch Done
           
          		DM-9057

          Check int vs float interoperability in pybind11 afw.geom

          		 1 Jim Bosch Done
           
          		DM-9018

          Investigate downcasting of Table and Record types in pybind11

          		 2 Jim Bosch Done
           
          		DM-9003

          Wrap clone for all table classes

          		 0.5 Pim Schellart [X] (Inactive) Done
           
          		DM-8999

          Rebase pybind11 branch onto latest master

          		 2 Pim Schellart [X] (Inactive) Done
           
          		DM-8417

          Wrap tests that depend on both image and table with pybind11

          		 8 Russell Owen Done
           
          		DM-8415

          Finish wrapping of pickling with pybind11 in afw

          		 4 Pim Schellart [X] (Inactive) Done
           
          		DM-8631

          Robustify pybind11 holder type inheritance

          		 4 Pim Schellart [X] (Inactive) Done
           
          		DM-8619

          Enable testWarper.py with pybind11

          		 3 Russell Owen Done
           
          		DM-8612

          build_templates.py creates incorrect test list

          		 1 Fred Moolekamp Done
           
          		DM-8597

          The pybind11 wrapper for CoaddInputs segfaults

          		 4 Pim Schellart [X] (Inactive) Done
           
          		DM-8577

          Add pybind11 rules to DM dev-guide

          		 1 Pim Schellart [X] (Inactive) Done
           
          		DM-8500

          Rebase pybind11 branch on master and move to new top level ticket

          		 1 Pim Schellart [X] (Inactive) Done
           
          		DM-8466

          Wrap obs_subaru with pybind11

          		 0.5 Pim Schellart [X] (Inactive) Done
           
          		DM-8465

          Wrap meas_modelfit with pybind11

          		 5 Jim Bosch Done
           
          		DM-8463

          Wrap obs_sdss with pybind11

          		 0.5 Russell Owen Done
           
          		DM-8460

          Wrap ip_diffim with pybind11

          		 5 Russell Owen Done
           
          		DM-8459

          Wrap ip_isr with pybind11

          		 2 Russell Owen Done
           
          		DM-8458

          Wrap meas_astrom with pybind11

          		 6 Russell Owen Done
           
          		DM-8457

          Wrap meas_deblender with pybind11

          		 0.4 Fred Moolekamp Done
           
          		DM-8456

          Wrap meas_extensions_photometryKron with pybind11

          		 2 Russell Owen Done
           
          		DM-8455

          Wrap meas_extensions_psfex with pybind11

          		 1.4 Pim Schellart [X] (Inactive) Done
           
          		DM-8453

          Wrap meas_algorithms with pybind11

          		 13.2 Pim Schellart [X] (Inactive) Done
           
          		DM-8716

          Use visitor pattern to clean up afw::table pybind11 wrappers

          		 6 Jim Bosch Done
           
          		DM-8715

          Fix or remove pickling support for Match lists with pybind11

          		 2 Pim Schellart [X] (Inactive) Done
           
          		DM-8696

          Wrap testTicketDM-433 with pybind11

          		 1 Pim Schellart [X] (Inactive) Done
           
          		DM-8675

          Rebase pybind11 branch on master and strip GPU code

          		 1 Pim Schellart [X] (Inactive) Done
           
          		DM-8674

          Wrap testSourceTable with pybind11

          		 6 Russell Owen Done
           
          		DM-8652

          Wrap afw ds9 test with pybind11

          		 1 Pim Schellart [X] (Inactive) Done
           
          		DM-8866

          Wrap datarel with pybind11

          		 0.5 Pim Schellart [X] (Inactive) Done
           
          		DM-8818

          build_templates incorrectly adds a blank namespace

          		 1 Fred Moolekamp Done
           
          		DM-8805

          Metaclass for wrapped-template ABCs and class extension decorators

          		 4 Jim Bosch Done
           
          		DM-8751

          Fix skipped testPickle in testSourceTable.py

          		 2 Russell Owen Done
           
          		DM-8728

          SCons pybind11 builds re-link unnecessarily

          		 6 Jim Bosch Done
           
          		DM-8888

          Upgrade pybind11 to version 2.x

          		 2 Pim Schellart [X] (Inactive) Done
           
          		DM-8877

          Wrap skymap with pybind11

          		 2 Russell Owen Done
           
          		DM-8876

          Wrap shapelet with pybind11

          		 5 Pim Schellart [X] (Inactive) Done
           
          		DM-8874

          Wrap display_ds9 with pybind11

          		 3 Russell Owen Done
           
          		DM-8873

          Wrap ctrl_orca with pybind11

          		 0.1 Pim Schellart [X] (Inactive) Done
           
          		DM-8871

          Wrap ctrl_execute with pybind11

          		 0.1 Pim Schellart [X] (Inactive) Done
           
          		DM-8870

          Wrap ctrl_platform_lsst with pybind11

          		 0.1 Pim Schellart [X] (Inactive) Done
           
          		DM-8869

          Wrap ctrl_platform_gordon with pybind11

          		 0.1 Pim Schellart [X] (Inactive) Done
           
          		DM-8868

          Wrap ctrl_pool with pybind11

          		 0.1 Pim Schellart [X] (Inactive) Done
           
          		DM-8867

          Wrap lsst_apps with pybind11

          		 0 Pim Schellart [X] (Inactive) Done
           
          		DM-9528

          Cleanup pybind11 code in meas_deblender

          		 1 Fred Moolekamp Done
           
          		DM-9447

          Process HSC RC data using pybind11 prototype

          		 5 Lauren MacArthur Done
           
          		DM-9429

          Update lsst_dm_stack_demo for pybind11

          		 0.5 Russell Owen Done
           
          		DM-9428

          Update exceptions tutorial-level documentation for pybind11

          		 1 Pim Schellart [X] (Inactive) Done
           
          		DM-9392

          Wrap meas_extensions_convolved with pybind11 instead of swig

          		 0.5 Pim Schellart [X] (Inactive) Done
           
          		DM-9731

          Final pybind11 rebase on master

          		 0.1 Pim Schellart [X] (Inactive) Done
           
          		DM-9711

          Clean up meas_base pybind11 wrappers

          		 2 Russell Owen Done
           
          		DM-9701

          Cleanup pybind11 afw dependencies

          		 1 Pim Schellart [X] (Inactive) Done
           
          		DM-9585

          Rebase pybind11 onto latest master

          		 2 Pim Schellart [X] (Inactive) Done
           
          		DM-9583

          Update pex_exceptions to use pybind11::reinterpret_borrow / steal instead of object

          		 0.5 Pim Schellart [X] (Inactive) Done
           
          		DM-9279

          Wrap cat with pybind11 instead of swig

          		 0.1 Russell Owen Done
           
          		DM-9278

          Wrap ctrl_platform_lsstvc with pybind11 instead of swig

          		 0.1 Pim Schellart [X] (Inactive) Done
           
          		DM-9277

          Wrap obs_decam with pybind11 instead of swig

          		 0.5 Russell Owen Done
           
          		DM-9276

          Wrap obs_cfht with pybind11 instead of swig

          		 0.5 Russell Owen Done
           
          		DM-9258

          Modify pybind11 sconsUtils to only prepend underscore if not already present

          		 0.5 Russell Owen Done
           
          		DM-9251

          Rebase pybind11 branch onto latest master

          		 2 Pim Schellart [X] (Inactive) Done
           
          		DM-9188

          Cleanup pybind11 code in core meas and ip packages

          		 4 Pim Schellart [X] (Inactive) Done
           
          		DM-9182

          Cleanup pybind11 code in afw

          		 3 Jim Bosch Done
           
          		DM-9671

          Rename custom pybind11 namespace to python

          		 0.5 Pim Schellart [X] (Inactive) Done
           
          		DM-7057

          Complete afw port to pybind11

          		 10 Pim Schellart [X] (Inactive) Done
           
          		DM-7056

          Wrap afw::table with pybind11

          		 32 Fred Moolekamp Done
           
          		DM-6297

          Wrap afw::detection with pybind11

          		 5 Pim Schellart [X] (Inactive) Done
           
          		DM-7799

          Wrap afw::display with pybind11

          		 9 Pim Schellart [X] (Inactive) Done
           
          		DM-7797

          Wrap afw::image with pybind11

          		 12.9 Pim Schellart [X] (Inactive) Done
           
          		DM-7974

          Improve upstream pybind11 enum support

          		 2 Pim Schellart [X] (Inactive) Done
           
          		DM-8096

          Make "immediate=True" the default for butler.get()

          		 0.5 Russell Owen Done
           
          		DM-9121

          Investigate slow linking of pybind11 modules

          		 2 Jim Bosch Won't Fix
           
          		DM-8875

          Wrap ctrl_provenance with pybind11

          		 0.1 Pim Schellart [X] (Inactive) Won't Fix
           
          		DM-9696

          Fix lsst_sims on pybind11

          		 4 Unassigned Won't Fix
           
          		DM-9280

          Wrap ctrl_events with pybind11 instead of swig

          		 6 Russell Owen Won't Fix

            Activity

            Hide
            Permalink
            pschella Pim Schellart [X] (Inactive) added a comment -

            All tickets done and merged.

            Show
            pschella Pim Schellart [X] (Inactive) added a comment - All tickets done and merged.

              People

              Assignee:
              swinbank John Swinbank
              Reporter:
              swinbank John Swinbank
              Watchers:
              John Swinbank, Krzysztof Findeisen, Pim Schellart [X] (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.