Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-5855

Accessing invalid attributes produces unhelpful error message

    Details

      Description

      While trying to introspect afw.table, I tried "table.deepcopy()" and got a "Record data is not contiguous in memory." error. This was not helpful.

      In fact, table.ANYTHING (parentheses or not) gives the same error. Looking in tableLib __getattribute__, there should maybe be an except clause for the getattr(columns), to properly deal with this case and produce a more helpful error message?

      In [164]: xx.deepcopy()
      ---------------------------------------------------------------------------
      RuntimeError                              Traceback (most recent call last)
      <ipython-input-164-9d4dd7ea8bd1> in <module>()
      ----> 1 xx.deepcopy()
       
      /Users/parejkoj/lsst/lsstsw/stack/DarwinX86/afw/2.2016.10-9-gc64e176/python/lsst/afw/table/tableLib.pyc in __getattribute__(self, name)
         8142             return getattr(self.table, name)
         8143         except AttributeError:
      -> 8144             return getattr(self.columns, name)
         8145     table = property(getTable)
         8146     schema = property(getSchema)
       
      /Users/parejkoj/lsst/lsstsw/stack/DarwinX86/afw/2.2016.10-9-gc64e176/python/lsst/afw/table/tableLib.pyc in __getattribute__(self, name)
         8132         # We have to use __getattribute__ because SWIG overrides __getattr__.
         8133         try:
      -> 8134             return object.__getattribute__(self, name)
         8135         except AttributeError:
         8136             # self._columns is created the when self.columns is accessed -
       
      /Users/parejkoj/lsst/lsstsw/stack/DarwinX86/afw/2.2016.10-9-gc64e176/python/lsst/afw/table/tableLib.pyc in __getColumns(self)
         8097     def __getColumns(self):
         8098         if not hasattr(self, "_columns") or self._columns is None:
      -> 8099             self._columns = self.getColumnView()
         8100         return self._columns
         8101     columns = property(__getColumns, doc="a column view of the catalog")
       
      /Users/parejkoj/lsst/lsstsw/stack/DarwinX86/afw/2.2016.10-9-gc64e176/python/lsst/afw/table/tableLib.pyc in getColumnView(self)
         7970     def getColumnView(self):
         7971         """getColumnView(_SimpleCatalogBase self) -> SimpleColumnView"""
      -> 7972         val = _tableLib._SimpleCatalogBase_getColumnView(self)
         7973         self._columns = val
         7974
       
      RuntimeError:
        File "include/lsst/afw/table/BaseColumnView.h", line 205, in static lsst::afw::table::BaseColumnView lsst::afw::table::BaseColumnView::make(const boost::shared_ptr<BaseTable> &, InputIterator, InputIterator) [InputIterator = lsst::afw::table::CatalogIterator<std::__1::__wrap_iter<const boost::shared_ptr<lsst::afw::table::SimpleRecord> *> >]
          Record data is not contiguous in memory. {0}
      lsst::pex::exceptions::RuntimeError: 'Record data is not contiguous in memory.'
      

        Attachments

          Issue Links

            Activity

            Hide
            Parejkoj John Parejko added a comment -

            Adding a couple of pybind11ers as watchers: this is a SWIG problem, and may be already fixed as part of the port, or may be an easy fix for you now.

            Show
            Parejkoj John Parejko added a comment - Adding a couple of pybind11ers as watchers: this is a SWIG problem, and may be already fixed as part of the port, or may be an easy fix for you now.
            Hide
            krzys Krzysztof Findeisen added a comment -

            I've been able to reproduce this problem with dm5855.py (attached). Unfortunately, the behavior is unchanged in pybind11 (nor do I understand why this was described as a "Swig problem").

            Show
            krzys Krzysztof Findeisen added a comment - I've been able to reproduce this problem with dm5855.py (attached). Unfortunately, the behavior is unchanged in pybind11 (nor do I understand why this was described as a "Swig problem").
            Hide
            Parejkoj John Parejko added a comment -

            I believe the SWIGness of this problem was due to a combination of a getattr defined in the SWIG file on one thing, and a lack of a getattr in the SWIG file on another thing, plus something to do with exception flow through. The second paragraph of the description has the details.

            Show
            Parejkoj John Parejko added a comment - I believe the SWIGness of this problem was due to a combination of a getattr defined in the SWIG file on one thing, and a lack of a getattr in the SWIG file on another thing, plus something to do with exception flow through. The second paragraph of the description has the details.
            Hide
            Parejkoj John Parejko added a comment -

            Here's some code to trigger the bug. We do get a useful AttributeError first, but then we also get the not useful memory contiguity error.

            import lsst.afw.table
            import lsst.afw.geom
            schema = lsst.afw.table.SourceTable.makeMinimalSchema()
            catalog = lsst.afw.table.SourceCatalog(schema)
            record = catalog.addNew()
            record['coord_dec'] = lsst.afw.geom.degrees*(-5.0)
            record['coord_ra'] = lsst.afw.geom.degrees*(22)
            record['id'] = 8
            record['parent'] = 3
            catalog.addNew()
            catalog.addNew()
            del catalog[1]
            catalog.blah()
            

            Show
            Parejkoj John Parejko added a comment - Here's some code to trigger the bug. We do get a useful AttributeError first, but then we also get the not useful memory contiguity error. import lsst.afw.table import lsst.afw.geom schema = lsst.afw.table.SourceTable.makeMinimalSchema() catalog = lsst.afw.table.SourceCatalog(schema) record = catalog.addNew() record['coord_dec'] = lsst.afw.geom.degrees*(-5.0) record['coord_ra'] = lsst.afw.geom.degrees*(22) record['id'] = 8 record['parent'] = 3 catalog.addNew() catalog.addNew() del catalog[1] catalog.blah()
            Hide
            Parejkoj John Parejko added a comment -

            Bumping this, as I've run into it while trying to remove some "copy things to ensure contiguity" lines from jointcal that Paul Price claims are no longer necessary. I can't tell if the exceptions I'm getting are really due to contiguity problems, or due to this bug, while this bug still exists.

            Given Krzysztof Findeisen attached example and my and Meredith Rawls's example above, could someone who knows pybind11 better please investigate?

            Show
            Parejkoj John Parejko added a comment - Bumping this, as I've run into it while trying to remove some "copy things to ensure contiguity" lines from jointcal that Paul Price claims are no longer necessary. I can't tell if the exceptions I'm getting are really due to contiguity problems, or due to this bug, while this bug still exists. Given Krzysztof Findeisen attached example and my and Meredith Rawls 's example above, could someone who knows pybind11 better please investigate?
            Hide
            Parejkoj John Parejko added a comment -

            I've just converted the above code into a failing test case. Hoping some expert can make it not-failing?

            Show
            Parejkoj John Parejko added a comment - I've just converted the above code into a failing test case. Hoping some expert can make it not-failing?
            Hide
            jbosch Jim Bosch added a comment -

            Sorry I don't have time to fix this right now myself, but

            an except clause for the getattr(columns), to properly deal with this case and produce a more helpful error message

            does sound like the right fix if anyone else wants to take a stab at it.

            Show
            jbosch Jim Bosch added a comment - Sorry I don't have time to fix this right now myself, but an except clause for the getattr(columns), to properly deal with this case and produce a more helpful error message does sound like the right fix if anyone else wants to take a stab at it.

              People

              • Assignee:
                Unassigned
                Reporter:
                Parejkoj John Parejko
                Watchers:
                Colin Slater, Jim Bosch, John Parejko, John Swinbank, Krzysztof Findeisen, Meredith Rawls, Paul Price, Russell Owen, Simon Krughoff
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Summary Panel