Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-30426

Out-of-bounds array access in shapeHSM

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Story Points:
      6
    • Epic Link:
    • Team:
      Data Release Production
    • Urgent?:
      No

      Description

      Monika Adamow reports in DM-30365 a likely shapeHSM bug with the following traceback.

      INFO  2021-05-28T11:52:38.987-0500 characterizeImage.measurement (characterizeImage:{instrument: 'HSC', detector: 41, visit: 27032, ...})(sfm.py:290)- Measuring 369 sources (369 parents, 0 children) python: /software/lsstsw/stack_20210520/conda/miniconda3-py38_4.9.2/envs/lsst-scipipe/include/boost/gil/image_view.hpp:316: boost::gil::image_view<L>::x_iterator boost::gil::image_view<L>::x_at(boost::gil::image_view<L>::x_coord_t, boost::gil::image_view<L>::y_coord_t) const [with Loc = boost::gil::memory_based_2d_locator<boost::gil::memory_based_step_iterator<boost::gil::pixel<int, boost::gil::layout<boost::mp11::mp_list<boost::gil::gray_color_t> > >*> >; boost::gil::image_view<L>::x_iterator = boost::gil::pixel<int, boost::gil::layout<boost::mp11::mp_list<boost::gil::gray_color_t> > >*; boost::gil::image_view<L>::x_coord_t = long int; boost::gil::image_view<L>::y_coord_t = long int]: Assertion `0 <= y && y < height()' failed.Caught signal 6, backtrace follows:/software/lsstsw/stack_20210520/stack/miniconda3-py38_4.9.2-0.6.0/Linux64/utils/21.0.0-6-g0bf7090+18535a8d22/lib/libutils.so(+0x26389) [0x2ab0b72aa389]/lib64/libc.so.6(+0x36450) [0x2ab0afffb450]/lib64/libc.so.6(gsignal+0x37) [0x2ab0afffb3d7]/lib64/libc.so.6(abort+0x148) [0x2ab0afffcac8]/lib64/libc.so.6(+0x2f1a6) [0x2ab0afff41a6]/lib64/libc.so.6(+0x2f252) [0x2ab0afff4252]/software/lsstsw/stack_20210520/stack/miniconda3-py38_4.9.2-0.6.0/Linux64/meas_extensions_shapeHSM/21.0.0-4-gee9d251+82de3135b4/lib/libmeas_extensions_shapeHSM.so(void lsst::meas::extensions::shapeHSM::HsmMomentsAlgorithm::calculate<double>(lsst::afw::table::SourceRecord&, std::shared_ptr<lsst::afw::image::Image<double> > const&, std::shared_ptr<lsst::afw::image::Mask<int> > const&, lsst::geom::Box2I const&, lsst::geom::Point<double, 2> const&, int, float, bool, bool, bool) const+0x1390) [0x2ab0d7a189c0]/software/lsstsw/stack_20210520/stack/miniconda3-py38_4.9.2-0.6.0/Linux64/meas_extensions_shapeHSM/21.0.0-4-gee9d251+82de3135b4/lib/libmeas_extensions_shapeHSM.so(lsst::meas::extensions::shapeHSM::HsmPsfMomentsAlgorithm::measure(lsst::afw::table::SourceRecord&, lsst::afw::image::Exposure<float, int, float> const&) const+0x1c5) [0x2ab0d7a18e45]/software/lsstsw/stack_20210520/stack/miniconda3-py38_4.9.2-0.6.0/Linux64/meas_base/21.0.0-13-g64487bb+b4fa93c58b/python/lsst/meas/base/algorithm.so(+0x17b22) [0x2ab0d2c2fb22]/software/lsstsw/stack_20210520/stack/miniconda3-py38_4.9.2-0.6.0/Linux64/meas_base/21.0.0-13-g64487bb+b4fa93c58b/python/lsst/meas/base/algorithm.so(+0x18b9b) [0x2ab0d2c30b9b]python(PyCFunction_Call+0x54) [0x556950967df4]python((null)+0x31e) [0x556950976f2e]python(_PyObject_MakeTpCall+0x31e) [0x556950976f2e]python(+0x1b269e) [0x5569509ee69e]python((null)+0x4ca3) [0x556950a108c3]python(_PyEval_EvalFrameDefault+0x4ca3) [0x556950a108c3]python((null)+0x1a6) [0x5569509ed706]python(_PyFunction_Vectorcall+0x1a6) [0x5569509ed706]python(+0x1b1f91) [0x5569509edf91]python(PyObject_Call+0x5e) [0x5569509610be]python((null)+0x21c1) [0x556950a0dde1]python(_PyEval_EvalFrameDefault+0x21c1) [0x556950a0dde1]python((null)+0x2c3) [0x5569509ec503]python(_PyEval_EvalCodeWithName+0x2c3) [0x5569509ec503]python((null)+0x378) [0x5569509ed8d8]python(_PyFunction_Vectorcall+0x378) [0x5569509ed8d8]python(+0x1b1f91) [0x5569509edf91]python(PyObject_Call+0x5e) [0x5569509610be]python((null)+0x21c1) [0x556950a0dde1]python(_PyEval_EvalFrameDefault+0x21c1) [0x556950a0dde1]python((null)+0x2c3) [0x5569509ec503]python(_PyEval_EvalCodeWithName+0x2c3) [0x5569509ec503]python(+0x1b2007) [0x5569509ee007]python((null)+0x1782) [0x556950a0d3a2]python(_PyEval_EvalFrameDefault+0x1782) [0x556950a0d3a2]python((null)+0x2c3) [0x5569509ec503]python(_PyEval_EvalCodeWithName+0x2c3) [0x5569509ec503]python((null)+0x378) [0x5569509ed8d8]python(_PyFunction_Vectorcall+0x378) [0x5569509ed8d8]python((null)+0xa4b) [0x556950a0c66b]python(_PyEval_EvalFrameDefault+0xa4b) [0x556950a0c66b]python((null)+0x2c3) [0x5569509ec503]python(_PyEval_EvalCodeWithName+0x2c3) [0x5569509ec503]python((null)+0x378) [0x5569509ed8d8]python(_PyFunction_Vectorcall+0x378) [0x5569509ed8d8]python(PyObject_Call+0x5e) [0x5569509610be]python((null)+0x21c1) [0x556950a0dde1]python(_PyEval_EvalFrameDefault+0x21c1) [0x556950a0dde1]python((null)+0x2c3) [0x5569509ec503]python(_PyEval_EvalCodeWithName+0x2c3) [0x5569509ec503]python(+0x1b2007) [0x5569509ee007]python((null)+0x1782) [0x556950a0d3a2]python(_PyEval_EvalFrameDefault+0x1782) [0x556950a0d3a2]python((null)+0x2c3) [0x5569509ec503]python(_PyEval_EvalCodeWithName+0x2c3) [0x5569509ec503]python((null)+0x378) [0x5569509ed8d8]python(_PyFunction_Vectorcall+0x378) [0x5569509ed8d8]python(PyObject_Call+0x5e) [0x5569509610be]python((null)+0x21c1) [0x556950a0dde1]python(_PyEval_EvalFrameDefault+0x21c1) [0x556950a0dde1]python((null)+0x2c3) [0x5569509ec503]python(_PyEval_EvalCodeWithName+0x2c3) [0x5569509ec503]python((null)+0x378) [0x5569509ed8d8]python(_PyFunction_Vectorcall+0x378) [0x5569509ed8d8]python(+0x1b1f91) [0x5569509edf91]python(PyObject_Call+0x5e) [0x5569509610be]python((null)+0x21c1) [0x556950a0dde1]python(_PyEval_EvalFrameDefault+0x21c1) [0x556950a0dde1]python((null)+0x1a6) [0x5569509ed706]python(_PyFunction_Vectorcall+0x1a6) [0x5569509ed706]python((null)+0xa4b) [0x556950a0c66b]python(_PyEval_EvalFrameDefault+0xa4b) [0x556950a0c66b]python((null)+0x1a6) [0x5569509ed706]python(_PyFunction_Vectorcall+0x1a6) [0x5569509ed706]python((null)+0xa4b) [0x556950a0c66b]python(_PyEval_EvalFrameDefault+0xa4b) [0x556950a0c66b]python((null)+0x1a6) [0x5569509ed706]python(_PyFunction_Vectorcall+0x1a6) [0x5569509ed706]python((null)+0xa4b) [0x556950a0c66b]python(_PyEval_EvalFrameDefault+0xa4b) [0x556950a0c66b]python((null)+0x1a6) [0x5569509ed706]python(_PyFunction_Vectorcall+0x1a6) [0x5569509ed706]python((null)+0xa4b) [0x556950a0c66b]python(_PyEval_EvalFrameDefault+0xa4b) [0x556950a0c66b]python((null)+0x1a6) [0x5569509ed706]python(_PyFunction_Vectorcall+0x1a6) [0x5569509ed706]python((null)+0xa4b) [0x556950a0c66b]python(_PyEval_EvalFrameDefault+0xa4b) [0x556950a0c66b]python((null)+0x2c3) [0x5569509ec503]python(_PyEval_EvalCodeWithName+0x2c3) [0x5569509ec503]python((null)+0x378) [0x5569509ed8d8]python(_PyFunction_Vectorcall+0x378) [0x5569509ed8d8]python((null)+0xa4b) [0x556950a0c66b]python(_PyEval_EvalFrameDefault+0xa4b) [0x556950a0c66b]python((null)+0x2c3) [0x5569509ec503]python(_PyEval_EvalCodeWithName+0x2c3) [0x5569509ec503]python((null)+0x378) [0x5569509ed8d8]python(_PyFunction_Vectorcall+0x378) [0x5569509ed8d8]python(PyObject_Call+0x5e) [0x5569509610be]python((null)+0x21c1) [0x556950a0dde1]python(_PyEval_EvalFrameDefault+0x21c1) [0x556950a0dde1]python((null)+0x2c3) [0x5569509ec503]python(_PyEval_EvalCodeWithName+0x2c3) [0x5569509ec503]python((null)+0x378) [0x5569509ed8d8]python(_PyFunction_Vectorcall+0x378) [0x5569509ed8d8]python(PyObject_Call+0x5e) [0x5569509610be]python((null)+0x21c1) [0x556950a0dde1]python(_PyEval_EvalFrameDefault+0x21c1) [0x556950a0dde1]python((null)+0x2c3) [0x5569509ec503]python(_PyEval_EvalCodeWithName+0x2c3) [0x5569509ec503]python((null)+0x378) [0x5569509ed8d8]python(_PyFunction_Vectorcall+0x378) [0x5569509ed8d8]python(PyObject_Call+0x5e) [0x5569509610be]python((null)+0x21c1) [0x556950a0dde1]python(_PyEval_EvalFrameDefault+0x21c1) [0x556950a0dde1]python((null)+0x2c3) [0x5569509ec503]python(_PyEval_EvalCodeWithName+0x2c3) [0x5569509ec503]python((null)+0x378) [0x5569509ed8d8]python(_PyFunction_Vectorcall+0x378) [0x5569509ed8d8]python(+0x1b254c) [0x5569509ee54c]python(PyObject_Call+0x5e) [0x5569509610be]python((null)+0x21c1) [0x556950a0dde1]python(_PyEval_EvalFrameDefault+0x21c1) [0x556950a0dde1]python((null)+0x1a6) [0x5569509ed706]python(_PyFunction_Vectorcall+0x1a6) [0x5569509ed706]python((null)+0xa4b) [0x556950a0c66b]python(_PyEval_EvalFrameDefault+0xa4b) [0x556950a0c66b]python((null)+0x2c3) [0x5569509ec503]python(_PyEval_EvalCodeWithName+0x2c3) [0x5569509ec503]python((null)+0x378) [0x5569509ed8d8]python(_PyFunction_Vectorcall+0x378) [0x5569509ed8d8]python((null)+0xa4b) [0x556950a0c66b]python(_PyEval_EvalFrameDefault+0xa4b) [0x556950a0c66b]python((null)+0x2c3) [0x5569509ec503]python(_PyEval_EvalCodeWithName+0x2c3) [0x5569509ec503]python((null)+0x378) [0x5569509ed8d8]python(_PyFunction_Vectorcall+0x378) [0x5569509ed8d8]python(+0x1b1f91) [0x5569509edf91]python(PyObject_Call+0x5e) [0x5569509610be]python((null)+0x21c1) [0x556950a0dde1]python(_PyEval_EvalFrameDefault+0x21c1) [0x556950a0dde1]python((null)+0x2c3) [0x5569509ec503]python(_PyEval_EvalCodeWithName+0x2c3) [0x5569509ec503]python((null)+0x2c1) [0x55695095bdf1]python(_PyObject_FastCallDict+0x2c1) [0x55695095bdf1]python((null)+0x63) [0x556950966983]python(_PyObject_Call_Prepend+0x63) [0x556950966983]python(+0x181b99) [0x5569509bdb99]python((null)+0x31e) [0x556950976f2e]python(_PyObject_MakeTpCall+0x31e) [0x556950976f2e]python((null)+0x4f2e) [0x556950a10b4e]python(_PyEval_EvalFrameDefault+0x4f2e) [0x556950a10b4e]python((null)+0x1a6) [0x5569509ed706]python(_PyFunction_Vectorcall+0x1a6) [0x5569509ed706]python((null)+0x92f) [0x556950a0c54f]python(_PyEval_EvalFrameDefault+0x92f) [0x556950a0c54f]python((null)+0x2c3) [0x5569509ec503]python(_PyEval_EvalCodeWithName+0x2c3) [0x5569509ec503]python(PyEval_EvalCodeEx+0x39) [0x5569509ed559]python(PyEval_EvalCode+0x1b) [0x556950a909ab]python(+0x254a43) [0x556950a90a43]python(+0x26e6b3) [0x556950aaa6b3]python(+0x2735b2) [0x556950aaf5b2]python(PyRun_SimpleFileExFlags+0x1b2) [0x556950aaf792]python(Py_RunMain+0x36d) [0x556950aafd0d]python(Py_BytesMain+0x39) [0x556950aafec9]/lib64/libc.so.6((null)+0xf5) [0x2ab0affe7555]/lib64/libc.so.6(__libc_start_main+0xf5) [0x2ab0affe7555]python(+0x1e9369) [0x556950a25369]

      Sounds similar to DM-29863.  Investigate!

        Attachments

          Issue Links

          is triggered by

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-30365 Gen3 RC2 reprocessing with w_2021_22 and bps

          • Done

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            yusra Yusra AlSayyad added a comment -

            How to reproduce: 

            singleFrameDriver.py /datasets/hsc/repo --calib /datasets/hsc/repo/CALIB/ --rerun $RERUN --id visit=1290 ccd=17 --cores 1

            Show
            yusra Yusra AlSayyad added a comment - How to reproduce: singleFrameDriver.py /datasets/hsc/repo --calib /datasets/hsc/repo/CALIB/ --rerun $RERUN --id visit=1290 ccd=17 --cores 1
            Hide
            Permalink
            jmeyers314 Joshua Meyers added a comment -

            Since Jira doesn't seem to be able to find all the relevant PRs, posting here:

            https://github.com/lsst/afw/pull/593/files

            https://github.com/lsst/meas_extensions_piff/pull/3/files

            https://github.com/lsst/meas_extensions_shapeHSM/pull/35/files

             

            Show
            jmeyers314 Joshua Meyers added a comment - Since Jira doesn't seem to be able to find all the relevant PRs, posting here: https://github.com/lsst/afw/pull/593/files https://github.com/lsst/meas_extensions_piff/pull/3/files https://github.com/lsst/meas_extensions_shapeHSM/pull/35/files  
            Hide
            Permalink
            jmeyers314 Joshua Meyers added a comment -

            The bug came down to shapeHSM's attempt to infer the results of psf.computeImage(point)->getBBox() from psf.computeBBox(point) without actually executing psf.computeImage(point).  To fix, I added a computeImageBBox(point) method to afw::detection::Psf with a default method that just pays the price of calling computeImage(point), but can optionally be overridden for efficiency. I haven't added any overrides of this yet, but anticipate doing so for at least meas_extensions_piff.

            Show
            jmeyers314 Joshua Meyers added a comment - The bug came down to shapeHSM's attempt to infer the results of psf.computeImage(point)->getBBox() from psf.computeBBox(point) without actually executing psf.computeImage(point) .  To fix, I added a computeImageBBox(point) method to afw::detection::Psf with a default method that just pays the price of calling computeImage(point) , but can optionally be overridden for efficiency. I haven't added any overrides of this yet, but anticipate doing so for at least meas_extensions_piff .
            Hide
            Permalink
            jmeyers314 Joshua Meyers added a comment -

            Thanks Paul Price.  I agree with both of your comments: 

            - I recommend stripping out the useSourceCentroidOffset knob, which will simplify the code.

            - I think you should also put some effort into implementing this method properly and efficiently in the subclasses in meas_algorithms.

            For the moment though I'd like to make these new tickets.  That'll make it a bit speedier to get the current bugfix merged.  Sound okay?

            Show
            jmeyers314 Joshua Meyers added a comment - Thanks Paul Price .  I agree with both of your comments:  - I recommend stripping out the  useSourceCentroidOffset  knob, which will simplify the code. - I think you should also put some effort into implementing this method properly and efficiently in the subclasses in meas_algorithms. For the moment though I'd like to make these new tickets.  That'll make it a bit speedier to get the current bugfix merged.  Sound okay?
            Hide
            Permalink
            price Paul Price added a comment -

            I'll leave such management questions to Yusra AlSayyad.

            Show
            price Paul Price added a comment - I'll leave such management questions to Yusra AlSayyad .
            Hide
            Permalink
            jmeyers314 Joshua Meyers added a comment -

            I went ahead and created DM-30676 and DM-30677.

            Paul Price, I think I've addressed all your other review comments now.  Let me know if you want to continue to iterate.

            Show
            jmeyers314 Joshua Meyers added a comment - I went ahead and created DM-30676 and DM-30677 . Paul Price , I think I've addressed all your other review comments now.  Let me know if you want to continue to iterate.
            Hide
            Permalink
            jmeyers314 Joshua Meyers added a comment -

            Passing Jenkins run: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/34387/pipeline

            Show
            jmeyers314 Joshua Meyers added a comment - Passing Jenkins run:  https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/34387/pipeline
            Hide
            Permalink
            yusra Yusra AlSayyad added a comment -

            Would be good to get this bugfix in before the weekly is tagged tonight at midnight.

            Show
            yusra Yusra AlSayyad added a comment - Would be good to get this bugfix in before the weekly is tagged tonight at midnight.

              People

              Assignee:
              jmeyers314 Joshua Meyers
              Reporter:
              jmeyers314 Joshua Meyers
              Reviewers:
              Paul Price
              Watchers:
              Joshua Meyers, Morgan Schmitz, Paul Price, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.