Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: meas_deblender
-
Labels:None
-
Team:Data Facility
Description
The following code gives segfault:
import lsst.afw.image as afwImage
|
from lsst.afw.table import SourceTable
|
from lsst.meas.algorithms import SourceDetectionTask
|
from lsst.meas.deblender import SourceDeblendTask
|
|
|
exposure = afwImage.ExposureF('/datasets/hsc/repo/rerun/RC/w_2018_18/DM-14243/01111/HSC-Z/corr/CORR-0017962-101.fits')
|
|
|
schema = SourceTable.makeMinimalSchema()
|
detectTask = SourceDetectionTask(schema=schema)
|
|
|
# these two lines need to switch the order
|
table = SourceTable.make(schema)
|
deblendTask = SourceDeblendTask(schema=schema)
|
|
|
detectResult = detectTask.run(table, exposure)
|
catalog = detectResult.sources
|
deblendTask.run(exposure, catalog)
|
This is because the schema is modified when SourceDeblendTask is constructed. So the above code passes a catalog of a different schema to deblendTask than its schema.
Add a check to make the error clearer.
Attachments
Activity
Code looks good; thanks for taking care of this.
Could you describe the case where you thought you'd be able to reproduce but couldn't?
In meas_deblender there is an old test fits file meas_deblender/tests/data/ticket1738.fits
which is used to run one unit test (test_include.py).
Running the snippet in the ticket descriptions with random, recent calexp segfault (with slightly different information of segfaulting, but all were something bad in memory). However, running the snippet with meas_deblender/tests/data/ticket1738.fits finishes just fine. I wonder if this small fits file (976K) was specially made to test something and just happens not to reach the crashing point? Below is the log running this fits:
sourceDetection INFO: Detected 30 positive peaks in 16 footprints and 0 negative peaks in 0 footprints to 5 sigma |
sourceDetection.background WARN: Too few points in grid to constrain fit: min(nx, ny) < approxOrder) [min(3, 3) < 6] |
sourceDetection.background WARN: Reducing approxOrder to 2 |
sourceDetection INFO: Resubtracting the background after object detection
|
sourceDeblend INFO: Deblending 16 sources |
sourceDeblend INFO: Deblended: of 16 sources, 3 were deblended, creating 17 children, total 33 sources |
Hsin-Fang Chiang
added a comment - In meas_deblender there is an old test fits file meas_deblender/tests/data/ticket1738.fits
which is used to run one unit test ( test_include.py ).
Running the snippet in the ticket descriptions with random, recent calexp segfault (with slightly different information of segfaulting, but all were something bad in memory). However, running the snippet with meas_deblender/tests/data/ticket1738.fits finishes just fine. I wonder if this small fits file (976K) was specially made to test something and just happens not to reach the crashing point? Below is the log running this fits:
sourceDetection INFO: Detected 30 positive peaks in 16 footprints and 0 negative peaks in 0 footprints to 5 sigma
sourceDetection.background WARN: Too few points in grid to constrain fit: min(nx, ny) < approxOrder) [min( 3 , 3 ) < 6 ]
sourceDetection.background WARN: Reducing approxOrder to 2
sourceDetection INFO: Resubtracting the background after object detection
sourceDeblend INFO: Deblending 16 sources
sourceDeblend INFO: Deblended: of 16 sources, 3 were deblended, creating 17 children, total 33 sources
Do you know if that test data triggers the assert you just added? It might just be that segfaults in most cases (including this one) can't be relied upon as a failure mode; a memory access violation formally just means "undefined behavior", and that can certainly include not segfaulting.
Yes that test data trigger the assert. So that means I can add a test using it. Will do.
Hsin-Fang Chiang
added a comment - Yes that test data trigger the assert. So that means I can add a test using it. Will do. A new test has beed added. Would you please take another look? Thanks!
Hsin-Fang Chiang
added a comment - A new test has beed added. Would you please take another look? Thanks! Looks great! Above and beyond the call of duty. Thank you.
Thanks for the review! Merged to master after running jenkins.
Hsin-Fang Chiang
added a comment - Thanks for the review! Merged to master after running jenkins . 
May you please review the two-line change?
I was thinking to add a test, but somehow the existing fits does not reproduce the problem (!?) I'm not sure this is worth adding a full calexp in the package.
Jenkins is running: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/27963/pipeline