# astrometry_net conda binary segfaults

XMLWordPrintable

#### Details

• Type: Bug
• Status: Won't Fix
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Team:
SQuaRE

#### Description

Running processCcd on monocam data segfaults during astrometry.

Steps to reproduce:

• conda install v12.1 of the stack
• setup ticket branch DM-8401 of obs_monocam
•  ingestImages.py path/to/downloaded/file/2016-05-12CAblank5_y4_70.fits 

•  processCcd.py filename --rerun test_1 --id visit=1333 --doraise -c isr.doLinearize=False isr.doBias=False isr.doDark=False isr.doFlat=False 

Should see segfault after

 processCcd.calibrate.astrometry.solver INFO: Number of selected sources for astrometry : 552 

.

Interestingly, running processCcd with -c calibrate.doAstrometry=False results in the astrometric solver still being run, and a segfault still occurring, but in a different place. Just mentioning as this may or may not be useful debug info.

#### Attachments

1. 2016-05-12CAblank5_y4_70.fits.zip
13.35 MB

#### Activity

Hide
John Swinbank added a comment -

Using astrometry_net_data sdss-dr9-fink-v5b I surmise. Am I correct?

Show
John Swinbank added a comment - Using astrometry_net_data sdss-dr9-fink-v5b I surmise. Am I correct?
Hide
John Swinbank added a comment -

Ran this using the shared stack (ie, built from source with eups distrib, not installed through conda) on lsst-dev01. It burns CPU for several minutes trying to match, then ultimately fails but doesn't segfault:

 processCcd.calibrate.astrometry.solver INFO: Number of selected sources for astrometry : 552 processCcd.calibrate.astrometry.solver WARN: Did not get an astrometric solution from Astrometry.net Traceback (most recent call last):  File "/ssd/lsstsw/stack/Linux64/pipe_tasks/12.1+1/bin/processCcd.py", line 25, in   ProcessCcdTask.parseAndRun()  File "/ssd/lsstsw/stack/Linux64/pipe_base/12.1+1/python/lsst/pipe/base/cmdLineTask.py", line 472, in parseAndRun  resultList = taskRunner.run(parsedCmd)  File "/ssd/lsstsw/stack/Linux64/pipe_base/12.1+1/python/lsst/pipe/base/cmdLineTask.py", line 208, in run  resultList = list(mapFunc(self, targetList))  File "/ssd/lsstsw/stack/Linux64/pipe_base/12.1+1/python/lsst/pipe/base/cmdLineTask.py", line 343, in __call__  result = task.run(dataRef, **kwargs)  File "/ssd/lsstsw/stack/Linux64/pipe_base/12.1+1/python/lsst/pipe/base/timer.py", line 121, in wrapper  res = func(self, *args, **keyArgs)  File "/ssd/lsstsw/stack/Linux64/pipe_tasks/12.1+1/python/lsst/pipe/tasks/processCcd.py", line 181, in run  icSourceCat = charRes.sourceCat,  File "/ssd/lsstsw/stack/Linux64/pipe_base/12.1+1/python/lsst/pipe/base/timer.py", line 121, in wrapper  res = func(self, *args, **keyArgs)  File "/ssd/lsstsw/stack/Linux64/pipe_tasks/12.1+1/python/lsst/pipe/tasks/calibrate.py", line 383, in run  icSourceCat=icSourceCat,  File "/ssd/lsstsw/stack/Linux64/pipe_tasks/12.1+1/python/lsst/pipe/tasks/calibrate.py", line 462, in calibrate  sourceCat=sourceCat,  File "/ssd/lsstsw/stack/Linux64/pipe_base/12.1+1/python/lsst/pipe/base/timer.py", line 121, in wrapper  res = func(self, *args, **keyArgs)  File "/ssd/lsstsw/stack/Linux64/meas_astrom/12.1+1/python/lsst/meas/astrom/anetAstrometry.py", line 195, in run  return self.solve(exposure=exposure, sourceCat=sourceCat)  File "/ssd/lsstsw/stack/Linux64/pipe_base/12.1+1/python/lsst/pipe/base/timer.py", line 121, in wrapper  res = func(self, *args, **keyArgs)  File "/ssd/lsstsw/stack/Linux64/meas_astrom/12.1+1/python/lsst/meas/astrom/anetAstrometry.py", line 226, in solve  results = self._astrometry(sourceCat=sourceCat, exposure=exposure, bbox=bbox)  File "/ssd/lsstsw/stack/Linux64/pipe_base/12.1+1/python/lsst/pipe/base/timer.py", line 121, in wrapper  res = func(self, *args, **keyArgs)  File "/ssd/lsstsw/stack/Linux64/meas_astrom/12.1+1/python/lsst/meas/astrom/anetAstrometry.py", line 391, in _astrometry  astrom = self.solver.determineWcs(sourceCat=sourceCat, exposure=exposure, bbox=bbox)  File "/ssd/lsstsw/stack/Linux64/meas_astrom/12.1+1/python/lsst/meas/astrom/anetBasicAstrometry.py", line 468, in determineWcs  return self.determineWcs2(sourceCat=sourceCat, **margs)  File "/ssd/lsstsw/stack/Linux64/meas_astrom/12.1+1/python/lsst/meas/astrom/anetBasicAstrometry.py", line 490, in determineWcs2  wcs, qa = self.getBlindWcsSolution(sourceCat, **kwargs)  File "/ssd/lsstsw/stack/Linux64/meas_astrom/12.1+1/python/lsst/meas/astrom/anetBasicAstrometry.py", line 578, in getBlindWcsSolution  raise RuntimeError("Unable to match sources with catalog.") RuntimeError: Unable to match sources with catalog. 

I'll see if I can replicate the segfault using the Conda stack.

Show
Hide
Merlin Fisher-Levine added a comment -

Sorry, yes, you are correct, using astrometry_net_data sdss-dr9-fink-v5b.

And, whilst it's not the desired behaviour, the "correct" behaviour for this with the current state of the the astrometry_net matching, is this to sit and burn CPU and then crash.

Show
Merlin Fisher-Levine added a comment - Sorry, yes, you are correct, using astrometry_net_data sdss-dr9-fink-v5b. And, whilst it's not the desired behaviour, the "correct" behaviour for this with the current state of the the astrometry_net matching, is this to sit and burn CPU and then crash.
Hide
John Swinbank added a comment - - edited

Confirmed this fails with the stack installed through conda on lsst-dev01.

The segfault happens in astrometry_net's starxy_sort_by_flux, ie not in LSST written code. Easy to reproduce, without using Monocam: from a fresh stack installation, we can just run

 $python${MEAS_ASTROM_DIR}/tests/testCreateWcsWithSip.py 

which segfaults within a fraction of a second.

I built and installed my own astrometry_net into the conda-provided stack. That is:

 $git clone https://github.com/lsst/astrometry_net.git $ cd astrometry_net $eupspkg -er fetch $ eupspkg -er prep $eupspkg -er config $ eupspkg -er build $eupspkg -er install $ eupspkg -er decl -F 

With that, everything seems to run fine (insofar as failing to match and throwing a RuntimeError on the Monocam data is "fine"; seems like it's expected, at least).

Given that the code in question wasn't written by LSST and that it works fine when not using the binary distribution, I'm not inclined to spend much more time trying to debug it. J Matt Peterson [X], do you have any idea what the problem might be? Perhaps this might be resolved in a future binary distribution?

Show
John Swinbank added a comment - - edited Confirmed this fails with the stack installed through conda on lsst-dev01. The segfault happens in astrometry_net's starxy_sort_by_flux , ie not in LSST written code. Easy to reproduce, without using Monocam: from a fresh stack installation, we can just run $python${MEAS_ASTROM_DIR}/tests/testCreateWcsWithSip.py which segfaults within a fraction of a second. I built and installed my own astrometry_net into the conda-provided stack. That is: $git clone https://github.com/lsst/astrometry_net.git$ cd astrometry_net $eupspkg -er fetch$ eupspkg -er prep $eupspkg -er config$ eupspkg -er build $eupspkg -er install$ eupspkg -er decl -F With that, everything seems to run fine (insofar as failing to match and throwing a RuntimeError on the Monocam data is "fine"; seems like it's expected, at least). Given that the code in question wasn't written by LSST and that it works fine when not using the binary distribution, I'm not inclined to spend much more time trying to debug it. J Matt Peterson [X] , do you have any idea what the problem might be? Perhaps this might be resolved in a future binary distribution?
Hide
John Swinbank added a comment -

Setting team to SQuaRE since this seems to be an issue with the conda packaging.

Show
John Swinbank added a comment - Setting team to SQuaRE since this seems to be an issue with the conda packaging.
Hide
John Swinbank added a comment -

Hey J Matt Peterson [X], just watned to be sure you're aware of this. It's not an immediate crisis, but until we can get it resolved it renders binaries a lot less useful than they might otherwise be.

Show
John Swinbank added a comment - Hey J Matt Peterson [X] , just watned to be sure you're aware of this. It's not an immediate crisis, but until we can get it resolved it renders binaries a lot less useful than they might otherwise be.

#### People

Assignee:
Unassigned
Reporter:
Merlin Fisher-Levine
Watchers:
J Matt Peterson [X] (Inactive), John Swinbank, Merlin Fisher-Levine