John Swinbank added a comment - 23/Nov/16 5:31 PM Using astrometry_net_data sdss-dr9-fink-v5b I surmise. Am I correct?

John Swinbank added a comment - 23/Nov/16 6:24 PM Ran this using the shared stack (ie, built from source with eups distrib, not installed through conda) on lsst-dev01. It burns CPU for several minutes trying to match, then ultimately fails but doesn't segfault: processCcd.calibrate.astrometry.solver INFO: Number of selected sources for astrometry : 552 processCcd.calibrate.astrometry.solver WARN: Did not get an astrometric solution from Astrometry.net Traceback (most recent call last): File "/ssd/lsstsw/stack/Linux64/pipe_tasks/12.1+1/bin/processCcd.py", line 25, in <module> ProcessCcdTask.parseAndRun() File "/ssd/lsstsw/stack/Linux64/pipe_base/12.1+1/python/lsst/pipe/base/cmdLineTask.py", line 472, in parseAndRun resultList = taskRunner.run(parsedCmd) File "/ssd/lsstsw/stack/Linux64/pipe_base/12.1+1/python/lsst/pipe/base/cmdLineTask.py", line 208, in run resultList = list(mapFunc(self, targetList)) File "/ssd/lsstsw/stack/Linux64/pipe_base/12.1+1/python/lsst/pipe/base/cmdLineTask.py", line 343, in __call__ result = task.run(dataRef, **kwargs) File "/ssd/lsstsw/stack/Linux64/pipe_base/12.1+1/python/lsst/pipe/base/timer.py", line 121, in wrapper res = func(self, *args, **keyArgs) File "/ssd/lsstsw/stack/Linux64/pipe_tasks/12.1+1/python/lsst/pipe/tasks/processCcd.py", line 181, in run icSourceCat = charRes.sourceCat, File "/ssd/lsstsw/stack/Linux64/pipe_base/12.1+1/python/lsst/pipe/base/timer.py", line 121, in wrapper res = func(self, *args, **keyArgs) File "/ssd/lsstsw/stack/Linux64/pipe_tasks/12.1+1/python/lsst/pipe/tasks/calibrate.py", line 383, in run icSourceCat=icSourceCat, File "/ssd/lsstsw/stack/Linux64/pipe_tasks/12.1+1/python/lsst/pipe/tasks/calibrate.py", line 462, in calibrate sourceCat=sourceCat, File "/ssd/lsstsw/stack/Linux64/pipe_base/12.1+1/python/lsst/pipe/base/timer.py", line 121, in wrapper res = func(self, *args, **keyArgs) File "/ssd/lsstsw/stack/Linux64/meas_astrom/12.1+1/python/lsst/meas/astrom/anetAstrometry.py", line 195, in run return self.solve(exposure=exposure, sourceCat=sourceCat) File "/ssd/lsstsw/stack/Linux64/pipe_base/12.1+1/python/lsst/pipe/base/timer.py", line 121, in wrapper res = func(self, *args, **keyArgs) File "/ssd/lsstsw/stack/Linux64/meas_astrom/12.1+1/python/lsst/meas/astrom/anetAstrometry.py", line 226, in solve results = self._astrometry(sourceCat=sourceCat, exposure=exposure, bbox=bbox) File "/ssd/lsstsw/stack/Linux64/pipe_base/12.1+1/python/lsst/pipe/base/timer.py", line 121, in wrapper res = func(self, *args, **keyArgs) File "/ssd/lsstsw/stack/Linux64/meas_astrom/12.1+1/python/lsst/meas/astrom/anetAstrometry.py", line 391, in _astrometry astrom = self.solver.determineWcs(sourceCat=sourceCat, exposure=exposure, bbox=bbox) File "/ssd/lsstsw/stack/Linux64/meas_astrom/12.1+1/python/lsst/meas/astrom/anetBasicAstrometry.py", line 468, in determineWcs return self.determineWcs2(sourceCat=sourceCat, **margs) File "/ssd/lsstsw/stack/Linux64/meas_astrom/12.1+1/python/lsst/meas/astrom/anetBasicAstrometry.py", line 490, in determineWcs2 wcs, qa = self.getBlindWcsSolution(sourceCat, **kwargs) File "/ssd/lsstsw/stack/Linux64/meas_astrom/12.1+1/python/lsst/meas/astrom/anetBasicAstrometry.py", line 578, in getBlindWcsSolution raise RuntimeError("Unable to match sources with catalog.") RuntimeError: Unable to match sources with catalog. I'll see if I can replicate the segfault using the Conda stack.

Merlin Fisher-Levine added a comment - 25/Nov/16 2:46 PM Sorry, yes, you are correct, using astrometry_net_data sdss-dr9-fink-v5b. And, whilst it's not the desired behaviour, the "correct" behaviour for this with the current state of the the astrometry_net matching, is this to sit and burn CPU and then crash.

John Swinbank added a comment - 25/Nov/16 4:59 PM - edited Confirmed this fails with the stack installed through conda on lsst-dev01. The segfault happens in astrometry_net's starxy_sort_by_flux , ie not in LSST written code. Easy to reproduce, without using Monocam: from a fresh stack installation, we can just run $ python ${MEAS_ASTROM_DIR}/tests/testCreateWcsWithSip.py which segfaults within a fraction of a second. I built and installed my own astrometry_net into the conda-provided stack. That is: $ git clone https://github.com/lsst/astrometry_net.git $ cd astrometry_net $ eupspkg -er fetch $ eupspkg -er prep $ eupspkg -er config $ eupspkg -er build $ eupspkg -er install $ eupspkg -er decl -F With that, everything seems to run fine (insofar as failing to match and throwing a RuntimeError on the Monocam data is "fine"; seems like it's expected, at least). Given that the code in question wasn't written by LSST and that it works fine when not using the binary distribution, I'm not inclined to spend much more time trying to debug it. J Matt Peterson [X] , do you have any idea what the problem might be? Perhaps this might be resolved in a future binary distribution?

John Swinbank added a comment - 28/Nov/16 6:29 AM Setting team to SQuaRE since this seems to be an issue with the conda packaging.

John Swinbank added a comment - 05/Dec/16 7:57 AM Hey J Matt Peterson [X] , just watned to be sure you're aware of this. It's not an immediate crisis, but until we can get it resolved it renders binaries a lot less useful than they might otherwise be.