Details
-
Type:
Bug
-
Status: Won't Fix
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: Developer Infrastructure, psfex, sconsUtils
-
Labels:None
-
Team:SQuaRE
Description
Building lsst_apps -t w_2015_35 fails on my laptop:
SCONSFLAGS="opt=3 -Q cc=clang" eups distrib install lsst_apps -t w_2015_35
|
[ 1/53 ] cfitsio 3360.lsst1 (already installed) done.
|
[ 2/53 ] doxygen 1.8.5 (already installed) done.
|
[ 3/53 ] eigen 3.2.0 (already installed) done.
|
[ 4/53 ] fftw 3.3.3-1-g8fdba61 (already installed) done.
|
[ 5/53 ] gsl 1.16.lsst1 (already installed) done.
|
[ 6/53 ] minuit2 5.28.00 (already installed) done.
|
[ 7/53 ] mysqlclient 5.1.73.lsst1-1-gb8bcc43 (already installed) done.
|
[ 8/53 ] python 0.0.1 (already installed) done.
|
[ 9/53 ] swig 3.0.2.lsst1 (already installed) done.
|
[ 10/53 ] xpa 2.1.15.lsst2 (already installed) done.
|
[ 11/53 ] boost 1.55.0.1.lsst2+3 (already installed) done.
|
[ 12/53 ] mysqlpython 1.2.3+17 (already installed) done.
|
[ 13/53 ] numpy 0.0.1 (already installed) done.
|
[ 14/53 ] scons 2.3.5 (already installed) done.
|
[ 15/53 ] wcslib 4.14+7 (already installed) done.
|
[ 16/53 ] astrometry_net 0.50.1+6 (already installed) done.
|
[ 17/53 ] matplotlib 0.0.1 (already installed) done.
|
[ 18/53 ] pyfits 3.2.4.lsst1+3 (already installed) done.
|
[ 19/53 ] sconsUtils 10.1-7-ga43ce92+1 (already installed) done.
|
[ 20/53 ] astrometry_net_data 10.0+30 (already installed) done.
|
[ 21/53 ] base 10.1-1-g8080078+19 (already installed) done.
|
[ 22/53 ] geom 10.0+29 (already installed) done.
|
[ 23/53 ] psfex master-ge3d792d24f ...
|
|
|
***** error: from /Users/lsst/products/EupsBuildDir/DarwinX86/psfex-master-ge3d792d24f/build.log:
|
"_fftwf_execute", referenced from:
|
_fft_conv in fft.os
|
_fft_rtf in fft.os
|
_fft_ctf in fft.os
|
"_fftwf_free", referenced from:
|
_fft_conv in fft.os
|
"_fftwf_malloc", referenced from:
|
_fft_conv in fft.os
|
_fft_rtf in fft.os
|
"_fftwf_plan_dft_2d", referenced from:
|
_fft_ctf in fft.os
|
"_fftwf_plan_dft_c2r_2d", referenced from:
|
_fft_conv in fft.os
|
"_fftwf_plan_dft_r2c_2d", referenced from:
|
_fft_conv in fft.os
|
_fft_rtf in fft.os
|
ld: symbol(s) not found for architecture x86_64
|
clang: error: linker command failed with exit code 1 (use -v to see invocation)
|
scons: *** [lib/libpsfex.dylib] Error 1
|
+ exit -4
|
eups distrib: Failed to build psfex-master-ge3d792d24f.eupspkg: Command:
|
source /Users/rhl/eups/bin/setups.sh; export EUPS_PATH=/Users/lsst/products; (/Users/lsst/products/EupsBuildDir/DarwinX86/psfex-master-ge3d792d24f/build.sh) >> /Users/lsst/products/EupsBuildDir/DarwinX86/psfex-master-ge3d792d24f/build.log 2>&1 4>/Users/lsst/products/EupsBuildDir/DarwinX86/psfex-master-ge3d792d24f/build.msg
|
exited with code 252
|
The linker command is:
clang -o lib/libpsfex.dylib -dynamiclib -Wl,-install_name -Wl,libpsfex.dylib -Wl,-headerpad_max_install_names src/levmar/lmbc.os src/levmar/Axb.os src/wcs_utils.os src/sample_utils.os src/fits/fitsmisc.os src/dummies.os src/diagnostic.os src/levmar/lm.os src/levmar/lmblec.os src/levmar/lmbleic.os src/field_utils.os src/fft.os src/psf.os src/field.os src/makeit2.os src/context.os src/misc.os src/homo.os src/wcs/poly.os src/xml.os src/prefs.os src/vignet.os src/pca.os src/levmar/misc.os src/levmar/lmlec.os -Llib -L/Users/lsst/products/DarwinX86/fftw/3.3.3-1-g8fdba61/lib -lfftw3 -llapackstub
|
and the symbols do seem to be defined in the library:
nm -oa /Users/lsst/products/DarwinX86/fftw/3.3.3-1-g8fdba61/lib/* 2>/dev/null | grep fftwf_free
|
/Users/lsst/products/DarwinX86/fftw/3.3.3-1-g8fdba61/lib/libfftw3f.3.dylib: 00000000000cb4d0 T _fftwf_free
|
/Users/lsst/products/DarwinX86/fftw/3.3.3-1-g8fdba61/lib/libfftw3f.a:malloc.o: 0000000000000010 T _fftwf_free
|
/Users/lsst/products/DarwinX86/fftw/3.3.3-1-g8fdba61/lib/libfftw3f.dylib: 00000000000cb4d0 T _fftwf_free
|
So maybe the linker's finding some other fftw?
Attachments
Issue Links
Activity
The configure command appears to be:
$ ./configure --prefix=/Users/lsst/products/DarwinX86/psfex/master-ge3d792d24f --disable-threads --with-clapack=/Users/lsst/products/EupsBuildDir/DarwinX86/psfex-master-ge3d792d24f/psfex-master-ge3d792d24f/lapack_functions --with-fftw-incdir=/Users/lsst/products/DarwinX86/fftw/3.3.3-1-g8fdba61/include --with-fftw-libdir=/Users/lsst/products/DarwinX86/fftw/3.3.3-1-g8fdba61/lib
|
which looks fine. But it ends up with -lfftw3 not -lfftw3f and that doesn't have the required symbols.
I thought that fftw.cfg saying that it provides both fftw3 and fftw3f would have gotten both into linker lines, but I guess something is messing that up here. Strange that it doesn't happen on CentOS...
The problem seems to be in sconsUtils, and git bisect fingers
$ git bisect bad
|
e7553b2526b609eb8c594018fa360c22ee3c716b is the first bad commit
|
commit e7553b2526b609eb8c594018fa360c22ee3c716b
|
Author: Joshua Hoblitt <josh@hoblitt.com>
|
Date: Fri Jul 17 16:55:16 2015 -0700
|
|
|
fix duplicate loading of <product>_DIR into cfgPath
|
|
|
:040000 040000 2d75123343ed1adbb2beb42bde11dd5f596b5126 dc8c5dafc4caf03d6e6f3798cb7795c54b3f08d7 M python
|
i.e.
git show e7553b2526
|
commit e7553b2526b609eb8c594018fa360c22ee3c716b
|
Author: Joshua Hoblitt <josh@hoblitt.com>
|
Date: Fri Jul 17 16:55:16 2015 -0700
|
|
|
fix duplicate loading of <product>_DIR into cfgPath
|
|
|
diff --git a/python/lsst/sconsUtils/dependencies.py b/python/lsst/sconsUtils/dependencies.py
|
index 92840bd..0887b20 100644
|
--- a/python/lsst/sconsUtils/dependencies.py
|
+++ b/python/lsst/sconsUtils/dependencies.py
|
@@ -483,10 +483,7 @@ class PackageTree(object):
|
def _tryImport(self, name):
|
"""Search for and import an individual configuration module from file."""
|
filenames = []
|
- envVars = ("%s_DIR_EXTRA" % name.upper(), "%s_DIR" % name.upper())
|
- paths = [os.path.join(os.environ[v], "ups") for v in envVars if v in os.environ]
|
- paths.extend(self.cfgPath)
|
- for path in paths:
|
+ for path in self.cfgPath:
|
filename = os.path.join(path, name + ".cfg")
|
if os.path.exists(filename):
|
try:
|
... and the reason why that matters is because it changes the place in the search path where $LSST_CFG_PATH is inserted. I have an fftwf.cfg files on that path (it's set to export LSST_CFG_PATH=$HOME/LSST/devenv/buildFiles which is git@github.com:lsst/buildFiles).
I'm sympathetic with Josh on this; it's an obscure path variable designed to handle things like fftw or python that need .cfg files and for which you don't want to declare the package to eups. This dates back to the early days of .cfg files and can probably be retired (you can always declare a pseudo-package if needs be).
Robert Lupton This is an unintentional regression. I discovered that the path search logic was completely duplicated while running scons under pdb and removed one of the implementations. It sounds like the expected behavior for LSST_CFG_PATH is to be searched first? Ie., the order of these two loops should be inverted:
I don't especially care what the order is, but I do care that things changed!
As I implied above, Jim and I would be OK with an RFC to remove LSST_CFG_PATH
Can we retitle this ticket to reflect the agreed problem?
Since I think this has been broken for four years, I don't think it should take an RFC to remove this code. I will do that. Marking this as WON'T FIX (and in fact actively breaking more...).
Interesting a build I did last night using lsst-build worked fine. Jonathan Sick did have a problem building it with eups distrb using the same tagged weekly as you are using but interestingly his build error was different to yours because it got confused when PLplot was installed on his system. We should probably disable PLplot detection in our fork of psfex. It may be that he would have got as far as your FFTW error if he had got past the PLplot problem.