Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-3655

LSST_CFG_PATH support broken because of recent sconsUtils change

    XMLWordPrintable

Details

    • SQuaRE

    Description

      Building lsst_apps -t w_2015_35 fails on my laptop:

      SCONSFLAGS="opt=3 -Q cc=clang" eups distrib install lsst_apps -t w_2015_35
        [  1/53 ]  cfitsio 3360.lsst1 (already installed)                     done. 
        [  2/53 ]  doxygen 1.8.5 (already installed)                          done. 
        [  3/53 ]  eigen 3.2.0 (already installed)                            done. 
        [  4/53 ]  fftw 3.3.3-1-g8fdba61 (already installed)                  done. 
        [  5/53 ]  gsl 1.16.lsst1 (already installed)                         done. 
        [  6/53 ]  minuit2 5.28.00 (already installed)                        done. 
        [  7/53 ]  mysqlclient 5.1.73.lsst1-1-gb8bcc43 (already installed)    done. 
        [  8/53 ]  python 0.0.1 (already installed)                           done. 
        [  9/53 ]  swig 3.0.2.lsst1 (already installed)                       done. 
        [ 10/53 ]  xpa 2.1.15.lsst2 (already installed)                       done. 
        [ 11/53 ]  boost 1.55.0.1.lsst2+3 (already installed)                 done. 
        [ 12/53 ]  mysqlpython 1.2.3+17 (already installed)                   done. 
        [ 13/53 ]  numpy 0.0.1 (already installed)                            done. 
        [ 14/53 ]  scons 2.3.5 (already installed)                            done. 
        [ 15/53 ]  wcslib 4.14+7 (already installed)                          done. 
        [ 16/53 ]  astrometry_net 0.50.1+6 (already installed)                done. 
        [ 17/53 ]  matplotlib 0.0.1 (already installed)                       done. 
        [ 18/53 ]  pyfits 3.2.4.lsst1+3 (already installed)                   done. 
        [ 19/53 ]  sconsUtils 10.1-7-ga43ce92+1 (already installed)           done. 
        [ 20/53 ]  astrometry_net_data 10.0+30 (already installed)            done. 
        [ 21/53 ]  base 10.1-1-g8080078+19 (already installed)                done. 
        [ 22/53 ]  geom 10.0+29 (already installed)                           done. 
        [ 23/53 ]  psfex master-ge3d792d24f ...
       
      ***** error: from /Users/lsst/products/EupsBuildDir/DarwinX86/psfex-master-ge3d792d24f/build.log:
        "_fftwf_execute", referenced from:
            _fft_conv in fft.os
            _fft_rtf in fft.os
            _fft_ctf in fft.os
        "_fftwf_free", referenced from:
            _fft_conv in fft.os
        "_fftwf_malloc", referenced from:
            _fft_conv in fft.os
            _fft_rtf in fft.os
        "_fftwf_plan_dft_2d", referenced from:
            _fft_ctf in fft.os
        "_fftwf_plan_dft_c2r_2d", referenced from:
            _fft_conv in fft.os
        "_fftwf_plan_dft_r2c_2d", referenced from:
            _fft_conv in fft.os
            _fft_rtf in fft.os
      ld: symbol(s) not found for architecture x86_64
      clang: error: linker command failed with exit code 1 (use -v to see invocation)
      scons: *** [lib/libpsfex.dylib] Error 1
      + exit -4
      eups distrib: Failed to build psfex-master-ge3d792d24f.eupspkg: Command:
              source /Users/rhl/eups/bin/setups.sh; export EUPS_PATH=/Users/lsst/products; (/Users/lsst/products/EupsBuildDir/DarwinX86/psfex-master-ge3d792d24f/build.sh) >> /Users/lsst/products/EupsBuildDir/DarwinX86/psfex-master-ge3d792d24f/build.log 2>&1 4>/Users/lsst/products/EupsBuildDir/DarwinX86/psfex-master-ge3d792d24f/build.msg 
      exited with code 252
      

      The linker command is:

      clang -o lib/libpsfex.dylib -dynamiclib -Wl,-install_name -Wl,libpsfex.dylib -Wl,-headerpad_max_install_names src/levmar/lmbc.os src/levmar/Axb.os src/wcs_utils.os src/sample_utils.os src/fits/fitsmisc.os src/dummies.os src/diagnostic.os src/levmar/lm.os src/levmar/lmblec.os src/levmar/lmbleic.os src/field_utils.os src/fft.os src/psf.os src/field.os src/makeit2.os src/context.os src/misc.os src/homo.os src/wcs/poly.os src/xml.os src/prefs.os src/vignet.os src/pca.os src/levmar/misc.os src/levmar/lmlec.os -Llib -L/Users/lsst/products/DarwinX86/fftw/3.3.3-1-g8fdba61/lib -lfftw3 -llapackstub
      

      and the symbols do seem to be defined in the library:

      nm -oa /Users/lsst/products/DarwinX86/fftw/3.3.3-1-g8fdba61/lib/* 2>/dev/null | grep fftwf_free
      /Users/lsst/products/DarwinX86/fftw/3.3.3-1-g8fdba61/lib/libfftw3f.3.dylib: 00000000000cb4d0 T _fftwf_free
      /Users/lsst/products/DarwinX86/fftw/3.3.3-1-g8fdba61/lib/libfftw3f.a:malloc.o: 0000000000000010 T _fftwf_free
      /Users/lsst/products/DarwinX86/fftw/3.3.3-1-g8fdba61/lib/libfftw3f.dylib: 00000000000cb4d0 T _fftwf_free
      

      So maybe the linker's finding some other fftw?

      Attachments

        Issue Links

          Activity

            ... and the reason why that matters is because it changes the place in the search path where $LSST_CFG_PATH is inserted. I have an fftwf.cfg files on that path (it's set to export LSST_CFG_PATH=$HOME/LSST/devenv/buildFiles which is git@github.com:lsst/buildFiles).

            I'm sympathetic with Josh on this; it's an obscure path variable designed to handle things like fftw or python that need .cfg files and for which you don't want to declare the package to eups. This dates back to the early days of .cfg files and can probably be retired (you can always declare a pseudo-package if needs be).

            rhl Robert Lupton added a comment - ... and the reason why that matters is because it changes the place in the search path where $LSST_CFG_PATH is inserted. I have an fftwf.cfg files on that path (it's set to export LSST_CFG_PATH=$HOME/LSST/devenv/buildFiles which is git@github.com:lsst/buildFiles). I'm sympathetic with Josh on this; it's an obscure path variable designed to handle things like fftw or python that need .cfg files and for which you don't want to declare the package to eups. This dates back to the early days of .cfg files and can probably be retired (you can always declare a pseudo-package if needs be).

            rhl This is an unintentional regression. I discovered that the path search logic was completely duplicated while running scons under pdb and removed one of the implementations. It sounds like the expected behavior for LSST_CFG_PATH is to be searched first? Ie., the order of these two loops should be inverted:

            https://github.com/lsst/sconsUtils/blob/9e06e55fe10fb4c150a970f40748afb96f2e45d3/python/lsst/sconsUtils/state.py#L117-L142

            jhoblitt Joshua Hoblitt added a comment - rhl This is an unintentional regression. I discovered that the path search logic was completely duplicated while running scons under pdb and removed one of the implementations. It sounds like the expected behavior for LSST_CFG_PATH is to be searched first? Ie., the order of these two loops should be inverted: https://github.com/lsst/sconsUtils/blob/9e06e55fe10fb4c150a970f40748afb96f2e45d3/python/lsst/sconsUtils/state.py#L117-L142

            I don't especially care what the order is, but I do care that things changed!

            As I implied above, Jim and I would be OK with an RFC to remove LSST_CFG_PATH

            rhl Robert Lupton added a comment - I don't especially care what the order is, but I do care that things changed! As I implied above, Jim and I would be OK with an RFC to remove LSST_CFG_PATH
            tjenness Tim Jenness added a comment -

            Can we retitle this ticket to reflect the agreed problem?

            tjenness Tim Jenness added a comment - Can we retitle this ticket to reflect the agreed problem?
            ktl Kian-Tat Lim added a comment -

            Since I think this has been broken for four years, I don't think it should take an RFC to remove this code.  I will do that.  Marking this as WON'T FIX (and in fact actively breaking more...).

            ktl Kian-Tat Lim added a comment - Since I think this has been broken for four years, I don't think it should take an RFC to remove this code.  I will do that.  Marking this as WON'T FIX (and in fact actively breaking more...).

            People

              ktl Kian-Tat Lim
              rhl Robert Lupton
              John Swinbank, Jonathan Sick, Joshua Hoblitt, Kian-Tat Lim, Robert Lupton, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Jenkins

                  No builds found.