Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-3655

LSST_CFG_PATH support broken because of recent sconsUtils change

    Details

    • Team:
      SQuaRE

      Description

      Building lsst_apps -t w_2015_35 fails on my laptop:

      SCONSFLAGS="opt=3 -Q cc=clang" eups distrib install lsst_apps -t w_2015_35
        [  1/53 ]  cfitsio 3360.lsst1 (already installed)                     done. 
        [  2/53 ]  doxygen 1.8.5 (already installed)                          done. 
        [  3/53 ]  eigen 3.2.0 (already installed)                            done. 
        [  4/53 ]  fftw 3.3.3-1-g8fdba61 (already installed)                  done. 
        [  5/53 ]  gsl 1.16.lsst1 (already installed)                         done. 
        [  6/53 ]  minuit2 5.28.00 (already installed)                        done. 
        [  7/53 ]  mysqlclient 5.1.73.lsst1-1-gb8bcc43 (already installed)    done. 
        [  8/53 ]  python 0.0.1 (already installed)                           done. 
        [  9/53 ]  swig 3.0.2.lsst1 (already installed)                       done. 
        [ 10/53 ]  xpa 2.1.15.lsst2 (already installed)                       done. 
        [ 11/53 ]  boost 1.55.0.1.lsst2+3 (already installed)                 done. 
        [ 12/53 ]  mysqlpython 1.2.3+17 (already installed)                   done. 
        [ 13/53 ]  numpy 0.0.1 (already installed)                            done. 
        [ 14/53 ]  scons 2.3.5 (already installed)                            done. 
        [ 15/53 ]  wcslib 4.14+7 (already installed)                          done. 
        [ 16/53 ]  astrometry_net 0.50.1+6 (already installed)                done. 
        [ 17/53 ]  matplotlib 0.0.1 (already installed)                       done. 
        [ 18/53 ]  pyfits 3.2.4.lsst1+3 (already installed)                   done. 
        [ 19/53 ]  sconsUtils 10.1-7-ga43ce92+1 (already installed)           done. 
        [ 20/53 ]  astrometry_net_data 10.0+30 (already installed)            done. 
        [ 21/53 ]  base 10.1-1-g8080078+19 (already installed)                done. 
        [ 22/53 ]  geom 10.0+29 (already installed)                           done. 
        [ 23/53 ]  psfex master-ge3d792d24f ...
       
      ***** error: from /Users/lsst/products/EupsBuildDir/DarwinX86/psfex-master-ge3d792d24f/build.log:
        "_fftwf_execute", referenced from:
            _fft_conv in fft.os
            _fft_rtf in fft.os
            _fft_ctf in fft.os
        "_fftwf_free", referenced from:
            _fft_conv in fft.os
        "_fftwf_malloc", referenced from:
            _fft_conv in fft.os
            _fft_rtf in fft.os
        "_fftwf_plan_dft_2d", referenced from:
            _fft_ctf in fft.os
        "_fftwf_plan_dft_c2r_2d", referenced from:
            _fft_conv in fft.os
        "_fftwf_plan_dft_r2c_2d", referenced from:
            _fft_conv in fft.os
            _fft_rtf in fft.os
      ld: symbol(s) not found for architecture x86_64
      clang: error: linker command failed with exit code 1 (use -v to see invocation)
      scons: *** [lib/libpsfex.dylib] Error 1
      + exit -4
      eups distrib: Failed to build psfex-master-ge3d792d24f.eupspkg: Command:
              source /Users/rhl/eups/bin/setups.sh; export EUPS_PATH=/Users/lsst/products; (/Users/lsst/products/EupsBuildDir/DarwinX86/psfex-master-ge3d792d24f/build.sh) >> /Users/lsst/products/EupsBuildDir/DarwinX86/psfex-master-ge3d792d24f/build.log 2>&1 4>/Users/lsst/products/EupsBuildDir/DarwinX86/psfex-master-ge3d792d24f/build.msg 
      exited with code 252
      

      The linker command is:

      clang -o lib/libpsfex.dylib -dynamiclib -Wl,-install_name -Wl,libpsfex.dylib -Wl,-headerpad_max_install_names src/levmar/lmbc.os src/levmar/Axb.os src/wcs_utils.os src/sample_utils.os src/fits/fitsmisc.os src/dummies.os src/diagnostic.os src/levmar/lm.os src/levmar/lmblec.os src/levmar/lmbleic.os src/field_utils.os src/fft.os src/psf.os src/field.os src/makeit2.os src/context.os src/misc.os src/homo.os src/wcs/poly.os src/xml.os src/prefs.os src/vignet.os src/pca.os src/levmar/misc.os src/levmar/lmlec.os -Llib -L/Users/lsst/products/DarwinX86/fftw/3.3.3-1-g8fdba61/lib -lfftw3 -llapackstub
      

      and the symbols do seem to be defined in the library:

      nm -oa /Users/lsst/products/DarwinX86/fftw/3.3.3-1-g8fdba61/lib/* 2>/dev/null | grep fftwf_free
      /Users/lsst/products/DarwinX86/fftw/3.3.3-1-g8fdba61/lib/libfftw3f.3.dylib: 00000000000cb4d0 T _fftwf_free
      /Users/lsst/products/DarwinX86/fftw/3.3.3-1-g8fdba61/lib/libfftw3f.a:malloc.o: 0000000000000010 T _fftwf_free
      /Users/lsst/products/DarwinX86/fftw/3.3.3-1-g8fdba61/lib/libfftw3f.dylib: 00000000000cb4d0 T _fftwf_free
      

      So maybe the linker's finding some other fftw?

        Attachments

          Issue Links

            Activity

            Hide
            rhl Robert Lupton added a comment -

            ... and the reason why that matters is because it changes the place in the search path where $LSST_CFG_PATH is inserted. I have an fftwf.cfg files on that path (it's set to export LSST_CFG_PATH=$HOME/LSST/devenv/buildFiles which is git@github.com:lsst/buildFiles).

            I'm sympathetic with Josh on this; it's an obscure path variable designed to handle things like fftw or python that need .cfg files and for which you don't want to declare the package to eups. This dates back to the early days of .cfg files and can probably be retired (you can always declare a pseudo-package if needs be).

            Show
            rhl Robert Lupton added a comment - ... and the reason why that matters is because it changes the place in the search path where $LSST_CFG_PATH is inserted. I have an fftwf.cfg files on that path (it's set to export LSST_CFG_PATH=$HOME/LSST/devenv/buildFiles which is git@github.com:lsst/buildFiles). I'm sympathetic with Josh on this; it's an obscure path variable designed to handle things like fftw or python that need .cfg files and for which you don't want to declare the package to eups. This dates back to the early days of .cfg files and can probably be retired (you can always declare a pseudo-package if needs be).
            Hide
            jhoblitt Joshua Hoblitt added a comment -

            Robert Lupton This is an unintentional regression. I discovered that the path search logic was completely duplicated while running scons under pdb and removed one of the implementations. It sounds like the expected behavior for LSST_CFG_PATH is to be searched first? Ie., the order of these two loops should be inverted:

            https://github.com/lsst/sconsUtils/blob/9e06e55fe10fb4c150a970f40748afb96f2e45d3/python/lsst/sconsUtils/state.py#L117-L142

            Show
            jhoblitt Joshua Hoblitt added a comment - Robert Lupton This is an unintentional regression. I discovered that the path search logic was completely duplicated while running scons under pdb and removed one of the implementations. It sounds like the expected behavior for LSST_CFG_PATH is to be searched first? Ie., the order of these two loops should be inverted: https://github.com/lsst/sconsUtils/blob/9e06e55fe10fb4c150a970f40748afb96f2e45d3/python/lsst/sconsUtils/state.py#L117-L142
            Hide
            rhl Robert Lupton added a comment -

            I don't especially care what the order is, but I do care that things changed!

            As I implied above, Jim and I would be OK with an RFC to remove LSST_CFG_PATH

            Show
            rhl Robert Lupton added a comment - I don't especially care what the order is, but I do care that things changed! As I implied above, Jim and I would be OK with an RFC to remove LSST_CFG_PATH
            Hide
            tjenness Tim Jenness added a comment -

            Can we retitle this ticket to reflect the agreed problem?

            Show
            tjenness Tim Jenness added a comment - Can we retitle this ticket to reflect the agreed problem?
            Hide
            ktl Kian-Tat Lim added a comment -

            Since I think this has been broken for four years, I don't think it should take an RFC to remove this code.  I will do that.  Marking this as WON'T FIX (and in fact actively breaking more...).

            Show
            ktl Kian-Tat Lim added a comment - Since I think this has been broken for four years, I don't think it should take an RFC to remove this code.  I will do that.  Marking this as WON'T FIX (and in fact actively breaking more...).

              People

              • Assignee:
                ktl Kian-Tat Lim
                Reporter:
                rhl Robert Lupton
                Watchers:
                John Swinbank, Jonathan Sick, Joshua Hoblitt, Kian-Tat Lim, Robert Lupton, Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel