Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-2769

sconsUtils has a hard dependency on EUPS for both tests and installation

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: sconsUtils
    • Labels:
      None

      Description

      After some discussion on Data Management, its clear that sconsUtils is a hard requirement on EUPS for both tests and installation. It was decided by RFC-44 that tests should not depend on EUPS. However, I'd argue that sconsUtils should also not depend on EUPS as any package that uses sconsUtils (the virtual entirety of the stack) can not build or run tests without the presence of EUPS. The current situation is that the complete stack has a hard dependency on EUPS.

      Attempting to build sconUtils without the presence of EUPS. The tests fail attempting to import the eups module.

      $ SCONSUTILS_DIR=. scons -Q
      Unable to import eups; guessing flavor
      CC is gcc version 4.8.3
      Checking for C++11 support
      C++11 supported with '-std=c++11'
      Unable to import eups; guessing flavor
      Doxygen is not setup; skipping documentation build.
      ImportError: No module named eups:
        File "/home/vagrant/sconsUtils/SConstruct", line 9:
          scripts.BasicSConstruct.initialize(packageName="sconsUtils")
        File "python/lsst/sconsUtils/scripts.py", line 106:
          SCons.Script.SConscript(os.path.join(root, "SConscript"))
        File "/usr/lib/scons/SCons/Script/SConscript.py", line 609:
          return method(*args, **kw)
        File "/usr/lib/scons/SCons/Script/SConscript.py", line 546:
          return _SConscript(self.fs, *files, **subst_kw)
        File "/usr/lib/scons/SCons/Script/SConscript.py", line 260:
          exec _file_ in call_stack[-1].globals
        File "/home/vagrant/sconsUtils/tests/SConscript", line 5:
          import eups
      

      Attempting to bypass the test failures:

      [vagrant@jenkins-el7-1 sconsUtils]$ rm -rf tests
      [vagrant@jenkins-el7-1 sconsUtils]$ SCONSUTILS_DIR=. scons -Q install
      Unable to import eups; guessing flavor
      CC is gcc version 4.8.3
      Checking for C++11 support
      C++11 supported with '-std=c++11'
      Error with git version: uncommitted changes
      Found problem with version number; update or specify force=True to proceed
      

        Attachments

          Issue Links

            Activity

            Hide
            jhoblitt Joshua Hoblitt added a comment -

            I've found another hard dep. on eups, if the install target is attempting to deploy a directory named 'ups' it shells out to invoke eups.

            https://github.com/lsst/sconsUtils/blob/54c983ffe9714a33657c4388de3506fe7a40518d/python/lsst/sconsUtils/installation.py#L366

            Show
            jhoblitt Joshua Hoblitt added a comment - I've found another hard dep. on eups, if the install target is attempting to deploy a directory named 'ups' it shells out to invoke eups. https://github.com/lsst/sconsUtils/blob/54c983ffe9714a33657c4388de3506fe7a40518d/python/lsst/sconsUtils/installation.py#L366
            Hide
            jhoblitt Joshua Hoblitt added a comment -

            I confess that I still don't a very deep understanding of scons so I'm requesting review "early" in hopes of getting some guidance by folks more familiar with it. I've posted a few questions on the GH PR.

            sconUtils is to the point that you can run `scons` without declaring any env vars or option flags. However, I did add a `--no-eups` flag to force eups off.

            [vagrant@localhost sconsUtils]$ scons
            scons: Reading SConscript files ...
            EUPS integration: disabled
            Unable to import eups; guessing flavor
            Checking who built the CC compiler...(cached) error: no result
            CC is gcc version 4.9.2
            Checking for C++11 support
            Checking whether the C++ compiler works... (cached) yes
            C++11 supported with '-std=c++11'
            Checking for C++ header file tr1/unordered_map... (cached) yes
            Setting up environment to build package 'sconsUtils'.
            Unable to import eups; guessing flavor
            Could not find EUPS product dir for 'sconsUtils'; using /vagrant/sconsUtils.
            warning: import of eups failed; tests will be skipped
            scons: done reading SConscript files.
            scons: Building targets ...
            scons: Nothing to be done for `python'.
            scons: `tests' is up to date.
            buildConfig(["doc/doxygen.conf"], ["doc/doxygen.conf.in"])
            doxygen /vagrant/sconsUtils/doc/doxygen.conf
            scons: done building targets.
            [vagrant@localhost sconsUtils]$ scons --prefix=/tmp/lsst install
            scons: Reading SConscript files ...
            EUPS integration: disabled
            Unable to import eups; guessing flavor
            Checking who built the CC compiler...(cached) error: no result
            CC is gcc version 4.9.2
            Checking for C++11 support
            Checking whether the C++ compiler works... (cached) yes
            C++11 supported with '-std=c++11'
            Checking for C++ header file tr1/unordered_map... (cached) yes
            Setting up environment to build package 'sconsUtils'.
            Unable to import eups; guessing flavor
            Could not find EUPS product dir for 'sconsUtils'; using /vagrant/sconsUtils.
            warning: import of eups failed; tests will be skipped
            scons: done reading SConscript files.
            scons: Building targets ...
            buildConfig(["doc/doxygen.conf"], ["doc/doxygen.conf.in"])
            doxygen /vagrant/sconsUtils/doc/doxygen.conf
            DirectoryInstaller(["/tmp/lsst/ups"], ["ups"])
            DirectoryInstaller(["/tmp/lsst/python"], ["python"])
            DirectoryInstaller(["/tmp/lsst/bin"], ["bin"])
            DirectoryInstaller(["/tmp/lsst/doc"], ["doc"])
            DirectoryInstaller(["/tmp/lsst/tests"], ["tests"])
            scons: done building targets.
            

            Show
            jhoblitt Joshua Hoblitt added a comment - I confess that I still don't a very deep understanding of scons so I'm requesting review "early" in hopes of getting some guidance by folks more familiar with it. I've posted a few questions on the GH PR. sconUtils is to the point that you can run `scons` without declaring any env vars or option flags. However, I did add a `--no-eups` flag to force eups off. [vagrant @localhost sconsUtils]$ scons scons: Reading SConscript files ... EUPS integration: disabled Unable to import eups; guessing flavor Checking who built the CC compiler...(cached) error: no result CC is gcc version 4.9 . 2 Checking for C++ 11 support Checking whether the C++ compiler works... (cached) yes C++ 11 supported with '-std=c++11' Checking for C++ header file tr1/unordered_map... (cached) yes Setting up environment to build package 'sconsUtils' . Unable to import eups; guessing flavor Could not find EUPS product dir for 'sconsUtils' ; using /vagrant/sconsUtils. warning: import of eups failed; tests will be skipped scons: done reading SConscript files. scons: Building targets ... scons: Nothing to be done for `python'. scons: `tests' is up to date. buildConfig([ "doc/doxygen.conf" ], [ "doc/doxygen.conf.in" ]) doxygen /vagrant/sconsUtils/doc/doxygen.conf scons: done building targets. [vagrant @localhost sconsUtils]$ scons --prefix=/tmp/lsst install scons: Reading SConscript files ... EUPS integration: disabled Unable to import eups; guessing flavor Checking who built the CC compiler...(cached) error: no result CC is gcc version 4.9 . 2 Checking for C++ 11 support Checking whether the C++ compiler works... (cached) yes C++ 11 supported with '-std=c++11' Checking for C++ header file tr1/unordered_map... (cached) yes Setting up environment to build package 'sconsUtils' . Unable to import eups; guessing flavor Could not find EUPS product dir for 'sconsUtils' ; using /vagrant/sconsUtils. warning: import of eups failed; tests will be skipped scons: done reading SConscript files. scons: Building targets ... buildConfig([ "doc/doxygen.conf" ], [ "doc/doxygen.conf.in" ]) doxygen /vagrant/sconsUtils/doc/doxygen.conf DirectoryInstaller([ "/tmp/lsst/ups" ], [ "ups" ]) DirectoryInstaller([ "/tmp/lsst/python" ], [ "python" ]) DirectoryInstaller([ "/tmp/lsst/bin" ], [ "bin" ]) DirectoryInstaller([ "/tmp/lsst/doc" ], [ "doc" ]) DirectoryInstaller([ "/tmp/lsst/tests" ], [ "tests" ]) scons: done building targets.
            Hide
            jbosch Jim Bosch added a comment -

            Happy to review later, but I probably won't get to this until Wednesday at the earliest, as I've got a lot of prep work to do my my coadds talk and a few other high-priority tickets standing between me and starting that.

            Show
            jbosch Jim Bosch added a comment - Happy to review later, but I probably won't get to this until Wednesday at the earliest, as I've got a lot of prep work to do my my coadds talk and a few other high-priority tickets standing between me and starting that.
            Hide
            jbosch Jim Bosch added a comment -

            I've commented on everything I had an opinion on. My understanding is that this isn't actually done yet, so it's not appropriate to move it to Review Complete, but you have my permission to do that if I misunderstood and you've gotten sign-off from the other any reviewers you want feedback from.

            Show
            jbosch Jim Bosch added a comment - I've commented on everything I had an opinion on. My understanding is that this isn't actually done yet, so it's not appropriate to move it to Review Complete, but you have my permission to do that if I misunderstood and you've gotten sign-off from the other any reviewers you want feedback from.
            Hide
            tjenness Tim Jenness added a comment -

            I'm okay with this being merged once Jim's concerns are handled, although I'd like a new ticket filed on the treatment of _DIR environment variables so we don't forget.

            Show
            tjenness Tim Jenness added a comment - I'm okay with this being merged once Jim's concerns are handled, although I'd like a new ticket filed on the treatment of _DIR environment variables so we don't forget.
            Hide
            jhoblitt Joshua Hoblitt added a comment -

            Per discussion on hipchat with Kian-Tat Lim & Tim Jenness, I've decided to go ahead and merge this branch as-is. I've opened DM-3208 as trivial priority ticket to cover improving the probing logic for eups/doxygen.

            Show
            jhoblitt Joshua Hoblitt added a comment - Per discussion on hipchat with Kian-Tat Lim & Tim Jenness , I've decided to go ahead and merge this branch as-is. I've opened DM-3208 as trivial priority ticket to cover improving the probing logic for eups/doxygen.
            Hide
            tjenness Tim Jenness added a comment -

            Why was the assignee changed a year after the work was completed?

            Show
            tjenness Tim Jenness added a comment - Why was the assignee changed a year after the work was completed?
            Hide
            jmatt J Matt Peterson [X] (Inactive) added a comment -

            That was an accident. I was reading eups bugs looking for the problem I have right now.

            Show
            jmatt J Matt Peterson [X] (Inactive) added a comment - That was an accident. I was reading eups bugs looking for the problem I have right now.

              People

              • Assignee:
                jhoblitt Joshua Hoblitt
                Reporter:
                jhoblitt Joshua Hoblitt
                Reviewers:
                Jim Bosch, Kian-Tat Lim, Paul Price, Tim Jenness
                Watchers:
                Frossie Economou, Jim Bosch, J Matt Peterson [X] (Inactive), Joshua Hoblitt, Kian-Tat Lim, Paul Price, Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: