[DM-2769] sconsUtils has a hard dependency on EUPS for both tests and installation - Jira

XML

Word

Printable

Details

Type: Bug
Status: Done
Resolution: Done
Fix Version/s: None
Component/s: sconsUtils
Labels:
None

Story Points:
6
Epic Link:
sqre-ci-w16
Team:
SQuaRE

Description

Attempting to build sconUtils without the presence of EUPS. The tests fail attempting to import the eups module.

$ SCONSUTILS_DIR=. scons -Q

Unable to import eups; guessing flavor

CC is gcc version 4.8.3

Checking for C++11 support

C++11 supported with '-std=c++11'

Unable to import eups; guessing flavor

Doxygen is not setup; skipping documentation build.

ImportError: No module named eups:

  File "/home/vagrant/sconsUtils/SConstruct", line 9:

    scripts.BasicSConstruct.initialize(packageName="sconsUtils")

  File "python/lsst/sconsUtils/scripts.py", line 106:

    SCons.Script.SConscript(os.path.join(root, "SConscript"))

  File "/usr/lib/scons/SCons/Script/SConscript.py", line 609:

    return method(*args, **kw)

  File "/usr/lib/scons/SCons/Script/SConscript.py", line 546:

    return _SConscript(self.fs, *files, **subst_kw)

  File "/usr/lib/scons/SCons/Script/SConscript.py", line 260:

    exec _file_ in call_stack[-1].globals

  File "/home/vagrant/sconsUtils/tests/SConscript", line 5:

    import eups

Attempting to bypass the test failures:

[vagrant@jenkins-el7-1 sconsUtils]$ rm -rf tests

[vagrant@jenkins-el7-1 sconsUtils]$ SCONSUTILS_DIR=. scons -Q install

Unable to import eups; guessing flavor

CC is gcc version 4.8.3

Checking for C++11 support

C++11 supported with '-std=c++11'

Error with git version: uncommitted changes

Found problem with version number; update or specify force=True to proceed

Attachments

Issue Links

blocks

DM-2768 investigate decomposition of stack build into independent packages

Done

RFC-44 Remove build system dependencies from tests

Implemented

is duplicated by

DM-1164 remove dependency between sconsUtils and eups

Won't Fix

is triggered by

RFC-69 Refactor sconsUtils to enhance usability in CI

Adopted

is triggering

DM-3655 LSST_CFG_PATH support broken because of recent sconsUtils change

Won't Fix

relates to

DM-3208 sconsUtils "probe" refactoring

To Do

(1 relates to)

Activity

Ascending order - Click to sort in descending order

No builds found.

Joshua Hoblitt created issue - 20/May/15 5:57 PM

Joshua Hoblitt made changes - 20/May/15 6:04 PM

Field	Original Value	New Value
Link		This issue blocks ~~RFC-44~~ [ ~~RFC-44~~ ]

Joshua Hoblitt made changes - 20/May/15 6:05 PM

Assignee

Joshua Hoblitt [ jhoblitt ]

Jim Bosch [ jbosch ]

Hide

Permalink

Joshua Hoblitt added a comment - 20/May/15 6:25 PM

I've found another hard dep. on eups, if the install target is attempting to deploy a directory named 'ups' it shells out to invoke eups.

https://github.com/lsst/sconsUtils/blob/54c983ffe9714a33657c4388de3506fe7a40518d/python/lsst/sconsUtils/installation.py#L366

Show

Joshua Hoblitt added a comment - 20/May/15 6:25 PM I've found another hard dep. on eups, if the install target is attempting to deploy a directory named 'ups' it shells out to invoke eups. https://github.com/lsst/sconsUtils/blob/54c983ffe9714a33657c4388de3506fe7a40518d/python/lsst/sconsUtils/installation.py#L366

Joshua Hoblitt made changes - 20/May/15 7:02 PM

Link

This issue blocks ~~DM-2768~~ [ ~~DM-2768~~ ]

Joshua Hoblitt made changes - 21/May/15 9:09 AM

Assignee

Jim Bosch [ jbosch ]

Frossie Economou [ frossie ]

Tim Jenness made changes - 05/Jun/15 5:36 PM

Team

SQuaRE [ 10302 ]

Joshua Hoblitt made changes - 08/Jun/15 3:54 PM

Description

After some discussion on Data Management, its clear that sconsUtils is a hard requirement on EUPS for both tests and installation. It was decided by ~~RFC-44~~ that tests should not depend on EUPS. However, I'd argue that sconsUtils should also not depend on EUPS as any package that uses sconsUtils (the virtual entirety of the stack) can not build or run tests with the presence of EUPS. The current situation is that the complete stack has a hard dependency on EUPS.

Attempting to build sconUtils without the presence of EUPS. The tests fail attempting to import the eups module.
{code}
$ SCONSUTILS_DIR=. scons -Q
Unable to import eups; guessing flavor
CC is gcc version 4.8.3
Checking for C++11 support
C++11 supported with '-std=c++11'
Unable to import eups; guessing flavor
Doxygen is not setup; skipping documentation build.
ImportError: No module named eups:
  File "/home/vagrant/sconsUtils/SConstruct", line 9:
    scripts.BasicSConstruct.initialize(packageName="sconsUtils")
  File "python/lsst/sconsUtils/scripts.py", line 106:
    SCons.Script.SConscript(os.path.join(root, "SConscript"))
  File "/usr/lib/scons/SCons/Script/SConscript.py", line 609:
    return method(*args, **kw)
  File "/usr/lib/scons/SCons/Script/SConscript.py", line 546:
    return _SConscript(self.fs, *files, **subst_kw)
  File "/usr/lib/scons/SCons/Script/SConscript.py", line 260:
    exec _file_ in call_stack[-1].globals
  File "/home/vagrant/sconsUtils/tests/SConscript", line 5:
    import eups
{code}

Attempting to bypass the test failures:
{code}
[vagrant@jenkins-el7-1 sconsUtils]$ rm -rf tests
[vagrant@jenkins-el7-1 sconsUtils]$ SCONSUTILS_DIR=. scons -Q install
Unable to import eups; guessing flavor
CC is gcc version 4.8.3
Checking for C++11 support
C++11 supported with '-std=c++11'
Error with git version: uncommitted changes
Found problem with version number; update or specify force=True to proceed
{code}

After some discussion on Data Management, its clear that sconsUtils is a hard requirement on EUPS for both tests and installation. It was decided by ~~RFC-44~~ that tests should not depend on EUPS. However, I'd argue that sconsUtils should also not depend on EUPS as any package that uses sconsUtils (the virtual entirety of the stack) can not build or run tests without the presence of EUPS. The current situation is that the complete stack has a hard dependency on EUPS.

Attempting to build sconUtils without the presence of EUPS. The tests fail attempting to import the eups module.
{code}
$ SCONSUTILS_DIR=. scons -Q
Unable to import eups; guessing flavor
CC is gcc version 4.8.3
Checking for C++11 support
C++11 supported with '-std=c++11'
Unable to import eups; guessing flavor
Doxygen is not setup; skipping documentation build.
ImportError: No module named eups:
  File "/home/vagrant/sconsUtils/SConstruct", line 9:
    scripts.BasicSConstruct.initialize(packageName="sconsUtils")
  File "python/lsst/sconsUtils/scripts.py", line 106:
    SCons.Script.SConscript(os.path.join(root, "SConscript"))
  File "/usr/lib/scons/SCons/Script/SConscript.py", line 609:
    return method(*args, **kw)
  File "/usr/lib/scons/SCons/Script/SConscript.py", line 546:
    return _SConscript(self.fs, *files, **subst_kw)
  File "/usr/lib/scons/SCons/Script/SConscript.py", line 260:
    exec _file_ in call_stack[-1].globals
  File "/home/vagrant/sconsUtils/tests/SConscript", line 5:
    import eups
{code}

Attempting to bypass the test failures:
{code}
[vagrant@jenkins-el7-1 sconsUtils]$ rm -rf tests
[vagrant@jenkins-el7-1 sconsUtils]$ SCONSUTILS_DIR=. scons -Q install
Unable to import eups; guessing flavor
CC is gcc version 4.8.3
Checking for C++11 support
C++11 supported with '-std=c++11'
Error with git version: uncommitted changes
Found problem with version number; update or specify force=True to proceed
{code}

Joshua Hoblitt made changes - 11/Jun/15 2:25 PM

Assignee

Frossie Economou [ frossie ]

Joshua Hoblitt [ jhoblitt ]

Tim Jenness made changes - 26/Jun/15 10:27 AM

Link

This issue relates to ~~RFC-69~~ [ ~~RFC-69~~ ]

Hide

Permalink

Joshua Hoblitt added a comment - 17/Jul/15 1:46 PM

I confess that I still don't a very deep understanding of scons so I'm requesting review "early" in hopes of getting some guidance by folks more familiar with it. I've posted a few questions on the GH PR.

sconUtils is to the point that you can run `scons` without declaring any env vars or option flags. However, I did add a `--no-eups` flag to force eups off.

[vagrant@localhost sconsUtils]$ scons

scons: Reading SConscript files ...

EUPS integration: disabled

Unable to import eups; guessing flavor

Checking who built the CC compiler...(cached) error: no result

CC is gcc version 4.9.2

Checking for C++11 support

Checking whether the C++ compiler works... (cached) yes

C++11 supported with '-std=c++11'

Checking for C++ header file tr1/unordered_map... (cached) yes

Setting up environment to build package 'sconsUtils'.

Unable to import eups; guessing flavor

Could not find EUPS product dir for 'sconsUtils'; using /vagrant/sconsUtils.

warning: import of eups failed; tests will be skipped

scons: done reading SConscript files.

scons: Building targets ...

scons: Nothing to be done for `python'.

scons: `tests' is up to date.

buildConfig(["doc/doxygen.conf"], ["doc/doxygen.conf.in"])

doxygen /vagrant/sconsUtils/doc/doxygen.conf

scons: done building targets.

[vagrant@localhost sconsUtils]$ scons --prefix=/tmp/lsst install

scons: Reading SConscript files ...

EUPS integration: disabled

Unable to import eups; guessing flavor

Checking who built the CC compiler...(cached) error: no result

CC is gcc version 4.9.2

Checking for C++11 support

Checking whether the C++ compiler works... (cached) yes

C++11 supported with '-std=c++11'

Checking for C++ header file tr1/unordered_map... (cached) yes

Setting up environment to build package 'sconsUtils'.

Unable to import eups; guessing flavor

Could not find EUPS product dir for 'sconsUtils'; using /vagrant/sconsUtils.

warning: import of eups failed; tests will be skipped

scons: done reading SConscript files.

scons: Building targets ...

buildConfig(["doc/doxygen.conf"], ["doc/doxygen.conf.in"])

doxygen /vagrant/sconsUtils/doc/doxygen.conf

DirectoryInstaller(["/tmp/lsst/ups"], ["ups"])

DirectoryInstaller(["/tmp/lsst/python"], ["python"])

DirectoryInstaller(["/tmp/lsst/bin"], ["bin"])

DirectoryInstaller(["/tmp/lsst/doc"], ["doc"])

DirectoryInstaller(["/tmp/lsst/tests"], ["tests"])

scons: done building targets.

Show

Joshua Hoblitt added a comment - 17/Jul/15 1:46 PM I confess that I still don't a very deep understanding of scons so I'm requesting review "early" in hopes of getting some guidance by folks more familiar with it. I've posted a few questions on the GH PR. sconUtils is to the point that you can run `scons` without declaring any env vars or option flags. However, I did add a `--no-eups` flag to force eups off. [vagrant @localhost sconsUtils]$ scons scons: Reading SConscript files ... EUPS integration: disabled Unable to import eups; guessing flavor Checking who built the CC compiler...(cached) error: no result CC is gcc version 4.9 . 2 Checking for C++ 11 support Checking whether the C++ compiler works... (cached) yes C++ 11 supported with '-std=c++11' Checking for C++ header file tr1/unordered_map... (cached) yes Setting up environment to build package 'sconsUtils' . Unable to import eups; guessing flavor Could not find EUPS product dir for 'sconsUtils' ; using /vagrant/sconsUtils. warning: import of eups failed; tests will be skipped scons: done reading SConscript files. scons: Building targets ... scons: Nothing to be done for `python'. scons: `tests' is up to date. buildConfig([ "doc/doxygen.conf" ], [ "doc/doxygen.conf.in" ]) doxygen /vagrant/sconsUtils/doc/doxygen.conf scons: done building targets. [vagrant @localhost sconsUtils]$ scons --prefix=/tmp/lsst install scons: Reading SConscript files ... EUPS integration: disabled Unable to import eups; guessing flavor Checking who built the CC compiler...(cached) error: no result CC is gcc version 4.9 . 2 Checking for C++ 11 support Checking whether the C++ compiler works... (cached) yes C++ 11 supported with '-std=c++11' Checking for C++ header file tr1/unordered_map... (cached) yes Setting up environment to build package 'sconsUtils' . Unable to import eups; guessing flavor Could not find EUPS product dir for 'sconsUtils' ; using /vagrant/sconsUtils. warning: import of eups failed; tests will be skipped scons: done reading SConscript files. scons: Building targets ... buildConfig([ "doc/doxygen.conf" ], [ "doc/doxygen.conf.in" ]) doxygen /vagrant/sconsUtils/doc/doxygen.conf DirectoryInstaller([ "/tmp/lsst/ups" ], [ "ups" ]) DirectoryInstaller([ "/tmp/lsst/python" ], [ "python" ]) DirectoryInstaller([ "/tmp/lsst/bin" ], [ "bin" ]) DirectoryInstaller([ "/tmp/lsst/doc" ], [ "doc" ]) DirectoryInstaller([ "/tmp/lsst/tests" ], [ "tests" ]) scons: done building targets.

Joshua Hoblitt made changes - 17/Jul/15 1:46 PM

Reviewers		Jim Bosch, Kian-Tat Lim, Paul Price, Tim Jenness [ jbosch, ktl, price, tjenness ]
Status	To Do [ 10001 ]	In Review [ 10004 ]

Hide

Permalink

Jim Bosch added a comment - 17/Jul/15 2:13 PM

Happy to review later, but I probably won't get to this until Wednesday at the earliest, as I've got a lot of prep work to do my my coadds talk and a few other high-priority tickets standing between me and starting that.

Show

Jim Bosch added a comment - 17/Jul/15 2:13 PM Happy to review later, but I probably won't get to this until Wednesday at the earliest, as I've got a lot of prep work to do my my coadds talk and a few other high-priority tickets standing between me and starting that.

Hide

Permalink

Jim Bosch added a comment - 22/Jul/15 12:59 PM

I've commented on everything I had an opinion on. My understanding is that this isn't actually done yet, so it's not appropriate to move it to Review Complete, but you have my permission to do that if I misunderstood and you've gotten sign-off from the other any reviewers you want feedback from.

Show

Jim Bosch added a comment - 22/Jul/15 12:59 PM I've commented on everything I had an opinion on. My understanding is that this isn't actually done yet, so it's not appropriate to move it to Review Complete, but you have my permission to do that if I misunderstood and you've gotten sign-off from the other any reviewers you want feedback from.

Hide

Permalink

Tim Jenness added a comment - 22/Jul/15 1:08 PM

I'm okay with this being merged once Jim's concerns are handled, although I'd like a new ticket filed on the treatment of _DIR environment variables so we don't forget.

Show

Tim Jenness added a comment - 22/Jul/15 1:08 PM I'm okay with this being merged once Jim's concerns are handled, although I'd like a new ticket filed on the treatment of _DIR environment variables so we don't forget.

Joshua Hoblitt made changes - 23/Jul/15 3:28 PM

Link

This issue relates to DM-3208 [ DM-3208 ]

Hide

Permalink

Joshua Hoblitt added a comment - 23/Jul/15 3:29 PM

Per discussion on hipchat with Kian-Tat Lim & Tim Jenness, I've decided to go ahead and merge this branch as-is. I've opened DM-3208 as trivial priority ticket to cover improving the probing logic for eups/doxygen.

Show

Joshua Hoblitt added a comment - 23/Jul/15 3:29 PM Per discussion on hipchat with Kian-Tat Lim & Tim Jenness , I've decided to go ahead and merge this branch as-is. I've opened DM-3208 as trivial priority ticket to cover improving the probing logic for eups/doxygen.

Joshua Hoblitt made changes - 23/Jul/15 3:30 PM

Epic Link

~~DM-1141~~ [ 13950 ]

Joshua Hoblitt made changes - 23/Jul/15 3:30 PM

Story Points

Joshua Hoblitt made changes - 23/Jul/15 3:30 PM

Resolution		Done [ 10000 ]
Status	In Review [ 10004 ]	Done [ 10002 ]

Frossie Economou made changes - 07/Aug/15 5:56 PM

Epic Link

~~DM-1141~~ [ 13950 ]

~~DM-3425~~ [ 19388 ]

Tim Jenness made changes - 04/Sep/15 10:43 AM

Link

This issue is duplicated by ~~DM-1164~~ [ ~~DM-1164~~ ]

J Matt Peterson [X] (Inactive) made changes - 04/Nov/16 3:01 PM

Assignee

Joshua Hoblitt [ jhoblitt ]

J Matt Peterson [ jmatt ]

Hide

Permalink

Tim Jenness added a comment - 04/Nov/16 3:03 PM

Why was the assignee changed a year after the work was completed?

Show

Tim Jenness added a comment - 04/Nov/16 3:03 PM Why was the assignee changed a year after the work was completed?

Hide

Permalink

J Matt Peterson [X] (Inactive) added a comment - 04/Nov/16 3:47 PM

That was an accident. I was reading eups bugs looking for the problem I have right now.

Show

J Matt Peterson [X] (Inactive) added a comment - 04/Nov/16 3:47 PM That was an accident. I was reading eups bugs looking for the problem I have right now.

J Matt Peterson [X] (Inactive) made changes - 04/Nov/16 3:47 PM

Assignee

J Matt Peterson [ jmatt ]

Joshua Hoblitt [ jhoblitt ]

John Swinbank made changes - 07/Nov/16 5:09 AM

Summary

sconsUtil has a hard dependency on EUPS for both tests and installation

sconsUtils has a hard dependency on EUPS for both tests and installation

Tim Jenness made changes - 18/Jan/17 12:10 PM

Link

This issue is triggered by ~~RFC-69~~ [ ~~RFC-69~~ ]

Tim Jenness made changes - 18/Jan/17 12:11 PM

Link

This issue relates to ~~RFC-69~~ [ ~~RFC-69~~ ]

Tim Jenness made changes - 22/May/19 3:57 PM

Link

This issue is triggering ~~DM-3655~~ [ ~~DM-3655~~ ]

People

Assignee:: Joshua Hoblitt

Reporter:: Joshua Hoblitt

Reviewers:: Jim Bosch, Kian-Tat Lim, Paul Price, Tim Jenness

Watchers:: Frossie Economou, Jim Bosch, J Matt Peterson [X] (Inactive), Joshua Hoblitt, Kian-Tat Lim, Paul Price, Tim Jenness

Votes:: 0 Vote for this issue

Watchers:: 7 Start watching this issue

Dates

Created:: 20/May/15 5:57 PM

Updated:: 22/May/19 3:57 PM

Resolved:: 23/Jul/15 3:30 PM

Jenkins

No builds found.