Details

    • Type: Story
    • Status: Done
    • Priority: Major
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: sconsUtils
    • Labels:
      None
    • Templates:
    • Story Points:
      0.5
    • Epic Link:
    • Sprint:
      DRP F16-2
    • Team:
      Data Release Production

      Description

      By default, our builds are not optimised (-O0), which requires everyone who doesn't want to wait until the heat death of the universe to set SCONSFLAGS="opt=3", but other packages that are built with scons may not recognise this. This default is also contrary to the standard practise for open-source software, which is that by default builds are optimised. I will change the default optimisation level to opt=3 from the current opt=0. I will also add support for -Og.

      This change was approved in RFC-202.

        Issue Links

          Activity

          Hide
          price Paul Price added a comment -

          Tim Jenness, would you review this please? I've implemented what I threatened, and observed that it works (-O3 is default, can set -Og). I also fixed the annoying error: no result from the compiler classifier.

          price@price-laptop:~/LSST/sconsUtils (tickets/DM-7010=) $ git sub
          commit b5e6e6106b33172be28b07195fe5b3c63323fda7
          Author: Paul Price <price@astro.princeton.edu>
          Date:   Mon Jul 25 10:48:14 2016 -0400
           
              set default optimisation level to 3
              
              Our C++ code needs good optimisation to run efficiently, but this
              was not set by default.
           
           python/lsst/sconsUtils/state.py | 2 +-
           1 file changed, 1 insertion(+), 1 deletion(-)
           
          commit b6cc932bcadb347e719aaa45d0d24e75e0488729
          Author: Paul Price <price@astro.princeton.edu>
          Date:   Mon Jul 25 10:51:44 2016 -0400
           
              add support for -Og
              
              This optimisation flag is supported from gcc 4.8, and is apparently
              useful for debugging C++ code.
           
           python/lsst/sconsUtils/state.py | 6 +++---
           1 file changed, 3 insertions(+), 3 deletions(-)
           
          commit 29ad0e61099543c74249f1998e055ea5c6eab8a8
          Author: Paul Price <price@astro.princeton.edu>
          Date:   Mon Jul 25 11:31:21 2016 -0400
           
              ClassifyCc: fix "error: no result"
              
              We weren't setting a result, so the default result ("error: no result")
              was being used for us, which can confuse the user.
           
           python/lsst/sconsUtils/state.py | 2 ++
           1 file changed, 2 insertions(+)
          

          Show
          price Paul Price added a comment - Tim Jenness , would you review this please? I've implemented what I threatened, and observed that it works ( -O3 is default, can set -Og ). I also fixed the annoying error: no result from the compiler classifier. price@price-laptop:~/LSST/sconsUtils (tickets/DM-7010=) $ git sub commit b5e6e6106b33172be28b07195fe5b3c63323fda7 Author: Paul Price <price@astro.princeton.edu> Date: Mon Jul 25 10:48:14 2016 -0400   set default optimisation level to 3 Our C++ code needs good optimisation to run efficiently, but this was not set by default.   python/lsst/sconsUtils/state.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)   commit b6cc932bcadb347e719aaa45d0d24e75e0488729 Author: Paul Price <price@astro.princeton.edu> Date: Mon Jul 25 10:51:44 2016 -0400   add support for -Og This optimisation flag is supported from gcc 4.8, and is apparently useful for debugging C++ code.   python/lsst/sconsUtils/state.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)   commit 29ad0e61099543c74249f1998e055ea5c6eab8a8 Author: Paul Price <price@astro.princeton.edu> Date: Mon Jul 25 11:31:21 2016 -0400   ClassifyCc: fix "error: no result" We weren't setting a result, so the default result ("error: no result") was being used for us, which can confuse the user.   python/lsst/sconsUtils/state.py | 2 ++ 1 file changed, 2 insertions(+)
          Hide
          tjenness Tim Jenness added a comment -

          Looks good to me.

          Show
          tjenness Tim Jenness added a comment - Looks good to me.
          Hide
          price Paul Price added a comment -

          sims_utils keeps failing under Jenkins.

          ### sims_utils - testModifiedJulianDate.py.failed
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          tests/testModifiedJulianDate.py
           
          /home/jenkins-slave/workspace/stack-os-matrix/compiler/gcc/label/centos-6/python/py2/lsstsw/miniconda/lib/python2.7/site-packages/astropy/config/configuration.py:541: ConfigurationMissingWarning: Configuration defaults will be used due to OSError:Could not find unix home directory to search for astropy config dir on None
            warn(ConfigurationMissingWarning(msg))
          /home/jenkins-slave/workspace/stack-os-matrix/compiler/gcc/label/centos-6/python/py2/lsstsw/miniconda/lib/python2.7/site-packages/matplotlib/font_manager.py:273: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment.
            warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.')
          WARNING: CacheMissingWarning: Remote data cache could not be accessed due to OSError: Could not find unix home directory to search for astropy config dir [astropy.utils.data]
          Downloading http://maia.usno.navy.mil/ser7/finals2000A.all [Done]
          WARNING: CacheMissingWarning: (u'File downloaded to temporary location due to problem with cache directory and will not be cached.', '/tmp/tmpL9wmVf') [astropy.utils.data]
          .....F
          ======================================================================
          FAIL: test_warnings (__main__.MjdTest)
          ----------------------------------------------------------------------
          Traceback (most recent call last):
            File "tests/testModifiedJulianDate.py", line 134, in test_warnings
              self.assertIsInstance(w_list[1].message, UTCtoUT1Warning)
          AssertionError: ErfaWarning('ERFA function "utcut1" yielded 1 of "dubious year (Note 3)"',) is not an instance of <class 'lsst.sims.utils.ModifiedJulianDate.UTCtoUT1Warning'>
           
          ----------------------------------------------------------------------
          Ran 6 tests in 8.620s
           
          FAILED (failures=1)
          

          This looks like it's someone else's fault, but it's keeping me from finishing the Jenkins run.

          Show
          price Paul Price added a comment - sims_utils keeps failing under Jenkins. ### sims_utils - testModifiedJulianDate.py.failed ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ tests/testModifiedJulianDate.py   /home/jenkins-slave/workspace/stack-os-matrix/compiler/gcc/label/centos-6/python/py2/lsstsw/miniconda/lib/python2.7/site-packages/astropy/config/configuration.py:541: ConfigurationMissingWarning: Configuration defaults will be used due to OSError:Could not find unix home directory to search for astropy config dir on None warn(ConfigurationMissingWarning(msg)) /home/jenkins-slave/workspace/stack-os-matrix/compiler/gcc/label/centos-6/python/py2/lsstsw/miniconda/lib/python2.7/site-packages/matplotlib/font_manager.py:273: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment. warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.') WARNING: CacheMissingWarning: Remote data cache could not be accessed due to OSError: Could not find unix home directory to search for astropy config dir [astropy.utils.data] Downloading http://maia.usno.navy.mil/ser7/finals2000A.all [Done] WARNING: CacheMissingWarning: (u'File downloaded to temporary location due to problem with cache directory and will not be cached.', '/tmp/tmpL9wmVf') [astropy.utils.data] .....F ====================================================================== FAIL: test_warnings (__main__.MjdTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "tests/testModifiedJulianDate.py", line 134, in test_warnings self.assertIsInstance(w_list[1].message, UTCtoUT1Warning) AssertionError: ErfaWarning('ERFA function "utcut1" yielded 1 of "dubious year (Note 3)"',) is not an instance of <class 'lsst.sims.utils.ModifiedJulianDate.UTCtoUT1Warning'>   ---------------------------------------------------------------------- Ran 6 tests in 8.620s   FAILED (failures=1) This looks like it's someone else's fault, but it's keeping me from finishing the Jenkins run.
          Hide
          price Paul Price added a comment -

          It passes for me when I run it manually (though I fiddled with the environment some to get the necessary modules). Also note that there is no C++ in that module for my optimisation changes to impact. I believe there's something funky with the system, and have asked Joshua Hoblitt to look at it.

          Show
          price Paul Price added a comment - It passes for me when I run it manually (though I fiddled with the environment some to get the necessary modules). Also note that there is no C++ in that module for my optimisation changes to impact. I believe there's something funky with the system, and have asked Joshua Hoblitt to look at it.
          Hide
          price Paul Price added a comment -

          After a build of master sims_utils failed on Jenkins, we filed SIM-2067 to address that.

          Show
          price Paul Price added a comment - After a build of master sims_utils failed on Jenkins, we filed SIM-2067 to address that.
          Hide
          price Paul Price added a comment -

          From HipChat:

          [12:00 AM] Paul Price: OK, can I merge 7010 to master without running Jenkins on the sims side?
          [12:00 AM] K-T Lim: If they're broken, and you don't break them worse, then I'd say OK.

          Merged to master.

          Show
          price Paul Price added a comment - From HipChat: [12:00 AM] Paul Price: OK, can I merge 7010 to master without running Jenkins on the sims side? [12:00 AM] K-T Lim: If they're broken, and you don't break them worse, then I'd say OK. Merged to master.

            People

            • Assignee:
              price Paul Price
              Reporter:
              price Paul Price
              Reviewers:
              Tim Jenness
              Watchers:
              Paul Price, Tim Jenness
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development

                  Agile