Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-5105

new conda 'mkl' dependent packages break meas_base tests

    Details

    • Type: Bug
    • Status: Done
    • Priority: Blocker
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: Continuous Integration
    • Labels:
      None
    • Templates:
    • Story Points:
      4
    • Epic Link:
    • Team:
      SQuaRE

      Description

      Continuum release/rebuilt a number of packages last friday to depend on the the Intel MKL library.

      https://www.continuum.io/blog/developer-blog/anaconda-25-release-now-mkl-optimizations

      There are [new feature named] versions that continue to use openblas but the MKL versions appear to be installed by default. This causes at least multiple meas_base tests to fail.After extensive testing, I have confirmed that the meas_base tests do not fail with the equivalent 'nomkl' package.
      In addition, mkl is closed source software that requires you to accept and download a license file or it is time-bombed to stop working after a trial period.

          docker-centos-7: [ 36/36 ]  meas_base 2015_10.0-9-g6daf04b+7 ...
          docker-centos-7:
          docker-centos-7: ***** error: from /opt/lsst/software/stack/EupsBuildDir/Linux64/meas_base-2015_10.0-9-g6daf04b+7/build.log:
          docker-centos-7: tests/sincPhotSums.py
          docker-centos-7:
          docker-centos-7: tests/measureSources.py
          docker-centos-7:
          docker-centos-7: tests/testApertureFlux.py
          docker-centos-7:
          docker-centos-7: tests/testJacobian.py
          docker-centos-7:
          docker-centos-7: tests/testScaledApertureFlux.py
          docker-centos-7:
          docker-centos-7: The following tests failed:
          docker-centos-7: /opt/lsst/software/stack/EupsBuildDir/Linux64/meas_base-2015_10.0-9-g6daf04b+7/meas_base-2015_10.0-9-g6daf04b+7/tests/.tests/sincPhotSums.py.failed
          docker-centos-7: /opt/lsst/software/stack/EupsBuildDir/Linux64/meas_base-2015_10.0-9-g6daf04b+7/meas_base-2015_10.0-9-g6daf04b+7/tests/.tests/measureSources.py.failed
          docker-centos-7: /opt/lsst/software/stack/EupsBuildDir/Linux64/meas_base-2015_10.0-9-g6daf04b+7/meas_base-2015_10.0-9-g6daf04b+7/tests/.tests/testApertureFlux.py.failed
          docker-centos-7: /opt/lsst/software/stack/EupsBuildDir/Linux64/meas_base-2015_10.0-9-g6daf04b+7/meas_base-2015_10.0-9-g6daf04b+7/tests/.tests/testJacobian.py.failed
          docker-centos-7: /opt/lsst/software/stack/EupsBuildDir/Linux64/meas_base-2015_10.0-9-g6daf04b+7/meas_base-2015_10.0-9-g6daf04b+7/tests/.tests/testScaledApertureFlux.py.failed
          docker-centos-7: 5 tests failed
      

      (the exact cause of the test failures was not investigated as this should not have happened)

      This change has also broken the ability to import an existing conda env from 2016-02-05 or earlier that uses scipy due to some sort of package version resolution problem. Explicit declaring it as the scipy package without mkl fixes the resolution problem.

      There is a new 'nomkl' package, when installed, any subsequent package installations will default to versions without mkl. However, this does not fix any already installed packages.

      I am traumatized by the lack of reproducible build envs even within a few days of each other. After discussion in the Tucson office, I'm going to pin the lsstsw and newinstall.sh conda package versions with a commitment from square to update them on a monthly basis. I already have a test version of lsstsw/bin/deploy that defaults to a bundled package but with a option flag to use bleeding edge.

        Attachments

          Container Issues

            Issue Links

              Activity

                People

                • Assignee:
                  jhoblitt Joshua Hoblitt
                  Reporter:
                  jhoblitt Joshua Hoblitt
                  Reviewers:
                  Mario Juric, Tim Jenness
                  Watchers:
                  Frossie Economou, Jonathan Sick, Joshua Hoblitt, Kian-Tat Lim, Mario Juric, Tim Jenness
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  6 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Summary Panel