Continuum release/rebuilt a number of packages last friday to depend on the the Intel MKL library.

https://www.continuum.io/blog/developer-blog/anaconda-25-release-now-mkl-optimizations

There are [new feature named] versions that continue to use openblas but the MKL versions appear to be installed by default. This causes at least multiple meas_base tests to fail.After extensive testing, I have confirmed that the meas_base tests do not fail with the equivalent 'nomkl' package.
In addition, mkl is closed source software that requires you to accept and download a license file or it is time-bombed to stop working after a trial period.

    docker-centos-7: [ 36/36 ]  meas_base 2015_10.0-9-g6daf04b+7 ...

    docker-centos-7:

    docker-centos-7: ***** error: from /opt/lsst/software/stack/EupsBuildDir/Linux64/meas_base-2015_10.0-9-g6daf04b+7/build.log:

    docker-centos-7: tests/sincPhotSums.py

    docker-centos-7:

    docker-centos-7: tests/measureSources.py

    docker-centos-7:

    docker-centos-7: tests/testApertureFlux.py

    docker-centos-7:

    docker-centos-7: tests/testJacobian.py

    docker-centos-7:

    docker-centos-7: tests/testScaledApertureFlux.py

    docker-centos-7:

    docker-centos-7: The following tests failed:

    docker-centos-7: /opt/lsst/software/stack/EupsBuildDir/Linux64/meas_base-2015_10.0-9-g6daf04b+7/meas_base-2015_10.0-9-g6daf04b+7/tests/.tests/sincPhotSums.py.failed

    docker-centos-7: /opt/lsst/software/stack/EupsBuildDir/Linux64/meas_base-2015_10.0-9-g6daf04b+7/meas_base-2015_10.0-9-g6daf04b+7/tests/.tests/measureSources.py.failed

    docker-centos-7: /opt/lsst/software/stack/EupsBuildDir/Linux64/meas_base-2015_10.0-9-g6daf04b+7/meas_base-2015_10.0-9-g6daf04b+7/tests/.tests/testApertureFlux.py.failed

    docker-centos-7: /opt/lsst/software/stack/EupsBuildDir/Linux64/meas_base-2015_10.0-9-g6daf04b+7/meas_base-2015_10.0-9-g6daf04b+7/tests/.tests/testJacobian.py.failed

    docker-centos-7: /opt/lsst/software/stack/EupsBuildDir/Linux64/meas_base-2015_10.0-9-g6daf04b+7/meas_base-2015_10.0-9-g6daf04b+7/tests/.tests/testScaledApertureFlux.py.failed

    docker-centos-7: 5 tests failed

(the exact cause of the test failures was not investigated as this should not have happened)

This change has also broken the ability to import an existing conda env from 2016-02-05 or earlier that uses scipy due to some sort of package version resolution problem. Explicit declaring it as the scipy package without mkl fixes the resolution problem.

There is a new 'nomkl' package, when installed, any subsequent package installations will default to versions without mkl. However, this does not fix any already installed packages.

I am traumatized by the lack of reproducible build envs even within a few days of each other. After discussion in the Tucson office, I'm going to pin the lsstsw and newinstall.sh conda package versions with a commitment from square to update them on a monthly basis. I already have a test version of lsstsw/bin/deploy that defaults to a bundled package but with a option flag to use bleeding edge.