So here's what seems to be happening:

mariadbclient is configured using CMake. Specifically, it uses CMake's FindZLIB.cmake to find zlib.

FindZLIB.cmake uses CMake's find_library function to find libz.dylib.

find_library searches, among other places, "<prefix>/lib for each <prefix>/bin in PATH".

If Miniconda is installed and set up, Miniconda's bin directory is on the PATH, so CMake will search in Miniconda's lib directory.

Miniconda's lib directory contains libz.1.dylib version 1.2.8. CMake will find this and link libmariadbclient.dylib against it.

daf_persistence tries to link against libmariadbclient.dylib. Even if Miniconda is set up at the time, it does not add it's lib directory to the linker path. Instead, we figure out the path to link against libpython by asking the Python interpreter what we should use, and it points us to miniconda/lib/python2.7/config. This does not contain libz.dylib.

Therefore, we fall back to trying to link against the system /usr/lib/libz.dylib. This is an older version (1.2.5 vs the Miniconda supplied 1.2.8), and linking fails. Exactly why it reports such an unhelpful message when it fails isn't clear to me, but it does. You can even ask ld to show you the libraries it's loading using the -t flag: watch it load /usr/lib/libz.dylib then follow that up by telling you that libz.dylib is not found.

The workaround would seem to be to stop mariadbclient linking against Miniconda's zlib. However, that's easier said than done. Convincing CMake to not consider "each <prefix>/bin in PATH" seems to involve editing the FindZLIB.cmake that's distributed as part of CMake itself and adding NO_SYSTEM_ENVIRONMENT_PATH arguments to various function calls. That seems impractical.

An alternative is to pass CMAKE_SYSTEM_IGNORE_PATH=<PATH_TO_MINICONDA>/lib to the cmake executable on the command line. A simple-minded approach to this is here. One might worry about what happens when the python executable isn't provided by Miniconda; I guess some extra checks could insure against that.

Another option is to simply force CMake to find the system zlib on OSX by adding -DZLIB_ROOT=/usr to the CMake invocation. That's here. In this case, you might worry that the user could have some other zlib that they really want to be used instead – seems unlikely, though.

For what it's worth, neither of the above options seem elegant to me, but they both resolve the daf_persistence build issue on my system. Anybody got a better idea?

I assume a fix for this would also fix DM-5170 on Linux. There it's finding a libssl in Miniconda and getting confused when it can't find it in /usr/lib.

That seems to argue for the CMAKE_SYSTEM_IGNORE_PATH solution – should stop it picking up any of the libraries shipped with Miniconda.

If the choice is to tell mariadbclient to ignore miniconda, can we also tell it to ignore anaconda in the path? (many people use their own anaconda instead of just using the miniconda).

Presumably also affects mariadb.

Ok, proposed fix on u/swinbank/DM-5595 in mariadbclient.

Hsin-Fang Chiang, could you please verify this fixes your problems with daf_persistence/ZLIB?

Lynne Jones, I believe this will fix your problem with OpenSSL from DM-5587. Could you please check?

John Parejko, if I've correctly understood what Tim Jenness was telling me, I think this (correctly) will make your DM-5170 build fail early in mariadbclient, rather than (incorrectly) proceeding to daf_persistence then failing. Again, could you confirm please?

Yes, I confirm that daf_persistence builds fine on my machine with the u/swinbank/DM-5595 branch of mariadbclient. (and fails with master)

Unfortunately, it wasn't on my system but rather a problem I was reporting for a MAF/sims user, and they've moved on to using the conda binaries instead.
I could ask if they would go back and test, but if that doesn't happen, please don't wait on me.
I take that back. Installing from a branch would be a bit much to ask from outside, non-expert users I think.

Lynne Jones Fair enough – do you know any more details about the platform they were building on? I can try to reproduce.

Hsin-Fang Chiang Thank you!

I know it was "redhat" , but I can ask which version.
Oh - or I could send you the build.log file?

John Swinbank yes the same fix will be needed for mariadb.

Lynne Jones Can you get me the build.log for both daf_persistence and mariadbclient? Thank you!

Well, I can get the build.log file for daf_persistence .. but I can't get the build log for mariadb, since it built successfully.

build.log file from daf_persistence failure (when mariadbclient could not
ifnd the openssl libraries during runtime).

On Thu, Mar 24, 2016 at 2:35 PM John Swinbank (JIRA) <jira-dm@lsst.org>

mariadbclient build logs are probably in $MARIADBCLIENT_DIR/ups/build.log following a successful build. But I'm fairly convinced that this has addressed the problem, and we've already shut down DM-5587, so I don't think it's worth burning lots of time trying to dig up logs if it's not straightforward.

Tim Jenness, what do you think? This is what needs reviewing: single identical commits on tickets/DM-5595 in both mariadb and mariadbclient.

Yes, I think the fixes would be the same. It's the same code.

(To be clear, I was asking for a review, not just if you agreed with my statement above. Sorry for any confusion.)

Looks like a good first go. We can get cleverer if needed later on. This works for me. I assume these worked on Jenkins.

Thanks; confirmed it runs through Jenkins and merged.