Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-8129

Demo breaks with Conda v12.1 binaries because of missing libexpat.so.0 file.

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      When running the demo with the Conda v12.1 binaries this error happens:

      vagrant@ubuntu:/opt/var/lsst_dm_stack_demo$ ./bin/demo.sh
      Setting up: astrometry_net_data             Flavor: Linux64    Version: LOCAL:/opt/var/lsst_dm_stack_demo/astrometry_net_data
      Traceback (most recent call last):
        File "/opt/var/miniconda2/opt/lsst/pipe_tasks/bin/processCcd.py", line 23, in <module>
          from lsst.pipe.tasks.processCcd import ProcessCcdTask
        File "/opt/var/miniconda2/opt/lsst/pipe_tasks/python/lsst/pipe/tasks/processCcd.py", line 22, in <module>
          from lsst.ip.isr import IsrTask
        File "/opt/var/miniconda2/opt/lsst/ip_isr/python/lsst/ip/isr/__init__.py", line 24, in <module>
          from .isrLib import *
        File "/opt/var/miniconda2/opt/lsst/ip_isr/python/lsst/ip/isr/isrLib.py", line 27, in <module>
          _isrLib = swig_import_helper()
        File "/opt/var/miniconda2/opt/lsst/ip_isr/python/lsst/ip/isr/isrLib.py", line 26, in swig_import_helper
          return importlib.import_module('_isrLib')
        File "/opt/var/miniconda2/lib/python2.7/importlib/__init__.py", line 37, in import_module
          __import__(name)
      ImportError: No module named _isrLib
      

      This is caused because ip_isr eventually depends on log4cxx which can't find libexpat.so.0. See this python repl log for the error.

      vagrant@ubuntu:/opt/var/lsst_dm_stack_demo$ which python
      /opt/var/miniconda2/bin/python
      vagrant@ubuntu:/opt/var/lsst_dm_stack_demo$ python
      Python 2.7.12 |Continuum Analytics, Inc.| (default, Jul  2 2016, 17:42:40)
      [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
      Type "help", "copyright", "credits" or "license" for more information.
      Anaconda is brought to you by Continuum Analytics.
      Please check out: http://continuum.io/thanks and https://anaconda.org
      >>> import sys
      >>> sys.path.append("/opt/var/miniconda2/opt/lsst/ip_isr/python/lsst/ip/isr")
      >>> import lsst.ip.isr
      Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
        File "/opt/var/miniconda2/opt/lsst/ip_isr/python/lsst/ip/isr/__init__.py", line 24, in <module>
          from .isrLib import *
        File "/opt/var/miniconda2/opt/lsst/ip_isr/python/lsst/ip/isr/isrLib.py", line 27, in <module>
          _isrLib = swig_import_helper()
        File "/opt/var/miniconda2/opt/lsst/ip_isr/python/lsst/ip/isr/isrLib.py", line 26, in swig_import_helper
          return importlib.import_module('_isrLib')
        File "/opt/var/miniconda2/lib/python2.7/importlib/__init__.py", line 37, in import_module
          __import__(name)
      ImportError: libexpat.so.0: cannot open shared object file: No such file or directory
      

      This error happens because there was a major release of expat between CentOS5 (the conda build environment) and Ubuntu16.04 (the platform where the error happened).

      The fix is to add

      {pre-link.sh}

      and

      {pre-unlink.sh}

      to our own lsst-log4cxx Conda package that creates a softlink and removes the softlink to the missing file.

        Attachments

          Issue Links

            Activity

            Hide
            jmatt J Matt Peterson [X] (Inactive) added a comment -

            Spent about three hours last night plus today working on this. It took some additional work to apply it to the existing build.

            What did not work and should have is just building the new conda package and pushing it and updated meta files.

            Current steps to manually remove a bad Conda package and replace it with another one:

            1. Ensure conda build 1.20.0 is installed. conda install conda-build==1.2.0
            2. To use an existing recipe: cp -r ./recipes/<conda package name> ./etc/recipes/. Make sure to i
            3. Update recipe in ./etc/recipes/<conda package name>. Make sure to increment the build number and build number string.
            4. conda lsst make-recipe build:bNNNN <eups package name> --build the new version.
            5. Manually remove the older builds from: ./miniconda/conda-bld/linux-64/
            6. Manually update the ./miniconda/conda-bld/linux-64/.index.json
            7. Ensure conda build current is installed. conda install conda-build
            8. Make conda-build recreate index. conda index ./miniconda/conda-bld/linux-64
            9. Upload the new <conda package name>-<build number>.tar.bz2 to aws.
            10. Upload .index.json, repodata.json and repodata.json.bz2 to aws.

            When testing you can run conda clean --all to quickly reset an environment.

            Show
            jmatt J Matt Peterson [X] (Inactive) added a comment - Spent about three hours last night plus today working on this. It took some additional work to apply it to the existing build. What did not work and should have is just building the new conda package and pushing it and updated meta files. Current steps to manually remove a bad Conda package and replace it with another one: 1. Ensure conda build 1.20.0 is installed. conda install conda-build==1.2.0 2. To use an existing recipe: cp -r ./recipes/<conda package name> ./etc/recipes/ . Make sure to i 3. Update recipe in ./etc/recipes/<conda package name>. Make sure to increment the build number and build number string. 4. conda lsst make-recipe build:bNNNN <eups package name> --build the new version. 5. Manually remove the older builds from: ./miniconda/conda-bld/linux-64/ 6. Manually update the ./miniconda/conda-bld/linux-64/.index.json 7. Ensure conda build current is installed. conda install conda-build 8. Make conda-build recreate index. conda index ./miniconda/conda-bld/linux-64 9. Upload the new <conda package name>-<build number>.tar.bz2 to aws. 10. Upload .index.json, repodata.json and repodata.json.bz2 to aws. When testing you can run conda clean --all to quickly reset an environment.
            Hide
            tjenness Tim Jenness added a comment -

            J Matt Peterson [X] can you give this to someone else to review please? I had a quick look and I don't understand why the expat soft linking is required if you are linking against the conda expat library. More confusingly for my review, I don't understand how much of the content of these files is standard boiler plate that is shared by all the packages and how much of this is code that is specifically modified for log4cxx.

            Show
            tjenness Tim Jenness added a comment - J Matt Peterson [X] can you give this to someone else to review please? I had a quick look and I don't understand why the expat soft linking is required if you are linking against the conda expat library. More confusingly for my review, I don't understand how much of the content of these files is standard boiler plate that is shared by all the packages and how much of this is code that is specifically modified for log4cxx.
            Hide
            jmatt J Matt Peterson [X] (Inactive) added a comment -

            The Conda v12.1 (semantic version 0.12.1) binaries have been successfully created. The repository for this release is: http://conda.lsst.codes/stack/0.12.1

            Show
            jmatt J Matt Peterson [X] (Inactive) added a comment - The Conda v12.1 (semantic version 0.12.1) binaries have been successfully created. The repository for this release is: http://conda.lsst.codes/stack/0.12.1

              People

              • Assignee:
                jmatt J Matt Peterson [X] (Inactive)
                Reporter:
                jmatt J Matt Peterson [X] (Inactive)
                Reviewers:
                Frossie Economou
                Watchers:
                Frossie Economou, J Matt Peterson [X] (Inactive), Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel