Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-3856

matplotlib sometimes fails to load during tests

    XMLWordPrintable

Details

    • Architecture

    Description

      During some multi-threaded builds meas_algorithms is sometimes failing when loading the matplotlib font cache. We think this is due to two tests simultaneous using matplotlib infrastructure and the second one sometimes attempts to load the font cache when the first is using it.

         import matplotlib.colorbar
        File "/home/vagrant/stack/Linux64/anaconda/2.2.0/lib/python2.7/site-packages/matplotlib/colorbar.py", line 34, in <module>
          import matplotlib.collections as collections
        File "/home/vagrant/stack/Linux64/anaconda/2.2.0/lib/python2.7/site-packages/matplotlib/collections.py", line 27, in <module>
          import matplotlib.backend_bases as backend_bases
        File "/home/vagrant/stack/Linux64/anaconda/2.2.0/lib/python2.7/site-packages/matplotlib/backend_bases.py", line 56, in <module>
          import matplotlib.textpath as textpath
        File "/home/vagrant/stack/Linux64/anaconda/2.2.0/lib/python2.7/site-packages/matplotlib/textpath.py", line 19, in <module>
          import matplotlib.font_manager as font_manager
        File "/home/vagrant/stack/Linux64/anaconda/2.2.0/lib/python2.7/site-packages/matplotlib/font_manager.py", line 1415, in <module>
          _rebuild()
        File "/home/vagrant/stack/Linux64/anaconda/2.2.0/lib/python2.7/site-packages/matplotlib/font_manager.py", line 1402, in _rebuild
          pickle_dump(fontManager, _fmcache)
        File "/home/vagrant/stack/Linux64/anaconda/2.2.0/lib/python2.7/site-packages/matplotlib/font_manager.py", line 952, in pickle_dump
          with open(filename, 'wb') as fh:
      IOError: [Errno 2] No such file or directory: u'/tmp/matplotlib-vagrant/fontList.cache'
      

      Currently matplotlib import failures are trapped as follows:

      try:
          import matplotlib.pyplot as plt
          import matplotlib.colors
      except ImportError:
          plt = None
      

      but this only protects the case where matplotlib is not installed. It does not protect against a failure of an existing matplotlib to load.

      The matplotlib project has come across this problem itself when using Travis (https://github.com/matplotlib/matplotlib/commit/f0888de10e8740200c7cb8387a3733d4fe6af967) and they disable the font cache when $TRAVIS is set.

      To fix this we can therefore either fix every import to also trap an IOError or else modify sconsUtils to set the TRAVIS environment variable when it runs the tests.

      Attachments

        Issue Links

          Activity

            1 db egg cache failures + 7 matlplotlib failures out of 50 builds of lsst_distrib, b1690 - 5 eL7, 2 el6

            jhoblitt Joshua Hoblitt added a comment - 1 db egg cache failures + 7 matlplotlib failures out of 50 builds of lsst_distrib, b1690 - 5 eL7, 2 el6
            tjenness Tim Jenness added a comment -

            We adopted the approach of setting $TRAVIS. Running more than 100 test jobs indicate that this failure mode no longer occurs with this change.

            tjenness Tim Jenness added a comment - We adopted the approach of setting $TRAVIS . Running more than 100 test jobs indicate that this failure mode no longer occurs with this change.
            tjenness Tim Jenness added a comment -
            tjenness Tim Jenness added a comment - I have filed a matplotlib issue: https://github.com/matplotlib/matplotlib/issues/5226
            tjenness Tim Jenness added a comment -

            Upstream has been fixed in https://github.com/matplotlib/matplotlib/pull/5276 but unfortunately this was only applied to the v2 tree and has not been backported to v1.5.x.

            tjenness Tim Jenness added a comment - Upstream has been fixed in https://github.com/matplotlib/matplotlib/pull/5276 but unfortunately this was only applied to the v2 tree and has not been backported to v1.5.x.

            People

              tjenness Tim Jenness
              tjenness Tim Jenness
              Joshua Hoblitt, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Jenkins

                  No builds found.