UnicodeDecodeError: 'ascii' codec can't decode byte 0x8e in position 20: ordinal not in range(128) #9196

javadr · 2017-09-17T12:20:47Z

Bug report

When I want to use pyplot from matplotlib, it causes error.
There is no problem with import matplotlib but I cannot use pyplot

Code for reproduction

from matplotlib import pyplot

Actual outcome

---------------------------------------------
FileNotFoundErrorTraceback (most recent call last)
/usr/lib64/python3.6/site-packages/matplotlib/font_manager.py in <module>()
   1428         try:
-> 1429             fontManager = pickle_load(_fmcache)
   1430             if (not hasattr(fontManager, '_version') or

/usr/lib64/python3.6/site-packages/matplotlib/font_manager.py in pickle_load(filename)
    964     """
--> 965     with open(filename, 'rb') as fh:
    966         data = pickle.load(fh)

FileNotFoundError: [Errno 2] No such file or directory: '/home/javad/.cache/matplotlib/fontList.py3k.cache'

During handling of the above exception, another exception occurred:

UnicodeDecodeErrorTraceback (most recent call last)
<ipython-input-3-a371ea114746> in <module>()
----> 1 from matplotlib import pyplot

/usr/lib64/python3.6/site-packages/matplotlib/pyplot.py in <module>()
     27 from cycler import cycler
     28 import matplotlib
---> 29 import matplotlib.colorbar
     30 from matplotlib import style
     31 from matplotlib import _pylab_helpers, interactive

/usr/lib64/python3.6/site-packages/matplotlib/colorbar.py in <module>()
     34 import matplotlib.collections as collections
     35 import matplotlib.colors as colors
---> 36 import matplotlib.contour as contour
     37 import matplotlib.cm as cm
     38 import matplotlib.gridspec as gridspec

/usr/lib64/python3.6/site-packages/matplotlib/contour.py in <module>()
     20 import matplotlib.colors as colors
     21 import matplotlib.collections as mcoll
---> 22 import matplotlib.font_manager as font_manager
     23 import matplotlib.text as text
     24 import matplotlib.cbook as cbook

/usr/lib64/python3.6/site-packages/matplotlib/font_manager.py in <module>()
   1437             raise
   1438         except:
-> 1439             _rebuild()
   1440     else:
   1441         _rebuild()

/usr/lib64/python3.6/site-packages/matplotlib/font_manager.py in _rebuild()
   1416         global fontManager
   1417 
-> 1418         fontManager = FontManager()
   1419 
   1420         if _fmcache:

/usr/lib64/python3.6/site-packages/matplotlib/font_manager.py in __init__(self, size, weight)
   1070         self.afmfiles = findSystemFonts(paths, fontext='afm') + \
   1071             findSystemFonts(fontext='afm')
-> 1072         self.afmlist = createFontList(self.afmfiles, fontext='afm')
   1073         if len(self.afmfiles):
   1074             self.defaultFont['afm'] = self.afmfiles[0]

/usr/lib64/python3.6/site-packages/matplotlib/font_manager.py in createFontList(fontfiles, fontext)
    581             try:
    582                 try:
--> 583                     font = afm.AFM(fh)
    584                 finally:
    585                     fh.close()

/usr/lib64/python3.6/site-packages/matplotlib/afm.py in __init__(self, fh)
    347         """
    348         (dhead, dcmetrics_ascii, dcmetrics_name, dkernpairs, dcomposite) = \
--> 349             parse_afm(fh)
    350         self._header = dhead
    351         self._kern = dkernpairs

/usr/lib64/python3.6/site-packages/matplotlib/afm.py in parse_afm(fh)
    335     _sanity_check(fh)
    336     dhead = _parse_header(fh)
--> 337     dcmetrics_ascii, dcmetrics_name = _parse_char_metrics(fh)
    338     doptional = _parse_optional(fh)
    339     return dhead, dcmetrics_ascii, dcmetrics_name, doptional[0], doptional[1]

/usr/lib64/python3.6/site-packages/matplotlib/afm.py in _parse_char_metrics(fh)
    196         if not line:
    197             break
--> 198         line = line.rstrip().decode('ascii')  # Convert from byte-literal
    199         if line.startswith('EndCharMetrics'):
    200             return ascii_d, name_d

UnicodeDecodeError: 'ascii' codec can't decode byte 0x8e in position 20: ordinal not in range(128)

Expected outcome

using %matplotlib magic in jupyter notebook also produces the same error.

Matplotlib version

Operating System: Fedora 26, 64bit
Matplotlib Version: 2.0.0-2
Python Version: 2.7 / 3.6
Jupyter Version (if applicable): 4.1.0
Other Libraries:

The text was updated successfully, but these errors were encountered:

tacaswell · 2017-09-17T21:04:24Z

This is a duplicate of #3517 and which is fixed by #8021 which is in v2.0.1

tacaswell · 2017-09-17T21:14:34Z

@javadr Sorry you are having this issue and thank you for reporting it. This should be reported to Fedora to update their packaged version.

javadr · 2017-09-18T07:43:47Z

Appreciate your concern and your fast reply.

I updated the matplotlib to 2.1.0rc1 with pip, but the problem not solved yet. :(

WeatherGod · 2017-09-18T14:25:18Z

8000

Most likely, you have to clear out the old installation and reinstall the new version.

…

On Mon, Sep 18, 2017 at 3:43 AM, javadr ***@***.***> wrote: Appreciate your concern and your fast reply. I updated the matplotlib to 2.1.0rc1 with pip, but the problem not solved yet. :( — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#9196 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AARy-Mm82ILp2rXYOk5Vx-ClPd69WHNMks5sjh81gaJpZM4PaITb> .

tacaswell · 2017-09-18T17:55:30Z

@javadr can you drop into the debugger and determine which file it is failing on?

I was wrong, this is a different bug and the this decode is still on line 192 in afm.py

closes matplotlib#9196

javadr · 2017-09-19T04:57:20Z

thanks again, I hope that the following output help you.

Python 2.7.13 (default, Sep  5 2017, 08:53:59) 
[GCC 7.1.1 20170622 (Red Hat 7.1.1-3)] on linux2
Type "copyright", "credits" or "license()" for more information.
>>> import matplotlib.pyplot

Traceback (most recent call last):
  File "<pyshell#0>", line 1, in <module>
    import matplotlib.pyplot
  File "/usr/lib64/python2.7/site-packages/matplotlib/pyplot.py", line 29, in <module>
    import matplotlib.colorbar
  File "/usr/lib64/python2.7/site-packages/matplotlib/colorbar.py", line 36, in <module>
    import matplotlib.contour as contour
  File "/usr/lib64/python2.7/site-packages/matplotlib/contour.py", line 22, in <module>
    import matplotlib.font_manager as font_manager
  File "/usr/lib64/python2.7/site-packages/matplotlib/font_manager.py", line 1439, in <module>
    _rebuild()
  File "/usr/lib64/python2.7/site-packages/matplotlib/font_manager.py", line 1418, in _rebuild
    fontManager = FontManager()
  File "/usr/lib64/python2.7/site-packages/matplotlib/font_manager.py", line 1072, in __init__
    self.afmlist = createFontList(self.afmfiles, fontext='afm')
  File "/usr/lib64/python2.7/site-packages/matplotlib/font_manager.py", line 583, in createFontList
    font = afm.AFM(fh)
  File "/usr/lib64/python2.7/site-packages/matplotlib/afm.py", line 349, in __init__
    parse_afm(fh)
  File "/usr/lib64/python2.7/site-packages/matplotlib/afm.py", line 337, in parse_afm
    dcmetrics_ascii, dcmetrics_name = _parse_char_metrics(fh)
  File "/usr/lib64/python2.7/site-packages/matplotlib/afm.py", line 198, in _parse_char_metrics
    line = line.rstrip().decode('ascii')  # Convert from byte-literal
UnicodeDecodeError: 'ascii' codec can't decode byte 0x8e in position 20: ordinal not in range(128)

P.S.
There is no difference between python 2.7 and 3.6! both makes same results!

Python 3.6.2 (default, Aug 11 2017, 11:59:59) 
[GCC 7.1.1 20170622 (Red Hat 7.1.1-3)] on linux
Type "copyright", "credits" or "license()" for more information.
>>> import matplotlib.pyplot
Traceback (most recent call last):
  File "/usr/lib64/python3.6/site-packages/matplotlib/font_manager.py", line 1429, in <module>
    fontManager = pickle_load(_fmcache)
  File "/usr/lib64/python3.6/site-packages/matplotlib/font_manager.py", line 965, in pickle_load
    with open(filename, 'rb') as fh:
FileNotFoundError: [Errno 2] No such file or directory: '/home/javad/.cache/matplotlib/fontList.py3k.cache'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<pyshell#0>", line 1, in <module>
    import matplotlib.pyplot
  File "/usr/lib64/python3.6/site-packages/matplotlib/pyplot.py", line 29, in <module>
    import matplotlib.colorbar
  File "/usr/lib64/python3.6/site-packages/matplotlib/colorbar.py", line 36, in <module>
    import matplotlib.contour as contour
  File "/usr/lib64/python3.6/site-packages/matplotlib/contour.py", line 22, in <module>
    import matplotlib.font_manager as font_manager
  File "/usr/lib64/python3.6/site-packages/matplotlib/font_manager.py", line 1439, in <module>
    _rebuild()
  File "/usr/lib64/python3.6/site-packages/matplotlib/font_manager.py", line 1418, in _rebuild
    fontManager = FontManager()
  File "/usr/lib64/python3.6/site-packages/matplotlib/font_manager.py", line 1072, in __init__
    self.afmlist = createFontList(self.afmfiles, fontext='afm')
  File "/usr/lib64/python3.6/site-packages/matplotlib/font_manager.py", line 583, in createFontList
    font = afm.AFM(fh)
  File "/usr/lib64/python3.6/site-packages/matplotlib/afm.py", line 349, in __init__
    parse_afm(fh)
  File "/usr/lib64/python3.6/site-packages/matplotlib/afm.py", line 337, in parse_afm
    dcmetrics_ascii, dcmetrics_name = _parse_char_metrics(fh)
  File "/usr/lib64/python3.6/site-packages/matplotlib/afm.py", line 198, in _parse_char_metrics
    line = line.rstrip().decode('ascii')  # Convert from byte-literal
UnicodeDecodeError: 'ascii' codec can't decode byte 0x8e in position 20: ordinal not in range(128)

javadr · 2017-09-19T05:37:09Z

This is so weird, I've also installed Anaconda2 and I've got the same error!

Python 2.7.13 |Anaconda 4.4.0 (64-bit)| (default, Dec 20 2016, 23:09:15)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import matplotlib.pyplot
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/javad/anaconda2/lib/python2.7/site-packages/matplotlib/pyplot.py", line 29, in <module>
    import matplotlib.colorbar
  File "/home/javad/anaconda2/lib/python2.7/site-packages/matplotlib/colorbar.py", line 36, in <module>
    import matplotlib.contour as contour
  File "/home/javad/anaconda2/lib/python2.7/site-packages/matplotlib/contour.py", line 22, in <module>
    import matplotlib.font_manager as font_manager
  File "/home/javad/anaconda2/lib/python2.7/site-packages/matplotlib/font_manager.py", line 1439, in <module>
    _rebuild()
  File "/home/javad/anaconda2/lib/python2.7/site-packages/matplotlib/font_manager.py", line 1418, in _rebuild
    fontManager = FontManager()
  File "/home/javad/anaconda2/lib/python2.7/site-packages/matplotlib/font_manager.py", line 1072, in __init__
    self.afmlist = createFontList(self.afmfiles, fontext='afm')
  File "/home/javad/anaconda2/lib/python2.7/site-packages/matplotlib/font_manager.py", line 583, in createFontList
    font = afm.AFM(fh)
  File "/home/javad/anaconda2/lib/python2.7/site-packages/matplotlib/afm.py", line 349, in __init__
    parse_afm(fh)
  File "/home/javad/anaconda2/lib/python2.7/site-packages/matplotlib/afm.py", line 337, in parse_afm
    dcmetrics_ascii, dcmetrics_name = _parse_char_metrics(fh)
  File "/home/javad/anaconda2/lib/python2.7/site-packages/matplotlib/afm.py", line 198, in _parse_char_metrics
    line = line.rstrip().decode('ascii')  # Convert from byte-literal
UnicodeDecodeError: 'ascii' codec can't decode byte 0x8e in position 20: ordinal not in range(128)

I don't know is it something wrong with fedora 26 or not?!

javadr · 2017-09-19T06:19:06Z

I found the problem, It was because of my installed fonts! I removed some of them and now I can use matplotlib without any problem. As I noticed some of my fonts were duplicates! (Two distinct directories contained the same set of fonts, so I deleted one of them and it solved the problem.)

Sorry to cause you inconvenience.

tacaswell · 2017-09-19T13:01:10Z

@javadr which font (that is what I meant by 'which file' above, sorry for being unclear!)?

I am interested in looking at exactly what it has inside to understand why this is happening. We should be able to handle, or at least gracefully fail, on (possibly) malformed font!

javadr · 2017-09-24T08:46:44Z

Unfortunately, I removed many fonts, and now I don't know which one caused the problem , but if I find that problematic font, I'll send it.

tacaswell · 2017-09-24T17:43:31Z

Thanks @javadr .

Glad things are working for you again!

MDoYA · 2018-05-24T07:00:27Z

Hello All,
I have apparently the same problem with Pycharm IDE installed on Windows 10 (Anaconda, python 2.7). Matplotlib and utf-8 characters in my path (French) drive the python interpreter in issuing the exact same error as Javadr. Pycharm project and interpreter into it are set on utf-8.
Anyone could tell me if I should try to remove font one by one (how I do that, I did not add any I believe) or another solution has been found?

Thank you

PS : Anaconda is installing also Spyder IDE and the same code works perfectly on it.

operating system :
Matplotlib Version: 2.2.2, also tried 2.1.0, same error
Python Version: 2.7.14
IDE : Pycharm 2018.1.3

anntzer · 2018-05-24T07:52:54Z

Does it work if you replace the line

    if not line.startswith('  File "<frozen importlib._bootstrap'))

by

    if not line.startswith(b'  File "<frozen importlib._bootstrap'))

?
Or even better perhaps, remove the unicode_literals import at the very top of the file (basically, #10044).

Reopening although the issue is kind of different, but that's fine.

MDoYA · 2018-05-24T08:45:04Z

Hello anntzer,
I have tried your solution and then I just removed the unicode_literals import like this :

from __future__ import (absolute_import, division, print_function)#,
#                        unicode_literals)

It makes no difference. Just to be accurate, the following pic : is the Error I get if running the script with a ipython console launched from the path window :

Here is the mistake I get in the Python console of Pycharm :

I will try to go further in the comments of #10044

anntzer · 2018-05-24T09:07:05Z

You edited C:\Anaconda2\lib\site-packages\matplotlib\backends\__init__.py when removing the unicode_literals import, right?

What about the other proposed fix of using b' File ...'?

MDoYA · 2018-05-24T09:46:00Z

Yes I have firstly edited the script with your proposed fixture using b' File ...'
Then I have edited removing unicode_literals import.
I have restarted the Pycharm IDE inbetween to be sure to initialize properly the Matplotlib librairies.
Both ended with the same UnicodeError.

AndreySV · 2018-08-28T11:10:28Z

I've got the same problem. On Debian Buster with matplotlib 2.2.2.
The problem is that traceback.format_stack() returns list of 'str' objects. If some of the modules in the stack was located in directory with non-ascii character in path, then such error happens.

To address this problem I recommend to use traceback2, that always returns list of 'unicode'.
It's available in Debian and Ubuntu archives as well (python-traceback2, python3-traceback2) packages and of course in pypi.

Change is simple: replace 'import traceback' with 'import traceback2 as traceback' in matplotlib/backends/init.py

tacaswell · 2018-08-28T11:26:01Z

The code path in question no longer exists on master branch / 3.0 branch. This is a python2 only issue and mpl3.0+ will not support python2.

@AndreySV I am skeptical of picking up a new dependency on a micro release (ex 2.2.4). Does

_backend_loading_tb = b"".join(
    line for line in traceback.format_stack()
    # Filter out line noise from importlib line.
    if not line.startswith(b'  File "<frozen importlib._bootstrap'))

fix the problem? In either case, can you open a new issue for this issue which is an entirely different issue than the original.

AndreySV · 2018-08-28T13:38:57Z

Yes, convert to bytes both strings helps. I've opened for that separate issue #11955

tacaswell · 2018-08-28T14:12:16Z

I'm going to re-close this it is really two issues, the first of which is already fixed and the second of which now has it's own issue.

shanglidan · 2018-12-19T03:30:54Z

Before import matplotlib.pyplot, you can use the following code.
import sys
reload(sys)
sys.setdefaultencoding('utf8')

anntzer added the topic: text label Sep 17, 2017

tacaswell closed this as completed Sep 17, 2017

tacaswell added the status: duplicate label Sep 17, 2017

tacaswell reopened this Sep 18, 2017

tacaswell removed the status: duplicate label Sep 18, 2017

tacaswell added this to the 2.1 (next point release) milestone Sep 18, 2017

tacaswell added a commit to tacaswell/matplotlib that referenced this issue Sep 18, 2017

FIX: always decode byte strings from AFM files as utf8

7758b08

closes matplotlib#9196

tacaswell mentioned this issue Sep 18, 2017

FIX: always decode byte strings from AFM files as utf8 #9198

Merged

2 tasks

tacaswell added the Release critical For bugs that make the library unusable (segfaults, incorrect plots, etc) and major regressions. label Sep 18, 2017

tacaswell closed this as completed in e3e78be Sep 24, 2017

anntzer reopened this May 24, 2018

AndreySV mentioned this issue Aug 28, 2018

UnicodeDecodeError on importing pyplot in python2 #11955

Closed

tacaswell closed this as completed Aug 28, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

UnicodeDecodeError: 'ascii' codec can't decode byte 0x8e in position 20: ordinal not in range(128) #9196

UnicodeDecodeError: 'ascii' codec can't decode byte 0x8e in position 20: ordinal not in range(128) #9196

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

UnicodeDecodeError: 'ascii' codec can't decode byte 0x8e in position 20: ordinal not in range(128) #9196

UnicodeDecodeError: 'ascii' codec can't decode byte 0x8e in position 20: ordinal not in range(128) #9196

Comments

Uh oh!

Bug report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!