8000 fix cmr10 negative sign in cmsy10 (RuntimeWarning: Glyph 8722 missing) by casperdcl · Pull Request #18397 · matplotlib/matplotlib · GitHub
[go: up one dir, main page]

Skip to content

fix cmr10 negative sign in cmsy10 (RuntimeWarning: Glyph 8722 missing) #18397

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
May 14, 2021

Conversation

casperdcl
Copy link
Contributor
@casperdcl casperdcl commented Sep 2, 2020

PR Summary

Code

Before this PR, this code would not render negative signs on the axes tick labels.

import matplotlib
matplotlib.rcParams.update({
    'font.family': 'cmr10',
    'axes.formatter.use_mathtext: True
})
import matplotlib.pyplot as plt

plt.figure()
plt.plot(range(-1, 1), range(-1, 1))
plt.title(r"dash (-) $mathtext:negative (-)\bf{mathtext.bf:negative (-)}$")
plt.xlabel(r"dash (-) $mathtext:negative (-)\bf{mathtext.bf:negative (-)}$")

PR Checklist

  • Has pytest style unit tests (and pytest passes).
  • Is Flake 8 compliant (run flake8 on changed files to check).
  • [N/A] New features are documented, with examples if plot related.
  • [N/A] Documentation is sphinx and numpydoc compliant (the docs should build without error).
  • [N/A] Conforms to Matplotlib style conventions (install flake8-docstrings and pydocstyle<4 and run flake8 --docstring-convention=all).
  • [N/A] New features have an entry in doc/users/next_whats_new/ (follow instructions in README.rst there).
  • [N/A] API changes documented in doc/api/next_api_changes/ (follow instructions in README.rst there).
8000

@casperdcl casperdcl changed the title fix cmr10 negative sign in cmsy10 fix cmr10 negative sign in cmsy10 (RuntimeWarning: Glyph 8722 missing from current font) Sep 2, 2020
@casperdcl casperdcl changed the title fix cmr10 negative sign in cmsy10 (RuntimeWarning: Glyph 8722 missing from current font) fix cmr10 negative sign in cmsy10 (RuntimeWarning: Glyph 8722 missing) Sep 2, 2020
Copy link
Member
@dstansby dstansby left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for opening the PR! Could you add a test to lib/matplotlib/tests/test_mathtext.py that triggers this code path, to make sure no warnings are raised. Code similar to that in #17007 can probably be used.

8000
@casperdcl
Copy link
Contributor Author
casperdcl commented Sep 21, 2020

@dstansby would you recommend appending to the test_math_fontfamily function?

@image_comparison(baseline_images=['math_fontfamily_image.png'],
savefig_kwarg={'dpi': 40})
def test_math_fontfamily():
fig = plt.figure(figsize=(10, 3))
fig.text(0.2, 0.7, r"$This\ text\ should\ have\ one\ font$",
size=24, math_fontfamily='dejavusans')
fig.text(0.2, 0.3, r"$This\ text\ should\ have\ another$",
size=24, math_fontfamily='stix')

fork

Or would you prefer delving into the pytest fixtures?

@QuLogic
Copy link
Member
QuLogic commented Sep 22, 2020

That would involve changing a test image, which seems unnecessary, so a different (and/or new) test would be better. Also, I don't know why you would have to delve into pytest fixtures?

@jklymak
Copy link
Member
jklymak commented Nov 10, 2020

test failure is real and appears related.... lib/matplotlib/tests/test_mathtext.py::test_mathtext_fontfamily

@casperdcl
Copy link
Contributor Author

@jklymak should be fixed now

@QuLogic QuLogic added this to the v3.5.0 milestone Jan 27, 2021
@mapfiable
Copy link

I'm not sure if I'm doing something wrong, but here is an example where the fix suggested by #17007 (comment) doesn't work:

import numpy as np
import matplotlib.pyplot as plt
from matplotlib.tight_layout import get_renderer

plt.rcParams["axes.formatter.use_mathtext"] = True
plt.rcParams["font.family"] = "serif"
plt.rcParams["font.serif"] = "cmr10"
plt.rcParams["mathtext.fontset"] = "cm"


fig, ax = plt.subplots()

image_artist = ax.imshow(np.random.uniform(-0.001, 0.001, (1000, 1000)))
colorbar = fig.colorbar(image_artist)

colorbar.ax.ticklabel_format(axis='y', scilimits=(-1, 0))
renderer = get_renderer(fig)
colorbar.ax.get_tightbbox(renderer)
offset_text = colorbar.ax.yaxis.get_offset_text()
exp = offset_text.get_text().split('e')[1].replace('+', '')

colorbar.ax.set_ylabel(
    f'Brightness [$10^{{{exp}}}$ W/m$^2$/SR/nm]'
)
colorbar.ax.yaxis.offsetText.set_visible(False)
plt.show()

I edited the mentioned lines in mathtext.py, but setting plt.rcParams["axes.formatter.use_mathtext"] = True in my case returns the following error:

pyparsing.ParseFatalException: Unknown symbol: \mathd, found '\'  (at char 5), (line:1, col:6)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\mathtext.py", line 2613, in parse
    result = self._expression.parseString(s)
  File "C:\Users\mapfu\Anaconda3\lib\site-packages\pyparsing.py", line 1955, in parseString
    raise exc
  File "C:\Users\mapfu\Anaconda3\lib\site-packages\pyparsing.py", line 4065, in parseImpl
    raise ParseSyntaxException._from_exception(pe)
pyparsing.ParseSyntaxException: Unknown symbol: \mathd, found '\'  (at char 5), (line:1, col:6)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:/Users/mapfu/ownCloud/MPS/Python/CPT/core/temp2.py", line 26, in <module>
    plt.show()
  File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\pyplot.py", line 353, in show
    return _backend_mod.show(*args, **kwargs)
  File "C:\Program Files\JetBrains\PyCharm 2020.1.1\plugins\python\helpers\pycharm_matplotlib_backend\backend_interagg.py", line 29, in __call__
    manager.show(**kwargs)
  File "C:\Program Files\JetBrains\PyCharm 2020.1.1\plugins\python\helpers\pycharm_matplotlib_backend\backend_interagg.py", line 112, in show
    self.canvas.show()
  File "C:\Program Files\JetBrains\PyCharm 2020.1.1\plugins\python\helpers\pycharm_matplotlib_backend\backend_interagg.py", line 68, in show
    FigureCanvasAgg.draw(self)
  File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\backends\backend_agg.py", line 407, in draw
    self.figure.draw(self.renderer)
  File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\artist.py", line 41, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\figure.py", line 1864, in draw
    renderer, self, artists, self.suppressComposite)
  File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\image.py", line 131, in _draw_list_compositing_images
    a.draw(renderer)
  File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\artist.py", line 41, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\cbook\deprecation.py", line 411, in wrapper
    return func(*inner_args, **inner_kwargs)
  File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\axes\_base.py", line 2747, in draw
    mimage._draw_list_compositing_images(renderer, self, artists)
  File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\image.py", line 131, in _draw_list_compositing_images
    a.draw(renderer)
  File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\artist.py", line 41, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\axis.py", line 1178, in draw
    self.label.draw(renderer)
  File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\artist.py", line 41, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\text.py", line 681, in draw
    bbox, info, descent = textobj._get_layout(renderer)
  File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\text.py", line 296, in _get_layout
    clean_line, self._fontproperties, ismath=ismath)
  File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\backends\backend_agg.py", line 233, in get_text_width_height_descent
    self.mathtext_parser.parse(s, self.dpi, prop)
  File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\mathtext.py", line 3337, in parse
    s, dpi, prop, rcParams['ps.useafm'], rcParams['mathtext.fontset'])
  File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\mathtext.py", line 3360, in _parse_cached
    box = self._parser.parse(s, font_output, fontsize, dpi)
  File "C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\mathtext.py", line 2618, in parse
    str(err)])) from err
ValueError: 
10^{s\mathd}
     ^
Unknown symbol: \mathd, found '\'  (at char 5), (line:1, col:6)

Process finished with exit code 1

Not setting plt.rcParams["axes.formatter.use_mathtext"] = True simply renders the minus signs wrong and returns the familiar RuntimeWarning:

C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\backends\backend_agg.py:238: RuntimeWarning: Glyph 8722 missing from current font.
  font.set_text(s, 0.0, flags=flags)
C:\Users\mapfu\Anaconda3\lib\site-packages\matplotlib\backends\backend_agg.py:201: RuntimeWarning: Glyph 8722 missing from current font.
  font.set_text(s, 0, flags=flags)

image

@anntzer
Copy link
Contributor
anntzer commented Mar 18, 2021

AFIACT this is unrelated. Check the value of exp...

@mapfiable
Copy link
mapfiable commented Mar 18, 2021

What do you mean? For me it is -4 (str type). I don't have any issues if I use the standard font (i.e. disable all the rcParams changes):
image

EDIT

Oohh, now I get what you mean. With plt.rcParams["axes.formatter.use_mathtext"] = True:
offset_text.get_text() # $\times\mathdefault{10^{−4}}\mathdefault{}$
Without:
offset_text.get_text() # 1e−4
Sooo.. is there a clever work around? I mean I could of course hard code a solution, but I feel like the fix discussed here should work for every case that also works normally with any other font?

@jklymak
Copy link
Member
jklymak commented Apr 23, 2021

Is this still an issue? If so this PR needs to pass the tests (preferably rebased on our modern test).

@casperdcl
Copy link
Contributor Author
casperdcl commented Apr 23, 2021

It's still an issue but I've just been manually looping through axes & labels running .replace() whenever I use cmr10 to substitute out problematic glyphs.

I already had to change the tests once, not sure if I'll have time to rebase, get familiar with the "modern test"s and update again.

@jklymak
Copy link
Member
jklymak commented Apr 23, 2021

Yes sorry for the process. Things fall off the review queue so unless you ping, folks forget!

casperdcl and others added 3 commits April 23, 2021 17:58
Co-authored-by: Tim Hoffmann <2836374+timhoffm@users.noreply.github.com>
@casperdcl
Copy link
Contributor Author
casperdcl commented Apr 23, 2021

Well just rebased and naturally it fails. The fix no longer works. The tests are (correctly) failing.

Grrr.

@anntzer any ideas?

@anntzer
Copy link
Contributor
anntzer commented Apr 23, 2021

I don't know right now, but will throw the ball to gsoc-candidate @aitikgupta :-)

Copy link
Contributor
@aitikgupta aitikgupta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know why tests passed (if they did) previously, adding a suggestion

fix test config
Co-authored by: Aitik Gupta <aitikgupta@users.noreply.github.com>
@casperdcl casperdcl marked this pull request as ready for review April 24, 2021 14:40
@mapfiable
Copy link
mapfiable commented Apr 24, 2021

Hi sorry, because I'm still subscribed to this post I was reminded of it again due to the recent comments. So I thought I just point out one more isse that I ran into when using this fix, and which I forgot to report earlier. For some reason, the degree symbol ° is not displayed correctly, so in a polar plot e.g. you also need some manual work around:

import matplotlib.pyplot as plt
from matplotlib.ticker import FormatStrFormatter

# plt.rcParams["axes.formatter.use_mathtext"] = True
plt.rcParams["font.family"] = "serif"
plt.rcParams["font.serif"] = "cmr10"
plt.rcParams["mathtext.fontset"] = "cm"

fig = plt.figure()
ax = fig.add_subplot(111, projection='polar')
ax.set_title('Degree °')
# ax.get_xaxis().set_major_formatter(FormatStrFormatter('%d$\,\degree$'))

fig.show()
labels = [label.get_text() for label in ax.get_xticklabels()]
print(labels)

grafik

Unlike my previous issue this is not related to the use of mathtext. For some reason, it seems to only affect the tick labels (scrap that, I made a stupid mistake), but when printed in the console they look correct. I didn't manage to find a post regarding this specific problem, so i thought I make you aware of it.

@aitikgupta
Copy link
Contributor
aitikgupta commented Apr 25, 2021

For some reason, the degree symbol ° is not displayed correctly..

@mapfiable your bug more or less points to the fact that the glyph for ° just doesn't exist in "cmr10", and it is unrelated to mathtext since even the title (which is not wrapped with \mathtextdefault) can't render it.
This PR only implements falling to a different font file for a certain glyph using mathtext, what you need is a general-fallback. Related: #18883
(@anntzer can you confirm my assertion? - also, should this belong to a new issue?)

@mapfiable
Copy link

Hi @aitikgupta, thanks, but I think you're mistaken. This thread is dealing with the issue that the minus sign symbol (Glyph 8722) is missing from the cmr10 font and results in the error message pointed out in the title. The use of mathtext is then part of the proposed fix.
While my previous issue is a direct consequence of the fix using mathtext (and independed of the font used), it was argued that it is irrelevant to the problem (though I don't fully agree; while I guess it's fair to expect that you should be aware of the implications of using mathtext, I feel like a fix shouldn't produce unexpeced behaviour elsewhere, e.g. in my program I need to switch fonts occasionally and the fix forces me to implement font-dependent plotting routines, though ultimately I'm still very grateful for it).
The issue with the degree symbol however is not related to the use of mathtext, but to the use of the cmr10 font. Just like the minus sign, the degree symbol seems to be missing from the font, although curiously, there is no error message that is pointing this out. So I thought it might make sense to make you aware of the missing degree symbol here, because this thread is dealing with a very similar issue. Maybe this can be solved with the same fix by making some minor additions to it, but I really don't know.

@anntzer
Copy link
Contributor
anntzer commented Apr 26, 2021

From a very quick look (no guarantees...), I agree with @mapfiable here; I believe fixing the degree-sign problem basically requires a similar fix as for the minus sign, i.e. something like if font.family_name == "cmr10" and uniindex == ord("°"): font = <cmsy10.ttf>; uniindex = <whatever>.

As a reminder, the problem is the following:

  • The cmr10 font is basically not intended for standalone use, because many required glyphs are not present in that file but in others (in particular cmsy10).
  • Matplotlib non-mathtext rendering is absolutely incapable of rendering a string with glyph from multiple font files (Matplotlib would not try to apply all the font in font list to draw all characters in the given string. #18883, etc.); fixing this may be a component of @aitikgupta's GSOC work, but in any case it is not an easy problem to fix.
  • Therefore, cmr10 mostly only makes sense when used in mathtext (possibly, we could even emit a warning when cmr10 is used in a non-mathtext context, as that seems to be pretty common) -- possibly wrapped with \mathdefault{}, to look like normal text.
  • However, even there, \mathdefault just means "use whatever glyphs are present in cmr10.ttf". So in order for glyphs present in other fonts to be used, we must add manual fallbacks to them (which is what this PR does for unicode minus, and could perhaps also do for degree).
  • Even if we add a generic fallback system, it is unlikely to actually work for cmr10/cmsy10, because the font encoding of cm is basically ad hoc and different from all other fonts, so it'd need a separate fallback table at least, so we may as well start with some hard-coding here.

@aitikgupta
Copy link
Contributor
aitikgupta commented Apr 26, 2021

I think I misspoke in my statement, I agree with the fundamental issue here - with @mapfiable's example if we do a rcParams["axes.formatter.use_mathtext"] = True, the proposed codepath (that involves if font.family_name == "cmr10" and uniindex == ord("°"): font = <cmsy10.ttf>; uniindex = <whatever>) wouldn't be triggered - which is why I said it's unrelated to mathtext (which isn't entirely true, it's just true for this case because of below reason).

Axis ticks are wrapped around \mathdefault if we trigger the use_mathtext, however I don't think matplotlib.projections.polar.ThetaTick objects are.


To get around this, an interesting thing you did was to "force" the formatting to use mathtext, i.e., when you did:

ax.get_xaxis().set_major_formatter(FormatStrFormatter('%d$\,\degree$'))

However, the font-family in the if condition changed to STIXGeneral, so if you printed:

             if font is not None:
+                print("Not found!", font.family_name, uniindex)
                 glyphindex = font.get_char_index(uniindex)

For °, It'll give you Not found! STIXGeneral 176. (somehow user's font family is overridden)
Ultimately, the reason why it "worked" for you is because matlpotlib used the STIX font and not "cmr10" for your ticks.

Overriding a user-defined font shouldn't be the default behaviour right? @anntzer

@casperdcl
Copy link
Contributor Author
casperdcl commented Apr 26, 2021

Could matplotlib perhaps roll its own standalone cmr10 (including missing symbols from cmsy10)?

@anntzer
Copy link
Contributor
anntzer commented Apr 26, 2021

Overriding a user-defined font shouldn't be the default behaviour right?

Possibly some bad interaction with rcParams["mathtext.fallback"]? (Not sure)

Could matplotlib perhaps roll its own standalone cmr10 (including missing symbols from cmsy10)?

I think this would more or less mean latin modern math (http://www.gust.org.pl/projects/e-foundry/lm-math), which yes, I agree, should be considered as a replacement (I have argued for it elsewhere).

@mapfiable
Copy link
mapfiable commented Apr 26, 2021

To get around this, an interesting thing you did was to "force" the formatting to use mathtext, i.e., when you did: ...

That's super interesting. I didn't know this was happening at all. In fact, now that you've dissected it like that, I'm not even sure why I was thinking that it should work in the first place.. I guess I thought that ° and \degree are different symbols (what about ^\circ though?).. oh well I guess I just got lucky there. Though it irks me a bit now that I know that actually a different font is used.

EDIT ok just an aside, but something super strage just happened. I could have sworn that

ax.get_xaxis().set_major_formatter(FormatStrFormatter('%d$\,\degree$'))

did the trick, but I just ran it again and now somehow the values are not in degrees anymore but in multiples of pi (i.e. radians)?! I'm pretty sure this was not the case before because when I came up with the solution I had to run it multiple times to make it work and something this obvious should have caught my eye.. but maybe I'm just going crazy (though I really don't understand why the unit / values should change dependent on whether or not the StrFormatter is applied). Anyways, just for the record here is the non-radians solution:

fmt = lambda x, pos: f'{np.degrees(x):.0f}$\degree$'
ax.get_xaxis().set_major_formatter(FuncFormatter(fmt))

@jklymak jklymak requested a review from dstansby May 8, 2021 22:10
@jklymak
Copy link
Member
jklymak commented May 8, 2021

@aitikgupta If you had a chance to review this, that would be helpful. Thanks!

Copy link
Contributor
@aitikgupta aitikgupta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically users with font.family == 'cmr10' also need axes.formatter.use_mathtext == True.
(This PR is still required to ensure this works)

I'm not sure if forcing axes.formatter to use mathtext when the font is 'cmr10' is something which should be implemented within the library?

Maybe we'd be better off raising a warning when the font is 'cmr10' and use_mathtext is False (GitHub doesn't allow reviewing on unchanged code):

diff --git a/lib/matplotlib/ticker.py b/lib/matplotlib/ticker.py
--- a/lib/matplotlib/ticker.py
+++ b/lib/matplotlib/ticker.py
@@ -475,6 +475,15 @@ class ScalarFormatter(Formatter):
         self._usetex = mpl.rcParams['text.usetex']
         if useMathText is None:
             useMathText = mpl.rcParams['axes.formatter.use_mathtext']
+            if useMathText is False:
+                for family in mpl.rcParams['font.family']:
+                    if "cmr10" in mpl.rcParams[f'font.{family}']:
+                        _api.warn_external(
+                            'cmr10 font should ideally be used with '
+                            'mathtext, set axes.formatter.use_mathtext to True'
+                        )
+                        # or if you *really* want to implement it:
+                        # useMathText = True
         self.set_useMathText(useMathText)
         self.orderOfMagnitude = 0
         self.format = ''

(could potentially be moved into an outer scope, which includes all formatters and not just ScalarFormatter)

Since this involves something with _api, I believe this needs to go through a senior developer.

Other than this, I think we're good to go 🚀

Copy link
Member
@jklymak jklymak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll approve based on @aitikgupta review. This needs a second review...

@tacaswell tacaswell merged commit cb21d7d into matplotlib:master May 14, 2021
@aitikgupta
Copy link
Contributor
aitikgupta commented May 14, 2021

Note that the actual solution would still require the warning I mentioned in the above comment, to respect one of @anntzer's comments:

(probably a real patch would need to add a comment there)

Without the warning user wouldn't know that they need to pass rcParams["axes.formatter.use_mathtext"] = True when they use 'cmr10' font.

@tacaswell
Copy link
Member

@aitikgupta Can you open a follow on PR with the warning?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Computer Modern Glyph Error
9 participants
0