8000 hatch_demo.py does not correctly save all the hatches in pdf · Issue #12367 · matplotlib/matplotlib · GitHub
[go: up one dir, main page]
Skip to content

hatch_demo.py does not correctly save all the hatches in pdf #12367

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
ftesser opened this issue Oct 2, 2018 · 16 comments
Open

hatch_demo.py does not correctly save all the hatches in pdf #12367

ftesser opened this issue Oct 2, 2018 · 16 comments
Labels
backend: pdf keep Items to be ignored by the “Stale” Github Action status: confirmed bug
Milestone
future releases

Comments

@ftesser
Copy link
ftesser commented Oct 2, 2018 •
edited
Loading

Bug report

Bug summary

Code for reproduction
The same code available in

https://matplotlib.org/2.0.2/examples/pylab_examples/hatch_demo.html

with just two addition: matplotlib.use('Agg') and plt.savefig in different formats.

# Paste your code here
"""
Hatching (pattern filled polygons) is supported currently in the PS,
PDF, SVG and Agg backends only.
"""
import matplotlib
matplotlib.use('Agg') #the same happen also without this line backend TkAgg
import matplotlib.pyplot as plt
from matplotlib.patches import Ellipse, Polygon

fig = plt.figure()
ax1 = fig.add_subplot(131)
ax1.bar(range(1, 5), range(1, 5), color='red', edgecolor='black', hatch="/")
ax1.bar(range(1, 5), [6] * 4, bottom=range(1, 5), color='blue', edgecolor='black', hatch='//')
ax1.set_xticks([1.5, 2.5, 3.5, 4.5])

ax2 = fig.add_subplot(132)
bars = ax2.bar(range(1, 5), range(1, 5), color='yellow', ecolor='black') + \
    ax2.bar(range(1, 5), [6] * 4, bottom=range(1, 5), color='green', ecolor='black')
ax2.set_xticks([1.5, 2.5, 3.5, 4.5])

patterns = ('-', '+', 'x', '\\', '*', 'o', 'O', '.')
for bar, pattern in zip(bars, patterns):
    bar.set_hatch(pattern)

ax3 = fig.add_subplot(133)
ax3.fill([1, 3, 3, 1], [1, 1, 2, 2], fill=False, hatch='\\')
ax3.add_patch(Ellipse((4, 1.5), 4, 0.5, fill=False, hatch='*'))
ax3.add_patch(Polygon([[0, 0], [4, 1.1], [6, 2.5], [2, 1.4]], closed=True, fill=False, hatch='/'))
ax3.set_xlim((0, 6))
ax3.set_ylim((0, 2.5))

#plt.show()

plt.savefig('hatch_demo.pdf')
plt.savefig('hatch_demo.png')
plt.savefig('hatch_demo.eps')
print(matplotlib.__version__)
print(matplotlib.get_backend())

Actual outcome

The pdf saved image does not show all the hatch.

hatch_demo.pdf

Instead, the png and the eps ones show all the hatch correctly.

Expected outcome

The same shown in

https://matplotlib.org/2.0.2/mpl_examples/pylab_examples/hatch_demo.pdf

Matplotlib version

  • Operating system: Ubuntu 18.04 x86_64
  • Matplotlib version: v3.0.0 (but the same happen also whit v2.1.1)
  • Matplotlib backend (print(matplotlib.get_backend())): Agg (but the same happen also whit TkAgg)
  • Python version: Python 3.6.6
  • Jupyter version (if applicable):
  • Other libraries:

Matplotlib installed with pip
@ftesser ftesser changed the title hatch_demo.py does not save pdf hatch_demo.py does not correctly save all the hatches in pdf Oct 2, 2018
@QuLogic
Copy link
Member
QuLogic commented Oct 2, 2018 •
edited
Loading

This is strange because ax.bar produces Rectangles in a BarContainer, and Rectangle hatching for PDF is definitely tested here:

matplotlib/lib/matplotlib/tests/test_simplification.py

Lines 221 to 226 in 7bcf618

@image_comparison(baseline_images=['hatch_simplify'], remove_text=True)
def test_hatch():
fig, ax = plt.subplots()
ax.add_patch(plt.Rectangle((0, 0), 1, 1, fill=False, hatch="/"))
ax.set_xlim((0.45, 0.55))
ax.set_ylim((0.45, 0.55))

The only difference is the example sets hatching after the object is created. But modifying the test to do the same still passes.

Perhaps something to do with the BarContainer is messing things up.

@ftesser
Copy link
Author
ftesser commented Oct 2, 2018 •
edited
Loading

Hi, I have just realized that the embedded pdf visualizer in my web browser show me the pdf correctly.
So I have to reformulate my issue:

  • I can correctly view the generated pdf with chromium pdf viewer
  • I can not correctly view the pdf with evince (GNOME Document Viewer 3.28.2)

I have done another test, I noticed that the generated pdf file is a "PDF-1.4" file.
I tried to to convert the same file as "PDF-1.5" version and now I can see all the hatching also with evince.
Does it exist any way to force plt.savefig to save the figure using PDF CompatibilityLevel 1.5?

For convenience I attach the "PDF-1.5" version here: hatch_demo-PDF-1.5.pdf

@ImportanceOfBeingErnest
Copy link
Member
ImportanceOfBeingErnest commented Oct 2, 2018 •
edited
Loading

pdfSam: Blured hatching
image

Adobe Acrobat: No hatching at all
image

SumatraPDF: No hatching + some white lines

image

Strikingly, the shared hatch_demo-PDF-1.5.pdf looks perfect in all tested pdf viewers and browsers (Firefox, Opera, Vivaldi, Foxit Reader, Acrobat, pdfSam, SumatraPDF).

However, I cannot reproduce the conversion. Trying to convert via ghostscript9.16 as >gswin64 -sDEVICE=pdfwrite -dCompatibilityLevel=1.5 -dPDFSETTINGS=/screen -dNOPAUSE -dQUIET -dBATCH -sOutputFile=hatch-demo15.pdf hatch_demo.pdf gives a pdf which looks bad in all tested viewers.

Also using some online tool (https://docupub.de/pdfconvert) produces undesired output.

@ftesser Can you share how exactly you did the conversion?

@anntzer
Copy link
Contributor
anntzer commented Oct 2, 2018

Curiously, I get similar result with mplcairo: chromium shows the hatches, mupdf and zathura don't, and okular only shows the round ones (second from the left).

@ftesser
Copy link
Author
ftesser commented Oct 2, 2018 •
edited
Loading

@ImportanceOfBeingErnest I did the conversion like this:
I imported the file with inkscape (https://inkscape.org) and saved back (inkscape allows to "restrict to PDF version" 1.5).

@ImportanceOfBeingErnest
Copy link
Member

I see, that means that it's not enough to just change the version, but to convert it to svg first and then save to pdf again. (Did you try with "restrict to version 1.4"? It might just work as well.)

@ftesser
Copy link
Author
ftesser commented Oct 5, 2018

@ImportanceOfBeingErnest: I confirm, the same inkscape procedure but with "restrict to PDF version" 1.4 works as well.

@ImportanceOfBeingErnest
Copy link
Member
ImportanceOfBeingErnest commented Oct 5, 2018 •
edited
Loading

Good to know that it doesn't depend on the pdf version. It means that this is purely a problem of the pdf backend.

For a workaround one can indeed use inkscape as

def savepdfviasvg(fig, name, **kwargs):
    import subprocess
    fig.savefig(name+".svg", format="svg", **kwargs)
    incmd = ["inkscape", name+".svg", "--export-pdf={}.pdf".format(name),
             "--export-pdf-version=1.5"] #"--export-ignore-filters",
    subprocess.check_output(incmd)

    
savepdfviasvg(fig, "hatchtestconv")

which requires inkscape to live in the Path.

pepper-jk added a commit 8000 to pepper-jk/plot_interval_statistics that referenced this issue Oct 25, 2018
@pepper-jk
use inkscape workaround
30cd773 
matplotlib/matplotlib#12367
pepper-jk added a commit to pepper-jk/plot_interval_statistics that referenced this issue Oct 25, 2018
@pepper-jk
export plots as pdf
49d2552 
use inkscape workaround:
matplotlib/matplotlib#12367
Closed
@xhain
Copy link
xhain commented Nov 18, 2019

Bump.

I'm having a similar issue here: hatching is not exported to PDF.

@timhoffm timhoffm mentioned this issue Jan 1, 2020
Closed
@ShawnZhong
Copy link
Contributor

Adding alpha=1 to bar() seems to solve this problem.

import matplotlib.pyplot as plt

fig = plt.figure()

ax2 = fig.add_subplot(132)
bars = ax2.bar(range(1, 5), range(1, 5), color='yellow', ecolor='black', alpha=1) + \
    ax2.bar(range(1, 5), [6] * 4, bottom=range(1, 5), color='green', ecolor='black', alpha=1)
ax2.set_xticks([1.5, 2.5, 3.5, 4.5])

patterns = ('-', '+', 'x', '\\', '*', 'o', 'O', '.')
for bar, pattern in zip(bars, patterns):
    bar.set_hatch(pattern)

plt.savefig('hatch_demo.pdf')

The screenshots below are taken from Chrome. I can always see the hatches in Adobe Reader and the default Preview app in macOS with or without alpha=1.

Before After
image image

@ShawnZhong
Copy link
Contributor

Here is what I see when I open the following files in Chrome (105.0.5195.125 on arm64 macOS):

tests/baseline_images/test_legend/hatching.pdf tests/baseline_images/test_legend/hatching.png
image image
tests/baseline_images/test_artist/hatching.pdf tests/baseline_images/test_artist/hatching.png
image image

@ShawnZhong
Copy link
Contributor

I dig into this a bit more and found that the alpha value is indeed the problem.

The following code snippet generates two images default.pdf and alpha1.0.pdf. Only the latter one properly shows the hatches in Chrome.

import matplotlib.pyplot as plt
from matplotlib.patches import Rectangle

rectangles = {
    "default" : Rectangle((0, 0), 1, 1, hatch="x"),
    "alpha1.0": Rectangle((0, 0), 1, 1, hatch="x", alpha=1.0),
    # "alpha0.5":Rectangle((0, 0), 1, 1, hatch="x", alpha=0.5),
    # "no-fill": Rectangle((0, 0), 1, 1, hatch="x", fill=False),
}

for name, rect in rectangles.items():
    fig, ax = plt.subplots()
    ax.set_axis_off()
    fig.set_tight_layout(True)
    ax.add_patch(rect)
    fig.savefig(name + ".pdf", bbox_inches='tight', pad_inches=0)
    fig.savefig(name + ".png", bbox_inches='tight', pad_inches=0)

By diffing the generated PDF files. There is only a single line of difference:

diff <(xxd alpha1.0.pdf) <(xxd default.pdf)
41c41
< 00000280: 7874 4753 7461 7465 202f 4341 2031 202f  xtGState /CA 1 /
---
> 00000280: 7874 4753 7461 7465 202f 4341 2030 202f  xtGState /CA 0 /

One can also see this using a PDF parser:

from PyPDF2 import PdfReader
for name in rectangles.keys():
    reader = PdfReader(name + ".pdf")
    print(reader.pages[0]["/Resources"]["/ExtGState"])

# output
 {'/A1': {'/Type': '/ExtGState', '/CA': 0, '/ca': 1}}
 {'/A1': {'/Type': '/ExtGState', '/CA': 1, '/ca': 1}}

The reference from http://res.icesoft.org/docs/icepdf/v4_4_0/core/org/icepdf/core/pobjects/graphics/ExtGState.html shows:

CA: The current stroking alpha constant, specifying the constant shape or constant opacity value to be used for stroking operations in the transparent imaging model.

ca: Same as CA, but for nonstroking operations.

In terms of the code from matplotlib, it has something to do with the following lines:

matplotlib/lib/matplotlib/backends/backend_pdf.py

Lines 1991 to 1996 in 1d5221b

if gc._forced_alpha:
gc._effective_alphas = (gc._alpha, gc._alpha)
elif fillcolor is None or len(fillcolor) < 4:
gc._effective_alphas = (gc._rgb[3], 1.0)
else:
gc._effective_alphas = (gc._rgb[3], fillcolor[3])

@joschkabirk joschkabirk mentioned this issue Dec 12, 2022
Closed
@oscargus oscargus mentioned this issue Dec 21, 2022
Draft
6 tasks
@yipengsun yipengsun mentioned this issue Feb 11, 2023
Open
@github-actions
Copy link

This issue has been marked "inactive" because it has been 365 days since the last comment. If this issue is still present in recent Matplotlib releases, or the feature request is still wanted, please leave a comment and this label will be removed. If there are no updates in another 30 days, this issue will be automatically closed, but you are free to re-open or create a new issue if needed. We value issue reports, and this procedure is meant to help us resurface and prioritize issues that have not been addressed yet, not make them disappear. Thanks for your help!

@github-actions github-actions bot added the status: inactive Marked by the “Stale” Github Action label Sep 20, 2023
@oscargus oscargus added keep Items to be ignored by the “Stale” Github Action and removed status: inactive Marked by the “Stale” Github Action labels Sep 20, 2023
@oscargus
Copy link
Member

Thanks for these insights @ShawnZhong !

It seems like CA and ca is set here:

matplotlib/lib/matplotlib/backends/backend_pdf.py

Lines 1455 to 1466 in bd8198c

def alphaState(self, alpha):
"""Return name of an ExtGState that sets alpha to the given value."""
state = self.alphaStates.get(alpha, None)
if state is not None:
return state[0]
name = next(self._alpha_state_seq)
self.alphaStates[alpha] = \
(name, {'Type': Name('ExtGState'),
'CA': alpha[0], 'ca': alpha[1]})
return name

Which in turn is called from:
https://github.com/matplotlib/matplotlib/blob/bd8198c8b8b1a41c39e7f85ef164f7801682ecd0/lib/matplotlib/backends/backend_pdf.py#L2510C1-L2513C1

This is only called in:

matplotlib/lib/matplotlib/backends/backend_pdf.py

Lines 2589 to 2625 in bd8198c

def delta(self, other):
"""
Copy properties of other into self and return PDF commands
needed to transform *self* into *other*.
"""
cmds = []
fill_performed = False
for params, cmd in self.commands:
different = False
for p in params:
ours = getattr(self, p)
theirs = getattr(other, p)
try:
if ours is None or theirs is None:
different = ours is not theirs
else:
different = bool(ours != theirs)
except ValueError:
ours = np.asarray(ours)
theirs = np.asarray(theirs)
different = (ours.shape != theirs.shape or
np.any(ours != theirs))
if different:
break
# Need to update hatching if we also updated fillcolor
if params == ('_hatch', '_hatch_color') and fill_performed:
different = True
if different:
if params == ('_fillcolor',):
fill_performed = True
theirs = [getattr(other, p) for p in params]
cmds.extend(cmd(self, *theirs))
for p in params:
setattr(self, p, getattr(other, p))
return cmds

(as part of self.commands)

There is a bit related to hatches where maybe alpha should be dealt with in some way...

Now, checking #17049, I may be somewhat correct, but the easiest thing is probably to build on that.

@A-Vani
Copy link
A-Vani commented Mar 1, 2024

I still have this issue. An alternative is to generate svg files and then convert to pdf.

@tacaswell
Copy link
Member

I can confirm some of the reports in this thread on ~current main:

  • viewer dependent behavior (and wildly in okular the visibile hatching changes if when the window is resized!)
  • setting alpha=1 explicitly (rather than implicitly) fixes the makes all of the hatches visible

@tacaswell tacaswell added this to the future releases milestone Apr 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
backend: pdf keep Items to be ignored by the “Stale” Github Action status: confirmed bug
Projects
None yet
Milestone
future releases
Development

No branches or pull requests
9 participants
@tacaswell @QuLogic @ftesser @anntzer @ShawnZhong @oscargus @xhain @ImportanceOfBeingErnest @A-Vani
0