8000 ENH: Added FuncNorm and PiecewiseNorm classes in colors by alvarosg · Pull Request #7294 · matplotlib/matplotlib · GitHub
[go: up one dir, main page]

Skip to content

ENH: Added FuncNorm and PiecewiseNorm classes in colors #7294

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 33 commits into from
Closed
Show file tree
Hide file tree
Changes from 27 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
2d90c5a
Added ArbitaryNorm and RootNorm classes in colors, as well as example…
Oct 17, 2016
57aad3d
PEP8 formatting on examples, plotting using the object oriented appro…
Oct 18, 2016
8000
f818aff
Added title/description to the examples
Oct 18, 2016
ffe1b9d
Class attributes are now hidden
Oct 18, 2016
1d22b90
Major update: complete refactorization of code. A much more powerful …
Oct 19, 2016
b5801ea
Corrected lambda function syntax that was not compatible with python …
Oct 19, 2016
e93d82d
Added FuncNorm: now everything inherits from this. Changed the name o…
Oct 20, 2016
3749b0a
Forgot to uncomment an import
Oct 20, 2016
de62491
Improved the auto-tick feature, and corrected some pep8 issues
Oct 20, 2016
d148756
Improved examples, created a new file for generating sample data.'
alvarosg Oct 22, 2016
5373a98
Corrected a double line, and removed a comment
alvarosg Oct 22, 2016
13edeab
Tests for FuncNorm added, and bug corrected in FuncNorm
alvarosg Oct 22, 2016
21d5cd0
Added compatibility for python 3 string check, added tests for Piecew…
alvarosg Oct 22, 2016
d359a4e
Added tests on all classes, including all public methods
alvarosg Oct 22, 2016
4622829
Change type of arrays in tests from int to float
alvarosg Oct 22, 2016
30ff404
Corrected wrong `super()` for RootNorm
alvarosg Oct 22, 2016
df835cb
Solve problem with implicit int to float casting that was not working…
alvarosg Oct 22, 2016
dfaa0f8
Added documentation in the numpydoc format
alvarosg Oct 23, 2016
a386395
Improve style in the examples. Corrected intending problem in the doc…
alvarosg Oct 23, 2016
b9dafb0
Added example in `FuncNorm` docstring
alvarosg Oct 23, 2016
d10be73
Finished with the examples in the docstrings
alvarosg Oct 24, 2016
c85a14c
Implemented clipping behavoir. Refactored _func_parser
alvarosg Oct 26, 2016
7597ddd
It now allows some string functions with parameters. Added a test for…
alvarosg Oct 28, 2016
7fce503
Forgot to add a file...
alvarosg Oct 28, 2016
bcd7dd0
Forgot to add another file...
alvarosg Oct 28, 2016
a71e1e9
Improved tests, documentation, and exceptions
alvarosg Oct 31, 2016
33f57d1
Removed test_colors.py from __init__.py after including parametrize
alvarosg Oct 31, 2016
9687173
Moved the string function parser to its own class in cbook. Added tes…
alvarosg Nov 1, 2016
46395aa
Improved documentation
Nov 2, 2016
63dab61
Added new example
Nov 2, 2016
8abf2c2
Added example for PiecewiseNorm, and MirrorPiecewiseNorm. String in t…
alvarosg Nov 3, 2016
b4ecdb2
Removed sampledata.py no longer necessary, and changed examples in do…
alvarosg Nov 3, 2016
42007ee
Added examples for MirrorRootNorm and RootNorm
alvarosg Nov 3, 2016
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
"""
============================================
Examples of arbitrary colormap normalization
============================================

Here I plot an image array with data spanning for a large dynamic range,
using different normalizations. Look at how each of them enhances
different features.

"""


import matplotlib.cm as cm
import matplotlib.colors as colors
import matplotlib.pyplot as plt

from mpl_toolkits.mplot3d import Axes3D

import numpy as np

from sampledata import PiecewiseNormData

X, Y, data = PiecewiseNormData()
cmap = cm.spectral

# Creating functions for plotting


def make_plot(norm, label=''):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there anyway humanly possible to reduce this example to just being about the norms? Something like:

fig, (ax1, ax2, ax3, ax4) = plt.subplots(2,2)
 data = something easy and simple

pnorm = ...
im = ax1.imshow(data, norm=pnorm)
cb = ax1.colorbar(im)
cb.set_title("PiecewiseNorm")

mnorm = ...
ax2.imshow(data, norm=mnorm)
 etc...

I think the important piece (the new norm) gets swamped under all this boilerplate prettification stuff. And I think the 3D should be its own example.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

About the data, I do not think we can just have something easy and simple, because it needs to show the normalization in action, and we are dealing here with pretty sophisticated normalization.

Maybe I could split the examples in many files so each of them always compares the linear case to a different type of norm. But I also think the normalization line is very useful for the user to understand better what is happening. Maybe a (2,2), where the top axes correspond to the linear case (normalization line, and example image), and the bottom axes to the normalize case. In this case I could make fake data according to what I want to show on each of the examples, instead of using the same fake data for all. Nevertheless even in that case I will probably should make use of an external module to generate the fake data, like I am doing now. How would you feel about this?

And I think the 3D should be its own example.

I am not sure what you mean exactly by that.

Copy link
Member
@story645 story645 Nov 1, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think 3D plot should just be a seperate file.

About the data, I do not think we can just have something easy and simple, because it needs to show the normalization in action, and we are dealing here with pretty sophisticated normalization

I think this is sorta debatable. Like even with integer data 1-10, you can cook up a small piecewise rule in that range, etc. This doesn't have to be sensible in a real world way, it just has to show people what code will do what. You just don't want your fake data to become the example, 'cause it almost doesn't matter when showing norms.

I'd say if you're gonna cook up custom data, do that separately, save as a numpy file, put in cbook's sample data folder and just use cbook.get_sample_data like
data = cbook.get_sample_data
basically, the user shouldn't need to know how you generated your data 'cause they'll have their own data anyway. Though I agree in then doing a (1,2) to show the user what a linear norm looks like relative to your shiny norm.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the user shouldn't need to know how you generated your data 'cause they'll have their own data anyway.

I agree that the user does not need to do this. The reason I made a separate file (sampledata.py) for generating the fake data instead of using cbook.get_sample_data is that sampledata.py is a 3KB file, while the equivalent npy file is 1 MB, and even if I reduce the resolution a bit , or the variable type, it will not get smaller than 100 KB. Do you know if there is any place to instead store cooked data, to store code that generates fake data?

This doesn't have to be sensible in a real world way, it just has to show people what code will do what. You just don't want your fake data to become the example, 'cause it almost doesn't matter when showing norms.

Well I guess there is a manifold purpose for the examples:

  1. Show the user how to use the normalization from a syntactic point of view. For this, a simple clear example with non sophisticated data will suffice.
  2. Show the user what the normalization is actually doing to the data. For this having the line plot is really important, and most users will understand this better than nothing, because it visually shows without doubt the ranges used, and how the data will be compressed/mapped into the 0, 1 range.
  3. Show the user what can be done with the normalization. And for this we need the examples with different kinds of small features to amplify them. I thing this is important also to understand what the normalization is for, because sometime people knows the problem they want to solve (i.e. amplify stuff in an array), but not what is the tool to solve it.

If you really do not like the example showing this, I guess I would be happy to not use sophisticated cook data (number 2), as long as number 2 is covered. Otherwise by showing just the plots we are at risk of the user not following what is really going on.

Your thoughts?

Copy link
Member
@story645 story645 Nov 1, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

while the equivalent npy file is 1 MB, and even if I reduce the resolution a bit , or the variable type, it will not get smaller than 100 KB

What about if you zip it down? (And I think you can have a 100x100 array or something)

Can you address 2 in the docstring/a short comment? I think 3 is addressed with the linear normalization vs. fancy normalization plots.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about if you zip it down? (And I think you can have a 100x100 array or something)

A 100x100 array (which already looks very pixelated), with a float16, zipped, is still 12 KB, and contains much less meta information than having the script itself, in case it is later decided to show the example in a higher resolution format.

Can you address 2 in the docstring/a short comment?
Well of course the documentation explains what it does, but an image is worth a thousand words, and there is nothing like the line plots to understand what is going on. Plus some users may also decide to plot that along there normalised plots to explain the normalization.

I think 3 is addressed with the linear normalization vs. fancy normalization plots.
Yes, but only if the fake array is cooked carefully for this purpose.

I guess my general point, is that, if it took me making the plots above to convince people people here that what I am adding in this PR, then the users would need something similar to understand what is this for.

I think I will give a try at generating some examples following some of what we discussed, when I get home after work in a couple of hours, and maybe we can go through further rounds of feedback :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should add the data directly to the repository.
If that data is useful to other people, we can add the data generation function to the cbook module.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I was not aware that the cbook module could be used for that purpose as well! In principle the same data would be useful for multiple examples, and even more normalization classes in the future. The reason I wanted to have it not directly in the example was not only re-usability, but also avoiding having the long code for generating the data in the example itself. What about having it as it is now, in a file in the same folder as the examples. Would this be allowed?

fig, (ax1, ax2) = plt.subplots(1, 2, gridspec_kw={
'width_ratios': [1, 2]}, figsize=plt.figaspect(0.5))
fig.subplots_adjust(top=0.87, left=0.07, right=0.96)
fig.suptitle(label)

cax = ax2.pcolormesh(X, Y, data, cmap=cmap, norm=norm)
ticks = cax.norm.ticks() if norm else None
fig.colorbar(cax, format='%.3g', ticks=ticks)
ax2.set_xlim(X.min(), X.max())
ax2.set_ylim(Y.min(), Y.max())

data_values = np.linspace(cax.norm.vmin, cax.norm.vmax, 100)
cm_values = cax.norm(data_values)
ax1.plot(data_values, cm_values)
ax1.set_xlabel('Data values')
ax1.set_ylabel('Colormap values')


def make_3dplot(label=''):
fig = plt.figure()
fig.suptitle(label)
ax = fig.gca(projection='3d')
cax = ax.plot_surface(X, Y, data, rstride=1, cstride=1,
cmap=cmap, linewidth=0, antialiased=False)
ax.set_zlim(data.min(), data.max())
fig.colorbar(cax, shrink=0.5, aspect=5)
ax.view_init(20, 225)


# Showing how the data looks in linear scale
make_3dplot('Regular linear scale')
make_plot(None, 'Regular linear scale')

# Example of logarithm normalization using FuncNorm
norm = colors.FuncNorm(f=lambda x: np.log10(x),
finv=lambda x: 10.**(x), vmin=0.01, vmax=2)
make_plot(norm, "Log normalization using FuncNorm")
# The same can be achived with
# norm = colors.FuncNorm(f='log10', vmin=0.01, vmax=2)

# Example of root normalization using FuncNorm
norm = colors.FuncNorm(f='sqrt', vmin=0.0, vmax=2)
make_plot(norm, "Root normalization using FuncNorm")

# Performing a symmetric amplification of the features around 0
norm = colors.MirrorPiecewiseNorm(fpos='crt')
make_plot(norm, "Amplified features symetrically around \n"
"0 with MirrorPiecewiseNorm")


# Amplifying features near 0.6 with MirrorPiecewiseNorm
norm = colors.MirrorPiecewiseNorm(fpos='crt', fneg='crt',
center_cm=0.35,
center_data=0.6)
make_plot(norm, "Amplifying positive and negative features\n"
"standing on 0.6 with MirrorPiecewiseNorm")

# Amplifying features near both -0.4 and near 1.2 with PiecewiseNorm
norm = colors.PiecewiseNorm(flist=['cubic', 'crt', 'cubic', 'crt'],
refpoints_cm=[0.25, 0.5, 0.75],
refpoints_data=[-0.4, 1, 1.2])
make_plot(norm, "Amplifying positive and negative features standing\n"
" on -0.4 and 1.2 with PiecewiseNorm")

# Amplifying positive features near -1, -0.2 and 1.2 simultaneously with
# PiecewiseNorm
norm = colors.PiecewiseNorm(flist=['crt', 'crt', 'crt'],
refpoints_cm=[0.4, 0.7],
refpoints_data=[-0.2, 1.2])
make_plot(norm, "Amplifying only positive features standing on -1, -0.2\n"
" and 1.2 with PiecewiseNorm")


plt.show()
90 changes: 90 additions & 0 deletions examples/colormap_normalization/sampledata.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
"""
================================================================
Creating sample data for the different examples on normalization
================================================================

Data with special features tailored to the need of the different examples on
colormal normalization is created.

"""

import numpy as np


def PiecewiseNormData(NX=512, NY=256):
"""Sample data for the PiecewiseNorm class.

Returns a 2d array with sample data, along with the X and Y values for the
array.

Parameters
----------
NX : int
Number of samples for the data accross the horizontal dimension.
Default is 512.
NY : int
Number of samples for the data accross the vertical dimension.
Default is 256.

Returns
-------
X, Y, data : ndarray of shape (NX,NY)
Values for the `X` coordinates, the `Y` coordinates, and the `data`.

Examples
--------
>>> X,Y,Z=PiecewiseNormData()
"""

xmax = 16 * np.pi
x = np.linspace(0, xmax, NX)
y = np.linspace(-2, 2, NY)
X, Y = np.meshgrid(x, y)

data = np.zeros(X.shape)

def gauss2d(x, y, a0, x0, y0, wx, wy):
return a0 * np.exp(-(x - x0)**2 / wx**2 - (y - y0)**2 / wy**2)

maskY = (Y > -1) * (Y <= 0)
N = 31
for i in range(N):
maskX = (X > (i * (xmax / N))) * (X <= ((i + 1) * (xmax / N)))
mask = maskX * maskY
data[mask] += gauss2d(X[mask], Y[mask], 2. * i / (N - 1), (i + 0.5) *
(xmax / N), -0.25, xmax / (3 * N), 0.07)
data[mask] -= gauss2d(X[mask], Y[mask], 1. * i / (N - 1), (i + 0.5) *
(xmax / N), -0.75, xmax / (3 * N), 0.07)

maskY = (Y > 0) * (Y <= 1)
data[maskY] = np.cos(X[maskY]) * Y[maskY]**2

N = 61
maskY = (Y > 1) * (Y <= 2.)
for i, val in enumerate(np.linspace(-1, 1, N)):
if val < 0:
aux = val
if val > 0:
aux = val * 2

maskX = (X >= (i * (xmax / N))) * (X <= ((i + 1) * (xmax / N)))
data[maskX * maskY] = aux

N = 11
maskY = (Y <= -1)
for i, val in enumerate(np.linspace(-1, 1, N)):
if val < 0:
factor = 1
if val >= 0:
factor = 2
maskX = (X >= (i * (xmax / N))) * (X <= ((i + 1) * (xmax / N)))
mask = maskX * maskY
data[mask] = val * factor

if i != N - 1:
data[mask] += gauss2d(X[mask], Y[mask], 0.05 * factor, (i + 0.5) *
(xmax / N), -1.25, xmax / (3 * N), 0.07)
if i != 0:
data[mask] -= gauss2d(X[mask], Y[mask], 0.05 * factor, (i + 0.5) *
(xmax / N), -1.75, xmax / (3 * N), 0.07)
return X, Y, data
1 change: 0 additions & 1 deletion lib/matplotlib/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -1484,7 +1484,6 @@ def _jupyter_nbextension_paths():
'matplotlib.tests.test_coding_standards',
'matplotlib.tests.test_collections',
'matplotlib.tests.test_colorbar',
'matplotlib.tests.test_colors',
'matplotlib.tests.test_compare_images',
'matplotlib.tests.test_container',
'matplotlib.tests.test_contour',
Expand Down
Loading
0