8000 [Bug]: Image color limits not correctly updated with set_clim() IFF color bar present AND new norm.vmin > old norm.vmax · Issue #29522 · matplotlib/matplotlib · GitHub
[go: up one dir, main page]

Skip to content

[Bug]: Image color limits not correctly updated with set_clim() IFF color bar present AND new norm.vmin > old norm.vmax #29522

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
erikdendekker opened this issue Jan 25, 2025 · 15 comments · Fixed by #29590

Comments

@erikdendekker
Copy link

Bug summary

I experienced some specific odd cases where I noticed that the colors of an image were not correctly set after updating the image data and setting new color limits with set_clim(). After debugging I was able to reproduce with a minimal example,..

Code for reproduction

import numpy as np
from matplotlib import pyplot as plt


# Create test data
np.random.seed(19771105)
data_0 = np.round(np.random.rand(10, 20), 1)
data_1 = data_0 + 10

# Compute data range
range_0 = data_0.min(), data_0.max()
range_1 = data_1.min(), data_1.max()

# Plot data as image; Auto adjusts colorbar scale to value range
image = plt.imshow(data_0)
cbar = plt.colorbar(image)  # NOTE: Problem does not occur if colorbar not created!

print(f'expected norm = {range_0}')
print(f'actual   norm = {image.norm.vmin, image.norm.vmax}')

# Change image data and update image color limits
image.set_data(data_1)
image.set_clim(range_1)

print(f'expected norm = {range_1}')
print(f'actual   norm = {image.norm.vmin, image.norm.vmax}')  # Incorrect!

plt.show()

Actual outcome

expected initial norm = (0.0, 1.0)
computed initial norm = (0.0, 1.0)
expected updated norm = (10.0, 11.0)
computed updated norm = (0.9, 11.0)

Image

Expected outcome

expected initial norm = (0.0, 1.0)
computed initial norm = (0.0, 1.0)
expected updated norm = (10.0, 11.0)
computed updated norm = (10.0, 11.0)

Image

Additional information

Only happens if color bar is present; Removing the line that creates it from code lets you see the correct image (minus color bar).
Data dependent; Only occurs if the lowest value of the new image data is higher than the highest value of the old image data (Hence +10 in the code example).

Operating system

Windows 10

Matplotlib Version

3.6.0

Matplotlib Backend

QtAgg

Python version

3.11.9

Jupyter version

Installation

pip

@erikdendekker erikdendekker changed the title [Bug]: Image color limits not correctly updated IFF color bar present AND new norm.vmin > old norm.vmax [Bug]: Image color limits not correctly updated with set_clim() IFF color bar present AND new norm.vmin > old norm.vmax Jan 25, 2025
@erikdendekker
Copy link
Author
erikdendekker commented Jan 25, 2025

Workaround for this bug (based on example code):

#image.set_clim(range_1)
if image.norm.vmin < range_1[0]:
    image.norm.vmax = range_1[1]
    image.norm.vmin = range_1[0]
else:
    image.norm.vmin = range_1[0]
    image.norm.vmax = range_1[1]

@jklymak
Copy link
Member
jklymak commented Jan 26, 2025

I can confirm this on latest Matplotlib. Thanks for the clear minimal report. @greglucas I wonder if this has to do with your colorbar interactivity?

@greglucas
Copy link
Contributor

Yes, I think it does have to do with the signals on the norm callbacks. We trigger the signals when setting vmin originally, which then becomes greater than the original vmax (vmin > vmax temporarily). We emit a norm-has-changed signal and then the colorbar goes and processes values and then we have a non-singular expansion which swaps the vmin/vmax if they are inverted.

self.norm.vmin, self.norm.vmax = mtransforms.nonsingular(
self.norm.vmin, self.norm.vmax, expander=0.1)

@erikdendekker that is why you are seeing that behavior of the order of setting vmin/vmax mattering in your workaround.
#29522 (comment)

I'm not sure what exactly the proper fix is here. We are triggering callbacks, which trigger colorbar processing before we have finished updating all of our norm limits which the colorbar doesn't like. We could fix the set_clim() case because vmin/vmax come in together by suppressing the callbacks and only emitting one at the end. But, I think that a user should be able to directly call im.norm.vmin = 10, im.norm.vmax=11 and have that work too regardless of the order even though they are sequential calls and a user shouldn't need to worry about suppressing the callbacks. I wonder if there is anything we can do in colorbar to move the autoexpansion / limit updates around to help?

@prafulgulani
Copy link
Contributor

@erikdendekker in your workaround you have same statements in your if and else blocks,
and a simpler workaround would have been to use colorbar at last after setting the final range.

# Plot data as image; Auto adjusts colorbar scale to value range
image = plt.imshow(data_0)

print(f'expected norm = {range_0}')
print(f'actual   norm = {image.norm.vmin, image.norm.vmax}')

# Change image data and update image color limits
image.set_data(data_1)
image.set_clim(range_1)

cbar = plt.colorbar(image)  # use after final range has been set to avoid callbacks.

@tacaswell tacaswell added this to the v3.10.1 milestone Feb 7, 2025
@tacaswell
Copy link
Member

This happening within a single set_clim is clearly wrong and we should fix it, but I am not sure what should happen when setting the min and then the max one-at-a-time.

I think it comes down to a question of when should we enforce that the colormappable is in a consistent drawable state. The options I see are:

  • never, if vmin > vmax fail at draw time
  • at draw time, fix it if we have to when we go to render
  • at set/update-time, fix it before the next line of code can run

I not sure any of these are wrong (and we have behavior in the library of all three types). Currently setting im.norm.vmin to be greater than vax falls into the first category without a colorbar and the third with on which is the I think the real bug (as all three have tradeoffs but we should be consistent what ever we do).

This seems related to discussion we had about colorbars and auto-scaling for constant value data (however I am failing to find the issue), we should be consistent with our decision there.

@timhoffm
Copy link
Member
timhoffm commented Feb 7, 2025

This seems related to discussion we had about colorbars and auto-scaling for constant value data (however I am failing to find the issue),

That‘s #26307. The main point of discussion there is to which color value we should map, not so much when.

@anntzer
Copy link
Contributor
anntzer commented Feb 9, 2025

I think there's also the question of whether we could support norms with vmin > vmax (and e.g. treat them as reversing the colormap). Somewhat related to #12665 I guess, although that's a fairly old issue.

(edited typo, was initially incorrectly vmin < vmax)

@timhoffm
Copy link
Member
timhoffm commented Feb 9, 2025

You mean vmin > vmax? I'm a bit hesitant. Is there a use case where that's natural? I suspect that this may rather be indicative of a calculation or input error, and we should raise instead. If people want to explicitly reverse the colormap they should use the "..._r" version of the colormap.

@jklymak
Copy link
Member
jklymak commented Feb 9, 2025

For sure there are instances where the colorbar makes more sense if it is inverted, but keeping the values on the colorbar positive. Eg depth goes from 5000 to zero but you want the darkest blue to be 5000. Currently you'd need to reverse the colormap and reverse the colorbar, or use negative depths and change the ticks.

I don't think it's a big deal. Personally I just use negative depths in the colorbar, but if someone were more persnickety, you could imagine them being saved a song and dance if we allowed inverted norms.

@anntzer
Copy link
Contributor
anntzer commented Feb 9, 2025

Ah, now that I remember it; the other advantage of supporting vmin > 8000 ; vmax would be to draw inverted colorbars, e.g. the left half of this one (from a real paper, where D and eta are inversely related):
Image

@erikdendekker
Copy link
Author

@erikdendekker in your workaround you have same statements in your if and else blocks, and a simpler workaround would have been to use colorbar at last after setting the final range.

For the simple code that reproduces this bug you are right of course,.. In my real-world application, image data is interactively updated from an external source and shown to the user. I just encountered unexpected behaviour in some specific cases, was able to pinpoint it to this issue, and condensed it to the simplified example.

@timhoffm
Copy link
Member

Interestingly, the colorbar modifies the norm:

im = plt.imshow(a, vmin=0.8, vmax=0.2)
print(im.norm.vmin, im.norm.vmax)
plt.colorbar()
print(im.norm.vmin, im.norm.vmax)

Prints

0.8 0.2
0.18 0.8

This is definitively undesired. Norms itself currently raise if vmin > vmax (e.g. add im.norm(0.5) before addint the colorbar.


the other advantage of supporting vmin > vmax would be to draw inverted colorbars

This may be too clever. IMHO it makes reasoning on the mapping harder. Instead I suggest to invert the colorbar Axes.`

cb = plt.colorbar()
cb.ax.invert_yaxis()

We could even grow a kwarg on colorbar() if that's helpful. I think this is much clearer than exploiting vmin > vmax. The latter has the ambiguity that you don't know whether the vmin will still map to the lower end of the color range or to the higher end (and I believe there's no canoncial answer to that).

@greglucas
Copy link
Contributor

Interestingly, the colorbar modifies the norm:
This is definitively undesired. Norms itself currently raise if vmin > vmax (e.g. add im.norm(0.5) before addint the colorbar.

@timhoffm, I agree. In my view, the colorbar should only "query" the norm and use that (interactive axes updating the norm for colorbars is separate and explicit), otherwise we have a divergence between images with/without colorbars. If we can push all of this logic over into the mappable drawing (and perhaps call that similar to draw_without_rendering instead here) then I think we could at least maintain consistency between figures with/without colorbars.

# transform from 0-1 to vmin-vmax:
if self.mappable.get_array() is not None:
self.mappable.autoscale_None()
if not self.norm.scaled():
# If we still aren't scaled after autoscaling, use 0, 1 as default
self.norm.vmin = 0
self.norm.vmax = 1
self.norm.vmin, self.norm.vmax = mtransforms.nonsingular(
self.norm.vmin, self.norm.vmax, expander=0.1)
if (not isinstance(self.norm, colors.BoundaryNorm) and
(self.boundaries is None)):
b = self.norm.inverse(b)

@timhoffm
Copy link
Member
timhoffm commented Feb 10, 2025

I've not looked into the details but I'm hesitant to move the non-singular expansion into the mappable. This ties again into #26307. It's a fundamental question how to handle singularity:

  • data can be singular
  • norms may autoscale to data and then have vmin = vmax. We could prohibit that, which means failing if the the user explicitly calculates limits imshow(a, vmin=a.min(), a.max()). But on autoscale imshow(a) we could expand (slight annoyance is that the amout of expansion would be an arbitrary choice).
    If we don't prohibit, and support singular norms, this means (1) we have to define to which value everying is mapped (0?, 0.5?, 1). And then we need special handling in colorbars that they draw themselves in a particular way if the norm is singular (e.g. only ticks at both ends with the same value - maybe even put a special marker on the position that the singular norm maps to; i.e. if the norms maps to 0.5, mark the center of the colorbar).

@anntzer
Copy link
Contributor
anntzer commented Feb 11, 2025

re: supporting inverted norms (I commented on the other thread re: singular norms): assuming this does not introduce too many complications elsewhere (e.g. re: inverted colorbars), I think making norm.vmin = ...; norm.vmax = ... always work (regardless of the relative vmin/vmax values) without bothering with any special-casing would be a big plus consistency-wise and simplicity-wise.

prafulgulani added a commit to prafulgulani/matplotlib that referenced this issue Feb 11, 2025
@ksunden ksunden modified the milestones: v3.10.1, v3.10.2 Feb 21, 2025
timhoffm pushed a commit that referenced this issue Feb 28, 2025
…29590)

* Blocked set_clim() callbacks to prevent inconsistent state (#29522)

* added test for set_clim() callback count

* Relocated test_set_clim_emits_single_callback to test_colors.py

* test case corrections

---------

Co-authored-by: prafulgulani555 <59774145+prafulgulani555@users.noreply.github.com>
meeseeksmachine pushed a commit to meeseeksmachine/matplotlib that referenced this issue Feb 28, 2025
QuLogic added a commit that referenced this issue Feb 28, 2025
…590-on-v3.10.x

Backport PR #29590 on branch v3.10.x (Blocked set_clim() callbacks to prevent inconsistent state (#29522))
@ksunden ksunden mentioned this issue May 9, 2025
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants
0