8000 Update rescale_intensity to prevent under/overflow and produce proper output dtype by jni · Pull Request #4585 · scikit-image/scikit-image · GitHub
[go: up one dir, main page]
Skip to content

Update rescale_intensity to prevent under/overflow and produce proper output dtype #4585

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 15 commits into from
Apr 24, 2020
Merged

Update rescale_intensity to prevent under/overflow and produce proper output dtype #4585

merged 15 commits into from
Apr 24, 2020

Conversation

jni
Copy link
Member
@jni jni commented Apr 16, 2020

Description

Fixes #4583

  • All computations are now in float, preventing under/overflow
  • If out_range is a dtype, then the output will be in that dtype (currently it is the input dtype, which is insane).
  • If out_range is a range of numbers, the output will be of type float and the caller will be responsible for converting to the correct dtype. This is a breaking change in some circumstances. See our use of rescale_intensity within equalize_adapthist which I had to fix.

Checklist

For reviewers

  • Check that the PR title is short, concise, and will make sense 1 year
    later.
  • Check that new functions are imported in corresponding __init__.py.
  • Check that new features, API changes, and deprecations are mentioned in
    doc/release/release_dev.rst.

@jni jni mentioned this pull request Apr 16, 2020
Closed
@rfezzani rfezzani added the 🩹 type: Bug fix Fixes unexpected or incorrect behavior label Apr 16, 2020
emmanuelle
emmanuelle reviewed Apr 18, 2020
View reviewed changes
skimage/exposure/exposure.py
except TypeError: # uint10, uint12, uint14
# otherwise, return uint16
return np.uint16

Copy link
Member
@emmanuelle emmanuelle Apr 18, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add an else here to raise a TypeError for example in the case of typos (flat instead of float). With the current version, output_dtype will be None if none of the ifs is True, and np.array([1, 2, 3], dtype=None) does not error...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand and support the first part of your suggestion.

But why should output_dtype be None. The way I understand the new behavior is

  • out_range is an actual range -> output dtype is float64
  • out_range is already a dtype (corresponding to the input dtype) -> use same type for output

None would just result in the returned output being of dtype float64 wouldn't it?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lagru probably what I wrote was not clear, please tell me if it's clearer now. The function should never return None, it should return a valid value or error out.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, then we are in agreement. 👍

@emmanuelle
Copy link
Member

One possibility we did not consider would be to accept for output_range a tuple with 3 elements, the last one being a dtype, for example (0, 200, np.uint8). Not sure we want this, but this could avoid an astype call when wanting a specific dtype and not floats.

lagru
lagru reviewed Apr 18, 2020
View reviewed changes
Copy link
Member
@lagru lagru left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the work! Only some nitpicks, feel free to ignore. All in all I find this function quite hard to fully grasp.

@jni
Copy link
Member Author
jni commented Apr 19, 2020

@emmanuelle @lagru Thank you for the reviews! I've addressed all of your comments.

@emmanuelle

One possibility we did not consider would be to accept for output_range a tuple with 3 elements, the last one being a dtype, for example (0, 200, np.uint8)

I don't like that solution very much, it seems clunky and it will complicate the code quite a lot. I would consider adding dtype= as an additional keyword argument, but then the desired behaviour would be unclear if e.g. I pass out_range='uint16', dtype=np.uint32. I would say that the dtype= argument should override all previous options, but that's just one of many possible alternatives.

At any rate, I think we can discuss this separately and address it in a separate PR, since this is already a big improvement on existing behaviour. None of the above proposals would allow existing code to continue working when the output dtype was incorrectly being inferred from the input dtype (as in my fix for equalize_adapthist), so I don't think they are critical to get in now.

lagru
lagru reviewed Apr 19, 2020
View reviewed changes
@jni @lagru
fix docstring to match new param name
1e80704 
Co-Authored-By: Lars Grüter <lagru@users.noreply.github.com>
@pep8speaks
Copy link
pep8speaks commented Apr 19, 2020 •
edited
Loading

Hello @jni! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2020-04-24 07:14:28 UTC

@jni
Copy link
Member Author
jni commented Apr 19, 2020

Thanks for keeping me honest, @lagru! 😂

lagru
lagru approved these changes Apr 19, 2020
View reviewed changes
Copy link
Member
@lagru lagru left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jni You're welcome. 😉 Nice work!

@m-albert
Copy link
Contributor

Thanks for the fix! After this is merged I'd raise another issue (I think independent of this one) on this function with a proposed fix. Cheers

@jni
Copy link
Member Author
jni commented Apr 19, 2020

@m-albert why do you need to wait until this is merged?

@jni jni mentioned this pull request Apr 20, 2020
Closed
@m-albert
Copy link
Contributor
m-albert commented Apr 20, 2020

@jni Somehow I was assuming it would be cleaner to address it once this is merged, but actually I can just open the issue, sorry for the mystery. See #4596

@jni
Copy link
Member Author
jni commented Apr 24, 2020

@scikit-image/core I've addressed the concerns about changed behaviours by adding more explicit notes in the release notes, as well as .. versionchanged:: markers in the docstrings. Pending CI I think this is ready to be merged!

@emmanuelle
Copy link
Member

Thank you @jni, merging!

@emmanuelle emmanuelle merged commit 4443aae into scikit-image:master Apr 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@emmanuelle emmanuelle emmanuelle left review comments

@rfezzani rfezzani rfezzani left review comments

@alexdesiqueira alexdesiqueira alexdesiqueira left review comments

@lagru lagru lagru approved these changes

Assignees
No one assigned
Labels
🩹 type: Bug fix Fixes unexpected or incorrect behavior
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

funky behavior of rescale_intensity
7 participants
@jni @emmanuelle @pep8speaks @m-albert @rfezzani @alexdesiqueira @lagru
0