8000 MAINT: back printoptions with a true context variable by ngoldbaum · Pull Request #26846 · numpy/numpy · GitHub
[go: up one dir, main page]

Skip to content

MAINT: back printoptions with a true context variable #26846

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Jul 22, 2024

Conversation

ngoldbaum
Copy link
Member

This is a re-do of #26345, I'm taking over from @mtsokol because this is needed for the free-threaded work.

The new _printoptions.py file exists to avoid a circular import during setup of the multiarray module.

I'm guessing this adds some overhead to printing. I haven't benchmarked it because it wasn't clear to me: do we care about printing performance? I could certainly add some caching or a way to avoid repeatedly calling get_legacy_print_mode for every printed value. We could also keep the C global we had before but make it thread-local. I just thought it made things conceptually simpler to store all the printoptions state in the context variable.

I also looked at Sebastians comment from the old PR and I'm not totally sure what he wants. It's not immediately clear to me how to expose the context variable we're creating as an internal-only context manager, if that's even what Sebastian was getting at.

@ngoldbaum ngoldbaum added 56 - Needs Release Note. Needs an entry in doc/release/upcoming_changes 39 - free-threading PRs and issues related to support for free-threading CPython (a.k.a. no-GIL, PEP 703) labels Jul 3, 2024
@seberg
Copy link
Member
seberg commented Jul 4, 2024

I was wondering if we should stop passing around options as a dict and just look them up from the global contextvar instead wherever needed.
IIRC, it is a bit of maze which options get passed on where, and I suspect it is just simpler to stop it completely: If you need an option, look it up from the contextvar, if you need to mutate an option use the context manager just like the public API user.

Unlike the old code, contextvars should protect us from having to "freeze" the state early on to avoid threading issues.

Copy link
Contributor
@mhvk mhvk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can see the slight worry about going out to python for every repr, but this does have the advantage of being quite clean (plus printing is presumably quite slow).

My comments mostly seem to be about naming...

Copy link
Member
@seberg seberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, as Marten already reviewed anyway. A few comments, new context safety tests could be nice, but I can live without most changes here.

Overall, it is nice and simple (I do think this code can use more love, which this old comment was about, but I had expected that this might end up changing more, forcing other refactors a bit).

@ngoldbaum
Copy link
Member Author

Last push responds to all the review comments from last week. Thanks all for taking a look at this.

@rgommers rgommers added this to the 2.1.0 release milestone Jul 10, 2024
Copy link
Member
@seberg seberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, was considering to just fix it up, add a tiny release note and put it in. But, I think checking error for get_legacy_print_mode() is a slightly bigger change that is unfortunately needed for sure.

@ngoldbaum ngoldbaum force-pushed the printoptions-contextvar branch from b340c76 to 02506cc Compare July 19, 2024 21:45
@ngoldbaum
Copy link
Member Author

The last pushes respond to review comments and add a release note. I also figured out how to write async tests using the same patterns as the threading tests, cribbing off the errstate tests. It turns out asyncio.Barrier is a thing you can await.

Copy link
Contributor
@mhvk mhvk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't look super carefully, but only found three nitpicks now, so approving.

Not merging yet, though, since I did wonder a bit whether it would make sense to subclass ContextDict(ContextVar) and make it more dict-like - in particular, one would add __getitem__, __or__ and copy. But perhaps that would just hide too much for too little gain.

@ngoldbaum ngoldbaum force-pushed the printoptions-contextvar branch from a76d841 to fe9add9 Compare July 22, 2024 18:26
@ngoldbaum
Copy link
Member Author
ngoldbaum commented Jul 22, 2024

But perhaps that would just hide too much for too little gain.

I don't think it's worth it. The explicit .get() lets you know when you're syncing with the "global" ContextVar state, so I like leaving it explicit.

@seberg seberg merged commit b3feb3c into numpy:main Jul 22, 2024
66 of 68 checks passed
@seberg
Copy link
Member
seberg commented Jul 22, 2024

Thanks, @ngoldbaum. Yeah, if we had to get individual options a lot, a class might be nice. But as is, I also prefer it as is.

ArvidJB pushed a commit to ArvidJB/numpy that referenced this pull request Nov 1, 2024
This is a re-do of numpygh-26345, I'm taking over from @mtsokol because this is needed for the free-threaded work.

The new _printoptions.py file exists to avoid a circular import during setup of the multiarray module.

I'm guessing this adds some overhead to printing. I haven't benchmarked it because it wasn't clear to me: do we care about printing performance? I could certainly add some caching or a way to avoid repeatedly calling get_legacy_print_mode for every printed value. We could also keep the C global we had before but make it thread-local. I just thought it made things conceptually simpler to store all the printoptions state in the context variable.

Co-authored-by: Mateusz Sokół <mat646@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
03 - Maintenance 39 - free-threading PRs and issues related to support for free-threading CPython (a.k.a. no-GIL, PEP 703) 56 - Needs Release Note. Needs an entry in doc/release/upcoming_changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants
0