8000 gh-111662: Update socket module to use AC for optimizing performance by wrongnull · Pull Request #111661 · python/cpython · GitHub
[go: up one dir, main page]

Skip to content

gh-111662: Update socket module to use AC for optimizing performance #111661

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 17 commits into from
Nov 8, 2023

Conversation

wrongnull
Copy link
Contributor
@wrongnull wrongnull commented Nov 2, 2023

Currently socket.htons uses unnecessary METH_VARAGRS calling convention meanwhile it only accepts one argument. This patch will speed this function up by changing calling convention to METH_O
I ran this test before commit

./python -m timeit -s "from socket import htons" "htons(42)"
1000000 loops, best of 5: 199 nsec per loop

and after

./python -m timeit -s "from socket import htons" "htons(42)"
1000000 loops, best of 5: 126 nsec per loop

@ghost
Copy link
ghost commented Nov 2, 2023

All commit authors signed the Contributor License Agreement.
CLA signed

@bedevere-app
Copy link
bedevere-app bot commented Nov 2, 2023

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@wrongnull wrongnull changed the title Make socket.htons run faster gh-111662: Make socket.htons run faster Nov 2, 2023
Copy link
Member
@Eclips4 Eclips4 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Just one thing: I'm don't sure that we need here NEWS entry, so let's wait for other opinions.
Thanks for finding this.

Copy link
Member
@corona10 corona10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use AC instead of writing manual parsing.

@serhiy-storchaka
Do you think that it's worth changing this code?
If yes, I prefer to update socketmodules to use AC as much as possible for all methods.

@@ -6396,11 +6396,19 @@ Convert a 32-bit integer from network to host byte order.");


static PyObject *
socket_htons(PyObject *self, PyObject *args)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/*[clinic input]
_socket.socket.htons
    x: int
    /

Convert a 16-bit unsigned integer from host to network byte order.
[clinic start generated code]*/

Use AC instead of writing manual parsing.
I got a similar performance gain in my local environment.

AS-IS: 5000000 loops, best of 5: 60.9 nsec per loop
TO-BE: 10000000 loops, best of 5: 23.3 nsec per loop

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wrongnull
You should take a look at the https://devguide.python.org/development-tools/clinic/ if you're not familiar with Argument Clinic :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure a similar approach can be applied to all other functions with only one argument. If yes, I'll be happy to make such a commit in this branch.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@corona10 seems ready. clinic argument was also applied to other functions where it was beneficial

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if ValueError is more preferable if negative value is passed to ntohl and htonl functions

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if ValueError is more preferable if negative value is passed to ntohl and htonl functions

Please don't change the exception type; it will occur in user regression.
Can I know where the type of exception will be changed if AC is applied?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if ValueError is more preferable if negative value is passed to ntohl and htonl functions

Please don't change the exception type; it will occur in user regression. Can I know where the type of exception will be changed if AC is applied?

Exception type will change to ValueError from OverflowError in htonl and ntohl functions after applying usigned_long argument converter.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exception type will change to ValueError from OverflowError in htonl and ntohl functions after applying usigned_long argument converter.

In that case, I would like to recommend not to touch it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exception type will change to ValueError from OverflowError in htonl and ntohl functions after applying usigned_long argument converter.

In that case, I would like to recommend not to touch it.

OK, I reverted this changes

@bedevere-app
Copy link
bedevere-app bot commented Nov 3, 2023

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

@wrongnull
Copy link
Contributor Author

I have made the requested changes; please review again

@Eclips4 Eclips4 requested a review from corona10 November 6, 2023 20:01
Copy link
Member
@corona10 corona10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM Thanks!

@corona10 corona10 changed the title gh-111662: Make socket.htons run faster gh-111662: Update socket module to use AC for optimizing performance Nov 8, 2023
@corona10 corona10 merged commit 8fbe531 into python:main Nov 8, 2023
@wrongnull
Copy link
Contributor Author

@Eclips4 Thanks a lot

@wrongnull wrongnull deleted the meth_o-for-htons branch November 8, 2023 16:06
aisk pushed a commit to aisk/cpython that referenced this pull request Feb 11, 2024
Glyphack pushed a commit to Glyphack/cpython that referenced this pull request Sep 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants
0