8000 python wheel · Issue #457 · prometheus/client_python · GitHub
[go: up one dir, main page]
Skip to content

python wheel #457

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
gjcarneiro opened this issue Aug 28, 2019 · 11 comments
Closed

python wheel #457

gjcarneiro opened this issue Aug 28, 2019 · 11 comments

Comments

@gjcarneiro
Copy link

Would it be possible, from now on, to also generate and upload Python wheels? It will speed up installation of this package a lot...
@brian-brazil
Copy link
Contributor

This package is pure python, would it make a difference?

@gjcarneiro
Copy link
Author

Yes, it makes a huge difference, installation from Python wheel, even for pure Python, is so much faster, as installation is little more than downloading a zip file and uncompressing it directly to the final destination. A normal package installation, not only needs uncompressing to a temporary src directory, then it needs to execute python setup.py install, which is a lot slower.

https://pythonwheels.com

@brian-brazil
Copy link
Contributor

Can you quantify "a lot"? 200kB doesn't take a lot to process.

@xavfernandez
Copy link
Contributor
xavfernandez commented Sep 11, 2019 •
edited
Loading

It mainly saves pip from spawning a process to launch python setup.py bdist_wheel and automagically create the wheel :)

(tmp-f8893c0a100797) $ pip uninstall prometheus-client -y
Uninstalling prometheus-client-0.7.1:
 Successfully uninstalled prometheus-client-0.7.1
(tmp-f8893c0a100797) $ time pip install --no-cache-dir --no-index ~/Downloads/prometheus_client-0.7.1.tar.gz
Processing /home/xfernandez/Downloads/prometheus_client-0.7.1.tar.gz
Building wheels for collected packages: prometheus-client
 Building wheel for prometheus-client (setup.py) ... done
 Created wheel for prometheus-client: filename=prometheus_client-0.7.1-cp37-none-any.whl size=41402 sha256=66668319af083af049349c7e8e81f7def84260e36c9e0968b753e27fefd5c421
 Stored in directory: /tmp/pip-ephem-wheel-cache-sh8aaupe/wheels/c3/4f/4e/65ac82a9f5c3008dee932dbf792e6cf5f9a3ae9de9031735a9
Successfully built prometheus-client
Installing collected packages: prometheus-client
Successfully installed prometheus-client-0.7.1

real    0m0.760s
user    0m0.677s
sys     0m0.083s
(tmp-f8893c0a100797) $ pip uninstall prometheus-client -y
Uninstalling prometheus-client-0.7.1:
 Successfully uninstalled prometheus-client-0.7.1
(tmp-f8893c0a100797) [xfernandez@xaferiba tmp-f8893c0a100797]$ time pip install --no-cache-dir --no-index ~/Downloads/prometheus_client-0.7.1-py3-none-any.whl 
Processing /home/xfernandez/Downloads/prometheus_client-0.7.1-py3-none-any.whl
Installing collected packages: prometheus-client
Successfully installed prometheus-client-0.7.1

real    0m0.370s
user    0m0.321s
sys     0m0.047s

(the wheel was produced via pip wheel ~/Downloads/prometheus_client-0.7.1.tar.gz -w ~/Downloads/)

@xavfernandez
Copy link
Contributor

Since pip will automatically build the wheel at the first install, it would save quite a lot of computation (~300 ms multiplied by every fresh install) to build the wheel once during the release and upload it to PyPI.org.

This would mean switching from:

git checkout <tag>
pip install -U setuptools twine
python setup.py sdist
twine upload dist/prometheus_client-<version>.tar.gz

to

git checkout <tag>
pip install -U setuptools twine wheel
python setup.py sdist bdist_wheel --universal
twine upload dist/prometheus_client-<version>.tar.gz dist/prometheus_client-<version>-py2.py3-none-any.whl

@brian-brazil
Copy link
Contributor

I'm not sure complicating our build process is worth saving 300ms of install cost.

@xavfernandez
Copy link
Contributor

I'm not sure complicating our build process is worth saving 300ms of install cost.

It's 300 ms multiplied by the 100k+ installs of prometheus-client that happen everyday according to https://pypistats.org/packages/prometheus-client.
So providing a pure-python wheel could potentially save several hours of computation everyday :)

If you could point me to your build process, I'd be more than pleased to improve it.

@brian-brazil
Copy link
Contributor

I build and release by hand.

@gjcarneiro
Copy link
Author 8000

Indeed, it's usually trivial to build/upload a wheel: if now you are doing python setup.py sdist upload, just change to python setup.py sdist bdist_wheel upload. You may also need to create a setup.cfg file containing:

[bdist_wheel]
universal = 1

This tells it that the built wheel is pure Python and works in either Python 2 or Python 3 without changes.

@xavfernandez xavfernandez mentioned this issue Nov 14, 2019
Merged
xavfernandez added a commit to Polyconseil/client_python that referenced this issue Nov 14, 2019
@xavfernandez
Update setup.py to produce universal wheels
9f06113 
To ease the transition to prometheus#457:

    `python setup.py sdist bdist_wheel`

will produce the universal wheel in addition to the sdist.

Signed-off-by: Xavier Fernandez <xavier.fernandez@polyconseil.fr>
@xavfernandez
Copy link
Contributor

I've opened #487 if you don't like the additional setup.cfg.

As @gjcarneiro explained, if you're currently using python setup.py sdist upload you'd just have to switch to python setup.py sdist bdist_wheel upload (either with a setup.cfg file or with #487 merged).

brian-brazil pushed a commit that referenced this issue Nov 14, 2019
@xavfernandez @brian-brazil
Update setup.py to produce universal wheels (#487)
1521d81 
To ease the transition to #457:

    `python setup.py sdist bdist_wheel`

will produce the universal wheel in addition to the sdist.

Signed-off-by: Xavier Fernandez <xavier.fernandez@polyconseil.fr>
@ehashman
Copy link
ehashman commented Mar 9, 2020

Just wanted to echo a +1 for this. I noticed prometheus-client was not distributing wheels today when I was reviewing the list of projects without wheels on https://pythonwheels.com.

The longer-form advantages of wheels over sdists are elaborated upon in the wheel PEP: https://www.python.org/dev/peps/pep-0427/#rationale

Distributing wheels is only one extra step as part of the build and release process (include a bdist_wheel target and twine or whatever upload tool you use will take care of the rest), and provides a lot of advantages for end users, even in the case of pure Python wheels like this one 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@xavfernandez @gjcarneiro @ehashman @brian-brazil
0