Add experimental support for building wheels. #128
Conversation
| mandatory=True, | ||
| allow_empty=False, | ||
| doc="""\ | ||
| List of Python packages to include in the distribution. |
There was a problem hiding this comment.
This is controversial and perhaps worth discussing.
A typical package will have dependencies on other packages; some might be first party but packaged separately, some might be third party. From bazel point of view they will probably form a tree of py_library dependencies.
When creating a wheel package one does not want to include the whole tree, so we need to tell bazel where to cut the chain. I see three ways to do that:
- Don't follow dependencies. Only package up the libraries listed directly in deps.
- Follow dependencies and only package specific python packages
a) Including sub-packages, so e.g. if "foo" is listed then "foo.bar" gets packaged too
b) Not including sub-packages.
The implementation here does "2a", but I could see how other options could be preferred. I'm happy to change if you think other options are better.
|
@brandjon: any chances somebody with commit rights takes a look? Thanks! |
|
Would love to see this make it into the repo. Any progress here? Thanks |
|
Still waiting for the maintainers to respond.
Did you try out this patch? Did it work for you? Any issues? I'm currently
aware of encoding issues if one of the metadata fields (eg. author)
contains non-ascii letters that I still need to address, though I think
there's value in the patch even as it is now.
pon., 14 sty 2019, 16:40: Dan Sully <notifications@github.com> napisał(a):
… Would love to see this make it into the repo. Any progress here?
Thanks
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#128 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AqnE8YLsy6-49YJZorIWrtoSi9ZXIWZhks5vDKTPgaJpZM4YMTgk>
.
|
|
There's still some issues on systems where default python is py2, I'll try to get them fixed by the end of the week. |
Fixes: #7
Quoting @duggelz > From PEP 508 it appears that you can register packages with any capitalization, and install them with any capitalization, even if it's different. I.e. you can register package 'MyPackage' and install it with mypackage, MYPACKAGE, or even mYpAcKaGe. > > That is not an ideal spec, but is it the spec, and the tools implement it, so we should to.
`boto3`s `metadata.json` file doesn't delimit the version specifier the way other packages I've come across seem to, so support splitting on the kinds of characters this variation might foist upon us. Fixes: #20
The warning is: You are using pip version 1.5.4, however version 9.0.1 is avai 10000 lable. You should consider upgrading via the 'pip install --upgrade pip' command. In reality, pip is at 9.0.1, but its bundled version of pkg_resources doesn't know that. See google/subpar#38
Use the skylark `fail` function instead of letting Python throw a key-not-found exception.
This also fixes a couple bugs I hit after regenerating piptool.par: - os.environ['PYTHONPATH'] may result in a KeyError - The cert expansion logic was removed, but things fail without it (after I ./update_tools.sh). I thought this was intentional (should the PAR changes have fixes this?), but I'm restoring for now to keep HEAD in a good state.
"Extras" are additional dependencies of a given library, which are consumed by passing the "extra" name in brackets after the distribution name, for example: ``` mock[docs]==1.0.1 ``` We see this in the dependencies of several Google Cloud libraries, which depend on: `googleapis_common_protos[grpc]` I've added a simple test that the dependency structure we synthesize for this kind of thing is correct via an "extras" test that has a `requirements.txt` of: ``` google-cloud-language==0.27.0 ``` Fixes: #12
* Evaluate PEP 508 environment markers for package dependencies Previously any wheel dependencies that had an environment marker (such as 'python_version>3.3') were simply ignored, leading to missing packages in the Python environment constructed by bazel. Fixes #49 * Regenerate the piptool.par Required after making changes to whl.py * Pin the version of setuptools in piptool & extract whltool Some common operators in version markers (e.g., <=) are only supported in setuptools>=17.1. Rather than risk failing because the environment has an old setuptools version it's better to include it. Pinning to an exact version (currently the latest) to make things as predictable as possible. In addition, whl.py used during workspace setup also now depends on setuptools. We package this in a separate whltool.par to make this predictable as well.
* Use posixpath to build internal zip file paths * Use literal / instead of posixpath * Update piptool.par and whltool.par
Add proposals directory and proposal for 2/3 mode
It was already in review but I didn't realize that was considered a status change.
The test is brittle and shouldn't be used as a security guarantee anyway. See #117 (comment) for more context. Fixes #117.
This upgrades our dependencies to newer versions that are compatible with the new repo rules, and handles the fallout. Also add some comments to WORKSPACE. rules_python now builds under Bazel 0.19.1 with `--incompatible_remove_native_git_repository` and `incompatible_remove_native_http_archive`. Fixes #105.
Remaining issues are caused by subpar
Tested: bazel-0.21 test //... --all_incompatible_changes Fixes #134
Currently only pure python wheels are supported. This extension builds wheels directly, invoking a simple python script that creates the zip archive in desired format instead of using distutils/setuptools. This will make building platform-dependent wheels easier in the future, as bazel will have full control on how extension code is built.
|
So there's good news and bad news. 👍 The good news is that everyone that needs to sign a CLA (the pull request submitter and all commit authors) have done so. Everything is all good there. 😕 The bad news is that it appears that one or more commits were authored or co-authored by someone other than the pull request submitter. We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that here in the pull request. Note to project maintainer: This is a terminal state, meaning the |
|
|
Going to try with a new pull request to avoid all the merge mess here: |
Currently only pure python wheels are supported.
This extension builds wheels directly, invoking a simple python script that creates the zip archive in the desired format instead of using distutils/setuptools.
This will make building platform-dependent wheels easier in the future, as bazel
will have full control on how extension code is built.