8000 feat(bzlmod): Calling pip multiple times · bazel-contrib/rules_python@76aab0f · GitHub
[go: up one dir, main page]
Skip to content < 8000 script crossorigin="anonymous" type="application/javascript" src="https://github.githubassets.com/assets/keyboard-shortcuts-dialog-cf9f9950f389.js" defer="defer">

Commit 76aab0f

Browse files
chrislovecnmchrislovecnm
committed
feat(bzlmod): Calling pip multiple times
Implementing the capability to call pip bzlmod extension multiple times. We can now call the pip extension with the same hub name multiple times. This allows a user to have multiple different requirement files based on the Python version. feat(bzlmod): removing incompatible aliases flag for bzlmod With workspace, we need incompible aliases, because you get @pip//tabulate or @pip_tabulate//:pkg. The requirement() macro hides this, but if you're manually referencing things, then changing the flag ends up being a breaking change With bzlmod, though, the @pip_tabulate style isn't a realistic option (you'd have to use_repo everything, a huge pain). So we have choosen to always have `@pip//tabulate`. feat(bzlmod): Implement automatic python version detection This commit implements that capability for pip.parse to programtically determine the python version from the 10000 provided interpreter.
1 parent 3903d1a commit 76aab0f

23 files changed

+1294
-165
lines changed

‎.bazelrc

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,8 @@
33
# This lets us glob() up all the files inside the examples to make them inputs to tests
44
# (Note, we cannot use `common --deleted_packages` because the bazel version command doesn't support it)
55
# To update these lines, run tools/bazel_integration_test/update_deleted_packages.sh
6-
build --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_point,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_install,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,tests/compile_pip_requirements,tests/compile_pip_requirements_test_from_external_workspace,tests/ignore_root_user_error,tests/pip_repository_entry_points
7-
query --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_point,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_install,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,tests/compile_pip_requirements,tests/compile_pip_requirements_test_from_external_workspace,tests/ignore_root_user_error,tests/pip_repository_entry_points
6+
build --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod/entry_point,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_install,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,tests/compile_pip_requirements,tests/compile_pip_requirements_test_from_external_workspace,tests/ignore_root_user_error,tests/pip_repository_entry_points
7+
query --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod/entry_point,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_install,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,tests/compile_pip_requirements,tests/compile_pip_requirements_test_from_external_workspace,tests/ignore_root_user_error,tests/pip_repository_entry_points
88

99
test --test_output=errors
1010

‎BZLMOD_SUPPORT.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -31,8 +31,8 @@ A second example, in [examples/bzlmod_build_file_generation](examples/bzlmod_bui
3131

3232
This rule set does not have full feature partity with the older `WORKSPACE` type configuration:
3333

34-
1. Multiple pip extensions are not yet supported, as demonstrated in [this](examples/multi_python_versions) example.
35-
2. Gazelle does not support finding deps in sub-modules. For instance we can have a dep like ` "@our_other_module//other_module/pkg:lib",` in a `py_test` definition.
34+
1. Gazelle does not support finding deps in sub-modules. For instance we can have a dep like ` "@our_other_module//other_module/pkg:lib",` in a `py_test` definition.
35+
2. We have some features that are still not fully flushed out, and the user interface may change.
3636

3737
Check ["issues"](/bazelbuild/rules_python/issues) for an up to date list.
3838

‎README.md

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -96,8 +96,7 @@ use_repo(interpreter, "interpreter")
9696

9797
pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")
9898
pip.parse(
99-
name = "pip",
100-
incompatible_generate_aliases = True,
99+
hub_name = "pip",
101100
python_interpreter_target = "@interpreter//:python",
102101
requirements_lock = "//:requirements_lock.txt",
103102
requirements_windows = "//:requirements_windows.txt",
@@ -201,7 +200,7 @@ central external repo and individual wheel external repos.
201200

202201
```python
203202
pip.parse(
204-
name = "my_deps",
203+
hub_name = "my_deps",
205204
requirements_lock = "//:requirements_lock.txt",
206205
)
207206

‎docs/pip_repository.md

Lines changed: 23 additions & 4 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎examples/bzlmod/BUILD.bazel

Lines changed: 16 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -8,17 +8,28 @@
88
load("@bazel_skylib//rules:build_test.bzl", "build_test")
99
load("@pip//:requirements.bzl", "all_requirements", "all_whl_requirements", "requirement")
1010
load("@python_3_9//:defs.bzl", py_test_with_transition = "py_test")
11+
load("@python_aliases//3.10:defs.bzl", compile_pip_requirements_3_10 = "compile_pip_requirements")
12+
load("@python_aliases//3.9:defs.bzl", compile_pip_requirements_3_9 = "compile_pip_requirements")
1113
load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test")
12-
load("@rules_python//python:pip.bzl", "compile_pip_requirements")
1314

1415
# This stanza calls a rule that generates targets for managing pip dependencies
1516
# with pip-compile.
16-
compile_pip_requirements(
17-
name = "requirements",
17+
compile_pip_requirements_3_9(
18+
name = "requirements_3_9",
1819
extra_args = ["--allow-unsafe"],
1920
requirements_in = "requirements.in",
20-
requirements_txt = "requirements_lock.txt",
21-
requirements_windows = "requirements_windows.txt",
21+
requirements_txt = "requirements_lock_3_9.txt",
22+
requirements_windows = "requirements_windows_3_9.txt",
23+
)
24+
25+
# This stanza calls a rule that generates targets for managing pip dependencies
26+
# with pip-compile.
27+
compile_pip_requirements_3_10(
28+
name = "requirements_3_10",
29+
extra_args = ["--allow-unsafe"],
30+
requirements_in = "requirements.in",
31+
requirements_txt = "requirements_lock_3_10.txt",
32+
requirements_windows = "requirements_windows_3_10.txt",
2233
)
2334

2435
# The rules below are language specific rules defined in

‎examples/bzlmod/MODULE.bazel

Lines changed: 16 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@ python.toolchain(
4343

4444
# You only need to load this repositories if you are using muiltple Python versions.
4545
# See the tests folder for various examples.
46-
use_repo(python, PYTHON_NAME_39, "python_aliases")
46+
use_repo(python, PYTHON_NAME_39, PYTHON_NAME_310, "python_aliases")
4747

4848
# The interpreter extension discovers the platform specific Python binary.
4949
# It creates a symlink to the binary, and we pass the label to the following
@@ -63,16 +63,24 @@ interpreter.install(
6363
use_repo(interpreter, INTERPRETER_NAME_39, INTERPRETER_NAME_310)
6464

6565
pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")
66+
67+
# For each pip setup we call pip.parse. We can pass in various options
68+
# but typically we are passing in a requirements and an interpreter.
69+
# If you provide the python_interpreter_target we attempt to determine
70+
# the python_version automatically. Otherwise include the python_version.
6671
pip.parse(
67-
name = "pip",
68-
# Intentionally set it false because the "true" case is already
69-
# covered by examples/bzlmod_build_file_generation
70-
incompatible_generate_aliases = False,
72+
hub_name = "pip",
7173
python_interpreter_target = "@{}//:python".format(INTERPRETER_NAME_39),
72-
requirements_lock = "//:requirements_lock.txt",
73-
requirements_windows = "//:requirements_windows.txt",
74+
requirements_lock = "//:requirements_lock_3_9.txt",
75+
requirements_windows = "//:requirements_windows_3_9.txt",
76+
)
77+
pip.parse(
78+
hub_name = "pip",
79+
python_interpreter_target = "@{}//:python".format(INTERPRETER_NAME_310),
80+
requirements_lock = "//:requirements_lock_3_10.txt",
81+
requirements_windows = "//:requirements_windows_3_10.txt",
7482
)
75-
use_repo(pip, "pip")
83+
use_repo(pip, "pip", "pip_39")
7684

7785
bazel_dep(name = "other_module", version = "", repo_name = "our_other_module")
7886
local_path_override(

‎examples/bzlmod/entry_point/BUILD.bazel

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
load("@pip//:requirements.bzl", "entry_point")
1+
load("@pip_39//:requirements.bzl", "entry_point")
22
load("@rules_python//python:defs.bzl", "py_test")
33

44
alias(

‎examples/bzlmod/libs/my_lib/BUILD.bazel

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
load("@pip//:requirements.bzl", "requirement")
2+
load("@rules_python//python:defs.bzl", "py_library")
3+
4+
py_library(
5+
name = "my_lib",
6+
srcs = ["__init__.py"],
7+
visibility = ["@//tests:__pkg__"],
8+
deps = [requirement("websockets")],
9+
)

‎examples/bzlmod/libs/my_lib/__init__.py

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
# Copyright 2023 The Bazel Authors. All rights reserved.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
import websockets
16+
17+
18+
def websockets_is_for_python_version(sanitized_version_check):
19+
# We are checking that the name of the repository folders
20+
# match the expexted generated names. If we update the folder
21+
# structure or naming we will need to modify this test
22+
return f"pip_{sanitized_version_check}_websockets" in websockets.__file__

‎examples/bzlmod/requirements.in

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
websockets
12
requests~=2.25.1
23
s3cmd~=2.1.0
34
yamllint>=1.28.0

0 commit comments

Comments
 (0)
0