8000 cleanup: Typos, doc, and formatting cleanup in pip extension et al (#… · matts1/rules_python@cc43594 · GitHub
[go: up one dir, main page]
Skip to content

Commit cc43594

Browse files
rickeylevrickeylev
authored
cleanup: Typos, doc, and formatting cleanup in pip extension et al (bazel-contrib#1275)
* Corrects some typos in docs * Expands the user-facing documentation * Fixes errors having newlines in the middle of them * Renames some internal functions to be more self-describing.
1 parent 1a333ce commit cc43594

File tree

4 files changed

+92
-50
lines changed

4 files changed

+92
-50
lines changed

‎docs/pip_repository.md

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎examples/bzlmod/MODULE.bazel

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -34,16 +34,17 @@ python.toolchain(
3434
# You only need to load this repositories if you are using multiple Python versions.
3535
# See the tests folder for various examples on using multiple Python versions.
3636
# The names "python_3_9" and "python_3_10" are autmatically created by the repo
37-
# rules based on the python_versions.
37+
# rules based on the `python_version` arg values.
3838
use_repo(python, "python_3_10", "python_3_9", "python_aliases")
3939

4040
pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip")
4141

42-
# For each pip setup we call pip.parse. We can pass in various options
43-
# but typically we are passing in a requirements and an interpreter.
44-
# If you provide the python_version we will attempt to determine
45-
# the interpreter target automatically. Otherwise use python_interpreter_target
46-
# to override the lookup.
42+
# To fetch pip dependencies, use pip.parse. We can pass in various options,
43+
# but typically we pass requirements and the Python version. The Python
44+
# version must have been configured by a corresponding `python.toolchain()`
45+
# call.
46+
# Alternatively, `python_interpreter_target` can be used to directly specify
47+
# the Python interpreter to run to resolve dependencies.
4748
pip.parse(
4849
hub_name = "pip",
4950
python_version = "3.9",
@@ -57,7 +58,8 @@ pip.parse(
5758
requirements_windows = "//:requirements_windows_3_10.txt",
5859
)
5960

60-
# we use the pip_39 repo because entry points are not yet supported.
61+
# NOTE: The pip_39 repo is only used because the plain `@pip` repo doesn't
62+
# yet support entry points; see https://github.com/bazelbuild/rules_python/issues/1262
6163
use_repo(pip, "pip", "pip_39")
6264

6365
bazel_dep(name = "other_module", version = "", repo_name = "our_other_module")

‎python/extensions/pip.bzl

Lines changed: 81 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -27,21 +27,25 @@ load(
2727
)
2828
load("@rules_python//python/pip_install:requirements_parser.bzl", parse_requirements = "parse")
2929

30-
def _create_pip(module_ctx, pip_attr, whl_map):
30+
def _create_versioned_pip_and_whl_repos(module_ctx, pip_attr, whl_map):
3131
python_interpreter_target = pip_attr.python_interpreter_target
3232

3333
# if we do not have the python_interpreter set in the attributes
3434
# we programtically find it.
35+
hub_name = pip_attr.hub_name
3536
if python_interpreter_target == None:
3637
python_name = "python_{}".format(pip_attr.python_version.replace(".", "_"))
3738
if python_name not in INTERPRETER_LABELS.keys():
38-
fail("""
39-
Unable to find '{}' in the list of interpreters please update your pip.parse call with the correct python name
40-
""".format(pip_attr.python_name))
41-
39+
fail((
40+
"Unable to find interpreter for pip hub '{hub_name}' for " +
41+
"python_version={version}: Make sure a corresponding " +
42+
'`python.toolchain(python_version="{version}")` call exists'
43+
).format(
44+
hub_name = hub_name,
45+
version = pip_attr.python_version,
46+
))
4247
python_interpreter_target = INTERPRETER_LABELS[python_name]
4348

44-
hub_name = pip_attr.hub_name
4549
pip_name = hub_name + "_{}".format(pip_attr.python_version.replace(".", ""))
4650
requrements_lock = locked_requirements_label(module_ctx, pip_attr)
4751

@@ -93,10 +97,10 @@ Unable to find '{}' in the list of interpreters please update your pip.parse cal
9397
whl_map[hub_name][whl_name][pip_attr.python_version] = pip_name + "_"
9498

9599
def _pip_impl(module_ctx):
96-
"""Implmentation of a class tag that creates the pip hub(s) and corresponding pip spoke, alias and whl repositories.
100+
"""Implementation of a class tag that creates the pip hub(s) and corresponding pip spoke, alias and whl repositories.
97101
98-
This implmentation iterates through all of the "pip.parse" calls and creates
99-
different pip hubs repositories based on the "hub_name". Each of the
102+
This implmentation iterates through all of the `pip.parse` calls and creates
103+
different pip hub repositories based on the "hub_name". Each of the
100104
pip calls create spoke repos that uses a specific Python interpreter.
101105
102106
In a MODULES.bazel file we have:
@@ -115,13 +119,13 @@ def _pip_impl(module_ctx):
115119
)
116120
117121
118-
For instance we have a hub with the name of "pip".
122+
For instance, we have a hub with the name of "pip".
119123
A repository named the following is created. It is actually called last when
120124
all of the pip spokes are collected.
121125
122126
- @@rules_python~override~pip~pip
123127
124-
As show in the example code above we have the following.
128+
As shown in the example code above we have the following.
125129
Two different pip.parse statements exist in MODULE.bazel provide the hub_name "pip".
126130
These definitions create two different pip spoke repositories that are
127131
related to the hub "pip".
@@ -185,22 +189,32 @@ def _pip_impl(module_ctx):
185189

186190
for mod in module_ctx.modules:
187191
for pip_attr in mod.tags.parse:
188-
if pip_attr.hub_name in pip_hub_map:
192+
hub_name = pip_attr.hub_name
193+
if hub_name in pip_hub_map:
189194
# We cannot have two hubs with the same name in different
190195
# modules.
191-
if pip_hub_map[pip_attr.hub_name].module_name != mod.name:
192-
fail("""Unable to create pip with the hub_name '{}', same hub name
193-
in a different module found.""".format(pip_attr.hub_name))
194-
195-
if pip_attr.python_version in pip_hub_map[pip_attr.hub_name].python_versions:
196-
fail(
197-
"""Unable to create pip with the hub_name '{}', same hub name
198-
using the same Python repo name '{}' found in module '{}'.""".format(
199-
pip_attr.hub_name,
200-
pip_attr.python_version,
201-
mod.name,
202-
),
203-
)
196+
if pip_hub_map[hub_name].module_name != mod.name:
197+
fail((
198+
"Duplicate cross-module pip hub named '{hub}': pip hub " +
199+
"names must be unique across modules. First defined " +
200+
"by module '{first_module}', second attempted by " +
201+
"module '{second_module}'"
202+
).format(
203+
hub = hub_name,
204+
first_module = pip_hub_map[hub_name].module_name,
205+
second_module = mod.name,
206+
))
207+
208+
if pip_attr.python_version in pip_hub_map[hub_name].python_versions:
209+
fail((
210+
"Duplicate pip python version '{version}' for hub " +
211+
"'{hub}' in module '{module}': the Python versions " +
212+
"used for a hub must be unique"
213+
).format(
214+
hub = hub_name,
215+
module = mod.name,
216+
version = pip_attr.python_version,
217+
))
204218
else:
205219
pip_hub_map[pip_attr.hub_name].python_versions.append(pip_attr.python_version)
206220
else:
@@ -209,17 +223,19 @@ def _pip_impl(module_ctx):
209223
python_versions = [pip_attr.python_version],
210224
)
211225

212-
_create_pip(module_ctx, pip_attr, hub_whl_map)
226+
_create_versioned_pip_and_whl_repos(module_ctx, pip_attr, hub_whl_map)
213227

214228
for hub_name, whl_map in hub_whl_map.items():
215229
for whl_name, version_map in whl_map.items():
216230
if DEFAULT_PYTHON_VERSION not in version_map:
217-
fail(
218-
"""
219-
Unable to find the default python version in the version map, please update your requirements files
220-
to include Python '{}'.
221-
""".format(DEFAULT_PYTHON_VERSION),
222-
)
231+
fail((
232+
"Default python version '{version}' missing in pip " +
233+
"hub '{hub}': update your pip.parse() calls so that " +
234+
'includes `python_version = "{version}"`'
235+
).format(
236+
version = DEFAULT_PYTHON_VERSION,
237+
hub = hub_name,
238+
))
223239

224240
# Create the alias repositories which contains different select
225241
# statements These select statements point to the different pip
@@ -247,14 +263,34 @@ def _pip_parse_ext_attrs():
247263
"hub_name": attr.string(
248264
mandatory = True,
249265
doc = """
250-
The unique hub name. Mulitple pip.parse calls that contain the same hub name,
251-
create spokes for specific Python versions.
266+
The name of the repo pip dependencies will be accessible from.
267+
268+
This name must be unique between modules; unless your module is guaranteed to
269+
always be the root module, it's highly recommended to include your module name
270+
in the hub name. Repo mapping, `use_repo(..., pip="my_modules_pip_deps")`, can
271+
be used for shorter local names within your module.
272+
273+
Within a module, the same `hub_name` can be specified to group different Python
274+
versions of pip dependencies under one repository name. This allows using a
275+
Python version-agnostic name when referring to pip dependencies; the
276+
correct version will be automatically selected.
277+
278+
Typically, a module will only have a single hub of pip dependencies, but this
279+
is not required. Each hub is a separate resolution of pip dependencies. This
280+
means if different programs need different versions of some library, separate
281+
hubs can be created, and each program can use its respective hub's targets.
282+
Targets from different hubs should not be used together.
252283
""",
253284
),
254285
"python_version": attr.string(
255286
mandatory = True,
256287
doc = """
257-
The Python version for the pip spoke.
288+
The Python version to use for resolving the pip dependencies. If not specified,
289+
then the default Python version (as set by the root module or rules_python)
290+
will be used.
291+
292+
The version specified here must have a corresponding `python.toolchain()`
293+
configured.
258294
""",
259295
),
260296
}, **pip_repository_attrs)
@@ -270,12 +306,16 @@ The Python version for the pip spoke.
270306

271307
pip = module_extension(
272308
doc = """\
273-
This extension is used to create a pip hub and all of the spokes that are part of that hub.
274-
We can have multiple different hubs, but we cannot have hubs that have the same name in
275-
different modules. Each hub needs one or more spokes. A spoke contains a specific version
276-
of Python, and the requirement(s) files that are unquie to that Python version.
277-
In order to add more spokes you call this extension mulitiple times using the same hub
278-
name.
309+
This extension is used to make dependencies from pip available.
310+
311+
To use, call `pip.parse()` and specify `hub_name` and your requirements file.
312+
Dependencies will be downloaded and made available in a repo named after the
313+
`hub_name` argument.
314+
315+
Each `pip.parse()` call configures a particular Python version. Multiple calls
316+
can be made to configure different Python versions, and will be grouped by
317+
the `hub_name` argument. This allows the same logical name, e.g. `@pip//numpy`
318+
to automatically resolve to different, Python version-specific, libraries.
279319
""",
280320
implementation = _pip_impl,
281321
tag_classes = {

‎python/pip_install/pip_repository.bzl

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -363,7 +363,7 @@ pip_hub_repository_bzlmod_attrs = {
363363

364364
pip_hub_repository_bzlmod = repository_rule(
365365
attrs = pip_hub_repository_bzlmod_attrs,
366-
doc = """A rule for bzlmod mulitple pip repository creation. Intended for private use only.""",
366+
doc = """A rule for bzlmod mulitple pip repository creation. PRIVATE USE ONLY.""",
367367
implementation = _pip_hub_repository_bzlmod_impl,
368368
)
369369

0 commit comments

Comments
 (0)
0