8000 Use PEP 508 rules when setting deps from extras (#724) · bazel-contrib/rules_python@dae610b · GitHub
[go: up one dir, main page]
Skip to content

Commit dae610b

Browse files
mattoberleJonathon Belottialexeagle
authored
Use PEP 508 rules when setting deps from extras (#724)
* Use PEP 426 rules when setting deps from extras This commit addresses issue #720. [PEP 426](https://peps.python.org/pep-0426/#name) states that distribution names are case-insensitive and "-" is interchangeable with "_". The `pip-compile` command creates a lockfile where all package names are lowercase. The tool may also modify interchangeable characters. The following examples are all valid `requirements.txt` or `requirements_lock.txt` entries: ``` SQLAlchemy[postgresql_psycopg2binary]==1.4.36 sqlalchemy[postgresql_psycopg2binary]==1.4.36 sentry_sdk[flask]==1.5.8 sentry-sdk[flask]==1.5.8 ``` A distribution's `METADATA` file contains the stylization chosen by the publisher. By applying a "sanitise" function when building the `extras` dict and when performing lookups we can eliminate this difference as a concern. * Use PEP 503 rules when sanitising extras * Normalize distribution name with pkg_resources `pypa/installer` is used to parse Wheel metadata, but does not currently provide a method for normalizing distribution names: - pypa/installer#97 `pypa/pkg_resources` provides `Requirement.parse` which returns an instance of `Requirement` where `.key` is the canonical distribution name per PEP 503. The `Requirement` class can also parse `extras`, but it returns a normalized form that I believe could break the installation of the extras. * Use Requirement.parse to populate extra reqs * Revert "Use Requirement.parse to populate extra reqs" This reverts commit f0faa97. * Test for distribution name normalization in extras * Replace pkg_resources with packaging.utils This replaces `pkg_resources.Requirement.parse` with `packaging.utils.canonicalize_name`. Doing this pulls in a vendored requirement from `pip`, which may be undesirable. The code we want is just: ``` re.sub(r"[-_.]+", "-", name).lower() ``` This commit also leaves a reference to `pkg_resources` in `wheel.py` which does not canonicalize the name. Co-authored-by: Jonathon Belotti <jonathon@canva.com> Co-authored-by: Alex Eagle <alex@aspect.dev>
1 parent c98bc8f commit dae610b

File tree

3 files changed

+8
-2
lines changed

3 files changed

+8
-2
lines changed

‎python/pip_install/extract_wheels/lib/requirements.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
import re
22
from typing import Dict, Optional, Set, Tuple
33

4+
from pip._vendor.packaging.utils import canonicalize_name
5+
46

57
def parse_extras(requirements_path: str) -> Dict[str, Set[str]]:
68
"""Parse over the requirements.txt file to find extras requested.
@@ -38,7 +40,7 @@ def _parse_requirement_for_extra(
3840
matches = extras_pattern.match(requirement)
3941
if matches:
4042
return (
41-
matches.group(1),
43+
canonicalize_name(matches.group(1)),
4244
{extra.strip() for extra in matches.group(2).split(",")},
4345
)
4446

‎python/pip_install/extract_wheels/lib/requirements_test.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,8 @@ def test_parses_requirement_for_extra(self) -> None:
99
("name[foo]", ("name", frozenset(["foo"]))),
1010
("name[ Foo123 ]", ("name", frozenset(["Foo123"]))),
1111
(" name1[ foo ] ", ("name1", frozenset(["foo"]))),
12+
("Name[foo]", ("name", frozenset(["foo"]))),
13+
("name_foo[bar]", ("name-foo", frozenset(["bar"]))),
1214
(
1315
"name [fred,bar] @ http://foo.com ; python_version=='2.7'",
1416
("name", frozenset(["fred", "bar"])),

‎python/pip_install/extract_wheels/lib/wheel.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@
88

99
import installer
1010
import pkg_resources
11+
from pip._vendor.packaging.utils import canonicalize_name
1112

1213

1314
def current_umask() -> int:
@@ -38,7 +39,8 @@ def path(self) -> str:
3839
@property
3940
def name(self) -> str:
4041
# TODO Also available as installer.sources.WheelSource.distribution
41-
return str(self.metadata['Name'])
42+
name = str(self.metadata['Name'])
43+
return canonicalize_name(name)
4244

4345
@property
4446
def metadata(self) -> email.message.Message:

0 commit comments

Comments
 (0)
0