8000 bpo-34632: Add importlib.metadata (GH-12547) · python/cpython@1bbf7b6 · GitHub
[go: up one dir, main page]
Skip to content

Commit 1bbf7b6

Browse files
jaracowarsaw
jaraco
authored and
warsaw
committed
bpo-34632: Add importlib.metadata (GH-12547)
Add importlib.metadata module as forward port of the standalone importlib_metadata.
1 parent 6dbbe74 commit 1bbf7b6

File tree

15 files changed

+2048
-638
lines changed

15 files changed

+2048
-638
lines changed

‎Doc/library/importlib.metadata.rst

Lines changed: 257 additions & 0 deletions
9E88
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,257 @@
1+
.. _using:
2+
3+
==========================
4+
Using importlib.metadata
5+
==========================
6+
7+
.. note::
8+
This functionality is provisional and may deviate from the usual
9+
version semantics of the standard library.
10+
11+
``importlib.metadata`` is a library that provides for access to installed
12+
package metadata. Built in part on Python's import system, this library
13+
intends to replace similar functionality in the `entry point
14+
API`_ and `metadata API`_ of ``pkg_resources``. Along with
15+
``importlib.resources`` in `Python 3.7
16+
and newer`_ (backported as `importlib_resources`_ for older versions of
17+
Python), this can eliminate the need to use the older and less efficient
18+
``pkg_resources`` package.
19+
20+
By "installed package" we generally mean a third-party package installed into
21+
Python's ``site-packages`` directory via tools such as `pip
22+
<https://pypi.org/project/pip/>`_. Specifically,
23+
it means a package with either a discoverable ``dist-info`` or ``egg-info``
24+
directory, and metadata defined by `PEP 566`_ or its older specifications.
25+
By default, package metadata can live on the file system or in zip archives on
26+
``sys.path``. Through an extension mechanism, the metadata can live almost
27+
anywhere.
28+
29+
30+
Overview
31+
========
32+
33+
Let's say you wanted to get the version string for a package you've installed
34+
using ``pip``. We start by creating a virtual environment and installing
35+
something into it::
36+
37+
.. highlight:: none
38+
39+
$ python3 -m venv example
40+
$ source example/bin/activate
41+
(example) $ pip install wheel
42+
43+
You can get the version string for ``wheel`` by running the following::
44+
45+
.. highlight:: none
46+
47+
(example) $ python
48+
>>> from importlib.metadata import version # doctest: +SKIP
49+
>>> version('wheel') # doctest: +SKIP
50+
'0.32.3'
51+
52+
You can also get the set of entry points keyed by group, such as
53+
``console_scripts``, ``distutils.commands`` and others. Each group contains a
54+
sequence of :ref:`EntryPoint <entry-points>` objects.
55+
56+
You can get the :ref:`metadata for a distribution <metadata>`::
57+
58+
>>> list(metadata('wheel')) # doctest: +SKIP
59+
['Metadata-Version', 'Name', 'Version', 'Summary', 'Home-page', 'Author', 'Author-email', 'Maintainer', 'Maintainer-email', 'License', 'Project-URL', 'Project-URL', 'Project-URL', 'Keywords', 'Platform', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Requires-Python', 'Provides-Extra', 'Requires-Dist', 'Requires-Dist']
60+
61+
You can also get a :ref:`distribution's version number <version>`, list its
62+
:ref:`constituent files <files>`, and get a list of the distribution's
63+
:ref:`requirements`.
64+
65+
66+
Functional API
67+
==============
68+
69+
This package provides the following functionality via its public API.
70+
71+
72+
.. _entry-points:
73+
74+
Entry points
75+
------------
76+
77+
The ``entry_points()`` function returns a dictionary of all entry points,
78+
keyed by group. Entry points are represented by ``EntryPoint`` instances;
79+
each ``EntryPoint`` has a ``.name``, ``.group``, and ``.value`` attributes and
80+
a ``.load()`` method to resolve the value.
81+
82+
>>> eps = entry_points() # doctest: +SKIP
83+
>>> list(eps) # doctest: +SKIP
84+
['console_scripts', 'distutils.commands', 'distutils.setup_keywords', 'egg_info.writers', 'setuptools.installation']
85+
>>> scripts = eps['console_scripts'] # doctest: +SKIP
86+
>>> wheel = [ep for ep in scripts if ep.name == 'wheel'][0] # doctest: +SKIP
87+
>>> wheel # doctest: +SKIP
88+
EntryPoint(name='wheel', value='wheel.cli:main', group='console_scripts')
89+
>>> main = wheel.load() # doctest: +SKIP
90+
>>> main # doctest: +SKIP
91+
<function main at 0x103528488>
92+
93+
The ``group`` and ``name`` are arbitrary values defined by the package author
94+
and usually a client will wish to resolve all entry points for a particular
95+
group. Read `the setuptools docs
96+
<https://setuptools.readthedocs.io/en/latest/setuptools.html#dynamic-discovery-of-services-and-plugins>`_
97+
for more information on entrypoints, their definition, and usage.
98+
99+
100+
.. _metadata:
101+
102+
Distribution metadata
103+
---------------------
104+
105+
Every distribution includes some metadata, which you can extract using the
106+
``metadata()`` function::
107+
108+
>>> wheel_metadata = metadata('wheel') # doctest: +SKIP
109+
110+
The keys of the returned data structure [#f1]_ name the metadata keywords, and
111+
their values are returned unparsed from the distribution metadata::
112+
113+
>>> wheel_metadata['Requires-Python'] # doctest: +SKIP
114+
'>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*'
115+
116+
117+
.. _version:
118+
119+
Distribution versions
120+
---------------------
121+
122+
The ``version()`` function is the quickest way to get a distribution's version
123+
number, as a string::
124+
125+
>>> version('wheel') # doctest: +SKIP
126+
'0.32.3'
127+
128+
129+
.. _files:
130+
131+
Distribution files
132+
------------------
133+
134+
You can also get the full set of files contained within a distribution. The
135+
``files()`` function takes a distribution package name and returns all of the
136+
files installed by this distribution. Each file object returned is a
137+
``PackagePath``, a `pathlib.Path`_ derived object with additional ``dist``,
138+
``size``, and ``hash`` properties as indicated by the metadata. For example::
139+
140+
>>> util = [p for p in files('wheel') if 'util.py' in str(p)][0] # doctest: +SKIP
141+
>>> util # doctest: +SKIP
142+
PackagePath('wheel/util.py')
143+
>>> util.size # doctest: +SKIP
144+
859
145+
>>> util.dist # doctest: +SKIP
146+
<importlib.metadata._hooks.PathDistribution object at 0x101e0cef0>
147+
>>> util.hash # doctest: +SKIP
148+
<FileHash mode: sha256 value: bYkw5oMccfazVCoYQwKkkemoVyMAFoR34mmKBx8R1NI>
149+
150+
Once you have the file, you can also read its contents::
151+
152+
>>> print(util.read_text()) # doctest: +SKIP
153+
import base64
154+
import sys
155+
...
156+
def as_bytes(s):
157+
if isinstance(s, text_type):
158+
return s.encode('utf-8')
159+
return s
160+
161+
162+
.. _requirements:
163+
164+
Distribution requirements
165+
-------------------------
166+
167+
To get the full set of requirements for a distribution, use the ``requires()``
168+
function. Note that this returns an iterator::
169+
170+
>>> list(requires('wheel')) # doctest: +SKIP
171+
["pytest (>=3.0.0) ; extra == 'test'"]
172+
173+
174+
Distributions
175+
=============
176+
177+
While the above API is the most common and convenient usage, you can get all
178+
of that information from the ``Distribution`` class. A ``Distribution`` is an
179+
abstract object that represents the metadata for a Python package. You can
180+
get the ``Distribution`` instance::
181+
182+
>>> from importlib.metadata import distribution # doctest: +SKIP
183+
>>> dist = distribution('wheel') # doctest: +SKIP
184+
185+
Thus, an alternative way to get the version number is through the
186+
``Distribution`` instance::
187+
188+
>>> dist.version # doctest: +SKIP
189+
'0.32.3'
190+
191+
There are all kinds of additional metadata available on the ``Distribution``
192+
instance::
193+
194+
>>> d.metadata['Requires-Python'] # doctest: +SKIP
195+
'>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*'
196+
>>> d.metadata['License'] # doctest: +SKIP
197+
'MIT'
198+
199+
The full set of available metadata is not described here. See `PEP 566
200+
<https://www.python.org/dev/peps/pep-0566/>`_ for additional details.
201+
202+
203+
Extending the search algorithm
204+
==============================
205+
206+
Because package metadata is not available through ``sys.path`` searches, or
207+
package loaders directly, the metadata for a package is found through import
208+
system `finders`_. To find a distribution package's metadata,
209+
``importlib.metadata`` queries the list of `meta path finders`_ on
210+
`sys.meta_path`_.
211+
212+
By default ``importlib.metadata`` installs a finder for distribution packages
213+
found on the file system. This finder doesn't actually find any *packages*,
214+
but it can find the packages' metadata.
215+
216+
The abstract class :py:class:`importlib.abc.MetaPathFinder` defines the
217+
interface expected of finders by Python's import system.
218+
``importlib.metadata`` extends this protocol by looking for an optional
219+
``find_distributions`` callable on the finders from
220+
``sys.meta_path``. If the finder has this method, it must return
221+
an iterator over instances of the ``Distribution`` abstract class. This
222+
method must have the signature::
223+
224+
def find_distributions(name=None, path=None):
225+
"""Return an iterable of all Distribution instances capable of
226+
loading the metadata for packages matching the name
227+
(or all names if not supplied) along the paths in the list
228+
of directories ``path`` (defaults to sys.path).
229+
"""
230+
231+
What this means in practice is that to support finding distribution package
232+
metadata in locations other than the file system, you should derive from
233+
``Distribution`` and implement the ``load_metadata()`` method. This takes a
234+
single argument which is the name of the package whose metadata is being
235+
found. This instance of the ``Distribution`` base abstract class is what your
236+
finder's ``find_distributions()`` method should return.
237+
238+
239+
.. _`entry point API`: https://setuptools.readthedocs.io/en/latest/pkg_resources.html#entry-points
240+
.. _`metadata API`: https://setuptools.readthedocs.io/en/latest/pkg_resources.html#metadata-api
241+
.. _`Python 3.7 and newer`: https://docs.python.org/3/library/importlib.html#module-importlib.resources
242+
.. _`importlib_resources`: https://importlib-resources.readthedocs.io/en/latest/index.html
243+
.. _`PEP 566`: https://www.python.org/dev/peps/pep-0566/
244+
.. _`finders`: https://docs.python.org/3/reference/import.html#finders-and-loaders
245+
.. _`meta path finders`: https://docs.python.org/3/glossary.html#term-meta-path-finder
246+
.. _`sys.meta_path`: https://docs.python.org/3/library/sys.html#sys.meta_path
247+
.. _`pathlib.Path`: https://docs.python.org/3/library/pathlib.html#pathlib.Path
248+
249+
250+
.. rubric:: Footnotes
251+
252+
.. [#f1] Technically, the returned distribution metadata object is an
253+
`email.message.Message
254+
<https://docs.python.org/3/library/email.message.html#email.message.EmailMessage>`_
255+
instance, but this is an implementation detail, and not part of the
256+
stable API. You should only use dictionary-like methods and syntax
257+
to access the metadata contents.

‎Doc/library/modules.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,3 +17,4 @@ The full list of modules described in this chapter is:
1717
modulefinder.rst
1818
runpy.rst
1919
importlib.rst
20+
importlib.metadata.rst

‎Doc/tools/susp-ignored.csv

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -350,3 +350,6 @@ whatsnew/3.7,,::,error::BytesWarning
350350
whatsnew/changelog,,::,error::BytesWarning
351351
whatsnew/changelog,,::,default::BytesWarning
352352
whatsnew/changelog,,::,default::DeprecationWarning
353+
library/importlib.metadata,,.. highlight:,.. highlight:: none
354+
library/importlib.metadata,,:main,"EntryPoint(name='wheel', value='wheel.cli:main', group='console_scripts')"
355+
library/importlib.metadata,,`,of directories ``path`` (defaults to sys.path).

‎Lib/importlib/_bootstrap_external.py

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1363,6 +1363,58 @@ def find_module(cls, fullname, path=None):
13631363
return None
13641364
return spec.loader
13651365

1366+
search_template = r'(?:{pattern}(-.*)?\.(dist|egg)-info|EGG-INFO)'
1367+
1368+
@classmethod
1369+
def find_distributions(cls, name=None, path=None):
1370+
"""
1371+
Find distributions.
1372+
1373+
Return an iterable of all Distribution instances capable of
1374+
loading the metadata for packages matching the ``name``
1375+
(or all names if not supplied) along the paths in the list
1376+
of directories ``path`` (defaults to sys.path).
1377+
"""
1378+
import re
1379+
from importlib.metadata import PathDistribution
1380+
if path is None:
1381+
path = sys.path
1382+
pattern = '.*' if name is None else re.escape(name)
1383+
found = cls._search_paths(pattern, path)
1384+
return map(PathDistribution, found)
1385+
1386+
@classmethod
1387+
def _search_paths(cls, pattern, paths):
1388+
"""Find metadata directories in paths heuristically."""
1389+
import itertools
1390+
return itertools.chain.from_iterable(
1391+
cls._search_path(path, pattern)
1392+
for path in map(cls._switch_path, paths)
1393+
)
1394+
1395+
@staticmethod
1396+
def _switch_path(path):
1397+
from contextlib import suppress
1398+
import zipfile
1399+
from pathlib import Path
1400+
with suppress(Exception):
1401+
return zipfile.Path(path)
1402+
return Path(path)
1403+
1404+
@classmethod
1405+
def _predicate(cls, pattern, root, item):
1406+
import re
1407+
return re.match(pattern, str(item.name), flags=re.IGNORECASE)
1408+
1409+
@classmethod
1410+
def _search_path(cls, root, pattern):
1411+
if not root.is_dir():
1412+
return ()
1413+
normalized = pattern.replace('-', '_')
1414+
matcher = cls.search_template.format(pattern=normalized)
1415+
return (item for item in root.iterdir()
1416+
if cls._predicate(matcher, root, item))
1417+
13661418

13671419
class FileFinder:
13681420

0 commit comments

Comments
 (0)
0