8000 bpo-45582: Port getpath[p].c to Python (GH-29041) · python/cpython@99fcf15 · GitHub
[go: up one dir, main page]
Skip to content

Commit 99fcf15

Browse files
zoobazooba
authored
bpo-45582: Port getpath[p].c to Python (GH-29041)
The getpath.py file is frozen at build time and executed as code over a namespace. It is never imported, nor is it meant to be importable or reusable. However, it should be easier to read, modify, and patch than the previous code. This commit attempts to preserve every previously tested quirk, but these may be changed in the future to better align platforms.
1 parent 9f2f7e4 commit 99fcf15

40 files changed

+3516
-3668
lines changed

‎.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,7 @@ Mac/pythonw
7474
Misc/python.pc
7575
Misc/python-embed.pc
7676
Misc/python-config.sh
77+
Modules/getpath.h
7778
Modules/Setup.config
7879
Modules/Setup.local
7980
Modules/Setup.stdlib

‎Doc/c-api/init_config.rst

Lines changed: 28 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -479,6 +479,9 @@ PyConfig
479479
480480
Fields which are already initialized are left unchanged.
481481
482+
Fields for :ref:`path configuration <init-path-config>` are no longer
483+
calculated or modified when calling this function, as of Python 3.11.
484+
482485
The :c:func:`PyConfig_Read` function only parses
483486
:c:member:`PyConfig.argv` arguments once: :c:member:`PyConfig.parse_argv`
484487
is set to ``2`` after arguments are parsed. Since Python arguments are
@@ -493,6 +496,12 @@ PyConfig
493496
parsed, and arguments are only parsed if
494497
:c:member:`PyConfig.parse_argv` equals ``1``.
495498
499+
.. versionchanged:: 3.11
500+
:c:func:`PyConfig_Read` no longer calculates all paths, and so fields
501+
listed under :ref:`Python Path Configuration <init-path-config>` may
502+
no longer be updated until :c:func:`Py_InitializeFromConfig` is
503+
called.
504+
496505
.. c:function:: void PyConfig_Clear(PyConfig *config)
497506
498507
Release configuration memory.
@@ -848,12 +857,19 @@ PyConfig
848857
849858
Default: value of the ``PLATLIBDIR`` macro which is set by the
850859
:option:`configure --with-platlibdir option <--with-platlibdir>`
851-
(default: ``"lib"``).
860+
(default: ``"lib"``, or ``"DLLs"`` on Windows).
852861
853862
Part of the :ref:`Python Path Configuration <init-path-config>` input.
854863
855864
.. versionadded:: 3.9
856865
866+
.. versionchanged:: 3.11
867+
This macro is now used on Windows to locate the standard
868+
library extension modules, typically under ``DLLs``. However,
869+
for compatibility, note that this value is ignored for any
870+
non-standard layouts, including in-tree builds and virtual
871+
environments.
872+
857873
.. c:member:: wchar_t* pythonpath_env
858874
859875
Module search paths (:data:`sys.path`) as a string separated by ``DELIM``
@@ -870,9 +886,9 @@ PyConfig
870886
871887
Module search paths: :data:`sys.path`.
872888
873-
If :c:member:`~PyConfig.module_search_paths_set` is equal to 0, the
874-
function calculating the :ref:`Python Path Configuration <init-path-config>`
875-
overrides the :c:member:`~PyConfig.module_search_paths` and sets
889+
If :c:member:`~PyConfig.module_search_paths_set` is equal to 0,
890+
:c:func:`Py_InitializeFromConfig` will replace
891+
:c:member:`~PyConfig.module_search_paths` and sets
876892
:c:member:`~PyConfig.module_search_paths_set` to ``1``.
877893
878894
Default: empty list (``module_search_paths``) and ``0``
@@ -944,16 +960,16 @@ PyConfig
944960
945961
.. c:member:: int pathconfig_warnings
946962
947-
On Unix, if non-zero, calculating the :ref:`Python Path Configuration
948-
<init-path-config>` can log warnings into ``stderr``. If equals to 0,
949-
suppress these warnings.
950-
951-
It has no effect on Windows.
963+
If non-zero, calculation of path configuration is allowed to log
964+
warnings into ``stderr``. If equals to 0, suppress these warnings.
952965
953966
Default: ``1`` in Python mode, ``0`` in isolated mode.
954967
955968
Part of the :ref:`Python Path Configuration <init-path-config>` input.
956969
970+
.. versionchanged:: 3.11
971+
Now also applies on Windows.
972+
957973
.. c:member:: wchar_t* prefix
958974
959975
The site-specific directory prefix where the platform independent Python
@@ -1305,10 +1321,9 @@ variables, command line arguments (:c:member:`PyConfig.argv` is not parsed)
13051321
and user site directory. The C standard streams (ex: ``stdout``) and the
13061322
LC_CTYPE locale are left unchanged. Signal handlers are not installed.
13071323
1308-
Configuration files are still used with this configuration. Set the
1309-
:ref:`Python Path Configuration <init-path-config>` ("output fields") to ignore these
1310-
configuration files and avoid the function computing the default path
1311-
configuration.
1324+
Configuration files are still used with this configuration to determine
1325+
paths that are unspecified. Ensure :c:member:`PyConfig.home` is specified
1326+
to avoid computing the default path configuration.
13121327
13131328
13141329
.. _init-python-config:

‎Include/cpython/fileutils.h

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -135,12 +135,6 @@ PyAPI_FUNC(wchar_t*) _Py_wrealpath(
135135
size_t resolved_path_len);
136136
#endif
137137

138-
#ifndef MS_WINDOWS
139-
PyAPI_FUNC(int) _Py_isabs(const wchar_t *path);
140-
#endif
141-
142-
PyAPI_FUNC(int) _Py_abspath(const wchar_t *path, wchar_t **abspath_p);
143-
144138
PyAPI_FUNC(wchar_t*) _Py_wgetcwd(
145139
wchar_t *buf,
146140
/* Number of characters of 'buf' buffer

‎Include/cpython/initconfig.h

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -213,6 +213,9 @@ typedef struct PyConfig {
213213
// If non-zero, disallow threads, subprocesses, and fork.
214214
// Default: 0.
215215
int _isolated_interpreter;
216+
217+
// If non-zero, we believe we're running from a source tree.
218+
int _is_python_build;
216219
} PyConfig;
217220

218221
PyAPI_FUNC(void) PyConfig_InitPythonConfig(PyConfig *config);

‎Include/internal/pycore_fileutils.h

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -74,14 +74,15 @@ extern int _Py_EncodeNonUnicodeWchar_InPlace(
7474
Py_ssize_t size);
7575
#endif
7676

77+
extern int _Py_isabs(const wchar_t *path);
78+
extern int _Py_abspath(const wchar_t *path, wchar_t **abspath_p);
7779
extern wchar_t * _Py_join_relfile(const wchar_t *dirname,
7880
const wchar_t *relfile);
7981
extern int _Py_add_relfile(wchar_t *dirname,
8082
const wchar_t *relfile,
8183
size_t bufsize);
8284
extern size_t _Py_find_basename(const wchar_t *filename);
83-
PyAPI_FUNC(int) _Py_normalize_path(const wchar_t *path,
84-
wchar_t *buf, const size_t buf_len);
85+
PyAPI_FUNC(wchar_t *) _Py_normpath(wchar_t *path, Py_ssize_t size);
8586

8687

8788
// Macros to protect CRT calls against instant termination when passed an

‎Include/internal/pycore_initconfig.h

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -166,6 +166,10 @@ extern PyStatus _PyConfig_SetPyArgv(
166166
PyAPI_FUNC(PyObject*) _PyConfig_AsDict(const PyConfig *config);
167167
PyAPI_FUNC(int) _PyConfig_FromDict(PyConfig *config, PyObject *dict);
168168

169+
extern void _Py_DumpPathConfig(PyThreadState *tstate);
170+
171+
PyAPI_FUNC(PyObject*) _Py_Get_Getpath_CodeObject();
172+
169173

170174
/* --- Function used for testing ---------------------------------- */
171175

‎Include/internal/pycore_pathconfig.h

Lines changed: 4 additions & 54 deletions
Original file line numberDiff line numberDiff line change
@@ -8,65 +8,15 @@ extern "C" {
88
# error "this header requires Py_BUILD_CORE define"
99
#endif
1010

11-
typedef struct _PyPathConfig {
12-
/* Full path to the Python program */
13-
wchar_t *program_full_path;
14-
wchar_t *prefix;
15-
wchar_t *exec_prefix;
16-
wchar_t *stdlib_dir;
17-
/* Set by Py_SetPath(), or computed by _PyConfig_InitPathConfig() */
18-
wchar_t *module_search_path;
19-
/* Python program name */
20-
wchar_t *program_name;
21-
/* Set by Py_SetPythonHome() or PYTHONHOME environment variable */
22-
wchar_t *home;
23-
#ifdef MS_WINDOWS
24-
/* isolated and site_import are used to set Py_IsolatedFlag and
25-
Py_NoSiteFlag flags on Windows in read_pth_file(). These fields
26-
are ignored when their value are equal to -1 (unset). */
27-
int isolated;
28-
int site_import;
29-
/* Set when a venv is detected */
30-
wchar_t *base_executable;
31-
#endif
32-
} _PyPathConfig;
33-
34-
#ifdef MS_WINDOWS
35-
# define _PyPathConfig_INIT \
36-
{.module_search_path = NULL, \
37-
.isolated = -1, \
38-
.site_import = -1}
39-
#else
40-
# define _PyPathConfig_INIT \
41-
{.module_search_path = NULL}
42-
#endif
43-
/* Note: _PyPathConfig_INIT sets other fields to 0/NULL */
44-
45-
PyAPI_DATA(_PyPathConfig) _Py_path_config;
46-
#ifdef MS_WINDOWS
47-
PyAPI_DATA(wchar_t*) _Py_dll_path;
48-
#endif
49-
50-
extern void _PyPathConfig_ClearGlobal(void);
11+
PyAPI_FUNC(void) _PyPathConfig_ClearGlobal(void);
12+
extern PyStatus _PyPathConfig_ReadGlobal(PyConfig *config);
13+
extern PyStatus _PyPathConfig_UpdateGlobal(const PyConfig *config);
14+
extern const wchar_t * _PyPathConfig_GetGlobalModuleSearchPath(void);
5115

52-
extern PyStatus _PyPathConfig_Calculate(
53-
_PyPathConfig *pathconfig,
54-
const PyConfig *config);
5516
extern int _PyPathConfig_ComputeSysPath0(
5617
const PyWideStringList *argv,
5718
PyObject **path0);
58-
extern PyStatus _Py_FindEnvConfigValue(
59-
FILE *env_file,
60-
const wchar_t *key,
61-
wchar_t **value_p);
62-
63-
#ifdef MS_WINDOWS
64-
extern wchar_t* _Py_GetDLLPath(void);
65-
#endif
6619

67-
extern PyStatus _PyConfig_WritePathConfig(const PyConfig *config);
68-
extern void _Py_DumpPathConfig(PyThreadState *tstate);
69-
extern PyObject* _PyPathConfig_AsDict(void);
7020

7121
#ifdef __cplusplus
7222
}

‎Lib/ntpath.py

Lines changed: 63 additions & 49 deletions
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,7 @@ def isabs(s):
7070
if s.replace('/', '\\').startswith('\\\\?\\'):
7171
return True
7272
s = splitdrive(s)[1]
73-
return len(s) > 0 and s[0] in _get_bothseps(s)
73+
return len(s) > 0 and s[0] and s[0] in _get_bothseps(s)
7474

7575

7676
# Join two (or more) paths.
@@ -268,11 +268,13 @@ def ismount(path):
268268
root, rest = splitdrive(path)
269269
if root and root[0] in seps:
270270
return (not rest) or (rest in seps)
271-
if rest in seps:
271+
if rest and rest in seps:
272272
return True
273273

274274
if _getvolumepathname:
275-
return path.rstrip(seps) == _getvolumepathname(path).rstrip(seps)
275+
x = path.rstrip(seps)
276+
y =_getvolumepathname(path).rstrip(seps)
277+
return x.casefold() == y.casefold()
276278
else:
277279
return False
278280

@@ -459,56 +461,68 @@ def expandvars(path):
459461
# Normalize a path, e.g. A//B, A/./B and A/foo/../B all become A\B.
460462
# Previously, this function also truncated pathnames to 8+3 format,
461463
# but as this module is called "ntpath", that's obviously wrong!
464+
try:
465+
from nt import _path_normpath
462466

463-
def normpath(path):
464-
"""Normalize path, eliminating double slashes, etc."""
465-
path = os.fspath(path)
466-
if isinstance(path, bytes):
467-
sep = b'\\'
468-
altsep = b'/'
469-
curdir = b'.'
470-
pardir = b'..'
471-
special_prefixes = (b'\\\\.\\', b'\\\\?\\')
472-
else:
473-
sep = '\\'
474-
altsep = '/'
475-
curdir = '.'
476-
pardir = '..'
477-
special_prefixes = ('\\\\.\\', '\\\\?\\')
478-
if path.startswith(special_prefixes):
479-
# in the case of paths with these prefixes:
480-
# \\.\ -> device names
481-
# \\?\ -> literal paths
482-
# do not do any normalization, but return the path
483-
# unchanged apart from the call to os.fspath()
484-
return path
485-
path = path.replace(altsep, sep)
486-
prefix, path = splitdrive(path)
487-
488-
# collapse initial backslashes
489-
if path.startswith(sep):
490-
prefix += sep
491-
path = path.lstrip(sep)
492-
493-
comps = path.split(sep)
494-
i = 0
495-
while i < len(comps):
496-
if not comps[i] or comps[i] == curdir:
497-
del comps[i]
498-
elif comps[i] == pardir:
499-
if i > 0 and comps[i-1] != pardir:
500-
del comps[i-1:i+1]
501-
i -= 1
502-
elif i == 0 and prefix.endswith(sep):
467+
except ImportError:
468+
def normpath(path):
469+
"""Normalize path, eliminating double slashes, etc."""
470+
path = os.fspath(path)
471+
if isinstance(path, bytes):
472+
sep = b'\\'
473+
altsep = b'/'
474+
curdir = b'.'
475+
pardir = b'..'
476+
special_prefixes = (b'\\\\.\\', b'\\\\?\\')
477+
else:
478+
sep = '\\'
479+
altsep = '/'
480+
curdir = '.'
481+
pardir = '..'
482+
special_prefixes = ('\\\\.\\', '\\\\?\\')
483+
if path.startswith(special_prefixes):
484+
# in the case of paths with these prefixes:
485+
# \\.\ -> device names
486+
# \\?\ -> literal paths
487+
# do not do any normalization, but return the path
488+
# unchanged apart from the call to os.fspath()
489+
return path
490+
path = path.replace(altsep, sep)
491+
prefix, path = splitdrive(path)
492+
493+
# collapse initial backslashes
494+
if path.startswith(sep):
495+
prefix += sep
496+
path = path.lstrip(sep)
497+
498+
comps = path.split(sep)
499+
i = 0
500+
while i < len(comps):
501+
if not comps[i] or comps[i] == curdir:
503502
del comps[i]
503+
elif comps[i] == pardir:
504+
if i > 0 and comps[i-1] != pardir:
505+
del comps[i-1:i+1]
506+
i -= 1
507+
elif i == 0 and prefix.endswith(sep):
508+
del comps[i]
509+
else:
510+
i += 1
504511
else:
505512
i += 1
506-
else:
507-
i += 1
508-
# If the path is now empty, substitute '.'
509-
if not prefix and not comps:
510-
comps.append(curdir)
511-
return prefix + sep.join(comps)
513+
# If the path is now empty, substitute '.'
514+
if not prefix and not comps:
515+
comps.append(curdir)
516+
return prefix + sep.join(comps)
517+
518+
else:
519+
def normpath(path):
520+
"""Normalize path, eliminating double slashes, etc."""
521+
path = os.fspath(path)
522+
if isinstance(path, bytes):
523+
return os.fsencode(_path_normpath(os.fsdecode(path))) or b"."
524+
return _path_normpath(path) or "."
525+
512526

513527
def _abspath_fallback(path):
514528
"""Return the absolute version of a path as a fallback function in case

0 commit comments

Comments
 (0)
0