8000 Merge branch 'main' into typing-subst-unpacked-vat-tuple · python/cpython@b091d97 · GitHub
[go: up one dir, main page]
Skip to content

Commit b091d97

Browse files
serhiy-storchakaserhiy-storchaka
committed
Merge branch 'main' into typing-subst-unpacked-vat-tuple
2 parents 7dcf277 + 6f8367d commit b091d97

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

58 files changed

+781
-302
lines changed

‎Doc/library/dis.rst

Lines changed: 18 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,12 @@
66

77
**Source code:** :source:`Lib/dis.py`
88

9+
.. testsetup::
10+
11+
import dis
12+
def myfunc(alist):
13+
return len(alist)
14+
915
--------------
1016

1117
The :mod:`dis` module supports the analysis of CPython :term:`bytecode` by
@@ -37,16 +43,17 @@ Example: Given the function :func:`myfunc`::
3743
return len(alist)
3844

3945
the following command can be used to display the disassembly of
40-
:func:`myfunc`::
46+
:func:`myfunc`:
4147

42-
>>> dis.dis(myfunc)
43-
1 0 RESUME 0
48+
.. doctest::
4449

45-
2 2 PUSH_NULL
46-
4 LOAD_GLOBAL 1 (NULL + len)
47-
6 LOAD_FAST 0 (alist)
48-
8 CALL 1
49-
18 RETURN_VALUE
50+
>>> dis.dis(myfunc)
51+
2 0 RESUME 0
52+
<BLANKLINE>
53+
3 2 LOAD_GLOBAL 1 (NULL + len)
54+
14 LOAD_FAST 0 (alist)
55+
16 CALL 1
56+
26 RETURN_VALUE
5057

5158
(The "2" is a line number).
5259

@@ -108,14 +115,15 @@ code.
108115
.. versionchanged:: 3.11
109116
Added the ``show_caches`` parameter.
110117

111-
Example::
118+
Example:
119+
120+
.. doctest::
112121

113122
>>> bytecode = dis.Bytecode(myfunc)
114123
>>> for instr in bytecode:
115124
... print(instr.opname)
116125
...
117126
RESUME
118-
PUSH_NULL
119127
LOAD_GLOBAL
120128
LOAD_FAST
121129
CALL

‎Doc/library/mailcap.rst

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,18 @@ standard. However, mailcap files are supported on most Unix systems.
6060
use) to determine whether or not the mailcap line applies. :func:`findmatch`
6161
will automatically check such conditions and skip the entry if the check fails.
6262

63+
.. versionchanged:: 3.11
64+
65+
To prevent security issues with shell metacharacters (symbols that have
66+
special effects in a shell command line), ``findmatch`` will refuse
67+
to inject ASCII characters other than alphanumerics and ``@+=:,./-_``
68+
into the returned command line.
69+
70+
If a disallowed character appears in *filename*, ``findmatch`` will always
71+
return ``(None, None)`` as if no entry was found.
72+
If such a character appears elsewhere (a value in *plist* or in *MIMEtype*),
73+
``findmatch`` will ignore all mailcap entries which use that value.
74+
A :mod:`warning <warnings>` will be raised in either case.
6375

6476
.. function:: getcaps()
6577

‎Doc/using/cmdline.rst

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -183,15 +183,37 @@ automatically enabled, if available on your platform (see
183183
Automatic enabling of tab-completion and history editing.
184184

185185

186+
.. _using-on-generic-options:
187+
186188
Generic options
187189
~~~~~~~~~~~~~~~
188190

189191
.. cmdoption:: -?
190192
-h
191193
--help
192194

193-
Print a short description of all command line options.
195+
Print a short description of all command line options and corresponding
196+
environment variables and exit.
197+
198+
.. cmdoption:: --help-env
199+
200+
Print a short description of Python-specific environment variables
201+
and exit.
202+
203+
.. versionadded:: 3.11
204+
205+
.. cmdoption:: --help-xoptions
194206

207+
Print a description of implementation-specific :option:`-X` options
208+
and exit.
209+
210+
.. versionadded:: 3.11
211+
212+
.. cmdoption:: --help-all
213+
214+
Print complete usage information and exit.
215+
216+
.. versionadded:: 3.11
195217

196218
.. cmdoption:: -V
197219
--version
@@ -212,6 +234,7 @@ Generic options
212234
.. versionadded:: 3.6
213235
The ``-VV`` option.
214236

237+
215238
.. _using-on-misc-options:
216239

217240
Miscellaneous options
@@ -460,6 +483,7 @@ Miscellaneous options
460483
See :ref:`warning-filter` and :ref:`describing-warning-filters` for more
461484
details.
462485

486+
463487
.. cmdoption:: -x
464488

465489
Skip the first line of the source, allowing use of non-Unix forms of
@@ -553,6 +577,7 @@ Miscellaneous options
553577
The ``-X frozen_modules`` option.
554578

555579

580+
556581
Options you shouldn't use
557582
~~~~~~~~~~~~~~~~~~~~~~~~~
558583

‎Include/boolobject.h

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,8 +19,8 @@ PyAPI_DATA(PyLongObject) _Py_FalseStruct;
1919
PyAPI_DATA(PyLongObject) _Py_TrueStruct;
2020

2121
/* Use these macros */
22-
#define Py_False ((PyObject *) &_Py_FalseStruct)
23-
#define Py_True ((PyObject *) &_Py_TrueStruct)
22+
#define Py_False _PyObject_CAST(&_Py_FalseStruct)
23+
#define Py_True _PyObject_CAST(&_Py_TrueStruct)
2424

2525
// Test if an object is the True singleton, the same as "x is True" in Python.
2626
PyAPI_FUNC(int) Py_IsTrue(PyObject *x);

‎Include/ceval.h

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ Py_DEPRECATED(3.9) PyAPI_FUNC(PyObject *) PyEval_CallObjectWithKeywords(
3131

3232
/* Deprecated since PyEval_CallObjectWithKeywords is deprecated */
3333
#define PyEval_CallObject(callable, arg) \
34-
PyEval_CallObjectWithKeywords(callable, arg, (PyObject *)NULL)
34+
PyEval_CallObjectWithKeywords(callable, arg, _PyObject_CAST(_Py_NULL))
3535

3636
Py_DEPRECATED(3.9) PyAPI_FUNC(PyObject *) PyEval_CallFunction(
3737
PyObject *callable, const char *format, ...);

‎Include/cpython/code.h

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,7 @@ typedef uint16_t _Py_CODEUNIT;
8888
PyObject *co_qualname; /* unicode (qualname, for reference) */ \
8989
PyObject *co_linetable; /* bytes object that holds location info */ \
9090
PyObject *co_weakreflist; /* to support weakrefs to code objects */ \
91+
void *_co_code; /* cached co_code object/attribute */ \
9192
/* Scratch space for extra data relating to the code object. \
9293
Type is a void* to keep the format private in codeobject.c to force \
9394
people to go through the proper APIs. */ \

‎Include/pyerrors.h

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@ PyAPI_FUNC(void) PyException_SetContext(PyObject *, PyObject *);
6262

6363
PyAPI_FUNC(const char *) PyExceptionClass_Name(PyObject *);
6464

65-
#define PyExceptionInstance_Class(x) ((PyObject*)Py_TYPE(x))
65+
#define PyExceptionInstance_Class(x) _PyObject_CAST(Py_TYPE(x))
6666

6767
#define _PyBaseExceptionGroup_Check(x) \
6868
PyObject_TypeCheck(x, (PyTypeObject *)PyExc_BaseExceptionGroup)

‎Include/pyport.h

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,9 +24,23 @@
2424
//
2525
// The type argument must not be a constant type.
2626
#ifdef __cplusplus
27+
#include <cstddef>
2728
# define _Py_STATIC_CAST(type, expr) static_cast<type>(expr)
2829
extern "C++" {
2930
namespace {
31+
template <typename type>
32+
inline type _Py_CAST_impl(long int ptr) {
33+
return reinterpret_cast<type>(ptr);
34+
}
35+
template <typename type>
36+
inline type _Py_CAST_impl(int ptr) {
37+
return reinterpret_cast<type>(ptr);
38+
}
39+
template <typename type>
40+
inline type _Py_CAST_impl(std::nullptr_t) {
41+
return static_cast<type>(nullptr);
42+
}
43+
3044
template <typename type, typename expr_type>
3145
inline type _Py_CAST_impl(expr_type *expr) {
3246
return reinterpret_cast<type>(expr);

‎Lib/dis.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -592,7 +592,7 @@ def _unpack_opargs(code):
592592
caches = _inline_cache_entries[deop]
593593
if deop >= HAVE_ARGUMENT:
594594
arg = code[i+1] | extended_arg
595-
extended_arg = (arg << 8) if op == EXTENDED_ARG else 0
595+
extended_arg = (arg << 8) if deop == EXTENDED_ARG else 0
596596
# The oparg is stored as a signed integer
597597
# If the value exceeds its upper limit, it will overflow and wrap
598598
# to a negative integer

‎Lib/logging/__init__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1498,7 +1498,7 @@ def info(self, msg, *args, **kwargs):
14981498
To pass exception information, use the keyword argument exc_info with
14991499
a true value, e.g.
15001500
1501-
logger.info("Houston, we have a %s", "interesting problem", exc_info=1)
1501+
logger.info("Houston, we have a %s", "notable problem", exc_info=1)
15021502
"""
15031503
if self.isEnabledFor(INFO):
15041504
self._log(INFO, msg, args, **kwargs)

‎Lib/mailcap.py

Lines changed: 24 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22

33
import os
44
import warnings
5+
import re
56

67
__all__ = ["getcaps","findmatch"]
78

@@ -19,6 +20,11 @@ def lineno_sort_key(entry):
1920
else:
2021
return 1, 0
2122

23+
_find_unsafe = re.compile(r'[^\xa1-\U0010FFFF\w@+=:,./-]').search
24+
25+
class UnsafeMailcapInput(Warning):
26+
"""Warning raised when refusing unsafe input"""
27+
2228

2329
# Part 1: top-level interface.
2430

@@ -171,15 +177,22 @@ def findmatch(caps, MIMEtype, key='view', filename="/dev/null", plist=[]):
171177
entry to use.
172178
173179
"""
180+
if _find_unsafe(filename):
181+
msg = "Refusing to use mailcap with filename %r. Use a safe temporary filename." % (filename,)
182+
warnings.warn(msg, UnsafeMailcapInput)
183+
return None, None
174184
entries = lookup(caps, MIMEtype, key)
175185
# XXX This code should somehow check for the needsterminal flag.
176186
for e in entries:
177187
if 'test' in e:
178188
test = subst(e['test'], filename, plist)
189+
if test is None:
190+
continue
179191
if test and os.system(test) != 0:
180192
continue
181193
command = subst(e[key], MIMEtype, filename, plist)
182-
return command, e
194+
if command is not None:
195+
return command, e
183196
return None, None
184197

185198
def lookup(caps, MIMEtype, key=None):
@@ -212,14 +225,23 @@ def subst(field, MIMEtype, filename, plist=[]):
212225
elif c == 's':
213226
res = res + filename
214227
elif c == 't':
228+
if _find_unsafe(MIMEtype):
229+
msg = "Refusing to substitute MIME type %r into a shell command." % (MIMEtype,)
230+
warnings.warn(msg, UnsafeMailcapInput)
231+
return None
215232
res = res + MIMEtype
216233
elif c == '{':
217234
start = i
218235
while i < n and field[i] != '}':
219236
i = i+1
220237
name = field[start:i]
221238
i = i+1
222-
res = res + findparam(name, plist)
239+
param = findparam(name, plist)
240+
if _find_unsafe(param):
241+
msg = "Refusing to substitute parameter %r (%s) into a shell command" % (param, name)
242+
warnings.warn(msg, UnsafeMailcapInput)
243+
return None
244+
res = res + param
223245
# XXX To do:
224246
# %n == number of parts if type is multipart/*
225247
# %F == list of alternating type and filename for parts

‎Lib/netrc.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
# Module and documentation by Eric S. Raymond, 21 Dec 1998
44

5-
import os, shlex, stat
5+
import os, stat
66

77
__all__ = ["netrc", "NetrcParseError"]
88

‎Lib/pathlib.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -443,6 +443,8 @@ def __getitem__(self, idx):
443443

444444
if idx >= len(self) or idx < -len(self):
445445
raise IndexError(idx)
446+
if idx < 0:
447+
idx += len(self)
446448
return self._pathcls._from_parsed_parts(self._drv, self._root,
447449
self._parts[:-idx - 1])
448450

‎Lib/test/_testcppext.cpp

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,13 @@ test_api_casts(PyObject *Py_UNUSED(module), PyObject *Py_UNUSED(args))
7474
Py_INCREF(strong_ref);
7575
Py_DECREF(strong_ref);
7676

77+
// gh-93442: Pass 0 as NULL for PyObject*
78+
Py_XINCREF(0);
79+
Py_XDECREF(0);
80+
// ensure that nullptr works too
81+
Py_XINCREF(nullptr);
82+
Py_XDECREF(nullptr);
83+
7784
Py_DECREF(obj);
7885
Py_RETURN_NONE;
7986
}

‎Lib/test/pythoninfo.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -144,7 +144,7 @@ def collect_sys(info_add):
144144
text = 'Yes (sys.getobjects() present)'
145145
else:
146146
text = 'No (sys.getobjects() missing)'
147-
info_add('build.Py_REF_DEBUG', text)
147+
info_add('build.Py_TRACE_REFS', text)
148148

149149

150150
def collect_platform(info_add):

‎Lib/test/support/__init__.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -305,6 +305,8 @@ def requires(resource, msg=None):
305305
if msg is None:
306306
msg = "Use of the %r resource not enabled" % resource
307307
raise ResourceDenied(msg)
308+
if resource in {"network", "urlfetch"} and not has_socket_support:
309+
raise ResourceDenied("No socket support")
308310
if resource == 'gui' and not _is_gui_available():
309311
raise ResourceDenied(_is_gui_available.reason)
310312

‎Lib/test/support/os_helper.py

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -171,9 +171,13 @@ def can_symlink():
171171
global _can_symlink
172172
if _can_symlink is not None:
173173
return _can_symlink
174-
symlink_path = TESTFN + "can_symlink"
174+
# WASI / wasmtime prevents symlinks with absolute paths, see man
175+
# openat2(2) RESOLVE_BENEATH. Almost all symlink tests use absolute
176+
# paths. Skip symlink tests on WASI for now.
177+
src = os.path.abspath(TESTFN)
178+
symlink_path = src + "can_symlink"
175179
try:
176-
os.symlink(TESTFN, symlink_path)
180+
os.symlink(src, symlink_path)
177181
can = True
178182
except (OSError, NotImplementedError, AttributeError):
179183
can = False

0 commit comments

Comments
 (0)
0