8000 Merge remote-tracking branch 'upstream/main' into specialize_load_method · python/cpython@020a326 · GitHub
[go: up one dir, main page]
Skip to content

Commit 020a326

Browse files
Fidget-SpinnerFidget-Spinner
committed
Merge remote-tracking branch 'upstream/main' into specialize_load_method
2 parents 6d806a2 + 4f51fa9 commit 020a326

File tree

71 files changed

+2500
-1409
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

71 files changed

+2500
-1409
lines changed

‎Doc/library/base64.rst

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
===============================================================
33

44
.. module:: base64
5-
:synopsis: RFC 3548: Base16, Base32, Base64 Data Encodings;
5+
:synopsis: RFC 4648: Base16, Base32, Base64 Data Encodings;
66
Base85 and Ascii85
77

88
**Source code:** :source:`Lib/base64.py`
@@ -16,10 +16,10 @@
1616
This module provides functions for encoding binary data to printable
1717
ASCII characters and decoding such encodings back to binary data.
1818
It provides encoding and decoding functions for the encodings specified in
19-
:rfc:`3548`, which defines the Base16, Base32, and Base64 algorithms,
19+
:rfc:`4648`, which defines the Base16, Base32, and Base64 algorithms,
2020
and for the de-facto standard Ascii85 and Base85 encodings.
2121

22-
The :rfc:`3548` encodings are suitable for encoding binary data so that it can
22+
The :rfc:`4648` encodings are suitable for encoding binary data so that it can be
2323
safely sent by email, used as parts of URLs, or included as part of an HTTP
2424
POST request. The encoding algorithm is not the same as the
2525
:program:`uuencode` program.
@@ -28,7 +28,7 @@ There are two interfaces provided by this module. The modern interface
2828
supports encoding :term:`bytes-like objects <bytes-like object>` to ASCII
2929
:class:`bytes`, and decoding :term:`bytes-like objects <bytes-like object>` or
3030
strings containing ASCII to :class:`bytes`. Both base-64 alphabets
31-
defined in :rfc:`3548` (normal, and URL- and filesystem-safe) are supported.
31+
defined in :rfc:`4648` (normal, and URL- and filesystem-safe) are supported.
3232

3333
The legacy interface does not support decoding from strings, but it does
3434
provide functions for encoding and decoding to and from :term:`file objects
@@ -287,6 +287,13 @@ An example usage of the module:
287287
>>> data
288288
b'data to be encoded'
289289

290+
.. _base64-security:
291+
292+
Security Considerations
293+
-----------------------
294+
295+
A new security considerations section was added to :rfc:`4648` (section 12); it's
296+
recommended to review the security section for any code deployed to production.
290297

291298
.. seealso::
292299

‎Doc/library/pty.rst

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,8 @@
22
========================================
33

44
.. module:: pty
5-
:platform: Linux
6-
:synopsis: Pseudo-Terminal Handling for Linux.
5+
:platform: Unix
6+
:synopsis: Pseudo-Terminal Handling for Unix.
77

88
.. moduleauthor:: Steen Lumholt
99
.. sectionauthor:: Moshe Zadka <moshez@zadka.site.co.il>
@@ -16,9 +16,9 @@ The :mod:`pty` module defines operations for handling the pseudo-terminal
1616
concept: starting another process and being able to write to and read from its
1717
controlling terminal programmatically.
1818

19-
Because pseudo-terminal handling is highly platform dependent, there is code to
20-
do it only for Linux. (The Linux code is supposed to work on other platforms,
21-
but hasn't been tested yet.)
19+
Pseudo-terminal handling is highly platform dependent. This code is mainly
20+
tested on Linux, FreeBSD, and macOS (it is supposed to work on other POSIX
21+
platforms but it's not been thoroughly tested).
2222

2323
The :mod:`pty` module defines the following functions:
2424

@@ -47,10 +47,14 @@ The :mod:`pty` module defines the following functions:
4747
spawned behind the pty will eventually terminate, and when it does *spawn*
4848
will return.
4949

50+
A loop copies STDIN of the current process to the child and data received
51+
from the child to STDOUT of the current process. It is not signaled to the
52+
child if STDIN of the current process closes down.
53+
5054
The functions *master_read* and *stdin_read* are passed a file descriptor
5155
which they should read from, and they should always return a byte string. In
5256
order to force spawn to return before the child process exits an
53-
:exc:`OSError` should be thrown.
57+
empty byte array should be returned to signal end of file.
5458

5559
The default implementation for both functions will read and return up to 1024
5660
bytes each time the function is called. The *master_read* callback is passed
@@ -65,10 +69,6 @@ The :mod:`pty` module defines the following functions:
6569
process will quit without any input, *spawn* will then loop forever. If
6670
*master_read* signals EOF the same behavior results (on linux at least).
6771

68-
If both callbacks signal EOF then *spawn* will probably never return, unless
69-
*select* throws an error on your platform when passed three empty lists. This
70-
is a bug, documented in `issue 26228 <https://bugs.python.org/issue26228>`_.
71-
7272
Return the exit status value from :func:`os.waitpid` on the child process.
7373

7474
:func:`waitstatus_to_exitcode` can be used to convert the exit status into

‎Doc/library/security_warnings.rst

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,8 @@ Security Considerations
77

88
The following modules have specific security considerations:
99

10+
* :mod:`base64`: :ref:`base64 security considerations <base64-security>` in
11+
:rfc:`4648`
1012
* :mod:`cgi`: :ref:`CGI security considerations <cgi-security>`
1113
* :mod:`hashlib`: :ref:`all constructors take a "usedforsecurity" keyword-only
1214
argument disabling known insecure and blocked algorithms

‎Doc/library/stdtypes.rst

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -529,6 +529,18 @@ class`. In addition, it provides a few more methods:
529529
given, an :exc:`OverflowError` is raised. The default value for *signed*
530530
is ``False``.
531531

532+
Equivalent to::
533+
534+
def to_bytes(n, length, byteorder, signed=False):
535+
if byteorder == 'little':
536+
order = range(length)
537+
elif byteorder == 'big':
538+
order = reversed(range(length))
539+
else:
540+
raise ValueError("byteorder must be either 'little' or 'big'")
541+
542+
return bytes((n >> i*8) & 0xff for i in order)
543+
532544
.. versionadded:: 3.2
533545

534546
.. classmethod:: int.from_bytes(bytes, byteorder, *, signed=False)
@@ -559,6 +571,22 @@ class`. In addition, it provides a few more methods:
559571
The *signed* argument indicates whether two's complement is used to
560572
represent the integer.
561573

574+
Equivalent to::
575+
576+
def from_bytes(bytes, byteorder, signed=False):
577+
if byteorder == 'little':
578+
little_ordered = list(bytes)
579+
elif byteorder == 'big':
580+
little_ordered = list(reversed(bytes))
581+
else:
582+
raise ValueError("byteorder must be either 'little' or 'big'")
583+
584+
n = sum(b << i*8 for i, b in enumerate(little_ordered))
585+
if signed and little_ordered and (little_ordered[-1] & 0x80):
586+
n -= 1 << 8*len(little_ordered)
587+
588+
return n
589+
562590
.. versionadded:: 3.2
563591

564592
.. method:: int.as_integer_ratio()

‎Doc/library/tkinter.rst

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,16 @@ demonstrating a simple Tk interface, letting you know that :mod:`tkinter` is
2020
properly installed on your system, and also showing what version of Tcl/Tk is
2121
installed, so you can read the Tcl/Tk documentation specific to that version.
2222

23+
Tkinter supports a range of Tcl/Tk versions, built either with or
24+
without thread support. The official Python binary release bundles Tcl/Tk 8.6
25+
threaded. See the source code for the :mod:`_tkinter` module
26+
for more information about supported versions.
27+
28+
Tkinter is not a thin wrapper, but adds a fair amount of its own logic to
29+
make the experience more pythonic. This documentation will concentrate on these
30+
additions and changes, and refer to the official Tcl/Tk documentation for
31+
details that are unchanged.
32+
2333
.. seealso::
2434

2535
Tkinter documentation:
@@ -62,6 +72,47 @@ installed, so you can read the Tcl/Tk documentation specific to that version.
6272
Brent Welch's encyclopedic book.
6373

6474

75+
Architecture
76+
------------
77+
78+
Tcl/Tk is not a single library but rather consists of a few distinct
79+
modules, each with a separate functionality and its own official
80+
documentation. Python's binary releases also ship an add-on module
81+
together with it.
82+
83+
Tcl
84+
Tcl is a dynamic interpreted programming language, just like Python. Though
85+
it can be used on its own as a general-purpose programming language, it is
86+
most commonly embedded into C applications as a scripting engine or an
87+
interface to the Tk toolkit. The Tcl library has a C interface to
88+
create and manage one or more instances of a Tcl interpreter, run Tcl
89+
commands and scripts in those instances, and add custom commands
90+
implemented in either Tcl or C. Each interpreter has an event queue,
91+
and there are facilities to send events to it and process them.
92+
Unlike Python, Tcl's execution model is designed around cooperative
93+
multitasking, and Tkinter bridges this difference
94+
(see `Threading model`_ for details).
95+
96+
Tk
97+
Tk is a `Tcl package <http://wiki.tcl.tk/37432>`_ implemented in C
98+
that adds custom commands to create and manipulate GUI widgets. Each
99+
:class:`Tk` object embeds its own Tcl interpreter instance with Tk loaded into
100+
it. Tk's widgets are very customizable, though at the cost of a dated appearance.
101+
Tk uses Tcl's event queue to generate and process GUI events.
102+
103+
Ttk
104+
Themed Tk (Ttk) is a newer family of Tk widgets that provide a much better
105+
appearance on different platforms than many of the classic Tk widgets.
106+
Ttk is distributed as part of Tk, starting with Tk version 8.5. Python
107+
bindings are provided in a separate module, :mod:`tkinter.ttk`.
108+
109+
Tix
110+
`Tix <https://core.tcl.tk/jenglish/gutter/packages/tix.html>`_ is an older
111+
third-party Tcl package, an add-on for Tk that adds several new widgets.
112+
Python bindings are found in the :mod:`tkinter.tix` module.
113+
It's deprecated in favor of Ttk.
114+
115+
65116
Tkinter Modules
66117
---------------
67118

@@ -377,6 +428,59 @@ Xlib (C)
377428
the Xlib library to draw graphics on the screen.
378429

379430

431+
Threading model
432+
---------------
433+
434+
Python and Tcl/Tk have very different threading models, which :mod:`tkinter`
435+
tries to bridge. If you use threads, you may need to be aware of this.
436+
437+
A Python interpreter may have many threads associated with it. In Tcl, multiple
438+
threads can be created, but each thread has a separate Tcl interpreter instance
439+
associated with it. Threads can also create more than one interpreter instance,
440+
though each interpreter instance can be used only by the one thread that created it.
441+
442+
Each :class:`Tk` object created by :mod:`tkinter` contains a Tcl interpreter.
443+
It also keeps track of which thread created that interpreter. Calls to
444+
:mod:`tkinter` can be made from any Python thread. Internally, if a call comes
445+
from a thread other than the one that created the :class:`Tk` object, an event
446+
is posted to the interpreter's event queue, and when executed, the result is
447+
returned to the calling Python thread.
448+
449+
Tcl/Tk applications are normally event-driven, meaning that after initialization,
450+
the interpreter runs an event loop (i.e. :func:`Tk.mainloop`) and responds to events.
451+
Because it is single-threaded, event handlers must respond quickly, otherwise they
452+
will block other events from being processed. To avoid this, any long-running
453+
computations should not run in an event handler, but are either broken into smaller
454+
pieces using timers, or run in another thread. This is different from many GUI
455+
toolkits where the GUI runs in a completely separate thread from all application
456+
code including event handlers.
457+
458+
If the Tcl interpreter is not running the event loop and processing events, any
459+
:mod:`tkinter` calls made from threads other than the one running the Tcl
460+
interpreter will fail.
461+
462+
A number of special cases exist:
463+
464+
* Tcl/Tk libraries can be built so they are not thread-aware. In this case,
465+
:mod:`tkinter` calls the library from the originating Python thread, even
466+
if this is different than the thread that created the Tcl interpreter. A global
467+
lock ensures only one call occurs at a time.
468+
469+
* While :mod:`tkinter` allows you to create more than one instance of a :class:`Tk`
470+
object (with its own interpreter), all interpreters that are part of the same
471+
thread share a common event queue, which gets ugly fast. In practice, don't create
472+
more than one instance of :class:`Tk` at a time. Otherwise, it's best to create
473+
them in separate threads and ensure you're running a thread-aware Tcl/Tk build.
474+
475+
* Blocking event handlers are not the only way to prevent the Tcl interpreter from
476+
reentering the event loop. It is even possible to run multiple nested event loops
477+
or abandon the event loop entirely. If you're doing anything tricky when it comes
478+
to events or threads, be aware of these possibilities.
479+
480+
* There are a few select :mod:`tkinter` functions that presently work only when
481+
called from the thread that created the Tcl interpreter.
482+
483+
380484
Handy Reference
381485
---------------
382486

‎Doc/tutorial/controlflow.rst

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -651,7 +651,7 @@ Here's an example that fails due to this restriction::
651651
>>> function(0, a=0)
652652
Traceback (most recent call last):
653653
File "<stdin>", line 1, in <module>
654-
TypeError: function() got multiple values for keyword argument 'a'
654+
TypeError: function() got multiple values for argument 'a'
655655

656656
When a final formal parameter of the form ``**name`` is present, it receives a
657657
dictionary (see :ref:`typesmapping`) containing all keyword arguments except for
@@ -787,7 +787,7 @@ parameters as there is a ``/`` in the function definition::
787787
>>> pos_only_arg(arg=1)
788788
Traceback (most recent call last):
789789
File "<stdin>", line 1, in <module>
790-
TypeError: pos_only_arg() got an unexpected keyword argument 'arg'
790+
TypeError: pos_only_arg() got some positional-only arguments passed as keyword arguments: 'arg'
791791

792792
The third function ``kwd_only_args`` only allows keyword arguments as indicated
793793
by a ``*`` in the function definition::
@@ -817,7 +817,7 @@ definition::
817817
>>> combined_example(pos_only=1, standard=2, kwd_only=3)
818818
Traceback (most recent call last):
819819
File "<stdin>", line 1, in <module>
820-
TypeError: combined_example() got an unexpected keyword argument 'pos_only'
820+
TypeError: combined_example() got some positional-only arguments passed as keyword arguments: 'pos_only'
821821

822822

823823
Finally, consider this function definition which has a potential collision between the positional argument ``name`` and ``**kwds`` which has ``name`` as a key::

‎Include/internal/pycore_code.h

Lines changed: 13 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -308,13 +308,21 @@ int _Py_Specialize_LoadGlobal(PyObject *globals, PyObject *builtins, _Py_CODEUNI
308308
int _Py_Specialize_LoadMethod(PyObject *owner, _Py_CODEUNIT *instr, PyObject *name, SpecializedCacheEntry *cache);
309309
int _Py_Specialize_BinarySubscr(PyObject *sub, PyObject *container, _Py_CODEUNIT *instr);
310310

311-
#define SPECIALIZATION_STATS 0
312-
#define SPECIALIZATION_STATS_DETAILED 0
313-
#define SPECIALIZATION_STATS_TO_FILE 0
311+
#define PRINT_SPECIALIZATION_STATS 0
312+
#define PRINT_SPECIALIZATION_STATS_DETAILED 0
313+
#define PRINT_SPECIALIZATION_STATS_TO_FILE 0
314+
315+
#ifdef Py_DEBUG
316+
#define COLLECT_SPECIALIZATION_STATS 1
317+
#define COLLECT_SPECIALIZATION_STATS_DETAILED 1
318+
#else
319+
#define COLLECT_SPECIALIZATION_STATS PRINT_SPECIALIZATION_STATS
320+
#define COLLECT_SPECIALIZATION_STATS_DETAILED PRINT_SPECIALIZATION_STATS_DETAILED
321+
#endif
314322

315323
#define SPECIALIZATION_FAILURE_KINDS 20
316324

317-
#if SPECIALIZATION_STATS
325+
#if COLLECT_SPECIALIZATION_STATS
318326

319327
typedef struct _stats {
320328
uint64_t specialization_success;
@@ -324,7 +332,7 @@ typedef struct _stats {
324332
uint64_t miss;
325333
uint64_t deopt;
326334
uint64_t unquickened;
327-
#if SPECIALIZATION_STATS_DETAILED
335+
#if COLLECT_SPECIALIZATION_STATS_DETAILED
328336
uint64_t specialization_failure_kinds[SPECIALIZATION_FAILURE_KINDS];
329337
#endif
330338
} SpecializationStats;

‎Include/opcode.h

Lines changed: 5 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)
0