8000 PEP 734: Multiple Interpreters in the Stdlib by ericsnowcurrently · Pull Request #3523 · python/peps · GitHub
[go: up one dir, main page]

Skip to content

PEP 734: Multiple Interpreters in the Stdlib #3523

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 87 commits into from
Nov 28, 2023
Merged
Changes from 1 commit
Commits
Show all changes
87 commits
Select commit Hold shift + click to select a range
c8d079f
Add a post-history entry.
ericsnowcurrently Nov 3, 2023
1b7f9eb
Add PEP 734.
ericsnowcurrently Nov 7, 2023
4d3f06e
Mark PEP 734 as the superseder of PEP 554.
ericsnowcurrently Nov 7, 2023
73f275a
Fix typos.
ericsnowcurrently Nov 7, 2023
bc0ab5c
Fix wording.
ericsnowcurrently Nov 7, 2023
3ebbd27
Remove the Content-Type header.
ericsnowcurrently Nov 7, 2023
dc192c9
Fix formatting.
ericsnowcurrently Nov 7, 2023
b6cd4ee
Clean up examples.
ericsnowcurrently Nov 7, 2023
5b6ca1f
Fix wording.
ericsnowcurrently Nov 7, 2023
2b6cba4
Clarify about Interpreter.exec().
ericsnowcurrently Nov 7, 2023
5dbfef2
Add Introduction and Motivation sections.
ericsnowcurrently Nov 13, 2023
1969079
Make InterpreterPoolExecutor part of the PEP.
ericsnowcurrently Nov 13, 2023
7e2947b
Do not use tables for the API summary.
ericsnowcurrently Nov 13, 2023
309e826
Clean up the builtin exec() comparison section.
ericsnowcurrently Nov 13, 2023
a91e77d
Move a note to the rationale.
ericsnowcurrently Nov 14, 2023
1f921af
Be more clear about channel serialization.
ericsnowcurrently Nov 14, 2023
8368c84
Move the note about exception propagation to the rationale.
ericsnowcurrently Nov 14, 2023
d038470
Drop text from the isolation section.
ericsnowcurrently Nov 14, 2023
0670951
Fix a link.
ericsnowcurrently Nov 14, 2023
5f9d802
fix executor
ericsnowcurrently Nov 14, 2023
a7f4c81
Clarify about channels.
ericsnowcurrently Nov 14, 2023
3b5a174
set_main_attrs() -> prepare___main__()
ericsnowcurrently Nov 14, 2023
3b55bd4
Clarify about the InterpreterID and ChannelID objects.
ericsnowcurrently Nov 14, 2023
99a10e0
Add a note about memoryview.
ericsnowcurrently Nov 14, 2023
84c7de8
Clarify about when to share channels.
ericsnowcurrently Nov 14, 2023
2bead4e
Elaborate on why objects cannot be directly shareable.
ericsnowcurrently Nov 14, 2023
e7e2472
Specify __hash__() and __eq__().
ericsnowcurrently Nov 14, 2023
b5fc432
Fix up the examples.
ericsnowcurrently Nov 14, 2023
16aabec
Drop the Documentation section.
ericsnowcurrently Nov 14, 2023
63cac3e
Expand the note about the propagated exception.
ericsnowcurrently Nov 14, 2023
cb6cd3e
formatting
ericsnowcurrently Nov 14, 2023
256eb4e
Add an example for how exec() ignores any return value.
ericsnowcurrently Nov 14, 2023
5879ff3
channels -> queues
ericsnowcurrently Nov 14, 2023
3ddbf02
Add a section about synchronization.
ericsnowcurrently Nov 15, 2023
7bc3315
Fix API spec formatting.
ericsnowcurrently Nov 15, 2023
5dd78bc
Tweak API spec formatting.
ericsnowcurrently Nov 15, 2023
5f8e4c7
Match queue.Queue more closely.
ericsnowcurrently Nov 15, 2023
408bc81
Update the Synchronization section.
ericsnowcurrently Nov 15, 2023
db82b9b
Add exception classes.
ericsnowcurrently Nov 15, 2023
8461f0f
Drop Queue.close().
ericsnowcurrently Nov 15, 2023
3f3188f
Drop queue put/get sync.
ericsnowcurrently Nov 15, 2023
799b28e
Fix the examples.
ericsnowcurrently Nov 15, 2023
11cc851
C-API -> C API.
ericsnowcurrently Nov 15, 2023
e0ee12b
formatting
ericsnowcurrently Nov 15, 2023
ab499fd
Clarify about OS threads vs. thread states.
ericsnowcurrently Nov 20, 2023
62607fb
Clarify some wording.
ericsnowcurrently Nov 20, 2023
89a205b
Clarify the relationship beween thread state and interpreters.
ericsnowcurrently Nov 20, 2023
2f76817
Clarify about the ID attributes.
ericsnowcurrently Nov 20, 2023
c517685
Clarify about Py_NewInterpreter().
ericsnowcurrently Nov 20, 2023
8560abb
Clarify about interpreter init.
ericsnowcurrently Nov 20, 2023
4d8ee95
Avoid "we".
ericsnowcurrently Nov 20, 2023
c1662e3
Clarify about the C API runtime context.
ericsnowcurrently Nov 21, 2023
f5fc9b5
Clarify about finalization.
ericsnowcurrently Nov 21, 2023
553e470
Distinguish between mutable and immutable shared state.
ericsnowcurrently Nov 21, 2023
b4bc397
prepare___main__() -> prepare_main()
ericsnowcurrently Nov 21, 2023
6115a05
Clarify the rationale for prepare_main().
ericsnowcurrently Nov 21, 2023
ba90c62
Clarify about RunFailedError.
ericsnowcurrently Nov 21, 2023
c5bf70a
Clarify about shareable objects.
ericsnowcurrently Nov 21, 2023
6ffb163
Clarify how interpreters and queues are exposed by ID.
ericsnowcurrently Nov 21, 2023
03afff2
Note that there is one Interpreter object per ID.
ericsnowcurrently Nov 21, 2023
f2529e6
Clarify that interpreters may have zero thread states.
ericsnowcurrently Nov 21, 2023
3a238e9
Use put() in the shared resource example.
ericsnowcurrently Nov 21, 2023
6d38e12
Drop the second synchronizing example.
ericsnowcurrently Nov 21, 2023
4f2e4e2
Clarify about RunFailedError.
ericsnowcurrently Nov 21, 2023
5df3837
Move the comparison section up to the introduction.
ericsnowcurrently Nov 22, 2023
fee27e4
identical -> strictly equivalent
ericsnowcurrently Nov 22, 2023
90c8913
Clarify about id.
ericsnowcurrently Nov 22, 2023
774c125
Fix a sentence.
ericsnowcurrently Nov 22, 2023
1798f24
Clarify about object uniqueness.
ericsnowcurrently Nov 22, 2023
e4ac108
Drop __eq__().
ericsnowcurrently Nov 22, 2023
0609ab3
Clarify about maxsize.
ericsnowcurrently Nov 22, 2023
7c629e6
Add a note about stability of getter return values.
ericsnowcurrently Nov 22, 2023
36b1e5d
Clarify about put_nowait().
ericsnowcurrently Nov 22, 2023
463259a
Clarify about shareable object equivalence.
ericsnowcurrently Nov 22, 2023
ff9342b
simplify
ericsnowcurrently Nov 22, 2023
ea5c75e
Clarify about sharing queues.
ericsnowcurrently Nov 22, 2023
2f3fa4f
Clean up about InterpreterPoolExecutor.
ericsnowcurrently Nov 22, 2023
256347c
Fix examples.
ericsnowcurrently Nov 22, 2023
1427ccb
Use .put() in examples instead of .put_nowait().
ericsnowcurrently Nov 22, 2023
e8a518a
Clarify Interpreter.exec().
ericsnowcurrently Nov 22, 2023
ae69298
Drop the bullet about exec() discarding return values.
ericsnowcurrently Nov 22, 2023
8d4676a
Drop the get_nowait() "default" arg.
ericsnowcurrently Nov 22, 2023
d31c379
Add a CODEOWNERS entry.
ericsnowcurrently Nov 28, 2023
421804d
Interpreter.exec() -> .exec_sync()
ericsnowcurrently Nov 28, 2023
0e3e5a0
Clarify about the propagated exception.
ericsnowcurrently Nov 28, 2023
d9ae6b1
Add Interpreter.run().
ericsnowcurrently Nov 28, 2023
8424321
RunFailedError -> ExecFailure
ericsnowcurrently Nov 28, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Clean up the builtin exec() comparison section.
  • Loading branch information
ericsnowcurrently committed Nov 14, 2023
commit 309e8267b913639cd415dba80ca7a70ccb47237a
61 changes: 27 additions & 34 deletions peps/pep-0734.rst
Original file line number Diff line number Diff line change
Expand Up @@ -304,40 +304,33 @@ Comparison with builtins.exec()

``Interpreter.exec()`` is essentially the same as the builtin
``exec()``, except it targets a different interpreter, using that
interpreter's isolated state.

The builtin ``exec()`` runs in the current OS thread and pauses
whatever was running there, which resumes when ``exec()`` finishes.
No other threads are affected. (To avoid pausing the current thread,
run ``exec()`` in a ``threading.Thread``.)

``Interpreter.exec()`` works the same way.

The builtin ``exec()`` executes against a namespace, by default the
``__dict__`` of the current module (i.e. ``globals()``).
It uses that namespace as-is and does not clear it before or after.

``Interpreter.exec()`` works the same way.

...where the "current" module is always the ``__main__`` module
of the target interpreter. This is the same as how it works
for code run from the Python command-line (e.g. ``-m``) or the REPL.

The builtin ``exec()`` discards any object returned from the
executed code.

``Interpreter.exec()`` works the same way.

The builtin ``exec()`` propagates any uncaught exception from the code
it ran. The exception is raised from the ``exec()`` call in the
thread that originally called ``exec()``.

``Interpreter.exec()`` works the same way.

...with one slight difference. Rather than propagate the uncaught
exception directly, we raise an ``interpreters.RunFailedError``
with a snapshot of the uncaught exception (including its traceback)
as the ``__cause__``. Directly raising (a proxy of) the exception
interpreter's distinct runtime state.

Here are the relevant characteristics of the builtin ``exec()``,
for comparison:

* It runs in the current OS thread and pauses whatever was running there,
which resumes when ``exec()`` finishes. No other threads are affected.
(To avoid pausing the current thread, run ``exec()``
in a ``threading.Thread``.)
* It executes against a namespace, by default the ``__dict__`` of the
current module (i.e. ``globals()``). It uses that namespace as-is
and does not clear it before or after.
* When code is run from the command-line (e.g. ``-m``) or the REPL,
the "current" module is always the ``__main__`` module
of the target (main) interpreter.
* It discards any object returned from the executed code.
* It propagates any uncaught exception from the code it ran.
The exception is raised from the ``exec()`` call in the thread
that originally called ``exec()``.

The only difference is that, rather than propagate the uncaught
exception directly, ``Interpreter.exec()`` raises an
``interpreters.RunFailedError`` with a snapshot
(``traceback.TracebackException``) of the uncaught exception
(including its traceback) as the ``__cause__``.

Directly raising (a proxy of) the exception
is problematic since it's harder to distinguish between an error
in the ``Interpreter.exec()`` call and an uncaught exception
from the subinterpreter.
Expand Down
0