8000 bpo-45953: Statically allocate the main interpreter (and initial thread state). by ericsnowcurrently · Pull Request #29883 · python/cpython · GitHub
[go: up one dir, main page]
Skip to content

bpo-45953: Statically allocate the main interpreter (and initial thread state). #29883

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 78 commits into from
Jan 12, 2022
Merged

bpo-45953: Statically allocate the main interpreter (and initial thread state). #29883

Changes from 1 commit
Commits
Show all changes
78 commits
Select commit Hold shift + click to select a range
5e6eda9
Return void from _PyEval_InitState().
ericsnowcurrently Nov 18, 2021
de12b98
Fix a memory leak.
ericsnowcurrently Nov 18, 2021
035eff3
Move thread-related interpreter state into a sub-struct.
ericsnowcurrently Nov 17, 2021
1569a53
Factor out init_threadstate().
ericsnowcurrently Nov 18, 2021
518e3f3
Do not initialize a thread state if already initialized.
ericsnowcurrently Nov 18, 2021
9995a84
Avoid allocating the first thread state.
ericsnowcurrently Nov 18, 2021
f22d0da
Factor out init_interpreter().
ericsnowcurrently Nov 18, 2021
cf39d57
Preallocate the main interpreter.
ericsnowcurrently Nov 18, 2021
b840213
Add a note about expectations for init_interpreter().
ericsnowcurrently Dec 1, 2021
3bf69b0
Add a NEWS entry.
ericsnowcurrently Dec 1, 2021
4de466e
Add _PyInterpreterState_Main().
ericsnowcurrently Dec 1, 2021
9bb32a6
Fix a check for the main interpreter.
ericsnowcurrently Dec 2, 2021
385f298
Ensure no subinterpreters are created until after the main interprete…
ericsnowcurrently Dec 2, 2021
354997f
Make sure the interpreter is fully initialized.
ericsnowcurrently Dec 2, 2021
9bbf8a0
Use PyMem_RawCalloc() to allocate thread states.
ericsnowcurrently Dec 2, 2021
8d1cf35
Make sure the thread state is fully initialized.
ericsnowcurrently Dec 2, 2021
188aed8
Clear the preallocated states when "deleted".
ericsnowcurrently Dec 2, 2021
e0da483
Statically initialize the "preallocated" states.
ericsnowcurrently Dec 2, 2021
46c7767
Fix a typo.
ericsnowcurrently Dec 3, 2021
7ca8d1c
Stop memset'ing the statically initialized _PyRuntime.
ericsnowcurrently Dec 3, 2021
ed6c273
Fix typos.
ericsnowcurrently Dec 3, 2021
29e1764
Only expose _PyThreadState_INIT in the internal API.
ericsnowcurrently Dec 3, 2021
1a83910
If needed, "statically" initialize the preallocated thread state.
ericsnowcurrently Dec 3, 2021
8631805
Check if _preallocated is initialized when initializing from other.
ericsnowcurrently Dec 3, 2021
a0569bb
init_interpreter_from_other -> init_interpreter_static_data.
ericsnowcurrently Dec 3, 2021
4c3c0f9
"Statically" initialize the preallocated thread state.
ericsnowcurrently Dec 3, 2021
47bfa7e
Clear _PyRuntime during finalization.
ericsnowcurrently Dec 3, 2021
dd69ffa
Track if interpreter or thread state has been initialized.
ericsnowcurrently Dec 3, 2021
ba75803
Zero-out states during re-init instead of fini.
ericsnowcurrently Dec 3, 2021
eef3194
Use Py_uintptr_t instead of ssize_t.
ericsnowcurrently Dec 3, 2021
8703297
Add Py*State._initialized.
ericsnowcurrently Dec 7, 2021
d1eddad
Factor out init_threadstate().
ericsnowcurrently Dec 7, 2021
8a8150c
Factor out free_threadstate().
ericsnowcurrently Dec 7, 2021
7a37237
Factor out init_interpreter().
ericsnowcurrently Dec 7, 2021
b2073ac
Factor out free_interpreter().
ericsnowcurrently Dec 7, 2021
21a522d
Factor out init_runtime().
ericsnowcurrently Dec 7, 2021
7b376d4
Factor out _PyRuntimeState_reset().
ericsnowcurrently Dec 7, 2021
562ee42
Merge branch 'main' into preallocate-main
ericsnowcurrently Dec 8, 2021
ed8f5cd
Add _PyInterpreterState_Main().
ericsnowcurrently Dec 1, 2021
6da97b7
Drop an outdated comment.
ericsnowcurrently Dec 8, 2021
cf24942
Fix a check for the main interpreter.
ericsnowcurrently Dec 2, 2021
fac6af6
Merge branch 'main' into preallocate-main
ericsnowcurrently Dec 8, 2021
4aa2073
Merge branch 'main' into preallocate-main
ericsnowcurrently Dec 8, 2021
f135168
Merge branch 'main' into preallocate-main
ericsnowcurrently Dec 14, 2021
2b4ba04
Move *_reset() out of the Include/.
ericsnowcurrently Dec 14, 2021
24d1de2
Pass the parent context to init_*_static_data().
ericsnowcurrently Dec 14, 2021
84ae7db
Drop reset_*() and init_*_static_data().
ericsnowcurrently Dec 14, 2021
e9de30c
Drop _Py_global_objects_reset().
ericsnowcurrently Dec 14, 2021
51a32ce
Drop _preallocated.initialized.
ericsnowcurrently Dec 16, 2021
7104518
Drop _preallocated_used.
ericsnowcurrently Dec 16, 2021
f7a8389
Drop init of PyThreadState._preallocated for now.
ericsnowcurrently Dec 16, 2021
18080a1
Note what could be pre-allocated.
ericsnowcurrently Dec 16, 2021
0457119
Add Py*State._static.
ericsnowcurrently Dec 16, 2021
923ca7a
Avoid an empty struct.
ericsnowcurrently Dec 17, 2021
99c2ae1
Drop PyInterpreterState._preallocated.initialized.
ericsnowcurrently Jan 4, 2022
3bc73ee
Drop an outdated comment.
ericsnowcurrently Jan 4, 2022
c1808ae
Drop outdated comments.
ericsnowcurrently Jan 4, 2022
7b98e59
Clarify what remains to be pre-allocated for _PyRuntimeState.
ericsnowcurrently Jan 4, 2022
0383eae
Explain what _preallocated is for.
ericsnowcurrently Jan 4, 2022
c959db0
Clarify why some data isn't pre-allocated.
ericsnowcurrently Jan 4, 2022
5b34b42
Fold _PyThreadState_INIT into _PyInterpreterState_INIT.
ericsnowcurrently Jan 10, 2022
8000
837bfe6
Fold _PyInterpreterState_INIT into _PyRuntimeState_INIT.
ericsnowcurrently Jan 10, 2022
ebefa35
Drop PyThreadState._preallocated.
ericsnowcurrently Jan 10, 2022
eb7a080
tstate -> initial_thread.
ericsnowcurrently Jan 11, 2022
1ef4a39
Mirror the top-level structure in _preallocated.
ericsnowcurrently Jan 11, 2022
771e574
Merge branch 'main' into preallocate-main
ericsnowcurrently Jan 11, 2022
b5c43f1
Clarify some comments.
ericsnowcurrently Jan 11, 2022
bfc3906
Fold in _PyRuntimeState._preallocated.
ericsnowcurrently Jan 11, 2022
9b25c8b
Make "initial" const.
ericsnowcurrently Jan 11, 2022
e776c31
Fold in PyInterpreterState._preallocated.
ericsnowcurrently Jan 11, 2022
c6bda1f
Move the initial thread to the bottom of PyInterpreterState.
ericsnowcurrently Jan 11, 2022
8ab6261
Split up _PyRuntimeState_INIT for less nesting and clutter.
ericsnowcurrently Jan 11, 2022
ff8de28
Fix a comment.
ericsnowcurrently Jan 11, 2022
b28afb3
Move the main interpreter to the bottom of _PyRuntimeState.
ericsnowcurrently Jan 11, 2022
0d30300
Clarify some comments.
ericsnowcurrently Jan 11, 2022
e193794
Add a comment for the _static field.
ericsnowcurrently Jan 12, 2022
e9f1262
Move the comments about pre-allocated data down.
ericsnowcurrently Jan 12, 2022
a6a6471
Fix a typo.
ericsnowcurrently Jan 12, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Only expose _PyThreadState_INIT in the internal API.
  • Loading branch information
@ericsnowcurrently
ericsnowcurrently committed Dec 3, 2021
commit 29e1764cb279acb5dd182495a763d1b930486f30
2 changes: 2 additions & 0 deletions Include/cpython/pystate.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -175,12 +175,14 @@ struct _ts {
} _preallocated;
};

#ifdef Py_BUILD_CORE
#define _PyThreadState_INIT \
{ \
._preallocated = { \
.initialized = 1, \
}, \
}
#endif

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can it be turned into a variant below?

#define _PyThreadState_INIT { ._preallocated.initialized = 1 }

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the tip. What's the advantage?

Copy link
Member
@arhadthedev arhadthedev 6E6E Dec 4, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, this is a minor adjustment done while it does not require a separate pull request.

More compact declaration is easier to grasp in a single eye swipe. Like _PyThreadState_INIT is the preallocated's initialized set to one versus _PyThreadState_INIT is... the preallocated's initialized set to one... ah, that's it. In isolation of few lines it sounds funny but when a programmer reads the whole file to assemble a picture what pystate is and what it is capable of, they read in zigzags only.

// Alias for backward compatibility with Python 3.8
#define _PyInterpreterState_Get PyInterpreterState_Get
Expand Down
0