-
-
Notifications
You must be signed in to change notification settings - Fork 32.4k
bpo-32124: Document C functions safe before init #4540
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 3 commits
b1375be
938684e
12b3b5f
b58aba4
b2fc166
e01c316
f1c305b
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -7,6 +7,227 @@ | |
Initialization, Finalization, and Threads | ||
***************************************** | ||
|
||
.. _pre-init-safe: | ||
|
||
Before Python Initialization | ||
============================ | ||
|
||
In an application embedding Python, the :c:func:`Py_Initialize` function must | ||
be called before using any other Python/C API functions; with the exception of | ||
a few functions and the :ref:`global configuration variables <global-conf-vars>`: | ||
|
||
The following functions can be safetely called before Python is initialized: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Typo :) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed. Crap. "safetely" looked weird, so I used my spell check: "Google safetely", but I found results. Maybe there is an issue with my spell checker? :-) |
||
|
||
* Configuration functions: | ||
|
||
* :c:func:`PyImport_AppendInittab` | ||
* :c:func:`PyImport_ExtendInittab` | ||
* :c:func:`PyInitFrozenExtensions` | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why is PyInitFrozenExtensions() in this list? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It was a request from @serhiy-storchaka. I don't know the rationale :-) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It is called before |
||
* :c:func:`PyMem_GetAllocator` | ||
* :c:func:`PyMem_SetAllocator` | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. How much of the PyMem_*() API do we really need to promise to embedders? AFAICS, we've only required them to use PyMem_RawFree() before initialization, due to Py_DecodeLocale(). The less we promise pre-init, the better. Ideally we would provide equivalent pre-init functions, along with pre-init allocator state, that embedders could use if they needed. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Setting the allocators pre-initialization is required, as that allows the embedding application to control exactly how CPython allocates memory during startup. |
||
* :c:func:`PyMem_SetupDebugHooks` | ||
* :c:func:`PyObject_GetArenaAllocator` | ||
* :c:func:`PyObject_SetArenaAllocator` | ||
* :c:func:`Py_GetBuildInfo` | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would group Py_GetBuildInfo() etc as "informative functions". |
||
* :c:func:`Py_GetCompiler` | ||
* :c:func:`Py_GetCopyright` | ||
* :c:func:`Py_GetPlatform` | ||
* :c:func:`Py_GetProgramName` | ||
* :c:func:`Py_GetVersion` | ||
* :c:func:`Py_SetPath` | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Py_GetPath() must not be called before Py_Initialize(). I added other Py_GetXXX() functions. |
||
* :c:func:`Py_SetProgramName` | ||
* :c:func:`Py_SetPythonHome` | ||
* :c:func:`Py_SetStandardStreamEncoding` | ||
|
||
* Utilities: | ||
|
||
* :c:func:`Py_DecodeLocale` | ||
|
||
* Memory allocators: | ||
|
||
* :c:func:`PyMem_RawMalloc` | ||
* :c:func:`PyMem_RawRealloc` | ||
* :c:func:`PyMem_RawCalloc` | ||
* :c:func:`PyMem_RawFree` | ||
* :c:func:`PyMem_Malloc` | ||
* :c:func:`PyMem_Realloc` | ||
* :c:func:`PyMem_Calloc` | ||
* :c:func:`PyMem_Free` | ||
* :c:func:`PyObject_Malloc` | ||
* :c:func:`PyObject_Realloc` | ||
* :c:func:`PyObject_Calloc` | ||
* :c:func:`PyObject_Free` | ||
|
||
|
||
.. note:: | ||
|
||
:c:func:`Py_GetPath`, :c:func:`Py_GetPrefix`, :c:func:`Py_GetExecPrefix` and | ||
:c:func:`Py_GetProgramFullPath` and :c:func:`Py_GetPythonHome` **should not | ||
be called** before :c:func:`Py_Initialize`. | ||
|
||
|
||
.. _global-conf-vars: | ||
|
||
Global configuration variables | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In some ways, I'd rather we not advertise these flags as global variables, and particularly not as stable pre-init state. If the goal is PEP 432 then further publicizing this state helps to entrench and persist the API from which we're trying to move away. On the other hand, PEP 432 and its API is not accepted (yet) and the status quo suggests that the documentation you've added here for existing state is useful. From my biased position, I'd favor leaving this part out for now, but only if we're serious about PEP 432. :) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. For PEP 432, I want us to consider "Don't break backwards compatibility with the current embedding API" as a hard constraint. Part of that process is going to be reverse engineering what the current embedding API actually is in practice, since we don't have it fully documented, and we definitely don't have it fully tested. So in a post-PEP-432 world, we'll end up with a situation where we have a "Legacy Embedding API" that we promise not to break (at least, not prior to Python 4.0 at the very earliest), and then a more fine-grained configuration-struct-based one. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
What do you mean by "not stable"? The inital state of these flags is well defined. These flags are already actively used in the wild, it's lilely to only way to tune some Python options. Not documenting them only makes Python harder to embed, it doesn't prevent users to use it. Moreover, variables are exported as public symbols (prefixed with "Py", not "_Py"). |
||
============================== | ||
|
||
Python has variables for the global configuration to control different features | ||
and options. By default, these flags are controlled by :ref:`command line | ||
options <using-on-interface-options>`. | ||
|
||
When a flag is set by an option, the value of the flag is the number of times | ||
that the option was set. For example, ``-b`` sets :c:data:`Py_BytesWarningFlag` | ||
to 1 and ``-bb`` sets :c:data:`Py_BytesWarningFlag` to 2. | ||
|
||
.. c:var:: Py_BytesWarningFlag | ||
|
||
Issue a warning when comparing :class:`bytes` or :class:`bytearray` with | ||
:class:`str` or :class:`bytes` with :class:`int`. Issue an error if greater | ||
or equal to ``2``. | ||
|
||
Set by the :option:`-b` option. | ||
|
||
.. c:var:: Py_DebugFlag | ||
|
||
Turn on parser debugging output (for wizards only, depending on compilation | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I see the word wizards and thought of magicians :) Do you mean something like setup wizard? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think Victor meant real wizards, like his and me. 🎩 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ok in that case I suggest something less ambiguous, like "for expert" or "for experienced core devs" only. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This wording already is used two times in the documentation. In particularly in the documentation for the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In sum, this option is only for those who understand this jargon. 😉 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I copied the doc from the -d option documentation. Ok, I replaced "for wizards only" with "for expert only". There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would keep the original wording. It is used also in the man page since the initial edition in 1994. |
||
options). | ||
|
||
Set by the :option:`-d` option and the :envvar:`PYTHONDEBUG` environment | ||
variable. | ||
|
||
.. c:var:: Py_DontWriteBytecodeFlag | ||
|
||
If set to non-zero, Python won't try to write ``.pyc`` files on the | ||
import of source modules. | ||
|
||
Set by the :option:`-B` option and the :envvar:`PYTHONDONTWRITEBYTECODE` | ||
environment variable. | ||
|
||
.. c:var:: Py_HashRandomizationFlag | ||
|
||
Set to ``1`` if the :envvar:`PYTHONHASHSEED` environment variable is set to | ||
a non-empty string. | ||
|
||
If the flag is non-zero, read the :envvar:`PYTHONHASHSEED` environment | ||
variable to initialize the secret hash seed. | ||
|
||
.. c:var:: Py_IgnoreEnvironmentFlag | ||
|
||
Ignore all :envvar:`PYTHON*` environment variables, e.g. | ||
:envvar:`PYTHONPATH` and :envvar:`PYTHONHOME`, that might be set. | ||
|
||
Set by the :option:`-E` and :option:`-I` options. | ||
|
||
.. c:var:: Py_InspectFlag | ||
|
||
When a script is passed as first argument or the :option:`-c` option is used, | ||
enter interactive mode after executing the script or the command, even when | ||
:data:`sys.stdin` does not appear to be a terminal. | ||
|
||
Set by the :option:`-i` option and the :envvar:`PYTHONINSPECT` environment | ||
variable. | ||
|
||
.. c:var:: Py_InteractiveFlag | ||
|
||
Set by the :option:`-i` option. | ||
|
||
.. c:var:: Py_IsolatedFlag | ||
|
||
Run Python in isolated mode. In isolated mode :data:`sys.path` contains | ||
neither the script's directory nor the user's site-packages directory. | ||
|
||
Set by the :option:`-I` option. | ||
|
||
.. versionadded:: 3.4 | ||
|
||
.. c:var:: Py_LegacyWindowsFSEncodingFlag | ||
|
||
If the flag is non-zero, use the ``mbcs`` encoding instead of the UTF-8 | ||
encoding for the filesystem encoding. | ||
|
||
Set to ``1`` if the :envvar:`PYTHONLEGACYWINDOWSFSENCODING` environment | ||
variable is set to a non-empty string. | ||
|
||
See also the :pep:`529`. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
(without "the") There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I picked: See :pep: |
||
|
||
Availability: Windows. | ||
|
||
.. c:var:: Py_LegacyWindowsStdioFlag | ||
|
||
If the flag is non-zero, use :class:`io.FileIO` instead of | ||
:class:`WindowsConsoleIO` for :mod:`sys` standard streams. | ||
|
||
Set to ``1`` if the :envvar:`PYTHONLEGACYWINDOWSSTDIO` environment | ||
variable is set to a non-empty string. | ||
|
||
See also the :pep:`528`. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Remove "the". |
||
|
||
Availability: Windows. | ||
|
||
.. c:var:: Py_NoSiteFlag | ||
|
||
Disable the import of the module :mod:`site` and the site-dependent | ||
manipulations of :data:`sys.path` that it entails. Also disable these | ||
manipulations if :mod:`site` is explicitly imported later (call | ||
:func:`site.main` if you want them to be triggered). | ||
|
||
Set by the :option:`-S` option. | ||
|
||
.. c:var:: Py_NoUserSiteDirectory | ||
|
||
Don't add the :data:`user site-packages directory <site.USER_SITE>` to | ||
:data:`sys.path`. | ||
|
||
Set by the :option:`-s` and :option:`-I` options, and the | ||
:envvar:`PYTHONNOUSERSITE` environment variable. | ||
|
||
.. c:var:: Py_OptimizeFlag | ||
|
||
Set by the :option:`-O` option and the :envvar:`PYTHONOPTIMIZE` environment | ||
variable. | ||
|
||
.. c:var:: Py_QuietFlag | ||
|
||
Don't display the copyright and version messages even in interactive mode. | ||
|
||
Set by the :option:`-q` option. | ||
|
||
.. versionadded:: 3.2 | ||
|
||
.. c:var:: Py_UnbufferedStdioFlag | ||
|
||
Force the stdout and stderr streams to be unbuffered. | ||
|
||
Set by the :option:`-u` option and the :envvar:`PYTHONUNBUFFERED` | ||
environment variable. | ||
|
||
.. c:var:: Py_VerboseFlag | ||
|
||
Print a message each time a module is initialized, showing the place | ||
(filename or built-in module) from which it is loaded. If greater or equal | ||
to ``2``, print a message for each file that is checked for when | ||
searching for a module. Also provides information on module cleanup at exit. | ||
|
||
Set by the :option:`-v` option and the :envvar:`PYTHONVERBOSE` environment | ||
variable. | ||
|
||
|
||
Other variables: | ||
|
||
.. c:var:: Py_FrozenFlag | ||
|
||
Suppress error messages when calculating the module search path in | ||
:c:func:`Py_GetPath`. | ||
|
||
Private flag used by ``_freeze_importlib`` and ``frozenmain`` programs. | ||
|
||
.. c:var:: Py_UseClassExceptionsFlag | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not sure that it's a good idea to document this flag, since I proposed to remove the flag https://bugs.python.org/issue32125 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Removed |
||
|
||
Deprecated and ignored flag, used to enable class exceptions prior Python | ||
2.0. | ||
|
||
.. deprecated:: 2.0 | ||
|
||
|
||
10000 | Initializing and finalizing the interpreter | |
=========================================== | ||
|
@@ -27,9 +248,11 @@ Initializing and finalizing the interpreter | |
single: PySys_SetArgvEx() | ||
single: Py_FinalizeEx() | ||
|
||
Initialize the Python interpreter. In an application embedding Python, this | ||
should be called before using any other Python/C API functions; with the | ||
exception of :c:func:`Py_SetProgramName`, :c:func:`Py_SetPythonHome` and :c:func:`Py_SetPath`. This initializes | ||
Initialize the Python interpreter. In an application embedding Python, | ||
this should be called before using any other Python/C API functions; see | ||
:ref:`Before Python Initialization <pre-init-safe>` for the few exceptions. | ||
|
||
This initializes | ||
the table of loaded modules (``sys.modules``), and creates the fundamental | ||
modules :mod:`builtins`, :mod:`__main__` and :mod:`sys`. It also initializes | ||
the module search path (``sys.path``). It does not set ``sys.argv``; use | ||
|
@@ -1092,7 +1315,7 @@ Python-level trace functions in previous versions. | |
+------------------------------+--------------------------------------+ | ||
|
||
|
||
.. c:var:: int PyTrace_CALL | ||
.. c:var::: int PyTrace_CALL | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Isn't the third colon redundant? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Oops, it's a mistake. Fixed. |
||
|
||
The value of the *what* parameter to a :c:type:`Py_tracefunc` function when a new | ||
call to a function or method is being reported, or a new entry into a generator. | ||
|
@@ -1101,7 +1324,7 @@ Python-level trace functions in previous versions. | |
frame. | ||
|
||
|
||
.. c:var:: int PyTrace_EXCEPTION | ||
.. c:var::: int PyTrace_EXCEPTION | ||
|
||
The value of the *what* parameter to a :c:type:`Py_tracefunc` function when an | ||
exception has been raised. The callback function is called with this value for | ||
|
@@ -1112,31 +1335,31 @@ Python-level trace functions in previous versions. | |
these events; they are not needed by the profiler. | ||
|
||
|
||
.. c:var:: int PyTrace_LINE | ||
.. c:var::: int PyTrace_LINE | ||
|
||
The value passed as the *what* parameter to a trace function (but not a | ||
profiling function) when a line-number event is being reported. | ||
|
||
|
||
.. c:var:: int PyTrace_RETURN | ||
.. c:var::: int PyTrace_RETURN | ||
|
||
The value for the *what* parameter to :c:type:`Py_tracefunc` functions when a | ||
call is returning without propagating an exception. | ||
|
||
|
||
.. c:var:: int PyTrace_C_CALL | ||
.. c:var::: int PyTrace_C_CALL | ||
|
||
The v 6651 alue for the *what* parameter to :c:type:`Py_tracefunc` functions when a C | ||
function is about to be called. | ||
|
||
|
||
.. c:var:: int PyTrace_C_EXCEPTION | ||
.. c:var::: int PyTrace_C_EXCEPTION | ||
|
||
The value for the *what* parameter to :c:type:`Py_tracefunc` functions when a C | ||
function has raised an exception. | ||
|
||
|
||
.. c:var:: int PyTrace_C_RETURN | ||
.. c:var::: int PyTrace_C_RETURN | ||
|
||
The value for the *what* parameter to :c:type:`Py_tracefunc` functions when a C | ||
function has returned. | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PyImport_AppendInittab(), PyImport_ExtendInittab(), PyInitFrozenExtensions().