8000 PEP 669: Low Impact Monitoring for CPython (version 2) by markshannon · Pull Request #2183 · python/peps · GitHub
[go: up one dir, main page]
Skip to content

PEP 669: Low Impact Monitoring for CPython (version 2) #2183

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Dec 7, 2021
Merged

PEP 669: Low Impact Monitoring for CPython (version 2) #2183

Changes from 1 commit
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
2b3da81
Initial draft of high performance debugging and profiling PEP.
markshannon Aug 20, 2021
b9c6233
Refine events and extend API a bit.
markshannon Aug 20, 2021
613d1ad
Fix number, dates and clean up text.
markshannon Sep 13, 2021
1dc252c
Fix formatting errors.
markshannon Sep 13, 2021
ceb9540
More edits to PEP 669.
markshannon Sep 13, 2021
c6dbfdd
Change title of PEP 669
markshannon Sep 13, 2021
9cc6799
Add myself as owner of PEPs 659 and 669.
markshannon Sep 14, 2021
7e76267
Edits based on feedback.
markshannon Dec 3, 2021
845d751
Rewrite PEP 669 to have the VM handle instrumentation.
markshannon Dec 6, 2021
0a3ed1e
Merge branch 'main' into pep-669-v2
markshannon Dec 6, 2021
54e913b
A couple of edits.
markshannon Dec 6, 2021
254aeb3
Minor edits.
markshannon Dec 7, 2021
8431692
Use correct date.
markshannon Dec 7, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Edits based on feedback.
  • Loading branch information
@markshannon
markshannon committed Dec 3, 2021
commit 7e762670d553ed09c0260d5a936db87d7d4df12c
218 changes: 135 additions & 83 deletions pep-0669.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,16 +13,13 @@ Abstract

Using a profiler or debugger in CPython can have a severe impact on
performance. Slowdowns by an order of magnitude are not uncommon.
It does not have this bad.

This PEP proposes an API for instrumentation and monitoring of Python
programs running on CPython that will enable the insertion of instrumentation
and monitoring at low cost.
programs running on CPython that will enable the insertion of
instrumentation and monitoring at low cost.

Using the new API, code run under a debugger on 3.11 should easily outperform
code run without a debugger on 3.10.

Profiling will still slow down execution, but by much less than in 3.10.
Although this PEP does not specify an implementation, it is expected that
it will be implemented using the quickening step of PEP 659 [1]_.

Motivation
==========
Expand All @@ -39,59 +36,82 @@ Rationale

The quickening mechanism provided by PEP 659 provides a way to dynamically
modify executing Python bytecode. These modifications have no cost beyond
the parts of the code that are modified and a relatively low cost to those
the parts of the code that are modified and a relatively low cost to those
parts that are modified. We can leverage this to provide an efficient
mechanism for instrumentation and monitoring that was not possible in 3.10
or earlier.

By using quickening, we expect that code run under a debugger on 3.11
should outperform code run without a debugger on 3.10.
Profiling will still slow down execution, but by much less than in 3.10.


Specification
=============

There are two parts to this specification, instrumentation and monitoring.

Instrumentation occurs early in a program's life cycle and persists through
the lifetime of the program. It is expected to be pervasive, but fixed.
Instrumentation is designed to support profiling and coverage tools that
expect to be active for the entire lifetime of the program.
Both instrumentation and monitoring are performed by insertion of
checkpoints in a code object and by registering a callback to be
called whenever a checkpoint is reached.

Instrumentation is designed to support profiling and coverage tools and
works by inserting sets checkpoints into a code object which respond
to certain events.

Monitoring can occur at any point in the program's life and be applied
anywhere in the program. Monitoring points are expected to few.
The capabilities of monitoring are a superset of that of profiling,
but bulk insertion of monitoring points will be *much* more
expensive than insertion of instrumentation.
Monitoring is designed to support debuggers and similar tools, and
works by inserting individual checkpoints into code objects.
There are no events associated with monitoring checkpoints.

Both instrumentation and monitoring is performed by insertion of
checkpoints in a code object.
Monitoring checkpoints can be turned on or off individually.
Instrumentation checkpoints cannot.

Inserting checkpoints into a code object before it has executed is
guaranteed to be a low cost operation. However, once a code object
has been executed, even once, inserting checkpoints may become expensive,
possibly taking hundreds of milliseconds.

Checkpoints
-----------

A checkpoint is simply a point in code defined by a
``(codeobject, offset)`` pair.
Every time a checkpoint is reached, the registered callable is called.
``offset`` is the byte offset in the ``code.co_code`` object.

Every time a checkpoint is reached,
the relevant registered callables are called.

Instrumentation
---------------

Instrumentation supports the bulk insertion of checkpoints, but does not
allow insertion or removal of checkpoints after code has started to execute.
Instrumentation works by bulk inserting checkpoints into a code object to
support different types of events.

Events
''''''

The events are::
As a code object executes various events occur that might be of interest
to tools. By instrumenting code objects and registering callback functions
tools can respond to these events in any way that suits them.

* BRANCH: A conditional branch is reached.
* JUMPBACK: A backwards, unconditional branch is reached.
* ENTER: A Python function is entered.
* EXIT: A Python function exits normally (without an exception).
* UNWIND: A Python function exits with an unhandled exception.
* C_CALL: A call to any object that is not a Python function.
* C_RETURN: A return from any object that is not a Python function.
* RAISE: An exception is raised.
* EXCEPT: An exception is handled.
Code objects can be instrumented for any (or all) of the following events:

* BRANCH: A conditional branch is reached.
* JUMPBACK: A backwards, unconditional branch is reached.
* ENTER: A Python function is entered.
* EXIT: A Python function exits normally (without an exception).
* UNWIND: A Python function exits with an unhandled exception.
* C_CALL: A call to any object that is not a Python function.
* C_RETURN: A return from any object that is not a Python function.
* RAISE: An exception is raised, for any reason.
* EXCEPT: Control is transferred to an exception handler.
* LINE: Whenever the first instruction in a line is reached.

For each ``ENTER`` event there will be a corresponding
``EXIT`` or ``UNWIND`` event.
For each ``C_CALL`` event there will be a corresponding
``C_RETURN`` or ``RAISE`` event.
``RAISE`` events will be followed by an ``EXCEPT`` or ``UNWIND`` event.

All events are integer powers of two and can be bitwise or-ed together to
instrument multiple events.
Expand All @@ -105,7 +125,8 @@ Code objects can be instrumented by calling::

Code objects must be instrumented before they are executed.
An exception will be raised if the code object has been executed before it
is instrumented.
is instrumented, or if ``instrumentation.instrument`` has already been
called on the same code object.

Register callback functions for instrumentation
'''''''''''''''''''''''''''''''''''''''''''''''
Expand All @@ -117,24 +138,26 @@ To register a callable for events call::
Functions can be unregistered by calling
``instrumentation.register(event, None)``.

Callback functions can be registered at any time.
Callback functions can be registered and unregistered at any time.

Callback function arguments
'''''''''''''''''''''''''''

When an event occurs the registered function will be called.
The arguments provided are as follows:
When an event in a code object instrumented for that event,
the registered function will be called.

The arguments provided are as follows::

* BRANCH: ``func(code: CodeType, offset: int, taken:bool)``
* JUMPBACK: ``func(code: CodeType, offset: int)``
* ENTER: ``func(code: CodeType, offset: int)``
* EXIT: ``func(code: CodeType, offset: int)``
* C_CALL: ``func(code: CodeType, offset: int, value: object)``
* C_RETURN: ``func(code: CodeType, offset: int, value: object)``
* C_EXCEPT: ``func(code: CodeType, offset: int, exception: BaseException)``
* RAISE: ``func(code: CodeType, offset: int, exception: BaseException)``
* EXCEPT: ``func(code: CodeType, offset: int)``
* UNWIND: ``func(code: CodeType)``
BRANCH: func(code: CodeType, offset: int, taken:bool)
JUMPBACK: func(code: CodeType, offset: int)
ENTER: func(code: CodeType, offset: int)
EXIT: func(code: CodeType, offset: int)
UNWIND: func(code: CodeType)
C_CALL: func(code: CodeType, offset: int, value: object)
C_RETURN: func(code: CodeType, offset: int, value: object)
RAISE: func(code: CodeType, offset: int, exception: BaseException)
EXCEPT: func(code: CodeType, offset: int)
LINE: func(code: CodeType, offset: int)

Monitoring
----------
Expand All @@ -146,38 +169,65 @@ The following functions are provided to insert monitoring points::

instrumentation.insert_monitors(codeobject, *offsets)
instrumentation.remove_monitors(codeobject, *offsets)
instrumentation.monitor_off(codeobject, offset)
instrumentation.monitor_on(codeobject, offset)

All functions return ``True`` if a monitor checkpoint was present,
or ``False`` if a monitor checkpoint was not present.
Turning on, or off, a non-existent checkpoint is a no-op;
no exception is raised.
Inserting a monitor where one was already present or removing one
that is not present is a no-op; no exception is raised.

Monitors can be queried and turned on or off with::

To register a callable for monitoring function events call::
instrumentation.is_monitor(codeobject, offset)
instrumentation.monitor_on(codeobject, offset, on)

Both functions return ``True`` if a monitor checkpoint is present at
``(codeobject, offset)``. ``instrumentation.monitor_on`` turns the
checkpoint on if ``bool(on)`` is ``True`` and turns it off otherwise.

To register a callable for monitoring::

instrumentation.monitor_register(func)

The callback function will be called with the code object and offset as arguments::
The callback function will be called with the code object and
offset as arguments::

func(code: CodeType, offset: int)

To register a callable for monitoring exceptions, regardless of
where the excpetion is raised::

instrumentation.monitor_exceptions(func)

The callback function will be called with the code object, offset and
exception as arguments::

func(code: CodeType, offset: int, exception: BaseException)

Performance
-----------

Insertion of instrumentation and monitors into a code object that has not
been executed should have neglible cost.
The impact on performance of those checkpoints will depend on how many
checkpoints there are. For a few monitors not in the hottest parts of a
program, the change in performance should very small.

For optimizing virtual machines, such as future versions of CPython
(and ``PyPy`` should they choose to support this API), a call to
``insert_monitors`` and ``remove_monitors`` in a long running program
could be quite expensive, possibly taking 100s of milliseconds as it
triggers de-optimizations. Repeated calls to ``insert_monitors``
and ``remove_monitors``, as may be required in an interactive debugger,
should be relatively inexpensive.
(and ``PyPy`` should they choose to support this API), adding instrumentation
or inserting or removing monitors in the midst of a long running program
could be quite expensive, possibly taking hundreds of milliseconds as it
triggers de-optimizations. Once such de-optimization has occurred, repeated
changes to the checkpoints of a code object, as may be required in an
interactive debugger, should be relatively inexpensive.

Combining Checkpoints
---------------------

Only one instrumentation checkpoint and one monitoring checkpoint is allowed
per bytecode instruction. It is possible to have both a monitoring and
instrumentation checkpoint on the same instruction; they are independent.
It is possible for a single checkpoint to support both instrumentation
and monitoring; they are independent.
Monitors will be called before instrumentation if both are present.

This allows some limited combination of tooling. For example, it
should be possible to debug a profiler.

Backwards Compatibility
=======================

Expand All @@ -199,13 +249,14 @@ All the functions listed above will trigger audit hooks.
Implementation
==============

The implementation of this PEP will be built on top of PEP 659 quickening.
The implementation of this PEP will be built on top of the quickening step of
PEP 659 [1]_.
Instrumentation or monitoring of a code object will cause it to be quickened.
Checkpoints will then be implemented by inserting one of several special
``CHECKPOINT`` instructions into the quickened code. These instructions
will call the registered callable before executing the original instruction.

Note that this can interfere with specialization, which will result in
Note that this may interfere with specialization, which will result in some
performance degradation in addition to the overhead of calling the
registered callable.

Expand All @@ -230,12 +281,12 @@ Then a monitor should be added for each of those offsets.
To avoid excessive overhead, a single call should be made to
``instrumentation.insert_monitors`` passing all the offsets at once.

Breakpoints can suspended with ``instrumentation.monitor_off``.
Breakpoints can suspended with
``instrumentation.monitor_on(code, offset, False)``.

Debuggers can break on exceptions being raised by registering a callable
for ``RAISE``:
Debuggers can break on exceptions being raised by registering a callable:

``instrumentation.register(RAISE, break_on_raise_handler)``
``instrumentation.monitor_exceptions(func)``

Stepping
''''''''
Expand All @@ -257,7 +308,7 @@ executed. To do this, they need to track most events and map those events
onto the control flow graph of the code object.
``BRANCH``, ``JUMPBACK``, ``START`` and ``RESUME`` events will inform which
basic blocks have started to execute.
The ``RAISE`` event with mark any blocks that did not complete.
The ``RAISE`` event will mark any blocks that did not complete.

This can be then be converted back into a line based report after execution
has completed.
Expand All @@ -268,24 +319,24 @@ Profilers
Simple profilers need to gather information about calls.
To do this profilers should register for the following events:

* ENTER
* EXIT
* UNWIND
* C_CALL
* C_RETURN
* RAISE
* ``ENTER``
* ``EXIT``
* ``UNWIND``
* ``C_CALL``
* ``C_RETURN``
* ``RAISE``

Line based profilers
''''''''''''''''''''

Line based profilers will also need to handle ``BRANCH`` and ``JUMPBACK``
events.
Beware that handling these extra events will have a large performance impact.
Line based profilers can use the ``LINE`` and ``JUMPBACK`` events.
Implementers of profilers should be aware that instrumenting ``LINE``
and ``JUMPBACK`` events will have a large impact on performance.

.. note::

Instrumenting profilers have a significant overhead and will distort the
results of profiling. Unless you need exact call counts,
Instrumenting profilers have significant overhead and will distort
the results of profiling. Unless you need exact call counts,
consider using a statistical profiler.

Open Issues
Expand All @@ -297,7 +348,9 @@ Open Issues
References
==========

[A collection of URLs used as references through the PEP.]
.. [1] Quickening in PEP 659
https://www.python.org/dev/peps/pep-0659/#quickening



Copyright
Expand All @@ -307,7 +360,6 @@ This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.



..
Local Variables:
mode: indented-text
Expand Down
0