Description
Describe the bug
PEP-669 (https://peps.python.org/pep-0669/) and its implementation in python/cpython#103083 change the internals of CPython's tracing in Py3.12. We need to adapt. Currently, profiling fails in Py3.12.
Compilation was already fixed in 03c498d (#5450), but that doesn't make the feature work again.
Test failure:
======================================================================
FAIL: pstats_profile_test ()
Doctest: pstats_profile_test
----------------------------------------------------------------------
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.12.0-beta.1/x64/lib/python3.12/doctest.py", line 2228, in runTest
raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for pstats_profile_test
File "/home/runner/work/cython/cython/TEST_TMP/6/run/c/pstats_profile_test/pstats_profile_test.cpython-312-x86_64-linux-gnu.so", line 1, in pstats_profile_test
----------------------------------------------------------------------
File "/home/runner/work/cython/cython/TEST_TMP/6/run/c/pstats_profile_test/pstats_profile_test.cpython-312-x86_64-linux-gnu.so", line 8, in pstats_profile_test
Failed example:
short_stats['f_def']
Exception raised:
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.12.0-beta.1/x64/lib/python3.12/doctest.py", line 1357, in __run
exec(compile(example.source, filename, "single",
File "<doctest pstats_profile_test[5]>", line 1, in <module>
short_stats['f_def']
~~~~~~~~~~~^^^^^^^^^
KeyError: 'f_def'
----------------------------------------------------------------------
File "/home/runner/work/cython/cython/TEST_TMP/6/run/c/pstats_profile_test/pstats_profile_test.cpython-312-x86_64-linux-gnu.so", line 10, in pstats_profile_test
Failed example:
short_stats['f_cdef']
Exception raised:
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.12.0-beta.1/x64/lib/python3.12/doctest.py", line 1357, in __run
exec(compile(example.source, filename, "single",
File "<doctest pstats_profile_test[6]>", line 1, in <module>
short_stats['f_cdef']
~~~~~~~~~~~^^^^^^^^^^
KeyError: 'f_cdef'
----------------------------------------------------------------------
File "/home/runner/work/cython/cython/TEST_TMP/6/run/c/pstats_profile_test/pstats_profile_test.cpython-312-x86_64-linux-gnu.so", line 12, in pstats_profile_test
Failed example:
short_stats['f_cpdef']
Exception raised:
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.12.0-beta.1/x64/lib/python3.12/doctest.py", line 1357, in __run
exec(compile(example.source, filename, "single",
File "<doctest pstats_profile_test[7]>", line 1, in <module>
short_stats['f_cpdef']
~~~~~~~~~~~^^^^^^^^^^^
KeyError: 'f_cpdef'
----------------------------------------------------------------------
File "/home/runner/work/cython/cython/TEST_TMP/6/run/c/pstats_profile_test/pstats_profile_test.cpython-312-x86_64-linux-gnu.so", line 14, in pstats_profile_test
Failed example:
short_stats['f_inline']
Exception raised:
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.12.0-beta.1/x64/lib/python3.12/doctest.py", line 1357, in __run
exec(compile(example.source, filename, "single",
File "<doctest pstats_profile_test[8]>", line 1, in <module>
short_stats['f_inline']
~~~~~~~~~~~^^^^^^^^^^^^
KeyError: 'f_inline'
----------------------------------------------------------------------
File "/home/runner/work/cython/cython/TEST_TMP/6/run/c/pstats_profile_test/pstats_profile_test.cpython-312-x86_64-linux-gnu.so", line 16, in pstats_profile_test
Failed example:
short_stats['f_inline_prof']
Exception raised:
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.12.0-beta.1/x64/lib/python3.12/doctest.py", line 1357, in __run
exec(compile(example.source, filename, "single",
File "<doctest pstats_profile_test[9]>", line 1, in <module>
short_stats['f_inline_prof']
~~~~~~~~~~~^^^^^^^^^^^^^^^^^
KeyError: 'f_inline_prof'
----------------------------------------------------------------------
File "/home/runner/work/cython/cython/TEST_TMP/6/run/c/pstats_profile_test/pstats_profile_test.cpython-312-x86_64-linux-gnu.so", line 22, in pstats_profile_test
Failed example:
short_stats['f_raise']
Exception raised:
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.12.0-beta.1/x64/lib/python3.12/doctest.py", line 1357, in __run
exec(compile(example.source, filename, "single",
File "<doctest pstats_profile_test[11]>", line 1, in <module>
short_stats['f_raise']
~~~~~~~~~~~^^^^^^^^^^^
KeyError: 'f_raise'
----------------------------------------------------------------------
File "/home/runner/work/cython/cython/TEST_TMP/6/run/c/pstats_profile_test/pstats_profile_test.cpython-312-x86_64-linux-gnu.so", line 25, in pstats_profile_test
Failed example:
short_stats['withgil_prof']
Exception raised:
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.12.0-beta.1/x64/lib/python3.12/doctest.py", line 1357, in __run
exec(compile(example.source, filename, "single",
File "<doctest pstats_profile_test[12]>", line 1, in <module>
short_stats['withgil_prof']
~~~~~~~~~~~^^^^^^^^^^^^^^^^
KeyError: 'withgil_prof'
----------------------------------------------------------------------
File "/home/runner/work/cython/cython/TEST_TMP/6/run/c/pstats_profile_test/pstats_profile_test.cpython-312-x86_64-linux-gnu.so", line 41, in pstats_profile_test
Failed example:
short_stats['f_raise']
Exception raised:
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.12.0-beta.1/x64/lib/python3.12/doctest.py", line 1357, in __run
exec(compile(example.source, filename, "single",
File "<doctest pstats_profile_test[16]>", line 1, in <module>
short_stats['f_raise']
~~~~~~~~~~~^^^^^^^^^^^
KeyError: 'f_raise'
----------------------------------------------------------------------
File "/home/runner/work/cython/cython/TEST_TMP/6/run/c/pstats_profile_test/pstats_profile_test.cpython-312-x86_64-linux-gnu.so", line 44, in pstats_profile_test
Failed example:
short_stats['m_def']
Exception raised:
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.12.0-beta.1/x64/lib/python3.12/doctest.py", line 1357, in __run
exec(compile(example.source, filename, "single",
File "<doctest pstats_profile_test[17]>", line 1, in <module>
short_stats['m_def']
~~~~~~~~~~~^^^^^^^^^
KeyError: 'm_def'
----------------------------------------------------------------------
File "/home/runner/work/cython/cython/TEST_TMP/6/run/c/pstats_profile_test/pstats_profile_test.cpython-312-x86_64-linux-gnu.so", line 46, in pstats_profile_test
Failed example:
short_stats['m_cdef']
Exception raised:
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.12.0-beta.1/x64/lib/python3.12/doctest.py", line 1357, in __run
exec(compile(example.source, filename, "single",
File "<doctest pstats_profile_test[18]>", line 1, in <module>
short_stats['m_cdef']
~~~~~~~~~~~^^^^^^^^^^
KeyError: 'm_cdef'
----------------------------------------------------------------------
File "/home/runner/work/cython/cython/TEST_TMP/6/run/c/pstats_profile_test/pstats_profile_test.cpython-312-x86_64-linux-gnu.so", line 48, in pstats_profile_test
Failed example:
short_stats['m_cpdef'] - (200 if CPDEF_METHODS_COUNT_TWICE else 0) # FIXME!
Exception raised:
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.12.0-beta.1/x64/lib/python3.12/doctest.py", line 1357, in __run
exec(compile(example.source, filename, "single",
File "<doctest pstats_profile_test[19]>", line 1, in <module>
short_stats['m_cpdef'] - (200 if CPDEF_METHODS_COUNT_TWICE else 0) # FIXME!
~~~~~~~~~~~^^^^^^^^^^^
KeyError: 'm_cpdef'
----------------------------------------------------------------------
File "/home/runner/work/cython/cython/TEST_TMP/6/run/c/pstats_profile_test/pstats_profile_test.cpython-312-x86_64-linux-gnu.so", line 56, in pstats_profile_test
Failed example:
sorted(callees(s, 'test_profile')) #doctest: +NORMALIZE_WHITESPACE
Expected:
['f_cdef', 'f_cpdef', 'f_def',
'f_inline', 'f_inline_prof',
'f_raise',
'm_cdef', 'm_cpdef', 'm_def',
'withgil_prof']
Got:
[]
----------------------------------------------------------------------
File "/home/runner/work/cython/cython/TEST_TMP/6/run/c/pstats_profile_test/pstats_profile_test.cpython-312-x86_64-linux-gnu.so", line 66, in pstats_profile_test
Failed example:
short_stats['generator']
Exception raised:
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.12.0-beta.1/x64/lib/python3.12/doctest.py", line 1357, in __run
exec(compile(example.source, filename, "single",
File "<doctest pstats_profile_test[25]>", line 1, in <module>
short_stats['generator']
~~~~~~~~~~~^^^^^^^^^^^^^
KeyError: 'generator'
----------------------------------------------------------------------
File "/home/runner/work/cython/cython/TEST_TMP/6/run/c/pstats_profile_test/pstats_profile_test.cpython-312-x86_64-linux-gnu.so", line 69, in pstats_profile_test
Failed example:
short_stats['generator_exception']
Exception raised:
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.12.0-beta.1/x64/lib/python3.12/doctest.py", line 1357, in __run
exec(compile(example.source, filename, "single",
File "<doctest pstats_profile_test[26]>", line 1, in <module>
short_stats['generator_exception']
~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
KeyError: 'generator_exception'
----------------------------------------------------------------------
File "/home/runner/work/cython/cython/TEST_TMP/6/run/c/pstats_profile_test/pstats_profile_test.cpython-312-x86_64-linux-gnu.so", line 72, in pstats_profile_test
Failed example:
short_stats['genexpr']
Exception raised:
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.12.0-beta.1/x64/lib/python3.12/doctest.py", line 1357, in __run
exec(compile(example.source, filename, "single",
File "<doctest pstats_profile_test[27]>", line 1, in <module>
short_stats['genexpr']
~~~~~~~~~~~^^^^^^^^^^^
KeyError: 'genexpr'
----------------------------------------------------------------------
File "/home/runner/work/cython/cython/TEST_TMP/6/run/c/pstats_profile_test/pstats_profile_test.cpython-312-x86_64-linux-gnu.so", line 75, in pstats_profile_test
Failed example:
sorted(callees(s, 'test_generators'))
Expected:
['call_generator', 'call_generator_exception', 'generator_expr']
Got:
[]
----------------------------------------------------------------------
File "/home/runner/work/cython/cython/TEST_TMP/6/run/c/pstats_profile_test/pstats_profile_test.cpython-312-x86_64-linux-gnu.so", line 78, in pstats_profile_test
Failed example:
list(callees(s, 'call_generator'))
Expected:
['generator']
Got:
[]
----------------------------------------------------------------------
File "/home/runner/work/cython/cython/TEST_TMP/6/run/c/pstats_profile_test/pstats_profile_test.cpython-312-x86_64-linux-gnu.so", line 87, in pstats_profile_test
Failed example:
list(callees(s, 'generator_expr'))
Expected:
['genexpr']
Got:
[]
----------------------------------------------------------------------
File "/home/runner/work/cython/cython/TEST_TMP/6/run/c/pstats_profile_test/pstats_profile_test.cpython-312-x86_64-linux-gnu.so", line 107, in pstats_profile_test
Failed example:
python_stats_dict['python_generator'] == cython_stats_dict['generator']
Exception raised:
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.12.0-beta.1/x64/lib/python3.12/doctest.py", line 1357, in __run
exec(compile(example.source, filename, "single",
File "<doctest pstats_profile_test[42]>", line 1, in <module>
python_stats_dict['python_generator'] == cython_stats_dict['generator']
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
KeyError: 'generator'
Code to reproduce the behaviour:
No response
Expected behaviour
No response
OS
No response
Python version
3.12.0b1
Cython version
current master (3.0.0b3)
Additional context
No response