8000 Backport Pdb closure evaluation fix (list comprehension, generators) … · ipython/ipython@2197228 · GitHub
[go: up one dir, main page]
Skip to content

Commit 2197228

Browse files
krassowskikrassowski
authored
Backport Pdb closure evaluation fix (list comprehension, generators) from Python 3.13 (#14933)
- [x] add a failing test case - [x] backport python/cpython#111094 into IPython codebase - fixes gotcha/ipdb#256 - solves the pdb-related comments on #62 ![image](https://github.com/user-attachments/assets/c1a33ad0-d8a5-44f2-84d3-73f708918747) ### Context IPython already includes bits of Pdb code from Python itself, as documented in: https://github.com/ipython/ipython/blob/1e9a2e8b5a584ffe1973318fdabf2e39395ec367/IPython/core/debugger.py#L92-L110 I am taking a more cautious approach and separating the backport to a new file to make it easy to delete in the (near) future (1 year from now).
2 parents 1e9a2e8 + c6ac674 commit 2197228

File tree

3 files changed

+245
-1
lines changed

3 files changed

+245
-1
lines changed

‎IPython/core/debugger.py‎

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -133,16 +133,26 @@
133133
from functools import lru_cache
134134

135135
from IPython import get_ipython
136+
from IPython.core.debugger_backport import PdbClosureBackport
136137
from IPython.utils import PyColorize
137138
from IPython.utils.PyColorize import TokenStream
138139

139140
from typing import TYPE_CHECKING
140141
from types import FrameType
141142

142143
# We have to check this directly from sys.argv, config struct not yet available
143-
from pdb import Pdb as OldPdb
144+
from pdb import Pdb as _OldPdb
144145
from pygments.token import Token
145146

147+
148+
if sys.version_info < (3, 13):
149+
150+
class OldPdb(PdbClosureBackport, _OldPdb):
151+
pass
152+
153+
else:
154+
OldPdb = _OldPdb
155+
146156
if TYPE_CHECKING:
147157
# otherwise circular import
148158
from IPython.core.interactiveshell import InteractiveShell

‎IPython/core/debugger_backport.py‎

Lines changed: 206 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,206 @@
1+
"""
2+
The code in this module is a backport of cPython changes in Pdb
3+
that were introduced in Python 3.13 by gh-83151: Make closure work on pdb
4+
https://github.com/python/cpython/pull/111094.
5+
This file should be removed once IPython drops supports for Python 3.12.
6+
7+
The only changes are:
8+
- reformatting by darker (black) formatter
9+
- addition of type-ignore comments to satisfy mypy
10+
11+
Copyright (c) 2001 Python Software Foundation; All Rights Reserved
12+
13+
PYTHON SOFTWARE FOUNDATION LICENSE VERSION 2
14+
--------------------------------------------
15+
16+
1. This LICENSE AGREEMENT is between the Python Software Foundation
17+
("PSF"), and the Individual or Organization ("Licensee") accessing and
18+
otherwise using this software ("Python") in source or binary form and
19+
its associated documentation.
20+
21+
2. Subject to the terms and conditions of this License Agreement, PSF hereby
22+
grants Licensee a nonexclusive, royalty-free, world-wide license to reproduce,
23+
analyze, test, perform and/or display publicly, prepare derivative works,
24+
distribute, and otherwise use Python alone or in any derivative version,
25+
provided, however, that PSF's License Agreement and PSF's notice of copyright,
26+
i.e., "Copyright (c) 2001 Python Software Foundation; All Rights Reserved"
27+
are retained in Python alone or in any derivative version prepared by Licensee.
28+
29+
3. In the event Licensee prepares a derivative work that is based on
30+
or incorporates Python or any part thereof, and wants to make
31+
the derivative work available to others as provided herein, then
32+
Licensee hereby agrees to include in any such work a brief summary of
33+
the changes made to Python.
34+
35+
4. PSF is making Python available to Licensee on an "AS IS"
36+
basis. PSF MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
37+
IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, PSF MAKES NO AND
38+
DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
39+
FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON WILL NOT
40+
INFRINGE ANY THIRD PARTY RIGHTS.
41+
42+
5. PSF SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON
43+
FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS
44+
A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON,
45+
OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.
46+
47+
6. This License Agreement will automatically terminate upon a material
48+
breach of its terms and conditions.
49+
50+
7. Nothing in this License Agreement shall be deemed to create any
51+
relationship of agency, partnership, or joint venture between PSF and
52+
Licensee. This License Agreement does not grant permission to use PSF
53+
trademarks or trade name in a trademark sense to endorse or promote
54+
products or services of Licensee, or any third party.
55+
56+
8. By copying, installing or otherwise using Python, Licensee
57+
agrees to be bound by the terms and conditions of this License
58+
Agreement.
59+
"""
60+
61+
import sys
62+
import types
63+
import codeop
64+
import textwrap
65+
from types import CodeType
66+
67+
68+
class PdbClosureBackport:
69+
def _exec_in_closure(self, source, globals, locals): # type: ignore[no-untyped-def]
70+
"""Run source code in closure so code object created within source
71+
can find variables in locals correctly
72+
returns True if the source is executed, False otherwise
73+
"""
74+
75+
# Determine if the source should be executed in closure. Only when the
76+
# source compiled to multiple code objects, we should use this feature.
77+
# Otherwise, we can just raise an exception and normal exec will be used.
78+
79+
code = compile(source, "<string>", "exec")
80+
if not any(isinstance(const, CodeType) for const in code.co_consts):
81+
return False
82+
83+
# locals could be a proxy which does not support pop
84+
# copy it first to avoid modifying the original locals
85+
locals_copy = dict(locals)
86+
87+
locals_copy["__pdb_eval__"] = {"result": None, "write_back": {}}
88+
89+
# If the source is an expression, we need to print its value
90+
try:
91+
compile(source, "<string>", "eval")
92+
except SyntaxError:
93+
pass
94+
else:
95+
source = "__pdb_eval__['result'] = " + source
96+
97+
# Add write-back to update the locals
98+
source = (
99+
"try:\n"
100+
+ textwrap.indent(source, " ")
101+
+ "\n"
102+
+ "finally:\n"
103+
+ " __pdb_eval__['write_back'] = locals()"
104+
)
105+
106+
# Build a closure source code with freevars from locals like:
107+
# def __pdb_outer():
108+
# var = None
109+
# def __pdb_scope(): # This is the code object we want to execute
110+
# nonlocal var
111+
# <source>
112+
# return __pdb_scope.__code__
113+
source_with_closure = (
114+
"def __pdb_outer():\n"
115+
+ "\n".join(f" {var} = None" for var in locals_copy)
116+
+ "\n"
117+
+ " def __pdb_scope():\n"
118+
+ "\n".join(f" nonlocal {var}" for var in locals_copy)
119+
+ "\n"
120+
+ textwrap.indent(source, " ")
121+
+ "\n"
122+
+ " return __pdb_scope.__code__"
123+
)
124+
125+
# Get the code object of __pdb_scope()
126+
# The exec fills locals_copy with the __pdb_outer() function and we can call
127+
# that to get the code object of __pdb_scope()
128+
ns = {}
129+
try:
130+
exec(source_with_closure, {}, ns)
131+
except Exception:
132+
return False
133+
code = ns["__pdb_outer"]()
134+
135+
cells = tuple(types.CellType(locals_copy.get(var)) for var in code.co_freevars)
136+
137+
try:
138+
exec(code, globals, locals_copy, closure=cells)
139+
except Exception:
140+
return False
141+
142+
# get the data we need from the statement
143+
pdb_eval = locals_copy["__pdb_eval__"]
144+
145+
# __pdb_eval__ should not be updated back to locals
146+
pdb_eval["write_back"].pop("__pdb_eval__")
147+
148+
# Write all local variables back to locals
149+
locals.update(pdb_eval["write_back"])
150+
eval_result = pdb_eval["result"]
151+
if eval_result is not None:
152+
print(repr(eval_result))
153+
154+
return True
155+
156+
def default(self, line): # type: ignore[no-untyped-def]
157+
if line[:1] == "!":
158+
line = line[1:].strip()
159+
locals = self.curframe_locals
160+
globals = self.curframe.f_globals
161+
try:
162+
buffer = line
163+
if (
164+
code := codeop.compile_command(line + "\n", "<stdin>", "single")
165+
) is None:
166+
# Multi-line mode
167+
with self._disable_command_completion():
168+
buffer = line
169+
continue_prompt = "... "
170+
while (
171+
code := codeop.compile_command(buffer, "<stdin>", "single")
172+
) is None:
173+
if self.use_rawinput:
174+
try:
175+
line = input(continue_prompt)
176+
except (EOFError, KeyboardInterrupt):
177+
self.lastcmd = ""
178+
print("\n")
179+
return
180+
else:
181+
self.stdout.write(continue_prompt)
182+
self.stdout.flush()
183+
line = self.stdin.readline()
184+
if not len(line):
185+
self.lastcmd = ""
186+
self.stdout.write("\n")
187+
self.stdout.flush()
188+
return
189+
else:
190+
line = line.rstrip("\r\n")
191+
buffer += "\n" + line
192+
save_stdout = sys.stdout
193+
save_stdin = sys.stdin
194+
save_displayhook = sys.displayhook
195+
try:
196+
sys.stdin = self.stdin
197+
sys.stdout = self.stdout
198+
sys.displayhook = self.displayhook
199+
if not self._exec_in_closure(buffer, globals, locals):
200+
exec(code, globals, locals)
201+
finally:
202+
sys.stdout = save_stdout
203+
sys.stdin = save_stdin
204+
sys.displayhook = save_displayhook
205+
except:
206+
self._error_exc()
Collapse file

‎tests/test_debugger.py‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -149,6 +149,34 @@ def test_ipdb_magics():
149149
'''
150150

151151

152+
def test_ipdb_closure():
153+
"""Test evaluation of expressions which depend on closure.
154+
155+
In [1]: old_trace = sys.gettrace()
156+
157+
Create a function which triggers ipdb.
158+
159+
In [2]: def trigger_ipdb():
160+
...: debugger.Pdb().set_ 2358 trace()
161+
162+
In [3]: with PdbTestInput([
163+
...: 'x = 1; sum(x * i for i in range(5))',
164+
...: 'continue',
165+
...: ]):
166+
...: trigger_ipdb()
167+
...> <doctest ...>(2)trigger_ipdb()
168+
1 def trigger_ipdb():
169+
----> 2 debugger.Pdb().set_trace()
170+
<BLANKLINE>
171+
ipdb> x = 1; sum(x * i for i in range(5))
172+
ipdb> continue
173+
174+
Restore previous trace function, e.g. for coverage.py
175+
176+
In [4]: sys.settrace(old_trace)
177+
"""
178+
179+
152180
def test_ipdb_magics2():
153181
"""Test ipdb with a very short function.
154182

0 commit comments

Comments
 (0)
0