8000 GH-96092: restore frame height of traceback.walk_stack to that of pyt… by graingert · Pull Request #99015 · python/cpython · GitHub
[go: up one dir, main page]

Skip to content

GH-96092: restore frame height of traceback.walk_stack to that of pyt… #99015

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

graingert
Copy link
Contributor
@graingert graingert commented Nov 2, 2022

@graingert graingert marked this pull request as ready for review November 2, 2022 16:56
@ammaraskar
Copy link
Member
ammaraskar commented Nov 2, 2022

Aah I think I see how these incorrect f_backs ended up in here (and maybe why it really should have been one f_back instead of two originally).

In StackSummary._extract_from_extended_frame_gen we iterate over the frame generator here:

for f, (lineno, end_lineno, colno, end_colno) in frame_gen:

Since walk_stack is a generator, the initial call to it in say

traceback.walk_stack(None), capture_locals=True, limit=1)

simply instantiates it. Only in _extract_from_extended_frame_gen does the code

if f is None:
    f = sys._getframe().f_back.f_back

get invoked and by that point we are already inside of the StackSummary.extract -> StackSummary._extract_from_extended_frame_gen -> extended_frame_gen -> walk_stack (generator) (hence the four f_backs.)

Since this code path was originally StackSummary.extract -> walk_stack, that would explain why it had two f_backs instead of just one, unlike other traceback methods like format_stack and extract_stack.

walk_stack was likely never meant to be used without a corresponding StackSummary.extract, the docs saying

Usually used with StackSummary.extract

and the lack of direct tests for it in the commit adding it seems to suggest so: 6bc2c1e#diff-8a52124ffceebd98f91bf96caa0f341ae109f79e190cd5acc1c56c8968d75252R514


One possible fix is to make walk_stack return a generator that returns the frames from when it was called, not when the generator is iterated over.

graingert added a commit to graingert/distributed that referenced this pull request Nov 3, 2022
see python/cpython#96092
and it's not really intended for use outside of StackSummary.extract,
see python/cpython#99015 (comment)
@ammaraskar
Copy link
Member

Here's my recommended fix for this:

diff --git a/Lib/test/test_traceback.py b/Lib/test/test_traceback.py
index 149d0234fe..48d5c1a809 100644
--- a/Lib/test/test_traceback.py
+++ b/Lib/test/test_traceback.py
@@ -2241,8 +2241,7 @@ class TestStack(unittest.TestCase):
     def test_walk_stack(self):
         def deeper():
             return list(traceback.walk_stack(None))
-        s1 = list(traceback.walk_stack(None))
-        s2 = deeper()
+        s1, s2 = list(traceback.walk_stack(None)), deeper()
         self.assertEqual(len(s2) - len(s1), 1)
         self.assertEqual(s2[1:], s1)

diff --git a/Lib/traceback.py b/Lib/traceback.py
index cf5f355ff0..701ce9e4bb 100644
--- a/Lib/traceback.py
+++ b/Lib/traceback.py
@@ -323,17 +323,21 @@ def line(self):
         return self._line.strip()


-def walk_stack(f):
+def walk_stack(frame):
     """Walk a stack yielding the frame and line number for each frame.

     This will follow f.f_back from the given frame. If no frame is given, the
     current stack is used. Usually used with StackSummary.extract.
     """
-    if f is None:
-        f = sys._getframe().f_back.f_back.f_back.f_back
-    while f is not None:
-        yield f, f.f_lineno
-        f = f.f_back
+    if frame is None:
+        frame = sys._getframe().f_back
+
+    def walk_stack_generator(f):
+        while f is not None:
+            yield f, f.f_lineno
+            f = f.f_back
+
+    return walk_stack_generator(frame)

As mentioned in the comment on the issue, this is a change in behavior we should definitely have a NEWS entry and documentation change. Code in practice is unlikely to break due to it. @graingert, would you like to take this on?

graingert added a commit to graingert/distributed that referenced this pull request Nov 9, 2022
see python/cpython#96092
and it's not really intended for use outside of StackSummary.extract,
see python/cpython#99015 (comment)
graingert added a commit to graingert/distributed that referenced this pull request Nov 17, 2022
see python/cpython#96092
and it's not really intended for use outside of StackSummary.extract,
see python/cpython#99015 (comment)
graingert added a commit to graingert/distributed that referenced this pull request Nov 23, 2022
see python/cpython#96092
and it's not really intended for use outside of StackSummary.extract,
see python/cpython#99015 (comment)
graingert added a commit to graingert/distributed that referenced this pull request Nov 24, 2022
see python/cpython#96092
and it's not really intended for use outside of StackSummary.extract,
see python/cpython#99015 (comment)
graingert added a commit to graingert/distributed that referenced this pull request Nov 30, 2022
see python/cpython#96092
and it's not really intended for use outside of StackSummary.extract,
see python/cpython#99015 (comment)
graingert added a commit to dask/distributed that referenced this pull request Dec 14, 2022
* use sys.executable for python binary

* pass the desired frame to traceback.walk_stack for 3.11

see python/cpython#96092
and it's not really intended for use outside of StackSummary.extract,
see python/cpython#99015 (comment)

* the GC overhead now depends on _PyType_PreHeaderSize

faster-cpython/ideas#125
python/cpython@8319114#diff-a3a5c73931235f7f344c072dc755d6508e13923db3f5d581c5e88652075871cbR1684

* GH-6785: asyncio.wait no longer calls ensure_future

python/cpython#95601

* test on python 3.11

* exclude macos py 3.10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants
0