8000 bpo-38070: Enhance visit_decref() debug trace by vstinner · Pull Request #16631 · python/cpython · GitHub
[go: up one dir, main page]

Skip to content

bpo-38070: Enhance visit_decref() debug trace #16631

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 8, 2019
Merged

bpo-38070: Enhance visit_decref() debug trace #16631

merged 1 commit into from
Oct 8, 2019

Conversation

vstinner
Copy link
Member
@vstinner vstinner commented Oct 7, 2019

subtract_refs() now pass the parent object to visit_decref() which
pass it to _PyObject_ASSERT(). So if the "is freed" assertion fails,
the parent is used in debug trace, rather than the freed object. The
parent object is more likely to contain useful information. Freed
objects cannot be inspected are are displayed as "" with no other detail.

https://bugs.python.org/issue38070

subtract_refs() now pass the parent object to visit_decref() which
pass it to _PyObject_ASSERT(). So if the "is freed" assertion fails,
the parent is used in debug trace, rather than the freed object. The
parent object is more likely to contain useful information. Freed
objects cannot be inspected are are displayed as "<object at xxx is
freed>" with no other detail.
@vstinner
Copy link
Member Author
vstinner commented Oct 7, 2019

Example with https://bugs.python.org/issue38400 bug.

Without the change, Python exit with:

Modules/gcmodule.c:378: visit_decref: Assertion "!_PyObject_IsFreed(op)" failed
<object at 0x7420a0 is freed>
Fatal Python error: _PyObject_AssertFailed
...

With this change, the output is more complete:

Modules/gcmodule.c:378: visit_decref: Assertion "!_PyObject_IsFreed(op)" failed
Enable tracemalloc to get the memory block allocation traceback

object address  : 0x7ff11b158b40
object refcount : 1
object type     : 0x740be0
object type name: dict
object repr     : {'__repr__': <slot wrapper '__repr__' of 'weakref' objects>, '__hash__': <slot wrapper '__hash__' of 'weakref' objects>, '__call__': <slot wrapper '__call__' of 'weakref' objects>, '__lt__': <slot wrapper '__lt__' of 'weakref' objects>, '__le__': <slot wrapper '__le__' of 'weakref' objects>, '__eq__': <slot wrapper '__eq__' of 'weakref' objects>, '__ne__': <slot wrapper '__ne__' of 'weakref' objects>, '__gt__': <slot wrapper '__gt__' of 'weakref' objects>, '__ge__': <slot wrapper '__ge__' of 'weakref' objects>, '__init__': <slot wrapper '__init__' of 'weakref' objects>, '__new__': <built-in method __new__ of type object at 0x749500>, '__callback__': <member '__callback__' of 'weakref' objects>, '__doc__': None}

Fatal Python error: _PyObject_AssertFailed
...

In this case, the bug was that None was seen as freed: the None object can be easily be recovered using a debugger in the visit_decref() frame.

@miss-islington
Copy link
Contributor

Thanks @vstinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.8.
🐍🍒⛏🤖

@miss-islington
Copy link
Contributor

Sorry, @vstinner, I could not cleanly backport this to 3.8 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 4d5f94b8cd20f804c7868c5395a15aa6032f874c 3.8

vstinner added a commit that referenced this pull request Oct 15, 2019
* bpo-36389: _PyObject_CheckConsistency() available in release mode (GH-16612)

bpo-36389, bpo-38376: The _PyObject_CheckConsistency() function is
now also available in release mode. For example, it can be used to
debug a crash in the visit_decref() function of the GC.

Modify the following functions to also work in release mode:

* _PyDict_CheckConsistency()
* _PyObject_CheckConsistency()
* _PyType_CheckConsistency()
* _PyUnicode_CheckConsistency()

Other changes:

* _PyMem_IsPtrFreed(ptr) now also returns 1 if ptr is NULL
  (equals to 0).
* _PyBytesWriter_CheckConsistency() now returns 1 and is only used
  with assert().
* Reorder _PyObject_Dump() to write safe fields first, and only
  attempt to render repr() at the end.

(cherry picked from commit 6876257)

* bpo-36389: Fix _PyBytesWriter in release mode (GH-16624)

Fix _PyBytesWriter API when Python is built in release mode with
assertions.

(cherry picked from commit 60ec6ef)

* bpo-38070: Enhance visit_decref() debug trace (GH-16631)

subtract_refs() now pass the parent object to visit_decref() which
pass it to _PyObject_ASSERT(). So if the "is freed" assertion fails,
the parent is used in debug trace, rather than the freed object. The
parent object is more likely to contain useful information. Freed
objects cannot be inspected are are displayed as "<object at xxx is
freed>" with no other detail.

(cherry picked from commit 4d5f94b)

* Fix also a typo in PYMEM_DEADBYTE macro comment

* bpo-36389: Add newline to _PyObject_AssertFailed() (GH-16629)

Add a newline between the verbose object dump and the Py_FatalError()
logs for readability.

(cherry picked from commit 7775349)
jacobneiltaylor pushed a commit to jacobneiltaylor/cpython that referenced this pull request Dec 5, 2019
subtract_refs() now pass the parent object to visit_decref() which
pass it to _PyObject_ASSERT(). So if the "is freed" assertion fails,
the parent is used in debug trace, rather than the freed object. The
parent object is more likely to contain useful information. Freed
objects cannot be inspected are are displayed as "<object at xxx is
freed>" with no other detail.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants
0