8000 [PEP 558 - WIP] bpo-30744: Trace hooks no longer reset closure state by ncoghlan · Pull Request #3640 · python/cpython · GitHub
[go: up one dir, main page]

Skip to content

[PEP 558 - WIP] bpo-30744: Trace hooks no longer reset closure state #3640

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
7626a0e
bpo-30744: Trace hooks no longer reset closure state
ncoghlan Sep 18, 2017
3cbb73c
Disable the current broken writeback logic
ncoghlan Nov 5, 2017
01f3f34
Failing test case for writeback functionality
ncoghlan Nov 5, 2017
4f6dd93
Initial skeleton for a write-through proxy
ncoghlan Nov 5, 2017
cd45da7
Merge remote-tracking branch 'origin/master' into bpo-30744-make-loca…
ncoghlan Apr 21, 2019
acbf587
Finish adding the write-through proxy
ncoghlan Apr 21, 2019
d1a9014
Merge remote-tracking branch 'origin/master' into bpo-30744-make-loca…
ncoghlan May 21, 2019
5ea8bcf
Add test case for the PEP 558 locals() behaviour
ncoghlan May 22, 2019
fe92891
Actually implement most of the PEP and fix the tests
ncoghlan May 22, 2019
ac9e0bf
Fix segfault on cleared frames
ncoghlan May 27, 2019
6774e71
Use correct printf formatting code
ncoghlan May 27, 2019
74c51e4
Initial skeleton for other mutable mapping methods
ncoghlan May 28, 2019
0e5fbf3
Break ref cycle when frame finishes executing
ncoghl 8000 an May 30, 2019
9e3ce53
Remove implicit frame locals update
ncoghlan May 30, 2019
9479557
Merge remote-tracking branch 'origin/master' into bpo-30744-make-loca…
ncoghlan May 30, 2019
8e886ef
Avoid double DECREF on error
ncoghlan May 30, 2019
5b63e7c
Attempt to make dealloc more robust under gc
ncoghlan May 30, 2019
f0ecee3
Merge remote-tracking branch 'origin/master' into bpo-30744-make-loca…
ncoghlan Dec 29, 2019
1752b54
Fix post-merge compilation errors
ncoghlan Dec 29, 2019
348a56d
Implement flp.pop()
ncoghlan Dec 29, 2019
93f512c
Merge remote-tracking branch 'origin/master' into bpo-30744-make-loca…
ncoghlan Dec 29, 2019
39ec4d8
Refactor ref map creation
ncoghlan Dec 29, 2019
7078632
Correctly manage fast local refcounts
ncoghlan Dec 29, 2019
ed5f86e
Allow closure updates after frame termination
ncoghlan Dec 29, 2019
a216747
Refactor frame post-eval cleanup
ncoghlan Dec 29, 2019
a0dc787
Use full word in API name
ncoghlan Dec 29, 2019
7b02bed
Update to match latest PEP draft
ncoghlan Dec 29, 2019
e9876b5
Update test_scope for snapshot semantics
ncoghlan Dec 29, 2019
0033c60
Update test_sys_settrace for snapshot semantics
ncoghlan Dec 29, 2019
a5a8b19
Fix pop/delete locals proxy bug
ncoghlan Dec 29, 2019
6c98f48
Update argument clinic output
ncoghlan Dec 30, 2019
617f6ab
Merge remote-tracking branch 'origin/master' into bpo-30744-make-loca…
ncoghlan Dec 30, 2019
729f121
Merge remote-tracking branch 'origin/master' into bpo-30744-make-loca…
ncoghlan Jan 18, 2020
619fb75
Merge remote-tracking branch 'origin/master' into bpo-30744-make-loca…
ncoghlan Feb 2, 2020
1fe964e
Migrate to revised public API design
ncoghlan Feb 2, 2020
b047ae4
Rename PyFrame_LocalsIsSnapshot to PyFrame_GetLocalsReturnsSnapshot
ncoghlan Feb 2, 2020
d1a8420
Mark fast locals proxy as an internal type
ncoghlan Feb 2, 2020
82108ff
Merge remote-tracking branch 'origin/master' into bpo-30744-make-loca…
ncoghlan Feb 15, 2020
eccb1ea
Update draft C API to match latest PEP text
ncoghlan Feb 16, 2020
c1933e7
Migrate exec() and eval() to PyLocals_Get()
ncoghlan Feb 16, 2020
161ad47
Merge remote-tracking branch 'origin/master' into bpo-30744-make-loca…
ncoghlan Feb 22, 2020
68f10ce
Avoid circular reference between locals proxy and frame
ncoghlan Feb 22, 2020
29ce344
Add back implicit view refresh in Python trace hook
ncoghlan Feb 22, 2020
ed6e53b
Attempt to tidy up Mac OS X compile warnings/errors
ncoghlan Feb 22, 2020
96c77cb
Merge remote-tracking branch 'origin/main' into bpo-30744-make-locals…
ncoghlan Jul 3, 2021
a348d08
Fix Argument Clinic checksum
ncoghlan Jul 3, 2021
69c8f19
Fix stable ABI minimum version
ncoghlan Jul 3, 2021
caeaf66
Bring implementation into line with latest PEP version
ncoghlan Jul 10, 2021
7ec5d26
Register new stable ABI additions
ncoghlan Jul 10, 2021
7ddc3eb
Add FLP str(), fix various value lookup issues
ncoghlan Jul 10, 2021
7400a46
Uninitialised fields will get you every time
ncoghlan Jul 10, 2021
5eae0d5
Fix refcounting, bdb segfault, pdb functionality
ncoghlan Jul 10, 2021
40db4e7
Delegate more operations to the full dynamic snapshot
ncoghlan Jul 10, 2021
74b97a3
Add TODO item for false positives in containment checks
ncoghlan Jul 10, 2021
9f16513
Only ensure frame snapshot is up to date in O(n) proxy operations
ncoghlan Jul 10, 2021
c477e24
Keep locals snapshot up to date when reading/writing individual keys
ncoghlan Jul 11, 2021
dd94608
Avoid false positives in FLP contains method
ncoghlan Jul 11, 2021
b7eb662
Merge remote-tracking branch 'origin/main' into bpo-30744-make-locals…
ncoghlan Jul 17, 2021
1484c10
Finish public C API, start dict API tests
ncoghlan Jul 17, 2021
3891c13
Merge remote-tracking branch 'origin/main' into bpo-30744-make-locals…
ncoghlan Jul 17, 2021
760ffa9
Remove debugging print statement
ncoghlan Jul 17, 2021
ae6b013
Regenerated stable ABI files
ncoghlan Jul 18, 2021
b03309b
Rename _PyLocals_Kind APIs to avoid potential confusion
ncoghlan Jul 21, 2021
cde5035
Merge remote-tracking branch 'origin/main' into bpo-30744-make-locals…
ncoghlan Jul 21, 2021
66d058c
PyLocals_GetReturnsCopy -> PyLocals_GetKind()
ncoghlan Jul 21, 2021
67c3958
Share fast_refs mapping between proxy objects
ncoghlan Aug 21, 2021
034345f
Remove debugging print
ncoghlan Aug 21, 2021
3c49ff8
Defer value cache refresh until needed, start fleshing out dict API t…
ncoghlan Aug 21, 2021
ea5f943
Add dict union operations to proxy
ncoghlan Aug 21, 2021
fcf99ca
Implement and test locals proxy clear() method
ncoghlan Aug 21, 2021
16e0581
Remove pointless print() call
ncoghlan Aug 21, 2021
c356949
Implement proxy pop() tests
ncoghlan Aug 21, 2021
8a4e788
Implement and test proxy popitem()
ncoghlan Aug 21, 2021
706eec4
Test popitem with cells and extra variables
ncoghlan Aug 21, 2021
35a017c
Implement and test setdefault()
ncoghlan Aug 21, 2021
06c406c
Implement and test proxy __sizeof__()
ncoghlan Aug 21, 2021
31493b9
Add C API test for the LocalsToFast exception
ncoghlan Aug 21, 2021
b587a41
Force enum size
ncoghlan Aug 23, 2021
e1b505d
Clarify code comment
ncoghlan Aug 23, 2021
2b27389
Keep track of defined names even on cleared frames
ncoghlan Aug 26, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
PyLocals_GetReturnsCopy -> PyLocals_GetKind()
  • Loading branch information
ncoghlan committed Jul 21, 2021
commit 66d058c1eda1009c0590bcb0f36eafdba4df5dd5
2 changes: 1 addition & 1 deletion Doc/data/stable_abi.dat
Original file line number Diff line number Diff line change
Expand Up @@ -334,7 +334,7 @@ function,PyList_Sort,3.2,
var,PyList_Type,3.2,
function,PyLocals_Get,3.11,
function,PyLocals_GetCopy,3.11,
function,PyLocals_GetReturnsCopy,3.11,
function,PyLocals_Kind,3.11,
function,PyLocals_GetView,3.11,
type,PyLongObject,3.2,
var,PyLongRangeIter_Type,3.2,
Expand Down
14 changes: 8 additions & 6 deletions Include/ceval.h
Original file line number Diff line number Diff line change
Expand Up @@ -43,20 +43,22 @@ PyAPI_FUNC(PyFrameObject *) PyEval_GetFrame(void);
* It returns a read/write reference or a shallow copy depending on the scope
* of the active frame.
*/
// TODO: Add API tests for this
PyAPI_FUNC(PyObject *) PyLocals_Get(void);

/* PyLocals_GetCopy() returns a fresh shallow copy of the active local namespace */
// TODO: Implement this, and add API tests
PyAPI_FUNC(PyObject *) PyLocals_GetCopy(void);

/* PyLocals_GetView() returns a read-only proxy for the active local namespace */
// TODO: Implement this, and add API tests
PyAPI_FUNC(PyObject *) PyLocals_GetView(void);

/* Returns true if PyLocals_Get() returns a shallow copy in the active scope */
// TODO: Implement this, and add API tests
PyAPI_FUNC(int) PyLocals_GetReturnsCopy(void);
/* PyLocals_GetKind()reports the behaviour of PyLocals_Get() in the active scope */
typedef enum {
PyLocals_UNDEFINED = -1, // Indicates error (e.g. no thread state defined)
PyLocals_DIRECT_REFERENCE = 0,
PyLocals_SHALLOW_COPY = 1
} PyLocals_Kind;

PyAPI_FUNC(PyLocals_Kind) PyLocals_GetKind(void);
#endif


Expand Down
4 changes: 2 additions & 2 deletions Include/cpython/frameobject.h
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ PyAPI_FUNC(PyFrameObject *) PyFrame_GetBack(PyFrameObject *frame);

/* Fast locals proxy allows for reliable write-through from trace functions */
// TODO: Perhaps this should be hidden, and API users told to query for
// PyFrame_GetLocalsReturnsCopy() instead. Having this available
// PyFrame_GetLocalsKind() instead. Having this available
// seems like a nice way to let folks write some useful debug assertions,
// though.
PyAPI_DATA(PyTypeObject) _PyFastLocalsProxy_Type;
Expand All @@ -93,7 +93,7 @@ PyAPI_DATA(PyTypeObject) _PyFastLocalsProxy_Type;
PyAPI_FUNC(PyObject *) PyFrame_GetLocals(PyFrameObject *);
PyAPI_FUNC(PyObject *) PyFrame_GetLocalsCopy(PyFrameObject *);
PyAPI_FUNC(PyObject *) PyFrame_GetLocalsView(PyFrameObject *);
PyAPI_FUNC(int) PyFrame_GetLocalsReturnsCopy(PyFrameObject *);
PyAPI_FUNC(PyLocals_Kind) PyFrame_GetLocalsKind(PyFrameObject *);

// Underlying API supporting PyEval_GetLocals()
PyAPI_FUNC(PyObject *) _PyFrame_BorrowLocals(PyFrameObject *);
Expand Down
14 changes: 7 additions & 7 deletions Lib/test/test_frame.py
Original file line number Diff line number Diff line change
Expand Up @@ -234,7 +234,7 @@ def test_active_frame_c_apis(self):
Py_IncRef = ctypes.pythonapi.Py_IncRef
PyEval_GetLocals = ctypes.pythonapi.PyEval_GetLocals
PyLocals_Get = ctypes.pythonapi.PyLocals_Get
PyLocals_GetReturnsCopy = ctypes.pythonapi.PyLocals_GetReturnsCopy
PyLocals_GetKind = ctypes.pythonapi.PyLocals_GetKind
PyLocals_GetCopy = ctypes.pythonapi.PyLocals_GetCopy
PyLocals_GetView = ctypes.pythonapi.PyLocals_GetView
for capi_func in (Py_IncRef,):
Expand All @@ -255,7 +255,7 @@ def test_active_frame_c_apis(self):
Py_IncRef(c_locals_cache) # Make the borrowed reference a real one
Py_IncRef(c_locals_cache) # Account for next check's borrowed reference
self.assertIs(PyEval_GetLocals(), c_locals_cache)
self.assertTrue(PyLocals_GetReturnsCopy())
self.assertEqual(PyLocals_GetKind(), 1) # PyLocals_SHALLOW_COPY
locals_get = PyLocals_Get()
self.assertIsInstance(locals_get, dict)
self.assertIsNot(locals_get, c_locals_cache)
Expand All @@ -271,7 +271,7 @@ class ExecFrame:
Py_IncRef(c_locals_cache) # Make the borrowed reference a real one
Py_IncRef(c_locals_cache) # Account for next check's borrowed reference
self.assertIs(PyEval_GetLocals(), c_locals_cache)
self.assertFalse(PyLocals_GetReturnsCopy())
self.assertEqual(PyLocals_GetKind(), 0) # PyLocals_DIRECT_REFERENCE
locals_get = PyLocals_Get()
self.assertIs(locals_get, c_locals_cache)
locals_copy = PyLocals_GetCopy()
Expand All @@ -286,11 +286,11 @@ def test_arbitrary_frame_c_apis(self):
Py_IncRef = ctypes.pythonapi.Py_IncRef
_PyFrame_BorrowLocals = ctypes.pythonapi._PyFrame_BorrowLocals
PyFrame_GetLocals = ctypes.pythonapi.PyFrame_GetLocals
PyFrame_GetLocalsReturnsCopy = ctypes.pythonapi.PyFrame_GetLocalsReturnsCopy
PyFrame_GetLocalsKind = ctypes.pythonapi.PyFrame_GetLocalsKind
PyFrame_GetLocalsCopy = ctypes.pythonapi.PyFrame_GetLocalsCopy
PyFrame_GetLocalsView = ctypes.pythonapi.PyFrame_GetLocalsView
for capi_func in (Py_IncRef, _PyFrame_BorrowLocals,
PyFrame_GetLocals, PyFrame_GetLocalsReturnsCopy,
PyFrame_GetLocals, PyFrame_GetLocalsKind,
PyFrame_GetLocalsCopy, PyFrame_GetLocalsView):
capi_func.argtypes = (ctypes.py_object,)
for capi_func in (_PyFrame_BorrowLocals, PyFrame_GetLocals,
Expand All @@ -314,7 +314,7 @@ def set_cls_frame(f):
class ExecFrame:
c_locals_cache = get_c_locals(func_frame)
self.assertIs(get_c_locals(func_frame), c_locals_cache)
self.assertTrue(PyFrame_GetLocalsReturnsCopy(func_frame))
self.assertEqual(PyFrame_GetLocalsKind(func_frame), 1) # PyLocals_SHALLOW_COPY
locals_get = PyFrame_GetLocals(func_frame)
self.assertIsInstance(locals_get, dict)
self.assertIsNot(locals_get, c_locals_cache)
Expand All @@ -330,7 +330,7 @@ class ExecFrame:
# Test querying an unoptimised frame from an optimised scope
c_locals_cache = get_c_locals(cls_frame)
self.assertIs(get_c_locals(cls_frame), c_locals_cache)
self.assertFalse(PyFrame_GetLocalsReturnsCopy(cls_frame))
self.assertEqual(PyFrame_GetLocalsKind(cls_frame), 0) # PyLocals_DIRECT_REFERENCE
locals_get = PyFrame_GetLocals(cls_frame)
self.assertIs(locals_get, c_locals_cache)
locals_copy = PyFrame_GetLocalsCopy(cls_frame)
Expand Down
2 changes: 1 addition & 1 deletion Misc/stable_abi.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2143,7 +2143,7 @@ function PyLocals_Get
added 3.11
function PyLocals_GetCopy
added 3.11
function PyLocals_GetReturnsCopy
function PyLocals_GetKind
added 3.11
function PyLocals_GetView
added 3.11
11 changes: 7 additions & 4 deletions Objects/frameobject.c
Original file line number Diff line number Diff line change
Expand Up @@ -81,13 +81,16 @@ PyFrame_GetLocals(PyFrameObject *f)
return updated_locals;
}

int
PyFrame_GetLocalsReturnsCopy(PyFrameObject *f)
PyLocals_Kind
PyFrame_GetLocalsKind(PyFrameObject *f)
{
// This frame API supports the stable PyLocals_GetReturnsCopy() API
// This frame API supports the stable PyLocals_GetKind() API
PyCodeObject *co = _PyFrame_GetCode(f);
assert(co);
return (co->co_flags & CO_OPTIMIZED);
if (co->co_flags & CO_OPTIMIZED) {
return PyLocals_SHALLOW_COPY;
}
return PyLocals_DIRECT_REFERENCE;
}

PyObject *
Expand Down
2 changes: 1 addition & 1 deletion PC/python3dll.c
Original file line number Diff line number Diff line change
Expand Up @@ -315,7 +315,7 @@ EXPORT_FUNC(PyList_Size)
EXPORT_FUNC(PyList_Sort)
EXPORT_FUNC(PyLocals_Get)
EXPORT_FUNC(PyLocals_GetCopy)
EXPORT_FUNC(PyLocals_GetReturnsCopy)
EXPORT_FUNC(PyLocals_GetKind)
EXPORT_FUNC(PyLocals_GetView)
EXPORT_FUNC(PyLong_AsDouble)
EXPORT_FUNC(PyLong_AsLong)
Expand Down
8 changes: 4 additions & 4 deletions Python/ceval.c
Original file line number Diff line number Diff line change
Expand Up @@ -5803,17 +5803,17 @@ PyLocals_Get(void)
return PyFrame_GetLocals(current_frame);
}

int
PyLocals_GetReturnsCopy(void)
PyLocals_Kind
PyLocals_GetKind(void)
{
PyThreadState *tstate = _PyThreadState_GET();
PyFrameObject *current_frame = tstate->frame;
if (current_frame == NULL) {
_PyErr_SetString(tstate, PyExc_SystemError, "frame does not exist");
return NULL;
return PyLocals_UNDEFINED;
}

return PyFrame_GetLocalsReturnsCopy(current_frame);
return PyFrame_GetLocalsKind(current_frame);
}

PyObject *
Expand Down
0