8000 gh-100554: Add ``Py_tp_vectorcall`` slot to set ``PyTypeObject.tp_vectorcall`` using the ``PyType_FromSpec`` function family. by wjakob · Pull Request #123332 · python/cpython · GitHub
[go: up one dir, main page]
Skip to content

gh-100554: Add Py_tp_vectorcall slot to set PyTypeObject.tp_vectorcall using the PyType_FromSpec function family. #123332

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Sep 13, 2024
Merged

gh-100554: Add Py_tp_vectorcall slot to set PyTypeObject.tp_vectorcall using the PyType_FromSpec function family. #123332

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
85c60d9
add Py_tp_vectorcall
wjakob Aug 26, 2024
e9c07c0
incorporate review suggestions
wjakob Aug 27, 2024
3ef669d
Documentation changes
encukou Aug 27, 2024
d4b67de
Merge branch 'main' into fix-issue-100554
wjakob Aug 29, 2024
ef81dc0
PEP-7 fixes
wjakob Sep 4, 2024
58f6759
Apply GH suggestions for docs
encukou Sep 13, 2024
dffd2a9
Merge branch 'main' into fix-issue-100554
encukou Sep 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
add Py_tp_vectorcall
  • Loading branch information
@wjakob
wjakob committed Aug 26, 2024
commit 85c60d9415e80271220df14cc3b388cf3e935691
9 changes: 5 additions & 4 deletions Doc/c-api/type.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -492,11 +492,12 @@ The following functions and structs are used to create
See :ref:`PyMemberDef documentation <pymemberdef-offsets>`
for details.

The following fields cannot be set at all when creating a heap type:
* The field :c:member:`~PyTypeObject.tp_vectorcall` can be set since
Python 3.14. On older versions, use :c:member:`~PyTypeObject.tp_new`
and/or :c:member:`~PyTypeObject.tp_init`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use .. versionchanged, like the 3.9 & 3.11 PyBufferProcs entries a few lines down.


* :c:member:`~PyTypeObject.tp_vectorcall`
(use :c:member:`~PyTypeObject.tp_new` and/or
:c:member:`~PyTypeObject.tp_init`)
The following internal fields cannot be set at all when creating a heap
type:

* Internal fields:
:c:member:`~PyTypeObject.tp_dict`,
Expand Down
4 changes: 4 additions & 0 deletions Include/typeslots.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,3 +86,7 @@
/* New in 3.10 */
#define Py_am_send 81
#endif
#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x030E0000
/* New in 3.14 */
#define Py_tp_vectorcall 82
#endif
7 changes: 7 additions & 0 deletions Lib/test/test_capi/test_misc.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -729,6 +729,13 @@ class Base(metaclass=metaclass):
self.assertTrue(issubclass(sub, Base))
self.assertIsInstance(sub, metaclass)

def test_heaptype_with_tp_vectorcall(self):
tp = _testcapi.HeapCTypeVectorcall
self.assertTrue(issubclass(tp, type))
value = tp()
self.assertIsInstance(value, int)
self.assertEqual(value, 123)

def test_multiple_inheritance_ctypes_with_weakref_or_dict(self):

with self.assertRaises(TypeError):
Expand Down
4 changes: 4 additions & 0 deletions Misc/NEWS.d/next/C_API/2024-08-26-13-01-20.gh-issue-100554.0ku85o.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Added a slot ``Py_tp_vectorcall`` to set
:c:member:`~PyTypeObject.tp_vectorcall` via the :c:func:`PyType_FromSpec`
function family. Limited API extensions can use this feature to provide more
efficient vector call-based implementation of ``__new__`` and ``__init__``.
2 changes: 2 additions & 0 deletions Misc/stable_abi.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2510,3 +2510,5 @@
added = '3.14'
[function.PyIter_NextItem]
added = '3.14'
[const.Py_tp_vectorcall]
added = '3.14'
22 changes: 22 additions & 0 deletions Modules/_testcapi/heaptype.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1008,6 +1008,24 @@ static PyType_Spec HeapCTypeSetattr_spec = {
HeapCTypeSetattr_slots
};

static PyObject *heapctype_vectorcall(PyObject *self, PyObject *const *args_in,
size_t nargsf, PyObject *kwargs_in) {
return PyLong_FromLong(123);
}

static PyType_Slot HeapCTypeVectorcall_slots[] = {
{Py_tp_vectorcall, heapctype_vectorcall},
{0, 0},
};
Copy link
Member
@encukou encukou Aug 26, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That looks like a good example of how not to do it, since it leaves tp_call from the base's metaclass:

>>> import _testcapi
>>> _testcapi.HeapCTypeVectorcall.__call__(_testcapi.HeapCTypeVectorcall, 'Sub', (), {})
<class '__m
10000
ain__.Sub'>

As per the docs, a class supporting vectorcall must also implement tp_call with the same semantics. Here, the overridden vectorcall function would need to match the behaviour of PyType_Type->tp_call (including any changes in future versions).

So, you do need to override tp_new, and things get even more tricky if you allow subclasses/instances of HeapCTypeVectorcall.

Copy link
Contributor Author
@wjakob wjakob Aug 26, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, the point of this PR is to be able to do exactly the same as what tp_call does, just more efficiently, so this difference in behavior would not arise in real-world usage.

But that is less straightforward to test from the outside, hence the the contrived example of returning 123, which is not even an instance of the type.

(To keep the test reasonably compact, I would prefer not to deal with metaclasses on top)

I will override tp_new and make the type non-inheritable.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm worried about leading users to a trap, and I'd prefer to see a full solution for the use case, to get an idea about where in the docs to put warnings, and what kinds of issues this opens.
Putting this in limited API means we should be extra careful about not exposing implementation details.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy to implement a more representative version but wonder about the following: how should the vectorcall version communicate to the test suite that it was used? If the equivalence requirement you stated is interpreted strictly, it should not be possible to detect any difference.

For example, would it be OK I return two different instances from each mode of construction and then add a comment saying that these are only different for test-related purposes, and that the contract of these operations is that they should be interchangeable?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's fine for tests.


static PyType_Spec HeapCTypeVectorcall_spec = {
"_testcapi.HeapCTypeVectorcall",
0,
0,
Py_TPFLAGS_DEFAULT,
HeapCTypeVectorcall_slots
};

PyDoc_STRVAR(HeapCCollection_doc,
"Tuple-like heap type that uses PyObject_GetItemData for items.");

Expand Down Expand Up @@ -1180,6 +1198,10 @@ _PyTestCapi_Init_Heaptype(PyObject *m) {
PyObject *HeapCTypeSetattr = PyType_FromSpec(&HeapCTypeSetattr_spec);
ADD("HeapCTypeSetattr", HeapCTypeSetattr);

PyObject *HeapCTypeVectorcall = PyType_FromSpecWithBases(
&HeapCTypeVectorcall_spec, (PyObject *) &PyType_Type);
ADD("HeapCTypeVectorcall", HeapCTypeVectorcall);

PyObject *subclass_with_finalizer_bases = PyTuple_Pack(1, HeapCTypeSubclass);
if (subclass_with_finalizer_bases == NULL) {
return -1;
Expand Down
1 change: 1 addition & 0 deletions Objects/typeslots.inc
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
0