8000 bpo-42990: Introduce 'frame constructor' struct to simplify API for PyEval_CodeEval and friends by markshannon · Pull Request #24298 · python/cpython · GitHub
[go: up one dir, main page]

Skip to content

bpo-42990: Introduce 'frame constructor' struct to simplify 8000 API for PyEval_CodeEval and friends #24298

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jan 29, 2021

Conversation

markshannon
Copy link
Member
@markshannon markshannon commented Jan 22, 2021

Rather than pass the contents of a PyFunctionObject as 8 different parameters, this PR passes a pointer to a struct, and embeds that struct in PyFunctionObject.

Only saves a few lines of code, but should enable more refactorings in _PyFunction_Vectorcall and similar functions.

https://bugs.python.org/issue42990

@markshannon
Copy link
Member Author
markshannon commented Jan 22, 2021

Skipping NEWS as this is just an internal cleanup

Comment on lines 24 to 41
PyObject *func_globals; /* A dictionary (other mappings won't do) */
PyObject *func_defaults; /* NULL or a tuple */
PyObject *func_kwdefaults; /* NULL or a dict */
PyObject *func_closure; /* NULL or a tuple of cell objects */
PyFrameConstructor func_descr; /* Frame descriptor fields for this function */
PyObject *func_doc; /* The __doc__ attribute, can be anything */
PyObject *func_name; /* The __name__ attribute, a string object */
PyObject *func_dict; /* The __dict__ attribute, a dict or NULL */
PyObject *func_weakreflist; /* List of weak references */
PyObject *func_module; /* The __module__ attribute, can be anything */
PyObject *func_annotations; /* Annotations, a dict or NULL */
PyObject *func_qualname; /* The qualified name */
vectorcallfunc vectorcall;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to my reading of PEP 387, this change should go through the incompatible changes process.

Copy link
Member Author
@markshannon markshannon Jan 22, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not according to my reading. The internal layout of the PyFunctionObject is not part of the API.

@markshannon
Copy link
Member Author

@encukou Are you OK with me merging this, or do you think it represents an incompatible change?

@encukou
Copy link
Member
encukou commented Jan 23, 2021

Well, there's nothing preventing people from using PyCFunction PyFunctionObject internals, as it's in the public header. AFAICS, the struct has been API-compatible (i.e. only new fields were added to it) for 25+ years. I wouldn't be surprised if projects are using these fields.
Formally, I don't see anything in PEP 387 that would exclude the fields from the C-API.

On the other hand, there are getter/setter functions for these fields, and those should be preferred.

@markshannon
Copy link
Member Author

The internals of C structs aren't part of the API. That would just be too fragile.
That's why all API parameters are PyObject *, rather than more specific types.
Sometimes structs need to be included in header files, for practical reasons, but that doesn't make them part of the API.
We are doing a better job of hiding them, largely thanks to @vstinner, but there are still some exposed structs.

https://docs.python.org/3/c-api/structures.html lists some structs, like PyMemberDef, which are used to pass information to the interpreter. But those are the exception, not the rule.

The internals of PyDictObject, PyFrameObject, PyCodeObject, and others, have all changed over recent years. None of these changes have been regarded as API changes.

@markshannon markshannon reopened this Jan 25, 2021
@encukou
8000 Copy link
Member
encukou commented Jan 26, 2021

The internals of PyDictObject, PyFrameObject, PyCodeObject, and others, have all changed over recent years.

All these are defined in Include/cpython/, but PyFunctionObject is directly in Include/funcobject.h. @vstinner, is there a reason for that, or is it just that nobody got around to moving it so far?

@markshannon markshannon requested a review from vstinner January 27, 2021 10:52
@markshannon
Copy link
Member Author

I've modified PyFunctionObject to support PyFrameConstructor and be backwards compatible, see https://bugs.python.org/issue43054 for explanation of why this is valid.

@encukou
Copy link
Member
encukou commented Jan 28, 2021

This looks good, but IMO it's the first use of anonymous structs/unions. How is the compiler support for them?
They'll need to be added to PEP 7, as they're not part of C89.

@markshannon
Copy link
Member Author

I've used a macro to make things C89, for now.

@markshannon markshannon merged commit d6c33fb into python:master Jan 29, 2021
@markshannon markshannon deleted the frame-descriptors branch January 29, 2021 13:39
Copy link
Member
@vstinner vstinner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is based on the fact that PyFunction_AS_FRAME_CONSTRUCTOR() is efficient, but it changes structures.

I suggest starting with a first PR which leaves structures unchanged and create a PyFrameConstructor from other structures by copying members. Something like:

PyFrameConstructor frame_constr = FrameConstructorFromFunction(func);

@vstinner
Copy link
Member

Oh. I didn't notice that this PR is already merged :-)

@vstinner
Copy link
Member

Well, we will see if anyone rely on the ABI of these structures ;-) I'm in favor of making these structures opaque: https://www.python.org/dev/peps/pep-0620/ :-)

adorilson pushed a commit to adorilson/cpython that referenced this pull request Mar 13, 2021
…yEval_CodeEval and friends (pythonGH-24298)

* Introduce 'frame constructor' to simplify API for frame creation

* Embed struct using a macro to conform to PEP 7
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants
0