diff --git a/DesignPrinciples.md b/DesignPrinciples.md index 0345449..220433b 100644 --- a/DesignPrinciples.md +++ b/DesignPrinciples.md @@ -56,7 +56,8 @@ have access to what state, improving robustness. ## API and ABI equivalence -All code written to the API will continue to work on future versions of Python +Provided that the extension is built in ABI compatible mode, then all code +written to the API will continue to work on future versions of Python without recompilation. Recompilation using newer versions may be more efficient, but code compiled to older versions of the API will continue to work. diff --git a/DesignRules.md b/DesignRules.md index b6fff23..0667ccd 100644 --- a/DesignRules.md +++ b/DesignRules.md @@ -22,6 +22,9 @@ For example `PyMemContext` is passed to destructor functions which prevents them from doing much more than freeing `PyRef`s and memory. +The exception to this rule is the set of funtions that get references +to per-process objects like `int` or `type`. + ## Opaque, linear references The C-API will refer to Python objects through opaque references @@ -48,50 +51,83 @@ The is a consequence of the "No invalid states" design principle. The legacy API allows inplace mutation of tuples, strings and other immutable object. These will not be allowed in the new API. -## All functions have an error "out" parameter, or return the error +## All functions must clearly show if an error has occurred. + +Whether an API function has produced an error must be clear +from the return value, and only from the return value. + +API functions must obey the following rules: + +* If no valid result can be returned, the the result value must be `PyRef_INVALID` or `-1`. +* If the result is passed though a pointer, and the an error occurs, then the result value must be untouched. +* If the result is valid, then an error must not have occured +* If an error has occurred, then the result of immediately calling `PyApi_GetLatestException()` must be that error. -### Functions that have results +Note that if an API function does not produce a result, the result of calling `PyApi_GetLatestException()` +is undefined. It will be a legal, safe to use value; it will just be meaningless. + + +### Functions that return references Many functions return a result, but may also raise an exception. -To handle this, all such API functions should have an `error` out -parameter, of the form `PyExceptionRef *error`. +To handle this case, a special value is used to indicate an error, +`PyRef_INVALID`. If an error occurs, `PyRef_INVALID` must be returned. +Conversely, if `PyRef_INVALID` is returned an error must have occurred. -The error pointer should always be the final parameter. +### Functions that return booleans. -API functions must obey the following rules: +These functions will return `int`, with -1 indicating an error. +0 and 1 indicate `False` and `True`, respectively. + +### Functions that return ints -* If no valid result can be returned, the the result value should be `PyRef_INVALID`. -* If the result is valid, then the memory pointer to by `error` should be untouched. +Functions that only return non-negative values can use -1 to indicate an error. +Functions that can return negative values must return an error code, +returning the real result via a pointer. -Some functions can fail without an error. Failure should be represented by returning -`PyRef_INVALID` and `*error = PyRef_NO_EXCEPTION`. +### Functions that may fail without an error + +Some functions can fail without an error. Those functions must return an `int` +error code and pass the result through a pointer. + +Success should be indicated by zero, and non-error failures by positive values. For example, to get a value from a dictionary might have the following API: ```C -PyRef PyAPi_Dict_Get(PyContext ctx, PyDictRef dict, PyRef key, PyExceptionRef *value); +int PyAPi_Dict_Get(PyContext ctx, PyDictRef dict, PyRef key, PyRef *result); ``` -If the result is `PyRef_INVALID` then the failure and error cases can be differentiated -by testing `*error == PyRef_NO_EXCEPTION`. +If the result is zero then the function has succeeded. +Otherwise the error and failure cases can be distinguished by whether +the result is negative. ### Functions without results Some functions, e.g. `PyApi_List_Append()` do not produce a result, but can raise. -Those functions should return a `PyExceptionRef`. +Those functions should return an `int`. ```C -PyExceptionRef PyApi_List_Append(PyContext ctx, PyListRef list, PyRef item); +int PyApi_List_Append(PyContext ctx, PyListRef list, PyRef item); ``` -Success is indicated by returning `PyRef_NO_EXCEPTION`. +Success is indicated by returning a negative value, usually -1. + +### Cleanup after an error + +In order to allow the cleanup of references after an error, +`PyRef_Dup`, `PyRef_Close` and `PyRef_Free` will not change the +result of `PyApi_GetLatestException()`. + ## Naming All API function and struct names should adhere to simple rules. For example, function names should take the form: -Prefix_NameSpace_Operation[_REF_CONSUMPTION] +Prefix_NameSpace_Operation[_REF_CONSUMPTION][_VERSION] E.g. ```C PyTupleRef PyApi_Tuple_FromArray(PyContext ctx, uintptr_t len, PyRef *array, PyExceptionRef *error); + +PyTupleRef PyApi_Tuple_FromNonEmptyArray_nC_v2(PyContext ctx, uintptr_tlen, PyRef *array); ``` ## Use C99 types @@ -160,11 +196,11 @@ differently, returning the empty tuple singleton. We handle this tension by providing an efficient, but difficult use ABI function: ```C -PyTupleRef PyApi_Tuple_FromNonEmptyArray_nC(PyContext ctx, uintptr_tlen, PyRef *array, PyExceptionRef *error); +PyTupleRef PyApi_Tuple_FromNonEmptyArray_nC(PyContext ctx, uintptr_tlen, PyRef *array); ``` and the easier to use API function ```C -PyTupleRef PyApi_Tuple_FromArray(PyContext ctx, uintptr_t len, PyRef *array, PyExceptionRef *error); +PyTupleRef PyApi_Tuple_FromArray(PyContext ctx, uintptr_t len, PyRef *array); ``` However, we can make this even easier to use by making a macro that diff --git a/Interop.md b/Interop.md index d5421c2..30bf47d 100644 --- a/Interop.md +++ b/Interop.md @@ -1,7 +1,8 @@ # Interoperability API The interopability API consists of functions for converting -`PyObject *` to `PyRef`, and vice versa +`PyObject *` to `PyRef`, and vice versa. +These functions are for internal use, so do not take a `PyContext`. ## Converting `PyObject *` to `PyRef` @@ -42,9 +43,8 @@ PyRef PyApi_Interop_FromObjectUnsafe_C(PyObject *obj) return (PyRef) { obj }; } -int PyApi_Interop_FromObject_C(PyObject *obj, PyRef *result) +PyRef PyApi_Interop_FromObject_C(PyObject *obj, PyExceptionRef *exc) { - int kind; if (PyErr_Occurred()) { PyObject *exception = get_normalized_exception(); if (object != NULL) { @@ -56,12 +56,10 @@ int PyApi_Interop_FromObject_C(PyObject *obj, PyRef *result) PyException_SetCause(new_exception, exception); Py_DECREF(exception); Py_DECREF(obj); - obj = new_exception; + exception = new_exception; } - else { - obj = exception; - } - kind = ERROR; + *exc = PyApi_Interop_FromException(exception); + return PyRef_INVALID; } else { if (obj == NULL) { @@ -70,16 +68,11 @@ int PyApi_Interop_FromObject_C(PyObject *obj, PyRef *result) PyExc_SystemError, "value is NULL without setting an exception" ); - obj = exception; - kind = ERROR; - } - else { - *result = obj; - kind = SUCCESS; + *exc = PyApi_Interop_FromException(exception); + return PyRef_INVALID; } } - *result = PyApi_Interop_FromObjectUnsafe(obj); - return kind; + return PyApi_Interop_FromObjectUnsafe(obj); } ``` diff --git a/README.md b/README.md index 560f181..999d211 100644 --- a/README.md +++ b/README.md @@ -18,7 +18,7 @@ We cannot incrementally change the current API, as each change will cause breakage and backwards incompatibility issues, leaving extension modules constantly broken. -Rather than change the existing C API is it better to create an entirely new API, +Rather than change the existing C API it is better to create an entirely new API, and deprecate the old one. This allows extension authors to switch to the new API at a time of their choosing, rather than constantly scrambling to keep up. @@ -73,21 +73,21 @@ To help porting we will provide an [interopability API](./Interop.md) This is very much provisional at this point. -* 2023 (3.12): New C API specification complete. Deprecation of unsafe parts of legacy C API. -* 2024 (3.13): New C API implementation in CPython complete. -* 2025 (3.14): Removal of unsafe parts of legacy C API. Deprecation of legacy C API. -* 2026 (3.15): Add warnings of parts of legacy C API with negative impact on performance. -* 2027 (3.16): Add warnings on any use of legacy C API -* 2028 (3.17): Remove parts of legacy C API with negative impact on performance. -* 2029 (3.18) +* 2024 (3.13): New C API specification complete. Deprecation of unsafe parts of legacy C API. +* 2025 (3.14): New C API implementation in CPython complete. +* 2026 (3.15): Removal of unsafe parts of legacy C API. Deprecation of legacy C API. +* 2027 (3.16): Add warnings of parts of legacy C API with negative impact on performance. +* 2028 (3.17): Add warnings on any use of legacy C API +* 2029 (3.18): Remove parts of legacy C API with negative impact on performance. * 2030 (3.19) -* 2031 (3.20/4.0): Removal of legacy C API +* 2031 (3.20) +* 2032 (3.21/4.0): Removal of legacy C API The removal of the legacy C API will happen in three stages: -* 2025 (3.14): Removal of the unsafe parts of the API: parts of the API that return borrowed references, or mutate immutable objects. -* 2028 (3.17): Removal of performance limiting parts of the API. For example, parts of the API that prevent improvements to internal data structures. -* 2031 (3.20/4.0): Removal of the rest of the legacy C API. +* 2026 (3.15): Removal of the unsafe parts of the API: parts of the API that return borrowed references, or mutate immutable objects. +* 2029 (3.18): Removal of performance limiting parts of the API. For example, parts of the API that prevent improvements to internal data structures. +* 2032 (3.21/4.0): Removal of the rest of the legacy C API. ## Documentation diff --git a/api/PyAPI.h b/api/PyAPI.h index 9a079ea..2ecff41 100644 --- a/api/PyAPI.h +++ b/api/PyAPI.h @@ -3,15 +3,17 @@ PyRef PyRef_INVALID; PyExceptionRef PyRef_NO_EXCEPTION; +PyRef PyRef_Dup(PyRef ref); +PyRef PyRef_Close(PyRef ref); /* Tuple */ PyTupleRef PyApi_Tuple_Empty(PyContext ctx); -PyTupleRef PyApi_Tuple_FromNonEmptyArray(PyContext ctx, PyRef array[], uintptr_t length, PyExceptionRef *exc); +PyTupleRef PyApi_Tuple_FromNonEmptyArray(PyContext ctx, PyRef array[], uintptr_t length); -PyTupleRef PyApi_Tuple_FromArray(PyContext ctx, PyRef array[], uintptr_t length, PyExceptionRef *exc); +PyTupleRef PyApi_Tuple_FromArray(PyContext ctx, PyRef array[], uintptr_t length); -PyRef PyApi_Tuple_GetItem(PyContext ctx, PyTupleRef self, uintptr_t index, PyExceptionRef *exc); +PyRef PyApi_Tuple_GetItem(PyContext ctx, PyTupleRef self, uintptr_t index); uintptr_t PyApi_Tuple_GetSize(PyContext ctx, PyTupleRef self); @@ -20,17 +22,17 @@ PyTupleRef PyApi_Tuple_UnsafeCast(PyRef ref); PyTupleRef PyApi_Tuple_DownCast(PyRef ref); PyRef PyApi_Tuple_UpCast(PyTupleRef ref); -typedef int (*PyApi_BinaryOperator_FuncPtr)(PyContext ctx, PyRef left, PyRef right, PyRef *result); +typedef PyRef (*PyApi_BinaryOperator_FuncPtr)(PyContext ctx, PyRef left, PyRef right); -typedef int (*PyApi_VectorCall_FuncPtr)(PyContext ctx, PyRef callable, PyRef args[], intptr_t nargsf, PyTupleRef kwnames, PyRef *result); +typedef PyRef (*PyApi_VectorCall_FuncPtr)(PyContext ctx, PyRef callable, PyRef args[], intptr_t nargsf, PyTupleRef kwnames); /* Str */ -PyStrRef PyApi_Str_FromUtfString(PyContext ctx, const UtfString data, uintptr_t length, PyExceptionRef *exc); +PyStrRef PyApi_Str_FromUtfString(PyContext ctx, const UtfString data, uintptr_t length); -PyStrRef PyApi_Str_Join(PyContext ctx, PyStrRef self, PyStrRef array[], PyExceptionRef *exc); +PyStrRef PyApi_Str_Join(PyContext ctx, PyStrRef self, PyStrRef array[]); -PyRef PyApi_Str_GetItem(PyContext ctx, PyStrRef self, uintptr_t index, PyExceptionRef *exc); +PyRef PyApi_Str_GetItem(PyContext ctx, PyStrRef self, uintptr_t index); uintptr_t PyApi_Str_GetSize(PyContext ctx, PyStrRef self); @@ -41,11 +43,11 @@ PyRef PyApi_Str_UpCast(PyStrRef ref); /* Class */ -PyExceptionRef PyApi_Class_AddBinaryOperator(PyContext ctx, uint8_t op, PyApi_BinaryOperator_FuncPtr func); +int PyApi_Class_AddBinaryOperator(PyContext ctx, uint8_t op, PyApi_BinaryOperator_FuncPtr func); -PyExceptionRef PyApi_Class_AddVectorCallMethod(PyContext ctx, PyStrRef name, PyApi_VectorCall_FuncPtr func); +int PyApi_Class_AddVectorCallMethod(PyContext ctx, PyStrRef name, PyApi_VectorCall_FuncPtr func); -PyRef PyApi_Class_New(PyContext ctx, PyClassRef self, PyExceptionRef *exc); +PyRef PyApi_Class_New(PyContext ctx, PyClassRef self); bool PyApi_IsAClass(PyRef ref); PyClassRef PyApi_Class_UnsafeCast(PyRef ref); @@ -54,180 +56,211 @@ PyRef PyApi_Class_UpCast(PyClassRef ref); bool PyApi_IsNone(PyContext ctx, PyRef obj); -PyRef PyApi_None(PyContext ctx); +PyRef PyApi_None(); bool PyApi_IsTrue(PyContext ctx, PyRef obj); -PyRef PyApi_True(PyContext ctx); +PyRef PyApi_True(); bool PyApi_IsFalse(PyContext ctx, PyRef obj); -PyRef PyApi_False(PyContext ctx); +PyRef PyApi_False(); -PyClassRef PyApi_TimeoutError(PyContext ctx); +PyClassRef PyApi_TimeoutError(); -PyClassRef PyApi_BaseException(PyContext ctx); +PyClassRef PyApi_bool(); +PyClassRef PyApi_memoryview(); -/* Exception */ -PyExceptionRef PyApi_Exception_FromString(PyContext ctx, PyClassRef cls, const UtfString message, PyExceptionRef *exc); +PyClassRef PyApi_bytearray(); -PyExceptionRef PyApi_Exception_FromValue(PyContext ctx, PyClassRef cls, PyRef message, PyExceptionRef *exc); +PyClassRef PyApi_bytes(); -void PyApi_Exception_Fatal(PyContext ctx, const UtfString message); +PyClassRef PyApi_classmethod(); -PyExceptionRef PyApi_Exception_FromErrnoWithFilename(PyContext ctx, PyClassRef cls, const UtfString filename, PyExceptionRef *exc); +PyClassRef PyApi_complex(); -/* Always fails. Returns, barring another error, - { PyRef PyRef_INVALID, new_exception } - */ -PyExceptionRef PyApi_Exception_RaiseFromString(PyContext ctx, PyClassRef cls, const UtfString message, PyExceptionRef *exc); +PyClassRef PyApi_dict(); -/* Always fails. Returns, barring another error, - { PyRef PyRef_INVALID, new_exception } - */ -PyExceptionRef PyApi_Exception_RaiseFromValue(PyContext ctx, PyClassRef cls, PyRef message, PyExceptionRef *exc); +PyClassRef PyApi_enumerate(); -bool PyApi_IsAnException(PyRef ref); -PyExceptionRef PyApi_Exception_UnsafeCast(PyRef ref); -PyExceptionRef PyApi_Exception_DownCast(PyRef ref); -PyRef PyApi_Exception_UpCast(PyExceptionRef ref); +PyClassRef PyApi_filter(); + +PyClassRef PyApi_float(); + +PyClassRef PyApi_frozenset(); + +PyClassRef PyApi_property(); + +PyClassRef PyApi_int(); + +PyClassRef PyApi_list(); + +PyClassRef PyApi_map(); + +PyClassRef PyApi_object(); + +PyClassRef PyApi_range(); + +PyClassRef PyApi_reversed(); + +PyClassRef PyApi_set(); + +PyClassRef PyApi_slice(); + +PyClassRef PyApi_staticmethod(); + +PyClassRef PyApi_str(); + +PyClassRef PyApi_super(); + +PyClassRef PyApi_tuple(); -PyClassRef PyApi_TypeError(PyContext ctx); +PyClassRef PyApi_type(); -PyClassRef PyApi_StopAsyncIteration(PyContext ctx); +PyClassRef PyApi_zip(); -PyClassRef PyApi_StopIteration(PyContext ctx); +PyClassRef PyApi_BaseException(); -PyClassRef PyApi_GeneratorExit(PyContext ctx); +PyClassRef PyApi_Exception(); -PyClassRef PyApi_SystemExit(PyContext ctx); +PyClassRef PyApi_TypeError(); -PyClassRef PyApi_KeyboardInterrupt(PyContext ctx); +PyClassRef PyApi_StopAsyncIteration(); -PyClassRef PyApi_ImportError(PyContext ctx); +PyClassRef PyApi_StopIteration(); -PyClassRef PyApi_ModuleNotFoundError(PyContext ctx); +PyClassRef PyApi_GeneratorExit(); -PyClassRef PyApi_OSError(PyContext ctx); +PyClassRef PyApi_SystemExit(); -PyClassRef PyApi_EnvironmentError(PyContext ctx); +PyClassRef PyApi_KeyboardInterrupt(); -PyClassRef PyApi_IOError(PyContext ctx); +PyClassRef PyApi_ImportError(); -PyClassRef PyApi_EOFError(PyContext ctx); +PyClassRef PyApi_ModuleNotFoundError(); -PyClassRef PyApi_RuntimeError(PyContext ctx); +PyClassRef PyApi_OSError(); -PyClassRef PyApi_RecursionError(PyContext ctx); +PyClassRef PyApi_EnvironmentError(); -PyClassRef PyApi_NotImplementedError(PyContext ctx); +PyClassRef PyApi_IOError(); -PyClassRef PyApi_NameError(PyContext ctx); +PyClassRef PyApi_EOFError(); -PyClassRef PyApi_UnboundLocalError(PyContext ctx); +PyClassRef PyApi_RuntimeError(); -PyClassRef PyApi_AttributeError(PyContext ctx); +PyClassRef PyApi_RecursionError(); -PyClassRef PyApi_SyntaxError(PyContext ctx); +PyClassRef PyApi_NotImplementedError(); -PyClassRef PyApi_IndentationError(PyContext ctx); +PyClassRef PyApi_NameError(); -PyClassRef PyApi_TabError(PyContext ctx); +PyClassRef PyApi_UnboundLocalError(); -PyClassRef PyApi_LookupError(PyContext ctx); +PyClassRef PyApi_AttributeError(); -PyClassRef PyApi_IndexError(PyContext ctx); +PyClassRef PyApi_SyntaxError(); -PyClassRef PyApi_KeyError(PyContext ctx); +PyClassRef PyApi_IndentationError(); -PyClassRef PyApi_ValueError(PyContext ctx); +PyClassRef PyApi_TabError(); -PyClassRef PyApi_UnicodeError(PyContext ctx); +PyClassRef PyApi_LookupError(); -PyClassRef PyApi_UnicodeEncodeError(PyContext ctx); +PyClassRef PyApi_IndexError(); -PyClassRef PyApi_UnicodeDecodeError(PyContext ctx); +PyClassRef PyApi_KeyError(); -PyClassRef PyApi_UnicodeTranslateError(PyContext ctx); +PyClassRef PyApi_ValueError(); -PyClassRef PyApi_AssertionError(PyContext ctx); +PyClassRef PyApi_UnicodeError(); -PyClassRef PyApi_ArithmeticError(PyContext ctx); +PyClassRef PyApi_UnicodeEncodeError(); -PyClassRef PyApi_FloatingPointError(PyContext ctx); +PyClassRef PyApi_UnicodeDecodeError(); -PyClassRef PyApi_OverflowError(PyContext ctx); +PyClassRef PyApi_UnicodeTranslateError(); -PyClassRef PyApi_ZeroDivisionError(PyContext ctx); +PyClassRef PyApi_AssertionError(); -PyClassRef PyApi_SystemError(PyContext ctx); +PyClassRef PyApi_ArithmeticError(); -PyClassRef PyApi_ReferenceError(PyContext ctx); +PyClassRef PyApi_FloatingPointError(); -PyClassRef PyApi_MemoryError(PyContext ctx); +PyClassRef PyApi_OverflowError(); -PyClassRef PyApi_BufferError(PyContext ctx); +PyClassRef PyApi_ZeroDivisionError(); -PyClassRef PyApi_Warning(PyContext ctx); +PyClassRef PyApi_SystemError(); -PyClassRef PyApi_UserWarning(PyContext ctx); +PyClassRef PyApi_ReferenceError(); -PyClassRef PyApi_DeprecationWarning(PyContext ctx); +PyClassRef PyApi_MemoryError(); -PyClassRef PyApi_PendingDeprecationWarning(PyContext ctx); +PyClassRef PyApi_BufferError(); -PyClassRef PyApi_SyntaxWarning(PyContext ctx); +PyClassRef PyApi_Warning(); -PyClassRef PyApi_RuntimeWarning(PyContext ctx); +PyClassRef PyApi_UserWarning(); -PyClassRef PyApi_FutureWarning(PyContext ctx); +PyClassRef PyApi_EncodingWarning(); -PyClassRef PyApi_ImportWarning(PyContext ctx); +PyClassRef PyApi_DeprecationWarning(); -PyClassRef PyApi_UnicodeWarning(PyContext ctx); +PyClassRef PyApi_PendingDeprecationWarning(); -PyClassRef PyApi_BytesWarning(PyContext ctx); +PyClassRef PyApi_SyntaxWarning(); -PyClassRef PyApi_ResourceWarning(PyContext ctx); +PyClassRef PyApi_RuntimeWarning(); -PyClassRef PyApi_ConnectionError(PyContext ctx); +PyClassRef PyApi_FutureWarning(); -PyClassRef PyApi_BlockingIOError(PyContext ctx); +PyClassRef PyApi_ImportWarning(); -PyClassRef PyApi_BrokenPipeError(PyContext ctx); +PyClassRef PyApi_UnicodeWarning(); -PyClassRef PyApi_ChildProcessError(PyContext ctx); +PyClassRef PyApi_BytesWarning(); -PyClassRef PyApi_ConnectionAbortedError(PyContext ctx); +PyClassRef PyApi_ResourceWarning(); -PyClassRef PyApi_ConnectionRefusedError(PyContext ctx); +PyClassRef PyApi_ConnectionError(); -PyClassRef PyApi_ConnectionResetError(PyContext ctx); +PyClassRef PyApi_BlockingIOError(); -PyClassRef PyApi_FileExistsError(PyContext ctx); +PyClassRef PyApi_BrokenPipeError(); -PyClassRef PyApi_FileNotFoundError(PyContext ctx); +PyClassRef PyApi_ChildProcessError(); -PyClassRef PyApi_IsADirectoryError(PyContext ctx); +PyClassRef PyApi_ConnectionAbortedError(); -PyClassRef PyApi_NotADirectoryError(PyContext ctx); +PyClassRef PyApi_ConnectionRefusedError(); -PyClassRef PyApi_InterruptedError(PyContext ctx); +PyClassRef PyApi_ConnectionResetError(); -PyClassRef PyApi_PermissionError(PyContext ctx); +PyClassRef PyApi_FileExistsError(); -PyClassRef PyApi_ProcessLookupError(PyContext ctx); +PyClassRef PyApi_FileNotFoundError(); -PyClassRef PyApi_TimeoutError(PyContext ctx); +PyClassRef PyApi_IsADirectoryError(); + +PyClassRef PyApi_NotADirectoryError(); + +PyClassRef PyApi_InterruptedError(); + +PyClassRef PyApi_PermissionError(); + +PyClassRef PyApi_ProcessLookupError(); + +PyClassRef PyApi_TimeoutError(); /* Bytes */ -uint8_t PyApi_Bytes_GetItem(PyContext ctx, PyBytesRef self, uintptr_t index, PyExceptionRef *exc); +uint8_t PyApi_Bytes_GetItem(PyContext ctx, PyBytesRef self, uintptr_t index); -PyBytesRef PyApi_Bytes_FromArray(PyContext ctx, const char * data, uintptr_t length, PyExceptionRef *exc); +PyBytesRef PyApi_Bytes_FromArray(PyContext ctx, const char * data, uintptr_t length); -uintptr_t PyApi_Bytes_GetSize(PyContext ctx, PyBytesRef self, PyExceptionRef *exc); +uintptr_t PyApi_Bytes_GetSize(PyContext ctx, PyBytesRef self); bool PyApi_IsABytes(PyRef ref); PyBytesRef PyApi_Bytes_UnsafeCast(PyRef ref); @@ -236,13 +269,13 @@ PyRef PyApi_Bytes_UpCast(PyBytesRef ref); /* StrBuilder */ -PyStrBuilderRef PyApi_StrBuilder_New(PyContext ctx, uintptr_t capacity, PyExceptionRef *exc); +PyStrBuilderRef PyApi_StrBuilder_New(PyContext ctx, uintptr_t capacity); -PyExceptionRef PyApi_StrBuilder_AppendStr(PyContext ctx, PyRef Self, PyStrRef s); +int PyApi_StrBuilder_AppendStr(PyContext ctx, PyRef Self, PyStrRef s); -PyExceptionRef PyApi_StrBuilder_AppendUtf8String(PyContext ctx, PyRef Self, const UtfString s); +int PyApi_StrBuilder_AppendUtf8String(PyContext ctx, PyRef Self, const UtfString s); -PyStrRef PyApi_StrBuilder_ToStr(PyContext ctx, PyRef Self, PyExceptionRef *exc); +PyStrRef PyApi_StrBuilder_ToStr(PyContext ctx, PyRef Self); bool PyApi_IsAStrBuilder(PyRef ref); PyStrBuilderRef PyApi_StrBuilder_UnsafeCast(PyRef ref); @@ -251,11 +284,11 @@ PyRef PyApi_StrBuilder_UpCast(PyStrBuilderRef ref); /* TupleBuilder */ -PyTupleBuilderRef PyApi_TupleBuilder_New(PyContext ctx, uintptr_t capacity, PyExceptionRef *exc); +PyTupleBuilderRef PyApi_TupleBuilder_New(PyContext ctx, uintptr_t capacity); -PyExceptionRef PyApi_TupleBuilder_Add(PyContext ctx, PyRef Self, PyStrRef s); +int PyApi_TupleBuilder_Add(PyContext ctx, PyRef Self, PyStrRef s); -PyStrRef PyApi_TupleBuilder_ToTuple(PyContext ctx, PyRef Self, PyExceptionRef *exc); +PyStrRef PyApi_TupleBuilder_ToTuple(PyContext ctx, PyRef Self); bool PyApi_IsATupleBuilder(PyRef ref); PyTupleBuilderRef PyApi_TupleBuilder_UnsafeCast(PyRef ref); @@ -264,57 +297,57 @@ PyRef PyApi_TupleBuilder_UpCast(PyTupleBuilderRef ref); /* Object */ -PyRef PyApi_Object_GetItem(PyContext ctx, PyRef obj, PyRef key, PyExceptionRef *exc); +PyRef PyApi_Object_GetItem(PyContext ctx, PyRef obj, PyRef key); -PyRef PyApi_Object_GetItem_i(PyContext ctx, PyRef obj, intptr_t key, PyExceptionRef *exc); +PyRef PyApi_Object_GetItem_i(PyContext ctx, PyRef obj, intptr_t key); -PyRef PyApi_Object_GetItem_s(PyContext ctx, PyRef obj, const UtfString key, PyExceptionRef *exc); +PyRef PyApi_Object_GetItem_s(PyContext ctx, PyRef obj, const UtfString key); -PyExceptionRef PyApi_Object_SetItem(PyContext ctx, PyRef obj, PyRef key, PyRef value); +int PyApi_Object_SetItem(PyContext ctx, PyRef obj, PyRef key, PyRef value); -PyExceptionRef PyApi_Object_SetItem_i(PyContext ctx, PyRef obj, intptr_t key, PyRef value); +int PyApi_Object_SetItem_i(PyContext ctx, PyRef obj, intptr_t key, PyRef value); -PyExceptionRef PyApi_Object_SetItem_s(PyContext ctx, PyRef obj, const UtfString key, PyRef value); +int PyApi_Object_SetItem_s(PyContext ctx, PyRef obj, const UtfString key, PyRef value); -PyRef PyApi_Object_GetAttr(PyContext ctx, PyRef obj, PyRef attr, PyExceptionRef *exc); +PyRef PyApi_Object_GetAttr(PyContext ctx, PyRef obj, PyRef attr); -PyRef PyApi_Object_GetAttr_s(PyContext ctx, PyRef obj, const UtfString attr, PyExceptionRef *exc); +PyRef PyApi_Object_GetAttr_s(PyContext ctx, PyRef obj, const UtfString attr); -int PyApi_Object_HasAttr(PyContext ctx, PyRef obj, PyRef attr, PyExceptionRef *exc); +int PyApi_Object_HasAttr(PyContext ctx, PyRef obj, PyRef attr); -int PyApi_Object_HasAttr_s(PyContext ctx, PyRef obj, const UtfString attr, PyExceptionRef *exc); +int PyApi_Object_HasAttr_s(PyContext ctx, PyRef obj, const UtfString attr); -PyExceptionRef PyApi_Object_SetAttr(PyContext ctx, PyRef obj, PyRef attr, PyRef value); +int PyApi_Object_SetAttr(PyContext ctx, PyRef obj, PyRef attr, PyRef value); -PyExceptionRef PyApi_Object_SetAttr_s(PyContext ctx, PyRef obj, const UtfString attr, PyRef value); +int PyApi_Object_SetAttr_s(PyContext ctx, PyRef obj, const UtfString attr, PyRef value); -int PyApi_Object_Contains(PyContext ctx, PyRef container, PyRef key, PyExceptionRef *exc); +int PyApi_Object_Contains(PyContext ctx, PyRef container, PyRef key); PyClassRef PyApi_Object_Type(PyContext ctx, PyRef obj); bool PyApi_Object_TypeCheck(PyContext ctx, PyRef obj, PyClassRef cls); -PyStrRef PyApi_Object_Repr(PyContext ctx, PyRef obj, PyExceptionRef *exc); +PyStrRef PyApi_Object_Repr(PyContext ctx, PyRef obj); -PyStrRef PyApi_Object_Str(PyContext ctx, PyRef obj, PyExceptionRef *exc); +PyStrRef PyApi_Object_Str(PyContext ctx, PyRef obj); -intptr_t PyApi_Object_Hash(PyContext ctx, PyRef obj, PyExceptionRef *exc); +intptr_t PyApi_Object_Hash(PyContext ctx, PyRef obj); -PyRef PyApi_Object_CallMethod(PyContext ctx, PyRef attr, PyRef args[], intptr_t nargsf, PyExceptionRef *exc); +PyRef PyApi_Object_CallMethod(PyContext ctx, PyRef attr, PyRef args[], intptr_t nargsf); -int PyApi_Object_Compare(PyContext ctx, uint8_t op, PyRef left, PyRef right, PyExceptionRef *exc); +int PyApi_Object_Compare(PyContext ctx, uint8_t op, PyRef left, PyRef right); -int PyApi_Object_IsIter(PyContext ctx, PyRef obj, PyExceptionRef *exc); +int PyApi_Object_IsIter(PyContext ctx, PyRef obj); -int PyApi_Object_IsAIter(PyContext ctx, PyRef obj, PyExceptionRef *exc); +int PyApi_Object_IsAnIter(PyContext ctx, PyRef obj); /* Dict */ -PyRef PyApi_Dict_Get(PyContext ctx, PyDictRef self, PyRef key, PyRef default, PyExceptionRef *exc); +PyRef PyApi_Dict_Get(PyContext ctx, PyDictRef self, PyRef key, PyRef default); -PyRef PyApi_Dict_GetItem(PyContext ctx, PyDictRef self, PyRef key, PyExceptionRef *exc); +PyRef PyApi_Dict_GetItem(PyContext ctx, PyDictRef self, PyRef key); -PyDictRef PyApi_Dict_New(PyContext ctx, PyExceptionRef *exc); +PyDictRef PyApi_Dict_New(PyContext ctx); bool PyApi_IsADict(PyRef ref); PyDictRef PyApi_Dict_UnsafeCast(PyRef ref); @@ -323,17 +356,17 @@ PyRef PyApi_Dict_UpCast(PyDictRef ref); /* Int */ -PyIntRef PyApi_Int_FromInt32(PyContext ctx, int32_t val, PyExceptionRef *exc); +PyIntRef PyApi_Int_FromInt32(PyContext ctx, int32_t val); -PyIntRef PyApi_Int_FromUInt32(PyContext ctx, int32_t val, PyExceptionRef *exc); +PyIntRef PyApi_Int_FromUInt32(PyContext ctx, int32_t val); -PyIntRef PyApi_Int_FromInt64(PyContext ctx, int32_t val, PyExceptionRef *exc); +PyIntRef PyApi_Int_FromInt64(PyContext ctx, int32_t val); -PyIntRef PyApi_Int_FromUInt64(PyContext ctx, int32_t val, PyExceptionRef *exc); +PyIntRef PyApi_Int_FromUInt64(PyContext ctx, int32_t val); -int32_t PyApi_Int_ToInt32(PyContext ctx, PyIntRef self, PyExceptionRef *exc); +int32_t PyApi_Int_ToInt32(PyContext ctx, PyIntRef self); -int64_t PyApi_Int_ToInt64(PyContext ctx, PyIntRef self, PyExceptionRef *exc); +int64_t PyApi_Int_ToInt64(PyContext ctx, PyIntRef self); bool PyApi_IsAnInt(PyRef ref); PyIntRef PyApi_Int_UnsafeCast(PyRef ref); @@ -342,15 +375,15 @@ PyRef PyApi_Int_UpCast(PyIntRef ref); /* List */ -PyListRef PyApi_List_New(PyContext ctx, PyExceptionRef *exc); +PyListRef PyApi_List_New(PyContext ctx); -PyExceptionRef PyApi_List_Append(PyContext ctx, PyListRef self, PyRef item); +int PyApi_List_Append(PyContext ctx, PyListRef self, PyRef item); -PyRef PyApi_List_GetItem(PyContext ctx, PyListRef self, uintptr_t index, PyExceptionRef *exc); +PyRef PyApi_List_GetItem(PyContext ctx, PyListRef self, uintptr_t index); uintptr_t PyApi_List_GetSize(PyContext ctx, PyListRef self); -PyRef PyApi_List_Pop(PyContext ctx, PyListRef self, PyExceptionRef *exc); +PyRef PyApi_List_Pop(PyContext ctx, PyListRef self); bool PyApi_IsAList(PyRef ref); PyListRef PyApi_List_UnsafeCast(PyRef ref); @@ -359,37 +392,68 @@ PyRef PyApi_List_UpCast(PyListRef ref); /* Operators */ -PyRef PyApi_Operators_UnaryOp(PyContext ctx, uint8_t op, PyRef argument, PyExceptionRef *exc); +PyRef PyApi_Operators_UnaryOp(PyContext ctx, uint8_t op, PyRef argument); + +PyRef PyApi_Operators_BinaryOp(PyContext ctx, uint8_t op, PyRef left, PyRef right); + +PyRef PyApi_Operators_Compare(PyContext ctx, PyRef left, PyRef right, uint8_t op); + +int PyApi_Operators_CompareBool(PyContext ctx, PyRef left, PyRef right, uint8_t op); + + +/* Exception */ +PyExceptionRef PyApi_Exception_FromString(PyContext ctx, PyClassRef cls, const UtfString message); -PyRef PyApi_Operators_BinaryOp(PyContext ctx, uint8_t op, PyRef left, PyRef right, PyExceptionRef *exc); +PyExceptionRef PyApi_Exception_FromValue(PyContext ctx, PyClassRef cls, PyRef message); -PyRef PyApi_Operators_Compare(PyContext ctx, PyRef left, PyRef right, uint8_t op, PyExceptionRef *exc); +void PyApi_Exception_Fatal(PyContext ctx, const UtfString message); + +PyExceptionRef PyApi_Exception_FromErrnoWithFilename(PyContext ctx, PyClassRef cls, const UtfString filename); + +/* Always fails. Barring another error, + will set *error to the newly created exception. + */ +PyExceptionRef PyApi_Exception_RaiseFromString(PyContext ctx, PyClassRef cls, const UtfString message); + +/* Always fails. Barring another error, + will set *error to the newly created exception. + */ +PyExceptionRef PyApi_Exception_RaiseFromValue(PyContext ctx, PyClassRef cls, PyRef message); + +bool PyApi_IsAnException(PyRef ref); +PyExceptionRef PyApi_Exception_UnsafeCast(PyRef ref); +PyExceptionRef PyApi_Exception_DownCast(PyRef ref); +PyRef PyApi_Exception_UpCast(PyExceptionRef ref); -int PyApi_Operators_CompareBool(PyContext ctx, PyRef left, PyRef right, uint8_t op, PyExceptionRef *exc); +/* If the immediately previous API call failed, + then return the exception set by that exception. + Otherwise, return either an arbitrary exception + or PyRef_NO_EXCEPTION */ +PyExceptionRef PyApi_GetLatestException(PyContext ctx); /* Call */ -int PyApi_Call_IsCallable(PyContext ctx, PyRef obj, PyExceptionRef *exc); +int PyApi_Call_IsCallable(PyContext ctx, PyRef obj); -PyRef PyApi_Call_TupleDict(PyContext ctx, PyRef callable, PyTupleRef args, PyDictRef kwargs, PyExceptionRef *exc); +PyRef PyApi_Call_TupleDict(PyContext ctx, PyRef callable, PyTupleRef args, PyDictRef kwargs); -PyRef PyApi_Call_Vector(PyContext ctx, PyRef callable, PyRef args[], intptr_t nargsf, PyTupleRef kwnames, PyExceptionRef *exc); +PyRef PyApi_Call_Vector(PyContext ctx, PyRef callable, PyRef args[], intptr_t nargsf, PyTupleRef kwnames); /* Iter */ -PyRef PyApi_Iter_Next(PyContext ctx, PyRef obj, PyExceptionRef *exc); +PyRef PyApi_Iter_Next(PyContext ctx, PyRef obj); /* If no more results are available then, instead of raising StopIteration, *error = PyRef_NO_EXCEPTION. This is more efficient than the plain Next function. */ -PyRef PyApi_Iter_NextX(PyContext ctx, PyRef obj, PyExceptionRef *exc); +PyRef PyApi_Iter_NextX(PyContext ctx, PyRef obj); -PyRef PyApi_Iter_Send(PyContext ctx, PyRef obj, PyExceptionRef *exc); +PyRef PyApi_Iter_Send(PyContext ctx, PyRef obj); /* If no exception is raised, then returned is set to 0 for a yield, and 1 for a return. This is more efficient than the plain Send function. */ -PyRef PyApi_Iter_SendX(PyContext ctx, PyRef obj, int * returned, PyExceptionRef *exc); +PyRef PyApi_Iter_SendX(PyContext ctx, PyRef obj, int * returned); /* Code */ @@ -400,11 +464,11 @@ PyRef PyApi_Code_UpCast(PyCodeRef ref); /* FrameStack */ -PyRef PyApi_FrameStack_GetLocal(PyContext ctx, uintptr_t depth, uintptr_t index, PyExceptionRef *exc); +PyRef PyApi_FrameStack_GetLocal(PyContext ctx, uintptr_t depth, uintptr_t index); -PyRef PyApi_FrameStack_GetLocalByName(PyContext ctx, uintptr_t depth, PyStrRef name, PyExceptionRef *exc); +PyRef PyApi_FrameStack_GetLocalByName(PyContext ctx, uintptr_t depth, PyStrRef name); -PyRef PyApi_FrameStack_GetLocalByCName(PyContext ctx, uintptr_t depth, const UtfString name, PyExceptionRef *exc); +PyRef PyApi_FrameStack_GetLocalByCName(PyContext ctx, uintptr_t depth, const UtfString name); -PyCodeRef PyApi_FrameStack_GetCode(PyContext ctx, uintptr_t depth, PyExceptionRef *exc); +PyCodeRef PyApi_FrameStack_GetCode(PyContext ctx, uintptr_t depth); diff --git a/api/generate/__main__.py b/api/generate/__main__.py index 9d82a43..00fc8c8 100644 --- a/api/generate/__main__.py +++ b/api/generate/__main__.py @@ -1,7 +1,7 @@ from __future__ import annotations from types import FunctionType -from .abi import is_namespace, Int, Self, Void, Size, uint8_t, no_result, cannot_fail, is_function_pointer +from .abi import is_namespace, Int, Self, Void, Size, uint8_t, no_result, cannot_fail, is_function_pointer, is_shared from . import api @@ -52,9 +52,8 @@ def generate_function_pointer(func): assert not cannot_fail(func) result_type = get_result_type(func) assert result_type is object - return_or_exception = [ "PyRef *result" ] - all_args = [ CONTEXT ] + flatten_args(arg_decls) + return_or_exception - print(f"typedef int (*{func_name})({', '.join(all_args)});\n") + all_args = [ CONTEXT ] + flatten_args(arg_decls) + print(f"typedef PyRef (*{func_name})({', '.join(all_args)});\n") def generate_api_func(namespace, func): @@ -66,17 +65,19 @@ def generate_api_func(namespace, func): annotations = func.__annotations__ arg_decls = [ get_arg_decl(name, namespace, annotations) for name in arg_names] return_type = get_result_type(func) + return_arg = None + if not cannot_fail(func): + if return_type is Void or return_type is bool: + return_type = int + if isinstance(cls, Int): + return_arg = return_type + return_type = int return_type = ctype_for_class(return_type, namespace) - if cannot_fail(func): - exception = [] - elif return_type == "void": - return_type = "PyExceptionRef" - exception = [] - else: - if return_type == "bool": - return_type = "int" - exception = [ "PyExceptionRef *exc" ] - all_args = [ CONTEXT ] + flatten_args(arg_decls) + exception + all_args = flatten_args(arg_decls) + if not is_shared(func): + all_args = [ CONTEXT ] + all_args + if return_arg is not None: + all_args.append(ctype_for_class(return_arg, namespace)+"*") if func.__doc__: print(f"/* {func.__doc__} */") print(f"{return_type} {func_name}({', '.join(all_args)});\n") @@ -88,6 +89,9 @@ def generate_api_func(namespace, func): print("PyRef PyRef_INVALID;") print("PyExceptionRef PyRef_NO_EXCEPTION;\n") +print("PyRef PyRef_Dup(PyRef ref);") +print("PyRef PyRef_Close(PyRef ref);") + for name, obj in api.__dict__.items(): if isinstance(obj, FunctionType): if is_function_pointer(obj): diff --git a/api/generate/abi.py b/api/generate/abi.py index 11d01db..eca0ff6 100644 --- a/api/generate/abi.py +++ b/api/generate/abi.py @@ -51,8 +51,12 @@ def no_result(func): def shared(func): "Marker for things that returned shared references." + func._shared = True return func +def is_shared(func): + return hasattr(func, "_shared") + exports = ( intptr_t, uintptr_t, uint8_t, int32_t, uint32_t, int64_t, uint64_t, abi, namespace, Self, Void, no_fail, function_pointer, shared diff --git a/api/generate/api.py b/api/generate/api.py index 0dd75ef..1bd9743 100644 --- a/api/generate/api.py +++ b/api/generate/api.py @@ -67,12 +67,12 @@ def false(): ... for name, cls in builtins.__dict__.items(): # Maybe we don't want every last exception class, # but that's the simplest approach for now. - if isinstance(cls, type) and issubclass(cls, builtin_base_exception): + if isinstance(cls, type) and name[0] != "_": @no_fail @shared def f() -> Class: ... f.__name__ = name - globals()[name] = f + globals()["class_" + name] = f del name, cls, builtin_base_exception class Str: @@ -165,7 +165,7 @@ def Compare(op: uint8_t, left, right) -> bool: ... def IsIter(obj) -> bool: ... - def IsAIter(obj) -> bool: ... + def IsAnIter(obj) -> bool: ... class Dict: @@ -229,15 +229,22 @@ def Fatal(message: utf_string) -> Void: ... def FromErrnoWithFilename(cls: Class, filename: utf_string) -> Self: ... def RaiseFromString(cls: Class, message: utf_string) -> Self: - """Always fails. Returns, barring another error, - { PyRef PyRef_INVALID, new_exception } + """Always fails. Barring another error, + will set *error to the newly created exception. """ def RaiseFromValue(cls: Class, message) -> Self: - """Always fails. Returns, barring another error, - { PyRef PyRef_INVALID, new_exception } + """Always fails. Barring another error, + will set *error to the newly created exception. """ +@no_fail +def GetLatestException() -> Exception: + """If the immediately previous API call failed, + then return the exception set by that exception. + Otherwise, return either an arbitrary exception + or PyRef_NO_EXCEPTION""" + @namespace class Call: