8000 gh-102471, PEP 757: Add PyLong import and export API by vstinner · Pull Request #121339 · python/cpython · GitHub
[go: up one dir, main page]

Skip to content

gh-102471, PEP 757: Add PyLong import and export API #121339

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 54 commits into from
Dec 13, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
f4fdbf2
gh-102471: Add PyLong import and export API
vstinner Jul 2, 2024
c2e568e
Add layout
vstinner Jul 23, 2024
f0d9525
Merge branch 'main' into long_export
vstinner Sep 3, 2024
b19764f
Rename word_endian to digits_order
vstinner Sep 3, 2024
6f7fd11
Replace Py_digit* type with void*
vstinner Sep 3, 2024
080e079
Add PyLongWriter_Discard() function
vstinner Sep 4, 2024
1a7902f
Fixes
vstinner Sep 5, 2024
b70a6dd
Use unsigned type for ndigits
vstinner Sep 5, 2024
07552a7
Remove again layout
vstinner Sep 5, 2024
0d0f942
Revert "Use unsigned type for ndigits"
vstinner Sep 6, 2024
762c33a
doc: adjust ndigits documentation
vstinner Sep 6, 2024
20be7a3
Update doc
vstinner Sep 13, 2024
d92bf1e
Make PyLong_DigitArray.obj private
vstinner Sep 16, 2024
b3b02a2
Remove reserved documentation
vstinner Sep 16, 2024
caca2d7
PyLong_FreeDigitArray() only clears _reserved
vstinner Sep 16, 2024
4221a49
Make PyLong_LAYOUT static
vstinner Sep 16, 2024
37b1d49
Add PyLong_AsDigitArray.value
vstinner Sep 16, 2024
d70a121
Inline PyLong_AsInt64() to avoid the exception
vstinner Sep 17, 2024
4aa25f6
Remove Py_digit type; update the doc
vstinner Sep 17, 2024
90973d4
Merge branch 'main' into long_export
vstinner Sep 17, 2024
5d3e224
Add long_asnativebytes() function
vstinner Sep 17, 2024
c7d7cb2
Remove reference to removed Py_digit type
vstinner Sep 17, 2024
a3d601a
Address Antoine's review
vstinner Sep 17, 2024
c049268
Merge branch 'main' into long_export
vstinner Sep 17, 2024
06b196b
Merge branch 'main' into long_export
skirpichev Sep 18, 2024
3e8d296
Apply suggestions from code review
skirpichev Sep 18, 2024
86c68c2
Merge branch 'main' into long_export
skirpichev Sep 18, 2024
a8fd669
Revert "Add long_asnativebytes() function"
vstinner Sep 18, 2024
a04f9d0
Use PyLong_AsLongAndOverflow()
vstinner Sep 18, 2024
b2be94a
Try PyLong_AsLongLongAndOverflow() first
vstinner Sep 18, 2024
ca98ad1
Merge branch 'main' into long_export
vstinner Sep 18, 2024
167d75e
Update Doc/c-api/long.rst
vstinner Sep 19, 2024
5e53a5b
Sync implementation with PEP (#8)
skirpichev Oct 16, 2024
c24789f
Merge branch 'main' into long_export
skirpichev Nov 13, 2024
0422f9d
fix NL in Doc/c-api/long.rst (sorry, damn web editor)
skirpichev Nov 13, 2024
a529a48
rename news
skirpichev Nov 13, 2024
3db44f3
Address Erlend's review
vstinner Nov 13, 2024
1d2863e
Address Sergey's review
vstinner Nov 13, 2024
d663511
Merge branch 'main' into long_export
vstinner Nov 13, 2024
816798d
Merge branch 'main' into long_export
vstinner Nov 28, 2024
033bd65
Update documentation from PEP 757
vstinner Nov 28, 2024
36b87d4
Update Modules/_testcapi/long.c
vstinner Nov 28, 2024
94d852e
Sync implementation with PEP (#9)
skirpichev Dec 9, 2024
a72ff83
Merge branch 'main' into long_export
vstinner Dec 9, 2024
53d584b
Cleanup
vstinner Dec 9, 2024
577598a
Update Doc/c-api/long.rst
vstinner Dec 9, 2024
b08cd55
Address Steve's review
vstinner Dec 9, 2024
eaebef3
Address Bénédikt's review
vstinner Dec 10, 2024
03248c7
Apply suggestions from code review
vstinner Dec 10, 2024
0a26f97
Address Steve's review
vstinner Dec 11, 2024
88a62fe
Add PyLong_Export to Doc/data/refcounts.dat
vstinner Dec 12, 2024
45517ab
Address Serhiy's review
vstinner Dec 12, 2024
92007d1
Address Petr's review
vstinner Dec 12, 2024
6d3cb80
Add PyLongWriter to Doc/data/refcounts.dat
vstinner Dec 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Remove Py_digit type; update the doc
  • Loading branch information
vstinner committed Sep 17, 2024
commit 4aa25f64eb01e00caa37bf4e9e445e47f1770993
67 changes: 23 additions & 44 deletions Doc/c-api/long.rst
Original file line number Diff line number Diff line change
Expand Up @@ -615,21 +615,6 @@ Export API

.. versionadded:: 3.14

.. c:type:: Py_digit

A single unsigned digit in the range [``0``; ``PyLong_BASE - 1``].

It is usually used in an *array of digits*, such as the
:c:member:`PyLong_DigitArray.digits` array.

Its size depend on the :c:macro:`!PYLONG_BITS_IN_DIGIT` macro:
see the ``configure`` :option:`--enable-big-digits` option.

See :c:member:`PyLongLayout.bits_per_digit` for the number of bits per
digit and :c:member:`PyLongLayout.digit_size` for the size of a digit (in
bytes).


.. c:struct:: PyLongLayout

Layout of an array of digits, used by Python :class:`int` object.
Expand Down Expand Up @@ -669,11 +654,23 @@ Export API
See the :c:struct:`PyLongLayout` structure.


.. c:struct:: PyLong_DigitArray
.. c:struct:: PyLongExport

Export of a Python :class:`int` object.

A Python :class:`int` object exported as an array of digits.
There are two cases:

.. c:member:: int negative
* If :c:member:`digits` is ``NULL``, only use the :c:member:`value` member.
Calling :c:func:`PyLong_FreeExport` is optional in this case.
* If :c:member:`digits` is not ``NULL``, use :c:member:`negative`,
:c:member:`ndigits` and :c:member:`digits` members.
Calling :c:func:`PyLong_FreeExport` is mandatory in this case.

.. c:member:: int64_t value

Integer value if :c:member:`digits` is ``NULL``.

.. c:member:: uint8_t negative

1 if the number is negative, 0 otherwise.

Expand All @@ -683,26 +680,26 @@ Export API

.. c:member:: const void *digits

Read-only array of unsigned digits.
Read-only array of unsigned digits. Can be ``NULL``.


.. c:function:: int PyLong_Export(PyObject *obj, PyLong_DigitArray *array)
.. c:function:: int PyLong_Export(PyObject *obj, PyLongExport *export_long)

Export a Python :class:`int` object as an array of digits.
Export a Python :class:`int` object.

On success, set *\*array* and return 0.
On success, set *\*export_long* and return 0.
On error, set an exception and return -1.

This function always succeeds if *obj* is a Python :class:`int` object or a
subclass.

:c:func:`PyLong_FreeExport` must be called once done with using
*array*.
If *export_long.digits* is not ``NULL, :c:func:`PyLong_FreeExport` must be
called when the export is no longer needed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
.. impl-detail::
This function always succeeds if *obj* is a Python :class:`int` object
or a subclass.

Lets see if we can restore this in a that way. It might be helpful for e.g. Sage, which doesn't support PyPy.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer to not add this note. It was controversial during PEP 757 design.

@serhiy-storchaka @encukou: What do you think? Would you be ok to declare that the PyLong_Export() function cannot fail if the argument is a Python int?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was controversial during PEP 757 design.

It was proposed unconditionally, not as CPython's implementation detail.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine with it, as an implementation detail.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


.. c:function:: void PyLong_FreeExport(PyLong_DigitArray *array)
.. c:function:: void PyLong_FreeExport(PyLongExport *export_long)

Release the export *array* created by :c:func:`PyLong_Export`.
Release the export *export_long* created by :c:func:`PyLong_Export`.


PyLongWriter API
Expand Down Expand Up @@ -748,21 +745,3 @@ The :c:type:`PyLongWriter` API can be used to import an integer.
.. c:function:: void PyLongWriter_Discard(PyLongWriter *writer)

Discard the internal object and destroy the writer instance.


Example creating an integer from an array of digits::

PyObject *
long_import(int negative, Py_ssize_t ndigits, Py_digit *digits)
{
void *writer_digits;
PyLongWriter *writer = PyLongWriter_Create(negative, ndigits,
&writer_digits);
if (writer == NULL) {
return NULL;
}

assert(layout.digit_size == sizeof(Py_digit));
memcpy(writer_digits, digits, ndigits * sizeof(Py_digit));
return PyLongWriter_Finish(writer);
}
4 changes: 2 additions & 2 deletions Doc/whatsnew/3.14.rst
Original file line number Diff line number Diff line change
Expand Up @@ -531,8 +531,8 @@ New Features
* Add a new import and export API for Python :class:`int` objects:

* :c:func:`PyLong_GetNativeLayout`;
* :c:func:`PyLong_AsDigitArray`;
* :c:func:`PyLong_FreeDigitArray`;
* :c:func:`PyLong_Export`;
* :c:func:`PyLong_FreeExport`;
* :c:func:`PyLongWriter_Create`;
* :c:func:`PyLongWriter_Finish`;
* :c:func:`PyLongWriter_Discard`.
Expand Down
6 changes: 2 additions & 4 deletions Include/cpython/longintrepr.h
Original file line number Diff line number Diff line change
Expand Up @@ -58,10 +58,8 @@ typedef long stwodigits; /* signed variant of twodigits */
#else
#error "PYLONG_BITS_IN_DIGIT should be 15 or 30"
#endif
#define PyLong_BASE ((Py_digit)1 << PyLong_SHIFT)
#define PyLong_MASK ((Py_digit)(PyLong_BASE - 1))

typedef digit Py_digit;
#define PyLong_BASE ((digit)1 << PyLong_SHIFT)
#define PyLong_MASK ((digit)(PyLong_BASE - 1))

/* Long integer representation.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
Add a new import and export API for Python :class:`int` objects:

* :c:func:`PyLong_GetNativeLayout`;
* :c:func:`PyLong_AsDigitArray`;
* :c:func:`PyLong_FreeDigitArray`;
* :c:func:`PyLong_Export`;
* :c:func:`PyLong_FreeExport`;
* :c:func:`PyLongWriter_Create`;
* :c:func:`PyLongWriter_Finish`;
* :c:func:`PyLongWriter_Discard`.
Expand Down
30 changes: 15 additions & 15 deletions Modules/_testcapi/long.c
Original file line number Diff line number Diff line change
Expand Up @@ -186,21 +186,21 @@ pylong_export(PyObject *module, PyObject *obj)
// PyLong_FreeExport() is not needed in this case
}

assert(PyLong_GetNativeLayout()->digit_size == sizeof(Py_digit));
const Py_digit *export_long_digits = export_long.digits;
assert(PyLong_GetNativeLayout()->digit_size == sizeof(digit));
const digit *export_long_digits = export_long.digits;

PyObject *digits = PyList_New(0);
for (Py_ssize_t i=0; i < export_long.ndigits; i++) {
PyObject *digit = PyLong_FromUnsignedLong(export_long_digits[i]);
if (digit == NULL) {
PyObject *item = PyLong_FromUnsignedLong(export_long_digits[i]);
if (item == NULL) {
goto error;
}

if (PyList_Append(digits, digit) < 0) {
Py_DECREF(digit);
if (PyList_Append(digits, item) < 0) {
Py_DECREF(item);
goto error;
}
Py_DECREF(digit);
Py_DECREF(item);
}

PyObject *res = Py_BuildValue("(iN)", export_long.negative, digits);
Expand Down Expand Up @@ -230,7 +230,7 @@ pylongwriter_create(PyObject *module, PyObject *args)
}
Py_ssize_t ndigits = PyList_GET_SIZE(list);

Py_digit *digits = PyMem_Malloc(ndigits * sizeof(Py_digit));
digit *digits = PyMem_Malloc(ndigits * sizeof(digit));
if (digits == NULL) {
PyErr_NoMemory();
return NULL;
Expand All @@ -239,16 +239,16 @@ pylongwriter_create(PyObject *module, PyObject *args)
for (Py_ssize_t i=0; i < ndigits; i++) {
PyObject *item = PyList_GET_ITEM(list, i);

long digit = PyLong_AsLong(item);
if (digit == -1 && PyErr_Occurred()) {
long num = PyLong_AsLong(item);
if (num == -1 && PyErr_Occurred()) {
goto error;
}

if (digit < 0 || digit >= PyLong_BASE) {
PyErr_SetString(PyExc_ValueError, "digit doesn't fit into Py_digit");
if (num < 0 || num >= PyLong_BASE) {
PyErr_SetString(PyExc_ValueError, "digit doesn't fit into digit");
goto error;
}
digits[i] = (Py_digit)digit;
digits[i] = (digit)num;
}

void *writer_digits;
Expand All @@ -257,8 +257,8 @@ pylongwriter_create(PyObject *module, PyObject *args)
if (writer == NULL) {
goto error;
}
assert(PyLong_GetNativeLayout()->digit_size == sizeof(Py_digit));
memcpy(writer_digits, digits, ndigits * sizeof(Py_digit));
assert(PyLong_GetNativeLayout()->digit_size == sizeof(digit));
memcpy(writer_digits, digits, ndigits * sizeof(digit));
PyObject *res = PyLongWriter_Finish(writer);
PyMem_Free(digits);

Expand Down
0