8000 gh-119127: functools.partial placeholders by dg-pb · Pull Request #119827 · python/cpython · GitHub
[go: up one dir, main page]

Skip to content

gh-119127: functools.partial placeholders #119827

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 78 commits into from
Sep 26, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
78 commits
Select commit Hold shift + click to select a range
ee7333c
Initial Implementation
dg-pb May 21, 2024
8bcc462
serialization fix
dg-pb May 21, 2024
c67c9b4
bug fix
dg-pb May 21, 2024
680d900
Bug 2 fix
dg-pb May 21, 2024
9591ff5
Py_TPFLAGS_IMMUTABLETYPE added
dg-pb May 21, 2024
067e938
placeholder added to state as opposed to being used as global constant
dg-pb May 23, 2024
8af20b3
static removed
dg-pb May 23, 2024
607a0b1
creating sentinel via PyType_Spec
dg-pb May 23, 2024
f55801e
more accurate variable name
dg-pb May 23, 2024
5894145
trailing trim bug and tests
dg-pb May 24, 2024
3722e07
Updated docs
dg-pb May 24, 2024
a79c2af
blurb
dg-pb May 24, 2024
12aaa72
Merge branch 'main' into implement-119127
May 24, 2024
92c767b
minor edit
dg-pb May 24, 2024
496a9d2
doc fix
dg-pb May 24, 2024
38d9c11
better variable names and mini corrections
dg-pb May 31, 2024
707b957
Merge branch 'implement-119127' into implement-119127-again
dg-pb May 31, 2024
14b38ca
review comments mostly
dg-pb Jun 8, 2024
32bca19
singleton sentinel and reduce
dg-pb Jun 8, 2024
8576493
python module sentinel better mimics the one of the extension
dg-pb Jun 8, 2024
a3fd2d6
Emulated None behaviour, but using PyType_FromModuleAndSpec
dg-pb Jun 8, 2024
0852993
review feedback
dg-pb Jun 8, 2024
6fea348
included constant into tsv
dg-pb Jun 8, 2024
caec6e8
documentation update
dg-pb Jun 8, 2024
115b8c5
review edits
dg-pb Jun 9, 2024
3f5f00b
trailing placeholder prohibition and small changes
dg-pb Jun 9, 2024
202c929
change constant name in ignores
dg-pb Jun 9, 2024
2c16d38
PlaceholderType Hidden
dg-pb Jun 11, 2024
400ff55
support 4-arg pre-placeholder state
dg-pb Jun 20, 2024
8ccc38f
better variable names
dg-pb Jun 20, 2024
e7c82c7
partialmethod impl
dg-pb Jun 20, 2024
c9b7ef3
fix tests
dg-pb Jun 20, 2024
e59d711
adjust inspect to partial Placeholders
dg-pb Jun 20, 2024
7bfc591
arg alignment
dg-pb Jun 20, 2024
7957a97
small fixes
dg-pb Jun 21, 2024
8aaee6a
pickle compatibility ensured
dg-pb Jun 23, 2024
fe8e0ad
trailing placeholder test for __setstate__
dg-pb Jun 23, 2024
00dd80e
rough implementation rolled back and example of successive applicatio…
dg-pb Jun 24, 2024
d352cfa
smal 8000 l doc edits
dg-pb Jun 24, 2024
9038ed5
example changes
dg-pb Jun 24, 2024
49b8c71
serialization issues addressed
dg-pb Jun 25, 2024
bc1fdbd
delete redundant references
dg-pb Jun 25, 2024
3067221
simplify PyPlaceholder implementation
dg-pb Jun 25, 2024
1185510
more optimal python functools.partial
dg-pb Jun 25, 2024
266b4fa
placeholder arg and pre-placeholder instance conversions to positional
dg-pb Jun 25, 2024
dd58a12
unittest.mock.ANY test for Placeholder
dg-pb Jun 25, 2024
5971fbb
functools.partial.__get__
dg-pb Jun 26, 2024
9033650
Revert "functools.partial.__get__"
dg-pb Jun 26, 2024
d31e5d1
Merge branch 'main' into implement-119127-again
dg-pb Jun 26, 2024
a3d39b0
review changes
dg-pb Jun 26, 2024
9e4c5df
factor out repr. same to be used for partial and partialmethod
dg-pb Jun 27, 2024
16f12f8
microopt & args.count(Placeholder) can not be used as it uses __eq__
dg-pb Jun 27, 2024
82dd600
simplify preparation
dg-pb Jun 27, 2024
f9cb653
whatsnew + minor doc edit
dg-pb Jun 27, 2024
d255524
typo fixes
dg-pb Jun 27, 2024
404044e
whatsnew edit
dg-pb Jun 27, 2024
800217b
revert stylistic changes
dg-pb Jun 27, 2024
38ee450
factor out full __new__
dg-pb Jun 28, 2024
11f47db
Merge branch 'main' into implement-119127-again
serhiy-storchaka Jul 3, 2024
3c872bd
Merge branch 'main' into implement-119127-again
serhiy-storchaka Aug 11, 2024
fd16189
CR part 1
dg-pb Aug 12, 2024
a6c6ef2
CR Part 2
dg-pb Aug 12, 2024
1c8d73e
remove ignored global var
dg-pb Aug 12, 2024
a8bd3ae
CR changes
dg-pb Aug 12, 2024
70e47ed
small CR changes and doc updates
dg-pb Aug 12, 2024
2eacf5e
push placeholder check to earlier place
dg-pb Aug 12, 2024
f78d8d3
more appropriate test functions and better doc example
dg-pb Aug 13, 2024
0a8640e
minor fixes; message, doc polish
dg-pb Aug 13, 2024
6e3d282
better doc example and small test changes
dg-pb Aug 13, 2024
66c305d
assertRaisesRegex exact match
dg-pb Aug 14, 2024
14bf68c
doc nits
dg-pb Aug 14, 2024
ee642d5
Add Placeholder to __all__
rhettinger Sep 25, 2024
8d6c28e
Update copyright span
rhettinger Sep 25, 2024
8744bcb
Minor doc edits and add doctests
rhettinger Sep 25, 2024
b896470
Merge branch 'main' into implement-119127-again
rhettinger Sep 25, 2024
4881ae6
Merge branch 'main' into implement-119127-again
rhettinger Sep 25, 2024
c3ad7d9
Having the separate dict was necessary to eliminate duplicate keywords
rhettinger Sep 26, 2024
5e5d484
Merge branch 'main' into implement-119127-again
rhettinger Sep 26, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
minor fixes; message, doc polish
  • Loading branch information
dg-pb committed Aug 13, 2024
commit 0a8640e400140fefd12d68d8599e8f449b3f03dd
4 changes: 2 additions & 2 deletions Doc/library/functools.rst
Original file line number Diff line number Diff line change
Expand Up @@ -348,7 +348,7 @@ The :mod:`functools` module defines the following functions:

The :func:`partial` is used for partial function application which "freezes"
some portion of a function's arguments and/or keywords resulting in a new object
with a simplified signature. For example, :func:`partial` can be used to create
with a simplified signature. For example, :func:`partial` can be used to create
a callable that behaves like the :func:`int` function where the *base* argument
defaults to two:

Expand All @@ -362,7 +362,7 @@ The :mod:`functools` module defines the following functions:
when :func:`partial` is called. This allows custom selection of positional arguments
to be pre-filled when constructing a :ref:`partial object <partial-objects>`.

If :data:`!Placeholder` sentinels are used, all of them must be filled at call time:
If :data:`!Placeholder` sentinels are present, all of them must be filled at call time:

>>> from functools import partial, Placeholder
>>> say_to_world = partial(print, Placeholder, Placeholder, "world!")
Expand Down
16 changes: 8 additions & 8 deletions Doc/whatsnew/3.14.rst
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,14 @@ Added support for converting any objects that have the
:meth:`!as_integer_ratio` method to a :class:`~fractions.Fraction`.
(Contributed by Serhiy Storchaka in :gh:`82017`.)

functools
---------

* Added support to :func:`functools.partial` and
:func:`functools.partialmethod` for :data:`functools.Placeholder` sentinels
to reserve a place for positional arguments.
(Contributed by Dominykas Grigonis in :gh:`119127`.)

json
----

Expand All @@ -133,14 +141,6 @@ operator
to ``obj is not None``.
(Contributed by Raymond Hettinger and Nico Mexis in :gh:`115808`.)

functools
---------

* :func:`functools.partial` and :func:`functools.partialmethod` now support
:data:`functools.Placeholder` sentinels to reserve a place for
positional arguments.
(Contributed by Dominykas Grigonis in :gh:`119127`)

os
--

Expand Down
10 changes: 5 additions & 5 deletions Lib/functools.py
Original file line number Diff line number Diff line change
Expand Up @@ -301,9 +301,9 @@ def __reduce__(self):
Placeholder = _PlaceholderType()

def _partial_prepare_merger(args):
nargs = len(args)
if not nargs:
if not args:
return 0, None
nargs = len(args)
order = []
j = nargs
for i, a in enumerate(args):
Expand All @@ -324,9 +324,9 @@ def _partial_new(cls, func, /, *args, **keywords):
else:
base_cls = partialmethod
# func could be a descriptor like classmethod which isn't callable
# assert issubclass(cls, partialmethod)
if not callable(func) and not hasattr(func, "__get__"):
raise TypeError(f"{func!r} is not callable or a descriptor")
raise TypeError(f"the first argument {func!r} must be a callable "
"or a descriptor")
if args and args[-1] is Placeholder:
raise TypeError("trailing Placeholders are not allowed")
if isinstance(func, base_cls):
Expand Down Expand Up @@ -459,7 +459,7 @@ def _method(cls_or_self, /, *args, **keywords):
args = args[phcount:]
except IndexError:
raise TypeError("missing positional arguments "
"in 'partial' call; expected "
"in 'partialmethod' call; expected "
f"at least {phcount}, got {len(args)}")
else:
pto_args = self.args
Expand Down
2 changes: 1 addition & 1 deletion Lib/inspect.py
Original file line number Diff line number Diff line change
Expand Up @@ -2464,7 +2464,7 @@ def _signature_from_callable(obj, *,
assert (not sig_params or
first_wrapped_param is not sig_params[0])
# If there were placeholders set,
# first param is transformaed to positional only
# first param is transformed to positional only
if partialmethod.args.count(functools.Placeholder):
first_wrapped_param = first_wrapped_param.replace(
kind=Parameter.POSITIONAL_ONLY)
Expand Down
23 changes: 20 additions & 3 deletions Lib/test/test_functools.py
Original file line number Diff line number Diff line change
Expand Up @@ -364,14 +364,19 @@ def test_setstate(self):
f = self.partial(signature)
f.__setstate__((capture, (PH, 1), dict(a=10), dict(attr=[])))
self.assertEqual(signature(f), (capture, (PH, 1), dict(a=10), dict(attr=[])))
with self.assertRaises(TypeError):
with self.assertRaises(TypeError) as cm:
self.assertEqual(f(), (PH, 1), dict(a=10))
expected = ("missing positional arguments in 'partial' call; "
"expected at least 1, got 0")
self.assertEqual(cm.exception.args[0], expected)
self.assertEqual(f(2), ((2, 1), dict(a=10)))

# Leading Placeholder error
# Trailing Placeholder error
f = self.partial(signature)
with self.assertRaises(TypeError):
with self.assertRaises(TypeError) as cm:
f.__setstate__((capture, (1, PH), dict(a=10), dict(attr=[])))
expected = "trailing Placeholders are not allowed"
self.assertEqual(cm.exception.args[0], expected)

def test_setstate_errors(self):
f = self.partial(signature)
Expand Down Expand Up @@ -556,10 +561,16 @@ class TestPartialPySubclass(TestPartialPy):
partial = PyPartialSubclass

def test_subclass_optimization(self):
# `partial` input to `partial` subclass
p = py_functools.partial(min, 2)
p2 = self.partial(p, 1)
self.assertIs(p2.func, min)
self.assertEqual(p2(0), 0)
# `partial` subclass input to `partial` subclass
p = self.partial(min, 2)
p2 = self.partial(p, 1)
self.assertIs(p2.func, min)
self.assertEqual(p2(0), 0)


class TestPartialMethod(unittest.TestCase):
Expand Down Expand Up @@ -702,10 +713,16 @@ def f(a, b, /):
def test_subclass_optimization(self):
class PartialMethodSubclass(functools.partialmethod):
pass
# `partialmethod` input to `partialmethod` subclass
p = functools.partialmethod(min, 2)
p2 = PartialMethodSubclass(p, 1)
self.assertIs(p2.func, min)
self.assertEqual(p2.__get__(0)(), 0)
# `partialmethod` subclass input to `partialmethod` subclass
p = PartialMethodSubclass(min, 2)
p2 = PartialMethodSubclass(p, 1)
self.assertIs(p2.func, min)
self.assertEqual(p2.__get__(0)(), 0)


class TestUpdateWrapper(unittest.TestCase):
Expand Down
Loading
0