From ed59bc4a706807acb105a1d4897bff0f90cad370 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 23 May 2024 13:56:10 -0500 Subject: [PATCH 01/16] Simplify takewhile() --- Doc/library/itertools.rst | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index 6d33748898361d..83acaa1860d4ba 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -676,10 +676,9 @@ loops that truncate the stream. def takewhile(predicate, iterable): # takewhile(lambda x: x<5, [1,4,6,4,1]) → 1 4 for x in iterable: - if predicate(x): - yield x - else: + if not predicate(x): break + yield x Note, the element that first fails the predicate condition is consumed from the input iterator and there is no way to access it. From acd55be461dd017ad8b2d61e4565240f754b785b Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 23 May 2024 13:58:10 -0500 Subject: [PATCH 02/16] Tighter sentence --- Doc/library/itertools.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index 83acaa1860d4ba..1866ee8d122569 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -134,7 +134,7 @@ loops that truncate the stream. total = func(total, element) yield total - There are a number of uses for the *func* argument. It can be set to + The *func* argument can be set to :func:`min` for a running minimum, :func:`max` for a running maximum, or :func:`operator.mul` for a running product. Amortization tables can be built by accumulating interest and applying payments: From 376b376287dde262b8bd20e9303011c8d65464df Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 23 May 2024 14:03:14 -0500 Subject: [PATCH 03/16] Remove redundant batched() example. Improve variable name. --- Doc/library/itertools.rst | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-) diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index 1866ee8d122569..1ca5c660268489 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -184,21 +184,14 @@ loops that truncate the stream. >>> unflattened [('roses', 'red'), ('violets', 'blue'), ('sugar', 'sweet')] - >>> for batch in batched('ABCDEFG', 3): - ... print(batch) - ... - ('A', 'B', 'C') - ('D', 'E', 'F') - ('G',) - Roughly equivalent to:: def batched(iterable, n, *, strict=False): # batched('ABCDEFG', 3) → ABC DEF G if n < 1: raise ValueError('n must be at least one') - it = iter(iterable) - while batch := tuple(islice(it, n)): + iterable = iter(iterable) + while batch := tuple(islice(iterable, n)): if strict and len(batch) != n: raise ValueError('batched(): incomplete batch') yield batch From ac27f16f04ae7ff0c2940f34501b6956362c97f9 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 23 May 2024 14:54:00 -0500 Subject: [PATCH 04/16] Grammar nits. --- Doc/library/itertools.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index 1ca5c660268489..d3b30667f72e6c 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -230,13 +230,13 @@ loops that truncate the stream. Return *r* length subsequences of elements from the input *iterable*. - The combination tuples are emitted in lexicographic ordering according to + The combination tuples are emitted in lexicographic order according to the order of the input *iterable*. So, if the input *iterable* is sorted, the output tuples will be produced in sorted order. Elements are treated as unique based on their position, not on their - value. So if the input elements are unique, there will be no repeated - values in each combination. + value. So, if the input elements are unique, there will be no repeated + values within each combination. Roughly equivalent to:: From ded2660b083da8ae3955a929da82d86a50421bf9 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 23 May 2024 14:54:41 -0500 Subject: [PATCH 05/16] Grammar nits. --- Doc/library/itertools.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index d3b30667f72e6c..c9c78659ba990a 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -279,12 +279,12 @@ loops that truncate the stream. Return *r* length subsequences of elements from the input *iterable* allowing individual elements to be repeated more than once. - The combination tuples are emitted in lexicographic ordering according to + The combination tuples are emitted in lexicographic order according to the order of the input *iterable*. So, if the input *iterable* is sorted, the output tuples will be produced in sorted order. Elements are treated as unique based on their position, not on their - value. So if the input elements are unique, the generated combinations + value. So, if the input elements are unique, the generated combinations will also be unique. Roughly equivalent to:: From 2a53903f72abfde520b0f34affe86c24f8b82098 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 23 May 2024 15:04:12 -0500 Subject: [PATCH 06/16] Improve comress() text and code equivalent. --- Doc/library/itertools.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index c9c78659ba990a..22dfce3bd4f7e1 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -325,13 +325,13 @@ loops that truncate the stream. .. function:: compress(data, selectors) Make an iterator that filters elements from *data* returning only those that - have a corresponding element in *selectors* that evaluates to ``True``. - Stops when either the *data* or *selectors* iterables has been exhausted. + have a corresponding element in *selectors* is true. + Stops when either the *data* or *selectors* iterables have been exhausted. Roughly equivalent to:: def compress(data, selectors): # compress('ABCDEF', [1,0,1,0,1,1]) → A C E F - return (d for d, s in zip(data, selectors) if s) + return (datum for datum, selector in zip(data, selectors) if selector) .. versionadded:: 3.1 From 83a4c37f87439adb24401d74dd4f4e5c108d8860 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 23 May 2024 15:22:25 -0500 Subject: [PATCH 07/16] Improve dropwhile() example with unique data elements --- Doc/library/itertools.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index 22dfce3bd4f7e1..3e079f61c79c6f 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -385,7 +385,7 @@ loops that truncate the stream. start-up time. Roughly equivalent to:: def dropwhile(predicate, iterable): - # dropwhile(lambda x: x<5, [1,4,6,4,1]) → 6 4 1 + # dropwhile(lambda x: x<5, [1,4,6,3,8]) → 6 3 8 iterable = iter(iterable) for x in iterable: if not predicate(x): From a1a864e0345221c1b785bc77d325dd72cb2a05a4 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 23 May 2024 15:26:09 -0500 Subject: [PATCH 08/16] Improve filterfalse() example with clearer predicate --- Doc/library/itertools.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index 3e079f61c79c6f..ba09edc4434d61 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -401,7 +401,7 @@ loops that truncate the stream. that are false. Roughly equivalent to:: def filterfalse(predicate, iterable): - # filterfalse(lambda x: x%2, range(10)) → 0 2 4 6 8 + # filterfalse(str.isupper, 'AbCdEf') → b d f if predicate is None: predicate = bool for x in iterable: From 8fc899c8676105dd6ead86fd6dda825a1b9dd2d6 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 23 May 2024 15:44:03 -0500 Subject: [PATCH 09/16] Clearer generator version of the rough code equivalent --- Doc/library/itertools.rst | 53 ++++++++++++++++++++------------------- 1 file changed, 27 insertions(+), 26 deletions(-) diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index ba09edc4434d61..16cfb5dc651660 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -437,36 +437,37 @@ loops that truncate the stream. :func:`groupby` is roughly equivalent to:: - class groupby: + def groupby(iterable, key=None): # [k for k, g in groupby('AAAABBBCCDAABBB')] → A B C D A B # [list(g) for k, g in groupby('AAAABBBCCD')] → AAAA BBB CC D - def __init__(self, iterable, key=None): - if key is None: - key = lambda x: x - self.keyfunc = key - self.it = iter(iterable) - self.tgtkey = self.currkey = self.currvalue = object() - - def __iter__(self): - return self - - def __next__(self): - self.id = object() - while self.currkey == self.tgtkey: - self.currvalue = next(self.it) # Exit on StopIteration - self.currkey = self.keyfunc(self.currvalue) - self.tgtkey = self.currkey - return (self.currkey, self._grouper(self.tgtkey, self.id)) - - def _grouper(self, tgtkey, id): - while self.id is id and self.currkey == tgtkey: - yield self.currvalue - try: - self.currvalue = next(self.it) - except StopIteration: + keyfunc = (lambda x: x) if key is None else key + iterator = iter(iterable) + exhausted = False + + def _grouper(target_key): + nonlocal curr_value, curr_key, exhausted + yield curr_value + for curr_value in iterator: + curr_key = keyfunc(curr_value) + if curr_key != target_key: return - self.currkey = self.keyfunc(self.currvalue) + yield curr_value + exhausted = True + + try: + curr_value = next(iterator) + except StopIteration: + return + curr_key = keyfunc(curr_value) + + while not exhausted: + target_key = curr_key + curr_group = _grouper(target_key) + yield curr_key, curr_group + if curr_key == target_key: + for _ in curr_group: + pass .. function:: islice(iterable, stop) From 9724d5e183f03995d7fe340eae2321618f2605bf Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 23 May 2024 15:48:49 -0500 Subject: [PATCH 10/16] Better argument ordering in islice() equivalent --- Doc/library/itertools.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index 16cfb5dc651660..538d523b80b25a 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -495,13 +495,14 @@ loops that truncate the stream. # islice('ABCDEFG', 2, 4) → C D # islice('ABCDEFG', 2, None) → C D E F G # islice('ABCDEFG', 0, None, 2) → A C E G + s = slice(*args) start = 0 if s.start is None else s.start stop = s.stop step = 1 if s.step is None else s.step if start < 0 or (stop is not None and stop < 0) or step <= 0: raise ValueError - indices = count() if stop is None else range(max(stop, start)) + indices = count() if stop is None else range(max(start, stop)) next_i = start for i, element in zip(indices, iterable): if i == next_i: From 3b8195b6d4b9ad0f7f0b3eb36f0573aaa712e294 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 23 May 2024 16:02:33 -0500 Subject: [PATCH 11/16] Improve clarity of takewhile() example using unique inputs --- Doc/library/itertools.rst | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index 538d523b80b25a..db75b9a0f1a057 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -544,7 +544,7 @@ loops that truncate the stream. the output tuples will be produced in sorted order. Elements are treated as unique based on their position, not on their - value. So if the input elements are unique, there will be no repeated + value. So, if the input elements are unique, there will be no repeated values within a permutation. Roughly equivalent to:: @@ -552,6 +552,7 @@ loops that truncate the stream. def permutations(iterable, r=None): # permutations('ABCD', 2) → AB AC AD BA BC BD CA CB CD DA DB DC # permutations(range(3)) → 012 021 102 120 201 210 + pool = tuple(iterable) n = len(pool) r = n if r is None else r @@ -575,7 +576,7 @@ loops that truncate the stream. return The code for :func:`permutations` can be also expressed as a subsequence of - :func:`product`, filtered to exclude entries with repeated elements (those + :func:`product` filtered to exclude entries with repeated elements (those from the same position in the input pool):: def permutations(iterable, r=None): @@ -669,7 +670,7 @@ loops that truncate the stream. predicate is true. Roughly equivalent to:: def takewhile(predicate, iterable): - # takewhile(lambda x: x<5, [1,4,6,4,1]) → 1 4 + # takewhile(lambda x: x<5, [1,4,6,3,8]) → 1 4 for x in iterable: if not predicate(x): break From 19de882c745de6a86d2fea1c972334a612dd49f6 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 23 May 2024 16:59:34 -0500 Subject: [PATCH 12/16] . --- Doc/library/itertools.rst | 3 +++ 1 file changed, 3 insertions(+) diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index db75b9a0f1a057..0ceeec068e59ab 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -502,6 +502,7 @@ loops that truncate the stream. step = 1 if s.step is None else s.step if start < 0 or (stop is not None and stop < 0) or step <= 0: raise ValueError + indices = count() if stop is None else range(max(start, stop)) next_i = start for i, element in zip(indices, iterable): @@ -729,10 +730,12 @@ loops that truncate the stream. def zip_longest(*iterables, fillvalue=None): # zip_longest('ABCD', 'xy', fillvalue='-') → Ax By C- D- + iterators = [iter(it) for it in iterables] num_active = len(iterators) if not num_active: return + while True: values = [] for i, iterator in enumerate(iterators): From 743d5c2f7bc1a3111aea7bdcad657ede81f7d71a Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Thu, 23 May 2024 17:06:41 -0500 Subject: [PATCH 13/16] Use map() to build the list of iterators --- Doc/library/itertools.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index 0ceeec068e59ab..3f3ed79a2db8a3 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -731,7 +731,7 @@ loops that truncate the stream. def zip_longest(*iterables, fillvalue=None): # zip_longest('ABCD', 'xy', fillvalue='-') → Ax By C- D- - iterators = [iter(it) for it in iterables] + iterators = list(map(iter, iterables)) num_active = len(iterators) if not num_active: return From 163428aa299600a50abb6264f87c4bc05d6c546b Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Fri, 24 May 2024 12:03:43 -0500 Subject: [PATCH 14/16] . --- Doc/library/itertools.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index 3f3ed79a2db8a3..5710b4819b99cb 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -559,9 +559,11 @@ loops that truncate the stream. r = n if r is None else r if r > n: return + indices = list(range(n)) cycles = list(range(n, n-r, -1)) yield tuple(pool[i] for i in indices[:r]) + while n: for i in reversed(range(r)): cycles[i] -= 1 From 0b19fb6cd5e8b85a0836727233427dfc3fb2be46 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Fri, 24 May 2024 12:29:44 -0500 Subject: [PATCH 15/16] . --- Doc/library/itertools.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index 5710b4819b99cb..d3b54654deaa16 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -682,7 +682,7 @@ loops that truncate the stream. Note, the element that first fails the predicate condition is consumed from the input iterator and there is no way to access it. This could be an issue if an application wants to further consume the - input iterator after takewhile has been run to exhaustion. To work + input iterator after *takewhile* has been run to exhaustion. To work around this problem, consider using `more-iterools before_and_after() `_ instead. From f2979330b874d55e87f89511bbeacac05c652572 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Fri, 24 May 2024 16:31:15 -0500 Subject: [PATCH 16/16] Sync the filterfalse() example with the dropwhile() example --- Doc/library/itertools.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index d3b54654deaa16..43432dae1623ce 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -401,7 +401,7 @@ loops that truncate the stream. that are false. Roughly equivalent to:: def filterfalse(predicate, iterable): - # filterfalse(str.isupper, 'AbCdEf') → b d f + # filterfalse(lambda x: x<5, [1,4,6,3,8]) → 6 8 if predicate is None: predicate = bool for x in iterable: