8000 Two argument form of iter() is not implemented. · Issue #5384 · micropython/micropython · GitHub
[go: up one dir, main page]
Skip to content
8000

Two argument form of iter() is not implemented. #5384

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
hardkrash opened this issue Dec 5, 2019 · 14 comments
Closed

Two argument form of iter() is not implemented. #5384

hardkrash opened this issue Dec 5, 2019 · 14 comments
Labels
docs py-core Relates to py/ directory in source

Comments

@hardkrash
Copy link

The builtin iter(callable, sentinel) from pep 234 is not implemented.

The use case is something like this.

with open('file.bin', 'rb') as f:
    for chunk in iter(lambda: f.read(128), b''):
        process_chunk(chunk)

The two parameter form allows an easy way to make a callable into an iterable by using a sentinel.

It allows for replacing the do while workarounds with a simple iterator.

with open('file.bin', 'rb') as f:
    chunk = f.read(128)
    while chunk != b'':
        process_chunk(chunk)
        chunk = f.read(128)

Credit to this stack overflow answer from jfs
https://stackoverflow.com/a/20014805
@mikewadsten
Copy link
Contributor

Sort of related to #4400.

@dpgeorge
Copy link
Member
dpgeorge commented Dec 6, 2019

Yes, this is simply not implemented, mainly because the same thing can be achieved in pure Python. As a workaround the following can be used:

def iter2(c, s):
    while True:
        v = c()
        if v == s:
            raise StopIteration
        yield v

(This would be simpler with the walrus operator!)

@dpgeorge dpgeorge added notimpl py-core Relates to py/ directory in source labels Dec 6, 2019
@hardkrash
Copy link
Author

While I agree on the fact that this can be worked around in many ways. it is unfortunate that this is not explicitly called out in the documentation as a difference from python's core syntax.

This would be a compatibility wrapper needed to make the two argument form of iter work without changing source code.

from compat import iter

Where compat.py has something like this.

S = some_singleton
_orig_iter = iter
def _compat_iter(c, s=S):
    if s is S:
        for x in _orig_iter(c):
            yield x
    else:
        while True:
            v = c()
            if v == s:
                raise StopIteration
            yield v

iter = _compat_iter

Would adding 2 argument form to the builtin iter function be an acceptable patch?

@dpgeorge
Copy link
Member

it is unfortunate that this is not explicitly called out in the documentation as a difference from python's core syntax.

It's not really a syntax difference, rather the fact that MicroPython implements a subset of the Python builtins. It could be added to the docs via tests/cpydiff (which automatically generates differences for the docs).

Would adding 2 argument form to the builtin iter function be an acceptable patch?

Implemented in C I think it'd be a large patch and therefore unlikely to be made part of default MicroPython builds. It could be made optional like the 2-arg form of next(), but I'm just not sure how generally useful it would be (it hasn't been requested so far).

@dpgeorge
Copy link
Member

This would be a good candidate for #5025.

@hex007
Copy link
hex007 commented Apr 19, 2023

Found this not implemented the hard way. next called with two params can just throw Not implemented exception would be much easier to understand than

TypeError: function takes 1 positional arguments but 2 were given

@projectgus
Copy link
Contributor

Labelling as docs, as I think the most sensible to do in the short term is to add this to the documented list of CPython differences.

@symbioquine
Copy link

I was also surprised to discover next not taking a second default/sentinel parameter - and not finding any mention of this in https://docs.micropython.org/en/latest/genrst/index.html#micropython-differences-from-cpython

Naively, I would have expected to see a section in the docs on differences in "builtin methods" above/below the section on differences in "builtin types".

Image

Again naively, this breaks my assumptions about MicroPython a bit because it seems surprising th 8000 at the logic to support the second parameter would increase the size noticeably, nor present any performance cost. Yes, it's easy to work-around, but I wish I didn't need to consider this among the things constraining code's "portability to MicroPython".

@Josverl Josverl mentioned this issue Mar 6, 2025
Open
1 task
@stinos
Copy link
Contributor
stinos commented Mar 7, 2025

Note that the 2-argument next is implemented, guarded by MICROPY_PY_BUILTINS_NEXT2, see 4286383

@symbioquine
Copy link

Okay, looking a bit harder, I actually found the relevant documentation: https://docs.micropython.org/en/latest/genrst/modules.html#second-argument-to-next-is-not-implemented

So I guess that documentation probably needs updating to clarify that it's now not available on some boards in the default builds and whether it is available is controlled by that compile time flag?

@stinos
Copy link
Contributor
stinos commented Mar 10, 2025

Yeah I don't think there's a real policy now for 'conditionally different from CPython'. There are multiple other occurrences in these docs like that.
8000

@dpgeorge
Copy link
Member

Looking at this again after many years, I see that the difference in implementation between 1-arg next and 2-arg next is very small.

I made a comment long ago:

Implemented in C I think it'd be a large patch and therefore unlikely to be made part of default MicroPython builds.

I think my reasoning back then was because I was thinking that the implementation would need a NLR block to catch the stop iteration. But it doesn't anymore because we have mp_iternext_allow_raise and mp_iternext.

In my opinion the 2-argument form of next() should be enabled at a much earlier feature level, like MICROPY_CONFIG_ROM_LEVEL_BASIC_FEATURES. Or even just unconditionally replace the 1-arg form.

@dpgeorge dpgeorge mentioned this issue Mar 11, 2025
Merged
@dpgeorge
Copy link
Member

See #16902.

dpgeorge added a commit to dpgeorge/micropython that referenced this issue Mar 27, 2025
@dpgeorge
py/mpconfig: Enable 2-argument built-in next() at basic feature level.
56e90cb 
This is a pretty fundamental built-in and having CPython-compatible
behaviour is beneficial.  The code size increase is not much, and
ports/boards can still disable it if needed to save space.

Addresses issue micropython#5384.

Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
Copy link
Member

Now implemented.

mseminatore pushed a commit to mseminatore/micropython that referenced this issue Mar 31, 2025
@dpgeorge
py/mpconfig: Enable 2-argument built-in next() at basic feature level.
a6994d9 
This is a pretty fundamental built-in and having CPython-compatible
behaviour is beneficial.  The code size increase is not much, and
ports/boards can still disable it if needed to save space.

Addresses issue micropython#5384.

Signed-off-by: Damien George <damien@micropython.org>
VynDragon pushed a commit to VynDragon/micropython that referenced this issue Apr 6, 2025
@dpgeorge @VynDragon
py/mpconfig: Enable 2-argument built-in next() at basic feature level.
0cf4681 
This is a pretty fundamental built-in and having CPython-compatible
behaviour is beneficial.  The code size increase is not much, and
ports/boards can still disable it if needed to save space.

Addresses issue micropython#5384.

Signed-off-by: Damien George <damien@micropython.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
docs py-core Relates to py/ directory in source
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@hardkrash @projectgus @hex007 @stinos @mikewadsten @dpgeorge @symbioquine
0