8000 IntFlag crashes with no single bit members · Issue #93035 · python/cpython · GitHub
[go: up one dir, main page]

Skip to content

IntFlag crashes with no single bit members #93035

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
tyehle opened this issue May 21, 2022 · 5 comments · Fixed by #93076
Closed

IntFlag crashes with no single bit members #93035

tyehle opened this issue May 21, 2022 · 5 comments · Fixed by #93076
Assignees
Labels
3.11 only security fixes 3.12 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@tyehle
Copy link
Contributor
tyehle commented May 21, 2022

Bug report

IntFlag causes a crash on 3.11.0b1 when there are no single bit members.

Minimal example is

from enum import IntFlag
class E(IntFlag):
  X = 3

this results in

Traceback (most recent call last):
  File "poc.py", line 2, in <module>
    class E(IntFlag):
    ^^^^^^^^^^^^^^^^^^^
  File "3.11.0b1/lib/python3.11/enum.py", line 580, in __new__
    member = list(enum_class)[0]
             ~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range

This is used in practice when values with only single bits set aren't valid, eg:

Expensive = 0b001
Cramped   = 0b010
Ugly      = 0b100

class HouseAttributes(IntFlag):
  Cheap = Cramped | Ugly
  Spacious = Expensive | Ugly
  Beautiful = Expensive | Cramped
  Standard = Expensive | Cramped | Ugly

Your environment

  • CPython versions tested on: 3.11.0b1
  • Operating system and architecture: Linux and Mac
@tyehle tyehle added the type-bug An unexpected behavior, bug, or error label May 21, 2022
@tyehle tyehle changed the title IntFlags IntFlag crashes with no single bit members May 21, 2022
@dignissimus
Copy link
Contributor
dignissimus commented May 21, 2022

83d544b introduced the error when creating the enum but the underlying issue is that multi-bit flags aren't included when iterating the enum. eea8148 Introduced support for multi-bit members but they weren't listed with the enum. On 3.10 this was fixed in 9bf7c2d and remained fixed in 2a9ab75. But it was never fixed in 3.11 and above.

sam@samtop ~/Documents/git/cpython (git)-[v3.11.0a5~234] % ./python                                             
Python 3.11.0a4+ (tags/v3.11.0a4-30-g83d544b929:83d544b929, May 21 2022, 09:41:53) [GCC 12.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from enum import IntFlag
>>> class E(IntFlag):
...     x = 3
... 
>>> list(E)
[]
>>> 
Python 3.10.4 (main, Mar 23 2022, 23:05:40) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from enum import IntFlag
>>> class E(IntFlag):
...     x = 3
... 
>>> list(E)
[<E.x: 3>]
>>> 

The problem might be around here

cpython/Lib/enum.py

Lines 713 to 727 in b96e20c

elif Flag is not None and issubclass(enum_class, Flag):
# ensure _all_bits_ is correct and there are no missing flags
single_bit_total = 0
multi_bit_total = 0
for flag in enum_class._member_map_.values():
flag_value = flag._value_
if _is_single_bit(flag_value):
single_bit_total |= flag_value
else:
# multi-bit flags are considered aliases
multi_bit_total |= flag_value
enum_class._flag_mask_ = single_bit_total
#
# set correct __iter__
member_list = [m._value_ for m in enum_class]

I will try looking more into it later and I think an extra test should be added for this.

@dignissimus
Copy link
Contributor

Actually, @ethanfurman, which is (or should be?) the correct behaviour when listing IntFlag enums with multi-bit members? This is what happens on Python 3.10.4

sam@samtop ~ % python          
Python 3.10.4 (main, Mar 23 2022, 23:05:40) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from enum import IntFlag
>>> class E(IntFlag):
...     x = 1
...     y = 3
... 
>>> list(E)
[<E.x: 1>, <E.y: 3>]
>>> 

And on trunk

sam@samtop ~/Documents/git/cpython (git)-[main] % ./python
Python 3.12.0a0 (heads/main:fa2b8b75eb, May 21 2022, 10:35:37) [GCC 12.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from enum import IntFlag
>>> class E(IntFlag):
...     x = 1
...     y = 3
... 
>>> list(E)
[<E.x: 1>]

@AlexWaygood AlexWaygood added stdlib Python modules in the Lib dir 3.11 < 8000 tool-tip id="tooltip-9a8766fb-968f-4cf7-877a-72eb184a5997" for="label-b51fe2" popover="manual" data-direction="s" data-type="description" data-view-component="true" class="sr-only position-absolute">only security fixes 3.12 only security fixes labels May 21, 2022
@ethanfurman
Copy link
Member

The iteration behavior in trunk is correct.

@dignissimus
Copy link
Contributor
dignissimus commented May 23, 2022

@ethanfurman How should the following work? (should it work?)

def test_bizarre(self):   
    class Bizarre(Flag):
        b = 3
        c = 4
        d = 6
    self.assertEqual(repr(Bizarre(7)), '<Bizarre.d|c|b: 7>')

My current patch produces the following

>>> repr(Bizarre(7))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/sam/Documents/git/cpython/Lib/enum.py", line 396, in __call__
    return cls.__new__(cls, value)
  File "/home/sam/Documents/git/cpython/Lib/enum.py", line 729, in __new__
    raise exc
  File "/home/sam/Documents/git/cpython/Lib/enum.py", line 711, in __new__
    result = cls._missing_(value)
  File "/home/sam/Documents/git/cpython/Lib/enum.py", line 857, in _missing_
    possible_member = cls._create_pseudo_member_(value)
  File "/home/sam/Documents/git/cpython/Lib/enum.py", line 872, in _create_pseudo_member_
    raise ValueError("%r is not a valid %s" % (value, cls.__qualname__))
ValueError: 7 is not a valid Bizarre

Behaviour with single-bit members is correct

>>> class Bizarre(Flag):
...     x = 1
...     y = 2
...     z = 4
... 
>>> Bizarre(7)
<Bizarre.z|y|x: 7>
>>> 

@dignissimus
Copy link
Contributor

If erroring is correct, then I think the error message may need to be amended

ethanfurman pushed a commit that referenced this issue May 25, 2022
`EnumType` attempts to create a custom docstring for each enum/flag, but that was failing with pathological flags that had no members (only multi-bit aliases).
miss-islington pushed a commit to miss-islington/cpython that referenced this issue May 25, 2022
…ythonGH-93076)

`EnumType` attempts to create a custom docstring for each enum/flag, but that was failing with pathological flags that had no members (only multi-bit aliases).
(cherry picked from commit 08cfc3d)

Co-authored-by: Tobin Yehle <tobinyehle@gmail.com>
ethanfurman pushed a commit that referenced this issue May 25, 2022
…H-93076) (GH-93197)

`EnumType` attempts to create a custom docstring for each enum/flag, but that was failing with pathological flags that had no members (only multi-bit aliases).
(cherry picked from commit 08cfc3d)

Co-authored-by: Tobin Yehle <tobinyehle@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.11 only security fixes 3.12 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants
0