object.eq and other comparisons should have return type Any #3685

jonasrauber · 2020-01-29T13:51:02Z

@JukkaL commented on Feb 1, 2017
I wonder if the return type of object.__eq__ in typeshed should be object, since the bool return type is not quite correct. The same applies to __lt__ and friends.

Given the implementation, I would suggest the return type should be Union[NoReturn, Any]. object.__eq__ always throws a NotImplementedError (hence the NoReturn) and Any allows sub-classes to override the return type to anything they like.

Originally posted by @Victor-Savu in python/mypy#2783 (comment)

Please fix the return type of __eq__ and others in typeshed as described in the comment above. __eq__ not returning a boolean is very common, most notably NumPy arrays performing elementwise comparisons.

The text was updated successfully, but these errors were encountered:

jonasrauber · 2020-01-29T13:55:09Z

Please see the thread linked above for details. The statement about raising NotImplementedError was corrected tot returning NotImplemented, so defining Any as the return type should be what we need.

jonasrauber · 2020-01-29T13:56:10Z

I would be happy to open a PR …

srittau · 2020-01-29T14:06:29Z

I am honestly not convinced. I would argue that the default assumption is that __eq__ and other binary comparison methods should return a boolean value. If some classes return another type, they can always override the return type, potentially using # type: ignore.

llchan · 2020-02-06T19:51:34Z

@jonasrauber I very much agree that something needs to be done about __eq__, but there are many nuances around it so I'd recommend discussing with the mypy maintainers before putting in significant time to prep a PR. It seems that there may be some baked-in assumptions and special handling beyond what typeshed dictates.

Related discussions:
python/mypy#2783
python/mypy#5704
python/mypy#6710

srittau

This is a case where # type: ignore should be used if non-standard behavior is desired.

kmillikin · 2022-02-08T13:44:40Z

NoReturn should be a bottom type, a subtype of every type. So Union[NoReturn, Any] is just Any.

Any is not what you want because then you have the undesirable property that fully statically typed code will contain dynamic types (types containing Any). object is the return type you want because (1) it faithfully matches the implementation and (2) it does allow subclasses to correctly override the method.

kmillikin · 2022-02-08T17:31:17Z

This is a case where # type: ignore should be used if non-standard behavior is desired.

OK, but I think you should reconsider closing this instead of fixing it.

The non-standard behavior is the annotation in typeshed. It's non-standard because it disagrees with the runtime semantics via implementation of CPython (the only real standard we have for Python). This program has a type error:

def f(x: object) -> int:
  return x == 0

But a static type checker that uses the typing in typeshed will not catch it without special handling of the type annotations in class object. I would say instead that the annotations in typeshed should be correct and then that checkers could implement something else if they wanted to.

JelleZijlstra · 2022-02-08T17:54:36Z

This program has a type error

No it doesn't. object.__eq__ returns a bool, and a bool is an int.

kmillikin · 2022-02-08T18:55:01Z

Are we both talking about the runtime behavior? x == 0 doesn't necessarily invoke object.__eq__. What matters is if the runtime ensures that x == 0 is always bool; and it doesn't. You can decide that the static type of x == 0 is bool but that seems like a bad description of the runtime behavior.

superbobry · 2022-02-09T10:36:09Z

I suspect the point was that

def f(x: object) -> int:
  return x == 0

has no type errors assuming that the type of x contains a compatible overload of object.__eq__. However, this is not true for all classes, including the very widely used np.ndarray. Imagine we call f with an instance of such class

class A:
  def __eq__(self, other) -> str:
    return "YES"

f(A())

A.__eq__ is clearly incompatible with object.__eq__, because str is not a subtype of bool. If we hide that incompatibility via # type: ignore, we will force the type checker to accept an otherwise unsafe f(...) call. Is this WAI?

joooeey · 2022-11-18T18:06:06Z

This is a case where # type: ignore should be used if non-standard behavior is desired.

Rich comparisons have been in Python since 2.1. There's nothing non-standard about it, as @kmillikin said.

I have some snippets here to speed up debugging (I do that with pytest . --pdb) that will require tons of # type: ignore statements if I use this more:

class Falsy:
    def __init__(self, reason: str) -> None:
        """Record description."""
        self.reason = reason

    def __bool__(self) -> typing.Literal[False]:
        """This class is always False."""
        return False

    def __eq__(self, other: typing.Any) -> bool:
        """Equal to falsy values."""
        return not other

    def __repr__(self) -> str:
        """Print the reason instead of just False."""
        return f'Falsy("{self.reason}")'


Boolean = Falsy | bool

class SomeClass:
    ...
    def __eq__(self, other: Any) -> Boolean:  # type: ignore
        for attr in self._defining_attributes:
            try:
                selfattr = getattr(self, attr)
                otherattr = getattr(other, attr)
            except AttributeError:
                return NotImplemented
            if selfattr != otherattr:
                return Falsy(f"{selfattr} != {otherattr}")
        return True

I also don't understand how this is supposed to work with Numpy.

Perhaps we need something like collections.abc.Boolean?

dstromberg · 2023-05-08T20:22:09Z

I'm here because SQLAlchemy also has __eq__ (and others) returning non-bools. I had to convince myself this works with a little test program and some googling, but I'm seeing SQLAlchemy return a BinaryExpression object for use in Object Relational Mapper (ORM) queries:
https://docs.sqlalchemy.org/en/20/core/sqlelement.html#sqlalchemy.sql.expression.BinaryExpression

Actually, I believe that Brython uses <= a little interestingly too:
https://brython.info/static_doc/en/faq.html

My first reaction to Brython's was sadness, but in the larger context of SQLAlchemy, Numpy and Brython, I'm getting over it. :)

Here's my test program:
$ cat /tmp/denb
below cmd output started 2023 Mon May 08 01:20:39 PM PDT

#!/usr/bin/env python3

class Foo:
    def __init__(self):
        pass

    def __eq__(self, other):
        return "abc"


foo1 = Foo()
foo2 = Foo()
print(foo1 == foo2)

above cmd output done    2023 Mon May 08 01:20:39 PM PDT
dstromberg@zareason2:~ x86_64-pc-linux-gnu 1421091

$ pythons --file /tmp/denb
below cmd output started 2023 Mon May 08 01:20:45 PM PDT
/usr/local/cpython-0.9/bin/python (unknown) bad
    Parsing error: file /tmp/denb, line 3:
    class Foo:
              ^
    Unhandled exception: run-time error: syntax error
/usr/local/cpython-1.0/bin/python (1.0.1) good 0
/usr/local/cpython-1.1/bin/python (1.1) good 0
/usr/local/cpython-1.2/bin/python (1.2) good 0
/usr/local/cpython-1.3/bin/python (1.3) good 0
/usr/local/cpython-1.4/bin/python (1.4) good 0
/usr/local/cpython-1.5/bin/python (1.5.2) good 0
/usr/local/cpython-1.6/bin/python (1.6.1) good 0
/usr/local/cpython-2.0/bin/python (2.0.1) good 0
/usr/local/cpython-2.1/bin/python (2.1.0) good abc
/usr/local/cpython-2.2/bin/python (2.2.0) good abc
/usr/local/cpython-2.3/bin/python (2.3.0) good abc
/usr/local/cpython-2.4/bin/python (2.4.0) good abc
/usr/local/cpython-2.5/bin/python (2.5.6) good abc
/usr/local/cpython-2.6/bin/python (2.6.9) good abc
/usr/local/cpython-2.7/bin/python (2.7.16) good abc
/usr/local/cpython-3.0/bin/python (3.0.1) good abc
/usr/local/cpython-3.1/bin/python (3.1.5) good abc
/usr/local/cpython-3.2/bin/python (3.2.5) good abc
/usr/local/cpython-3.3/bin/python (3.3.7) good abc
/usr/local/cpython-3.4/bin/python (3.4.8) good abc
/usr/local/cpython-3.5/bin/python (3.5.5) good abc
/usr/local/cpython-3.6/bin/python (3.6.13) good abc
/usr/local/cpython-3.7/bin/python (3.7.0) good abc
/usr/local/cpython-3.8/bin/python (3.8.0) good abc
/usr/local/cpython-3.9/bin/python (3.9.0) good abc
/usr/local/cpython-3.10/bin/python (3.10.0) good abc
/usr/local/cpython-3.11/bin/python (3.11.0) good abc
/usr/local/pypy3-7.3.9/bin/pypy3 (3.9.12) good abc

 1 bad  **
28 good ************************************************************

So apparently this has worked since 2.1.0.

Tishka17 · 2024-11-21T14:20:56Z

I have encountered the same problem. I am making a query builder for specific needs and it is strange that __eq__ should return bool while other operators shouldn't and there are really popular projects like SQLAlchemy and numpy using rich comparison.

Additionally, I want to quote pep-0207

This PEP proposes several new features for comparisons:

Allow separately overloading of <, >, <=, >=, ==, !=, both in classes and in C extensions.

Allow any of those overloaded operators to return something else besides a Boolean result.

kmillikin · 2024-11-21T15:24:25Z

One thing that really stands out about Python compared to other languages, is that it is a great host for embedded domain specific languages (DSLs). This bug means that type checkers that use typeshed for typing == will give false positives for such code, which is too bad.

Put another way: this typing is (gratuitously) unsound with respect to the runtime semantics.

srittau · 2024-11-21T15:46:47Z

This bug means that type checkers that use typeshed for typing == will give false positives for such code, which is too bad.

As mentioned before, type checkers will happily accept those comparisons if the LSP violation in the overloading classes is # type: ignored.

class Foo:
    def __eq__(self, other: object) -> str:  # type: ignore[override]
        return "Hey!"
    
reveal_type(Foo() == Foo())  # revealed type is `str`

While overloading the operators in common, it is an LSP violation and unsound. Using # type: ignore explicitly to signal intent is the best option. I don't see use changing this. The only completely safe option would be to change the return type to object, which is unreasonable.

Tishka17 · 2024-11-21T16:27:16Z

it is an LSP violation and unsound.

@srittau Can you please give us a reference to prove it? According to PEP207 it is not required to return bool for any comparison operator. I do not see there any difference between == and <= operators there

srittau · 2024-11-21T16:55:50Z

__le__ (and the other comparison operators) is not defined on object in typeshed, which is likely an oversight and should probably be fixed, although that might prove too disruptive. Apart from that I'm not interested in discussing this any further.

jonasrauber changed the title ~~> @JukkaL commented on Feb 1, 2017~~ object.__eq__ and other comparisons should have return type Union[NoReturn, Any] Jan 29, 2020

jonasrauber changed the title ~~object.__eq__ and other comparisons should have return type Union[NoReturn, Any]~~ object.__eq__ and other comparisons should have return type Any Jan 29, 2020

BvB93 mentioned this issue Apr 25, 2021

ENH: Add annotations to ndarray and generic numpy/numpy#17368

Closed

5 tasks

srittau mentioned this issue Feb 8, 2022

Should object.__eq__ return object? #7150

Closed

srittau closed this as completed Feb 8, 2022

lucascolley mentioned this issue Mar 28, 2025

add basedmypy, basedpyright quantity-dev/metrology-apis#13

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

object.eq and other comparisons should have return type Any #3685

object.eq and other comparisons should have return type Any #3685

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

object.__eq__ and other comparisons should have return type Any #3685

object.__eq__ and other comparisons should have return type Any #3685

Comments

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

object.eq and other comparisons should have return type Any #3685

object.eq and other comparisons should have return type Any #3685