8000 Convert AbstractContextManager from a protocol to an ABC by NeilGirdhar · Pull Request #11269 · python/typeshed · GitHub
[go: up one dir, main page]
Skip to content

Convert AbstractContextManager from a protocol to an ABC #11269

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

Convert AbstractContextManager from a protocol to an ABC #11269

wants to merge 1 commit into from

Conversation

NeilGirdhar
Copy link
Contributor
@NeilGirdhar NeilGirdhar commented Jan 12, 2024 •
edited
Loading

AbstractContextManager should not be a protocol or else the exit method, which is abstract, cannot be determined by type checkers to
be implemented. This prevents users from calling super. For details, see microsoft/pyright#6965 (comment) and microsoft/pyright#6965

@NeilGirdhar
8000 Copy link
Contributor Author

This decorator also caused a lot of confusion here: pylint-dev/pylint#1594

@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@NeilGirdhar
Copy link
Contributor Author

See https://discuss.python.org/t/can-we-make-abstractcontextmanager-exit-concrete/43025

@srittau
Copy link
Collaborator
srittau commented Jan 12, 2024

Apart from the necessary stubtest changes (and comment there), could you also add a comment to the stub to explain the situation for posterity?

@NeilGirdhar
Copy link
Contributor Author

Happy to do that, but there is some debate as to whether this is the right change, so I'll just wait until it's resolved.

@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
JelleZijlstra
JelleZijlstra reviewed Jan 12, 2024
View reviewed changes
stdlib/contextlib.pyi Outdated
@@ -40,7 +40,9 @@ _CM_EF = TypeVar("_CM_EF", bound=AbstractContextManager[Any] | _ExitFunc)
@runtime_checkable
class AbstractContextManager(Protocol[_T_co]):
def __enter__(self) -> _T_co: ...
@abstractmethod
# While __exit__ is decorated with @abstractmethod in the runtime, it is also implemented. Having the same decoration in the
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whatever the outcome, we should make the same change to AbstractAsyncContextManager.__aexit__.

@NeilGirdhar
Convert AbstractContextManager to an ABC
52758da 
AbstractContextManager should not be a protocol or else the exit method,
which is abstract, cannot be determined by type checkers to be
implemented.  This prevents users from calling super.  For details, see
this comment:
microsoft/pyright#6965 (comment)
@NeilGirdhar NeilGirdhar changed the title Make AbstractContextManager.__exit__ concrete Convert AbstractContextManager from a protocol to an ABC Jan 13, 2024
@github-actions GitHub Actions
Copy link
Contributor

Diff from mypy_primer, showing the effect of this PR on open source code:

pip (https://github.com/pypa/pip)
+ src/pip/_internal/utils/temp_dir.py:147: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "TempDirectory"; expected "AbstractContextManager[Never]"  [arg-type]
+ src/pip/_internal/operations/build/build_tracker.py:46: error: Never has no attribute "path"  [attr-defined]
+ src/pip/_internal/operations/build/build_tracker.py:46: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "TempDirectory"; expected "AbstractContextManager[Never]"  [arg-type]
+ src/pip/_internal/cli/req_command.py:95: error: Argument 1 to "enter_context" of "CommandContextMixIn" has incompatible type "PipSession"; expected "AbstractContextManager[Never]"  [arg-type]
+ src/pip/_internal/commands/install.py:320: error: Argument 1 to "enter_context" of "CommandContextMixIn" has incompatible type "TempDirectory"; expected "AbstractContextManager[Never]"  [arg-type]

prefect (https://github.com/PrefectHQ/prefect)
+ src/prefect/engine.py:1765: error: Argument 1 to "enter_async_context" of "AsyncExitStack" has incompatible type "PrefectClient"; expected "AbstractAsyncContextManager[PrefectClient]"  [arg-type]
+ src/prefect/engine.py:1765: error: Argument 1 to "enter_async_context" of "AsyncExitStack" has incompatible type "PrefectClient"; expected "AbstractAsyncContextManager[PrefectClient]"  [arg-type]

trio (https://github.com/python-trio/trio)
+ src/trio/_core/_run.py:1015: error: Incompatible return value type (got "NurseryManager", expected "AbstractAsyncContextManager[Nursery]")  [return-value]
+ src/trio/_tests/test_timeouts.py:130: error: "object" has no attribute "__enter__"  [attr-defined]
+ src/trio/_tests/test_timeouts.py:130: error: "object" has no attribute "__exit__"  [attr-defined]
+ src/trio/_core/_tests/test_run.py:735: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "CancelScope"; expected "AbstractContextManager[Never]"  [arg-type]
+ src/trio/_core/_tests/test_run.py:754: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "CancelScope"; expected "AbstractContextManager[Never]"  [arg-type]
+ src/trio/_core/_tests/test_run.py:2360: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "CancelScope"; expected "AbstractContextManager[Never]"  [arg-type]
+ src/trio/_core/_tests/test_run.py:2362: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "CancelScope"; expected "AbstractContextManager[Never]"  [arg-type]

tornado (https://github.com/tornadoweb/tornado)
+ tornado/template.py:752: error: Incompatible return value type (got "Indenter", expected "AbstractContextManager[Any]")  [return-value]
+ tornado/template.py:765: error: Incompatible return value type (got "IncludeTemplate", expected "AbstractContextManager[Any]")  [return-value]

pylint (https://github.com/pycqa/pylint)
+ pylint/lint/pylinter.py:431: error: Need type annotation for "output_file"  [var-annotated]
+ pylint/lint/pylinter.py:432: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "TextIOWrapper"; expected "AbstractContextManager[Never]"  [arg-type]

anyio (https://github.com/agronholm/anyio)
+ src/anyio/abc/_sockets.py:145: error: Argument 1 to "enter_async_context" of "AsyncExitStack" has incompatible type "TaskGroup"; expected "AbstractAsyncContextManager[TaskGroup | None]"  [arg-type]
+ src/anyio/abc/_sockets.py:149: error: Item "None" of "TaskGroup | None" has no attribute "start_soon"  [union-attr]
+ src/anyio/pytest_plugin.py:45: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "TestRunner"; expected "AbstractContextManager[Never]"  [arg-type]
+ src/anyio/_backends/_trio.py:1108: error: Incompatible return value type (got "_SignalReceiver", expected "AbstractContextManager[AsyncIterator[Signals]]")  [return-value]
+ src/anyio/_backends/_asyncio.py:2437: error: Incompatible return value type (got "_SignalReceiver", expected "AbstractContextManager[AsyncIterator[Signals]]")  [return-value]

starlette (https://github.com/encode/starlette)
+ starlette/_utils.py:48: error: All bases of a protocol must be protocols  [misc]
+ starlette/routing.py:658: error: Incompatible types in assignment (expression has type "_DefaultLifespan", variable has type "Union[Callable[[Any], AbstractAsyncContextManager[None]], Callable[[Any], AbstractAsyncContextManager[Mapping[str, Any]]]]")  [assignment]

Tanjun (https://github.com/FasterSpeeding/Tanjun)
+ tanjun/dependencies/data.py:128: error: Incompatible return value type (got "Lock", expec
8000
ted "AbstractAsyncContextManager[Any]")  [return-value]
+ tanjun/dependencies/limiters.py:215: error: Incompatible return value type (got "_CooldownAcquire", expected "AbstractAsyncContextManager[None]")  [return-value]
+ tanjun/dependencies/limiters.py:340: error: Incompatible return value type (got "_ConcurrencyAcquire", expected "AbstractAsyncContextManager[None]")  [return-value]

mkosi (https://github.com/systemd/mkosi)
+ mkosi/qemu.py:637:25: error: Need type annotation for "ovmf_vars"  [var-annotated]
+ mkosi/qemu.py:637:45: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "_TemporaryFileWrapper[bytes]"; expected "AbstractContextManager[Never]"  [arg-type]
+ mkosi/qemu.py:703:21: error: Need type annotation for "f"  [var-annotated]
+ mkosi/qemu.py:703:41: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "_TemporaryFileWrapper[str]"; expected "AbstractContextManager[Never]"  [arg-type]
+ mkosi/qemu.py:747:23: error: Need type annotation for "scratch"  [var-annotated]
+ mkosi/qemu.py:747:43: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "_TemporaryFileWrapper[bytes]"; expected "AbstractContextManager[Never]"  [arg-type]
+ mkosi/qemu.py:808:20: error: Need type annotation for "file"  [var-annotated]
+ mkosi/qemu.py:809:17: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "_TemporaryFileWrapper[bytes]"; expected "AbstractContextManager[Never]"  [arg-type]
+ mkosi/mounts.py:83:49: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "TemporaryDirectory[str]"; expected "AbstractContextManager[Never]"  [arg-type]
+ mkosi/mounts.py:89:33: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "TemporaryDirectory[str]"; expected "AbstractContextManager[Never]"  [arg-type]
+ mkosi/mounts.py:95:21: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "TemporaryDirectory[str]"; expected "AbstractContextManager[Never]"  [arg-type]
+ mkosi/__init__.py:3024:23: error: Need type annotation for "scratch"  [var-annotated]
+ mkosi/__init__.py:3024:43: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "TemporaryDirectory[str]"; expected "AbstractContextManager[Never]"  [arg-type]

pandas-stubs (https://github.com/pandas-dev/pandas-stubs)
+ tests/__init__.py:126: error: Incompatible return value type (got "WarningsChecker", expected "AbstractContextManager[Any]")  [return-value]

nionutils (https://github.com/nion-software/nionutils)
+ nion/utils/ReferenceCounting.py:51: error: Incompatible return value type (got "RefContextManager", expected "AbstractContextManager[ReferenceCounted]")  [return-value]
+ nion/utils/ListModel.py:332: error: Incompatible return value type (got "ChangeTracker", expected "AbstractContextManager[ChangeTracker]")  [return-value]
+ nion/utils/ListModel.py:744: error: Incompatible return value type (got "ChangeTracker", expected "AbstractContextManager[ChangeTracker]")  [return-value]
+ nion/utils/Stream.py:61: error: Incompatible return value type (got "RefContextManager", expected "AbstractContextManager[AbstractStream[T]]")  [return-value]

mongo-python-driver (https://github.com/mongodb/mongo-python-driver)
+ pymongo/client_session.py:755: error: Incompatible return value type (got "_TransactionContext", expected "AbstractContextManager[Any]")  [return-value]

urllib3 (https://github.com/urllib3/urllib3)
+ test/with_dummyserver/test_https.py:739: error: Incompatible types in assignment (expression has type "WarningsChecker", variable has type "AbstractContextManager[object]")  [assignment]

streamlit (https://github.com/streamlit/streamlit)
+ lib/tests/streamlit/web/server/server_test.py: note: In member "test_missing_file" of class "SslServerTest":
+ lib/tests/streamlit/web/server/server_test.py:374:23: error: Need type annotation for "tmp_dir"  [var-annotated]
+ lib/tests/streamlit/web/server/server_test.py:374:48: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "TemporaryDirectory[str]"; expected "AbstractContextManager[Never]"  [arg-type]
+ lib/tests/streamlit/web/server/server_test.py:389:20: error: Need type annotation for "logs"  [var-annotated]
+ lib/tests/streamlit/web/server/server_test.py:390:17: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "_AssertLogsContext[_LoggingWatcher]"; expected "AbstractContextManager[Never]"  [arg-type]
+ lib/tests/streamlit/web/server/server_test.py: note: In member "test_invalid_file_content" of class "SslServerTest":
+ lib/tests/streamlit/web/server/server_test.py:407:23: error: Need type annotation for "tmp_dir"  [var-annotated]
+ lib/tests/streamlit/web/server/server_test.py:407:48: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "TemporaryDirectory[str]"; expected "AbstractContextManager[Never]"  [arg-type]
+ lib/tests/streamlit/web/server/server_test.py:444:20: error: Need type annotation for "logs"  [var-annotated]
+ lib/tests/streamlit/web/server/server_test.py:445:17: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "_AssertLogsContext[_LoggingWatcher]"; expected "AbstractContextManager[Never]"  [arg-type]
+ lib/tests/streamlit/web/cli_test.py: note: In member "test_ssl" of class "HTTPServerIntegrationTest":
+ lib/tests/streamlit/web/cli_test.py:451:24: error: Need type annotation for "tmp_home"  [var-annotated]
+ lib/tests/streamlit/web/cli_test.py:451:49: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "TemporaryDirectory[str]"; expected "AbstractContextManager[Never]"  [arg-type]
+ lib/tests/streamlit/web/cli_test.py:495:29: error: Need type annotation for "https_session"  [var-annotated]
+ lib/tests/streamlit/web/cli_test.py:495:54: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "Session"; expected "AbstractContextManager[Never]"  [arg-type]
+ lib/tests/streamlit/web/cli_test.py:496:20: error: Need type annotation for "proc"  [var-annotated]
+ lib/tests/streamlit/web/cli_test.py:497:17: error: Argument 1 to "enter_context" of "ExitStack" has incompatible type "Popen[bytes]"; expected "AbstractContextManager[Never]"  [arg-type]

@NeilGirdhar NeilGirdhar marked this pull request as draft January 13, 2024 02:45
@NeilGirdhar
Copy link
Contributor Author
NeilGirdhar commented Jan 13, 2024 •
edited
Loading

Making this a draft. In an ideal world, we would have Self be a supported generic parameter? (Sorry for the noise.)

@JelleZijlstra
Copy link
Member

Closing this as it currently breaks more than it fixes and the PR has been open for a long time.

@tungol tungol mentioned this pull request Nov 12, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@JelleZijlstra JelleZijlstra JelleZijlstra left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@NeilGirdhar @srittau @JelleZijlstra
0