From 3b1af8643a04920d6813a9429fc9c081710c396b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Niklas=20Hamb=C3=BCchen?= Date: Sat, 28 Jun 2025 09:59:58 +0200 Subject: [PATCH] docs: pathlib: Mention that iterdir() is surprisingly not streaming This undocumented gotcha can cause excessive memory usage when "iterating" over very large directories. This is because iterdir() does entries = list(scandir_it) Fixing (if at all desired) will likel need least significant amounts of discussion and testing, so first document the behaviour. --- Doc/library/pathlib.rst | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/Doc/library/pathlib.rst b/Doc/library/pathlib.rst index 47986a2d9602ee..ff99b12d4ce101 100644 --- a/Doc/library/pathlib.rst +++ b/Doc/library/pathlib.rst @@ -1328,6 +1328,12 @@ Reading directories If the path is not a directory or otherwise inaccessible, :exc:`OSError` is raised. + .. warning:: + Despite its name and being a generator, this function currently + does not stream directory entries: It reads all directory + entries into memory before yielding the first entry. + If you need constant-memory iteration across a large number of + directory entries, use :func:`os.scandir` instead. .. method:: Path.glob(pattern, *, case_sensitive=None, recurse_symlinks=False)