8000 bpo-43757: make pathlib use os.path.realpath() to resolve all symlinks in a path by barneygale · Pull Request #25264 · python/cpython · GitHub
[go: up one dir, main page]
Skip to content

bpo-43757: make pathlib use os.path.realpath() to resolve all symlinks in a path #25264

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 23 commits into from
Apr 28, 2021
Merged

bpo-43757: make pathlib use os.path.realpath() to resolve all symlinks in a path #25264

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
164dbfd
bpo-43757: make pathlib use os.path.realpath() to resolve all symlink…
barneygale Apr 7, 2021
2894e0a
Adjust `os.path.realpath` to match `pathlib.Path.resolve()` behaviour…
barneygale Apr 8, 2021
495bd8b
Call `os.stat()` to raise `OSError(errno.ELOOP, ...)`.
barneygale Apr 8, 2021
492c66b
Remove unused import
barneygale Apr 8, 2021
c8bac15
Stop ignoring ERROR_CANT_RESOLVE_FILENAME error in _getfinalpathname_…
barneygale Apr 8, 2021
fe69688
Fix symlink detection and `posixpath` tests.
barneygale Apr 8, 2021
4114be9
Raise RuntimeError in Path.resolve() if realpath() raises ERROR_CANT_…
barneygale Apr 8, 2021
2a594ce
Remove unused import
barneygale Apr 8, 2021
fb36a40
Make documentation a little more general to accurately cover Windows.
barneygale Apr 8, 2021
0634486
First pass on fixing `ntpath` tests
barneygale Apr 8, 2021
600fc9e
Second pass on fixing `ntpath` tests
barneygale Apr 8, 2021
047471c
Fix indentation
barneygale Apr 8, 2021
bf3a3f7
Copy description of 'strict' from pathlib.
barneygale Apr 8, 2021
f839db0
Add tests
barneygale Apr 8, 2021
2d30bb7
Add NEWS
barneygale Apr 8, 2021
68c6533
Fix ntpath tests (pass 1)
barneygale Apr 8, 2021
9fa60eb
Add some notes on OS differences
barneygale Apr 10, 2021
f979947
Split NEWS
barneygale Apr 10, 2021
a82bc18
Do not suppress initial ERROR_CANT_RESOLVE_FILENAME error in non-stri…
barneygale Apr 10, 2021
86c318c
realpath(): only raise OSError for symlink loops in strict mode
barneygale Apr 13, 2021
7362f7e
Fix test_ntpath (pass 1)
barneygale Apr 13, 2021
72147c5
Fix test_ntpath (pass 2)
barneygale Apr 13, 2021
df04357
Split up exception handling in resolve() for greater clarity.
barneygale Apr 13, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
realpath(): only raise OSError for symlink loops in strict mode
  • Loading branch information
@barneygale
barneygale committed Apr 24, 2021
commit 86c318c61c8b9eb29e6f609af46fe25dcfd3ea61
11 changes: 4 additions & 7 deletions Doc/library/os.path.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -350,9 +350,10 @@ the :mod:`glob` module.)
links encountered in the path (if they are supported by the operating
system).

If the path doesn't exist and *strict* is ``True``, :exc:`FileNotFoundError`
is raised. If *strict* is ``False``, the path is resolved as far as possible
and any remainder is appended without checking whether it exists.
If a path doesn't exist or a symlink loop is encountered, and *strict* is
``True``, :exc:`OSError` is raised. If *strict* is ``False``, the path is
resolved as far as possible and any remainder is appended without checking
whether it exists.

.. note::
This function emulates the operating system's procedure for making a path
Expand All @@ -371,10 +372,6 @@ the :mod:`glob` module.)
.. versionchanged:: 3.10
The *strict* parameter was added.

.. versionchanged:: 3.10
Raises :exc:`OSError` when a symbolic link cycle occurs. Previously
returned one member of the cycle.


.. function:: relpath(path, start=os.curdir)

Expand Down
8 changes: 3 additions & 5 deletions Lib/ntpath.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -603,7 +603,8 @@ def _getfinalpathname_nonstrict(path):
# 87: ERROR_INVALID_PARAMETER
# 123: ERROR_INVALID_NAME
# 1920: ERROR_CANT_ACCESS_FILE
allowed_winerror = 1, 2, 3, 5, 21, 32, 50, 67, 87, 123, 1920
# 1921: ERROR_CANT_RESOLVE_FILENAME (implies unfollowable symlink)
allowed_winerror = 1, 2, 3, 5, 21, 32, 50, 67, 87, 123, 1920, 1921

# Non-strict algorithm is to find as much of the target directory
# as we can and join the rest.
Expand Down Expand Up @@ -659,10 +660,7 @@ def realpath(path, *, strict=False):
path = _getfinalpathname(path)
initial_winerror = 0
except OSError as ex:
# ERROR_CANT_RESOLVE_FILENAME (1921) is from exceeding the
# max allowed number of reparse attempts (currently 63), which
# is either due to a loop or a chain of links that's too long.
if strict or ex.winerror == 1921:
if strict:
raise
initial_winerror = ex.winerror
path = _getfinalpathname_nonstrict(path)
Expand Down
18 changes: 13 additions & 5 deletions Lib/pathlib.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1055,14 +1055,22 @@ def resolve(self, strict=False):
normalizing it (for example turning slashes into backslashes under
Windows).
"""

try:
p = self._accessor.realpath(self, strict=strict)
s = self._accessor.realpath(self, strict=strict)
p = self._from_parts((s,))

# In non-strict mode, realpath() doesn't raise on symlink loops.
# Ensure we get an exception by calling stat()
if not strict:
p.stat()
except OSError as ex:
winerr = getattr(ex, 'winerror', 0)
if ex.errno == ELOOP or winerr == _WINERROR_CANT_RESOLVE_FILENAME:
winerror = getattr(ex, 'winerror', 0)
if ex.errno == ELOOP or winerror == _WINERROR_CANT_RESOLVE_FILENAME:
raise RuntimeError("Symlink loop from %r", ex.filename)
raise
return self._from_parts((p,))
if strict:
raise
return p

def stat(self, *, follow_symlinks=True):
"""
Expand Down
16 changes: 11 additions & 5 deletions Lib/posixpath.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -391,7 +391,7 @@ def realpath(filename, *, strict=False):
"""Return the canonical path of the specified filename, eliminating any
symbolic links encountered in the path."""
filename = os.fspath(filename)
path = _joinrealpath(filename[:0], filename, strict, {})
path, ok = _joinrealpath(filename[:0], filename, strict, {})
return abspath(path)

# Join two paths, normalizing and eliminating any symbolic links
Expand Down Expand Up @@ -444,13 +444,19 @@ def _joinrealpath(path, rest, strict, seen):
# use cached value
continue
# The symlink is not resolved, so we must have a symlink loop.
# Raise OSError(errno.ELOOP)
os.stat(newpath)
if strict:
# Raise OSError(errno.ELOOP)
os.stat(newpath)
else:
# Return already resolved part + rest of the path unchanged.
return join(newpath, rest), False
seen[newpath] = None # not resolved symlink
path = _joinrealpath(path, os.readlink(newpath), strict, seen)
path, ok = _joinrealpath(path, os.readlink(newpath), strict, seen)
if not ok:
return join(path, rest), False
seen[newpath] = path # resolved symlink

return path
return path, True


supports_unicode_filenames = (sys.platform == 'darwin')
Expand Down
72 changes: 61 additions & 11 deletions Lib/test/test_ntpath.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -351,7 +351,9 @@ def test_realpath_broken_symlinks(self):
@os_helper.skip_unless_symlink
@unittest.skipUnless(HAVE_GETFINALPATHNAME, 'need _getfinalpathname')
def test_realpath_symlink_loops(self):
# Symlink loops raise OSError
# Symlink loops in non-strict mode are non-deterministic as to which
# path is returned, but it will always be the fully resolved path of
# one member of the cycle
ABSTFN = ntpath.abspath(os_helper.TESTFN)
self.addCleanup(os_helper.unlink, ABSTFN)
self.addCleanup(os_helper.unlink, ABSTFN + "1")
Expand All @@ -361,16 +363,16 @@ def test_realpath_symlink_loops(self):
self.addCleanup(os_helper.unlink, ABSTFN + "a")

os.symlink(ABSTFN, ABSTFN)
self.assertRaises(OSError, ntpath.realpath, ABSTFN)
self.assertPathEqual(ntpath.realpath(ABSTFN), ABSTFN)

os.symlink(ABSTFN + "1", ABSTFN + "2")
os.symlink(ABSTFN + "2", ABSTFN + "1")
self.assertRaises(OSError, ntpath.realpath, ABSTFN + "1")
self.assertRaises(OSError, ntpath.realpath, ABSTFN + "2")
self.assertRaises(OSError, ntpath.realpath, ABSTFN + "1\\x")
expected = (ABSTFN + "1", ABSTFN + "2")
self.assertPathIn(ntpath.realpath(ABSTFN + "1"), expected)
self.assertPathIn(ntpath.realpath(ABSTFN + "2"), expected)

# Windows eliminates '..' components before resolving links, so the
# following 3 realpath() calls are not expected to raise.
self.assertPathIn(ntpath.realpath(ABSTFN + "1\\x"),
(ntpath.join(r, "x") for r in expected))
self.assertPathEqual(ntpath.realpath(ABSTFN + "1\\.."),
ntpath.dirname(ABSTFN))
self.assertPathEqual(ntpath.realpath(ABSTFN + "1\\..\\x"),
Expand All @@ -379,18 +381,66 @@ def test_realpath_symlink_loops(self):
self.assertPathEqual(ntpath.realpath(ABSTFN + "1\\..\\"
+ ntpath.basename(ABSTFN) + "y"),
ABSTFN + "x")
self.assertPathIn(ntpath.realpath(ABSTFN + "1\\..\\"
+ ntpath.basename(ABSTFN) + "1"),
expected)

os.symlink(ntpath.basename(ABSTFN) + "a\\b", ABSTFN + "a")
self.assertPathEqual(ntpath.realpath(ABSTFN + "a"), ABSTFN + "a")

os.symlink("..\\" + ntpath.basename(ntpath.dirname(ABSTFN))
+ "\\" + ntpath.basename(ABSTFN) + "c", ABSTFN + "c")
self.assertPathEqual(ntpath.realpath(ABSTFN + "c"), ABSTFN + "c")

# Test using relative path as well.
self.assertPathEqual(ntpath.realpath(ntpath.basename(ABSTFN)), ABSTFN)

@os_helper.skip_unless_symlink
@unittest.skipUnless(HAVE_GETFINALPATHNAME, 'need _getfinalpathname')
def test_realpath_symlink_loops_strict(self):
# Symlink loops raise OSError in strict mode
ABSTFN = ntpath.abspath(os_helper.TESTFN)
self.addCleanup(os_helper.unlink, ABSTFN)
self.addCleanup(os_helper.unlink, ABSTFN + "1")
self.addCleanup(os_helper.unlink, ABSTFN + "2")
self.addCleanup(os_helper.unlink, ABSTFN + "y")
self.addCleanup(os_helper.unlink, ABSTFN + "c")
self.addCleanup(os_helper.unlink, ABSTFN + "a")

os.symlink(ABSTFN, ABSTFN)
self.assertRaises(OSError, ntpath.realpath, ABSTFN, strict=True)

os.symlink(ABSTFN + "1", ABSTFN + "2")
os.symlink(ABSTFN + "2", ABSTFN + "1")
self.assertRaises(OSError, ntpath.realpath, ABSTFN + "1", strict=True)
self.assertRaises(OSError, ntpath.realpath, ABSTFN + "2", strict=True)
self.assertRaises(OSError, ntpath.realpath, ABSTFN + "1\\x", strict=True)

# Windows eliminates '..' components before resolving links, so the
# following 3 realpath() calls are not expected to raise.
self.assertPathEqual(ntpath.realpath(ABSTFN + "1\\..", strict=True),
ntpath.dirname(ABSTFN))
self.assertPathEqual(ntpath.realpath(ABSTFN + "1\\..\\x", strict=True),
ntpath.dirname(ABSTFN) + "\\x")
os.symlink(ABSTFN + "x", ABSTFN + "y")
self.assertPathEqual(ntpath.realpath(ABSTFN + "1\\..\\"
+ ntpath.basename(ABSTFN) + "y",
strict=True),
ABSTFN + "x")
self.assertRaises(OSError, ntpath.realpath,
ABSTFN + "1\\..\\" + ntpath.basename(ABSTFN) + "1")
ABSTFN + "1\\..\\" + ntpath.basename(ABSTFN) + "1",
strict=True)

os.symlink(ntpath.basename(ABSTFN) + "a\\b", ABSTFN + "a")
self.assertRaises(OSError, ntpath.realpath, ABSTFN + "a")
self.assertRaises(OSError, ntpath.realpath, ABSTFN + "a", strict=True)

os.symlink("..\\" + ntpath.basename(ntpath.dirname(ABSTFN))
+ "\\" + ntpath.basename(ABSTFN) + "c", ABSTFN + "c")
self.assertRaises(OSError, ntpath.realpath, ABSTFN + "c")
self.assertRaises(OSError, ntpath.realpath, ABSTFN + "c", strict=True)

# Test using relative path as well.
self.assertRaises(OSError, ntpath.realpath, ntpath.basename(ABSTFN))
self.assertRaises(OSError, ntpath.realpath, ntpath.basename(ABSTFN),
strict=True)

@os_helper.skip_unless_symlink
@unittest.skipUnless(HAVE_GETFINALPATHNAME, 'need _getfinalpathname')
Expand Down
69 changes: 57 additions & 12 deletions Lib/test/test_posixpath.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -382,33 +382,78 @@ def test_realpath_relative(self):
"Missing symlink implementation")
@skip_if_ABSTFN_contains_backslash
def test_realpath_symlink_loops(self):
# Bug #43757, raise OSError if we get into an infinite symlink loop.
# Bug #930024, return the path unchanged if we get into an infinite
# symlink loop in non-strict mode (default).
try:
os.symlink(ABSTFN, ABSTFN)
self.assertRaises(OSError, realpath, ABSTFN)
self.assertEqual(realpath(ABSTFN), ABSTFN)

os.symlink(ABSTFN+"1", ABSTFN+"2")
os.symlink(ABSTFN+"2", ABSTFN+"1")
self.assertRaises(OSError, realpath, ABSTFN+"1")
self.assertRaises(OSError, realpath, ABSTFN+"2")
self.assertEqual(realpath(ABSTFN+"1"), ABSTFN+"1")
self.assertEqual(realpath(ABSTFN+"2"), ABSTFN+"2")

self.assertRaises(OSError, realpath, ABSTFN+"1/x")
self.assertRaises(OSError, realpath, ABSTFN+"1/..")
self.assertRaises(OSError, realpath, ABSTFN+"1/../x")
self.assertEqual(realpath(ABSTFN+"1/x"), ABSTFN+"1/x")
self.assertEqual(realpath(ABSTFN+"1/.."), dirname(ABSTFN))
self.assertEqual(realpath(ABSTFN+"1/../x"), dirname(ABSTFN) + "/x")
os.symlink(ABSTFN+"x", ABSTFN+"y")
self.assertRaises(OSError, realpath, ABSTFN+"1/../" + basename(ABSTFN) + "y")
self.assertRaises(OSError, realpath, ABSTFN+"1/../" + basename(ABSTFN) + "1")
self.assertEqual(realpath(ABSTFN+"1/../" + basename(ABSTFN) + "y"),
ABSTFN + "y")
self.assertEqual(realpath(ABSTFN+"1/../" + basename(ABSTFN) + "1"),
ABSTFN + "1")

os.symlink(basename(ABSTFN) + "a/b", ABSTFN+"a")
self.assertRaises(OSError, realpath, ABSTFN+"a")
self.assertEqual(realpath(ABSTFN+"a"), ABSTFN+"a/b")

os.symlink("../" + basename(dirname(ABSTFN)) + "/" +
basename(ABSTFN) + "c", ABSTFN+"c")
self.assertRaises(OSError, realpath, ABSTFN+"c")
self.assertEqual(realpath(ABSTFN+"c"), ABSTFN+"c")

# Test using relative path as well.
with os_helper.change_cwd(dirname(ABSTFN)):
self.assertRaises(OSError, realpath, basename(ABSTFN))
self.assertEqual(realpath(basename(ABSTFN)), ABSTFN)
finally:
os_helper.unlink(ABSTFN)
os_helper.unlink(ABSTFN+"1")
os_helper.unlink(ABSTFN+"2")
os_helper.unlink(ABSTFN+"y")
os_helper.unlink(ABSTFN+"c")
os_helper.unlink(ABSTFN+"a")

@unittest.skipUnless(hasattr(os, "symlink"),
"Missing symlink implementation")
@skip_if_ABSTFN_contains_backslash
def test_realpath_symlink_loops_strict(self):
# Bug #43757, raise OSError if we get into an infinite symlink loop in
# strict mode.
try:
os.symlink(ABSTFN, ABSTFN)
self.assertRaises(OSError, realpath, ABSTFN, strict=True)

os.symlink(ABSTFN+"1", ABSTFN+"2")
os.symlink(ABSTFN+"2", ABSTFN+"1")
self.assertRaises(OSError, realpath, ABSTFN+"1", strict=True)
self.assertRaises(OSError, realpath, ABSTFN+"2", strict=True)

self.assertRaises(OSError, realpath, ABSTFN+"1/x", strict=True)
self.assertRaises(OSError, realpath, ABSTFN+"1/..", strict=True)
self.assertRaises(OSError, realpath, ABSTFN+"1/../x", strict=True)
os.symlink(ABSTFN+"x", ABSTFN+"y")
self.assertRaises(OSError, realpath,
ABSTFN+"1/../" + basename(ABSTFN) + "y", strict=True)
self.assertRaises(OSError, realpath,
ABSTFN+"1/../" + basename(ABSTFN) + "1", strict=True)

os.symlink(basename(ABSTFN) + "a/b", ABSTFN+"a")
self.assertRaises(OSError, realpath, ABSTFN+"a", strict=True)

os.symlink("../" + basename(dirname(ABSTFN)) + "/" +
basename(ABSTFN) + "c", ABSTFN+"c")
self.assertRaises(OSError, realpath, ABSTFN+"c", strict=True)

# Test using relative path as well.
with os_helper.change_cwd(dirname(ABSTFN)):
self.assertRaises(OSError, realpath, basename(ABSTFN), strict=True)
finally:
os_helper.unlink(ABSTFN)
os_helper.unlink(ABSTFN+"1")
Expand Down
4 changes: 2 additions & 2 deletions Misc/NEWS.d/next/Library/2021-04-08-22-11-27.bpo-25264.b33fa0.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
:func:`os.path.realpath` now accepts a *strict* keyword-only argument.
When set to ``True``, :exc:`FileNotFoundException` is raised if a path
doesn't exist.
When set to ``True``, :exc:`OSError` is raised if a path doesn't exist
or a symlink loop is encountered.
2 changes: 0 additions & 2 deletions Misc/NEWS.d/next/Library/2021-04-08-22-11-27.bpo-25264.hEqA5d.rst
View file
Open in desktop

This file was deleted.

0