8000 bpo-28468: Add platform.freedesktop_osrelease by tiran · Pull Request #23492 · python/cpython · GitHub
[go: up one dir, main page]

Skip to content

bpo-28468: Add platform.freedesktop_osrelease #23492

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Nov 30, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions Doc/library/platform.rst
Original file line number Diff line number Diff line change
Expand Up @@ -253,3 +253,41 @@ Unix Platforms
using :program:`gcc`.

The file is read and scanned in chunks of *chunksize* bytes.


Linux Platforms
---------------

.. function:: freedesktop_os_release()

Get operating system identification from ``os-release`` file and return
it as a dict. The ``os-release`` file is a `freedesktop.org standard
<https://www.freedesktop.org/software/systemd/man/os-release.html>`_ and
is available in most Linux distributions. A noticeable exception is
Android and Android-based distributions.

Raises :exc:`OSError` or subclass when neither ``/etc/os-release`` nor
``/usr/lib/os-release`` can be read.

On success, the function returns a dictionary where keys and values are
strings. Values have their special characters like ``"`` and ``$``
unquoted. The fields ``NAME``, ``ID``, and ``PRETTY_NAME`` are always
defined according to the standard. All other fields are optional. Vendors
may include additional fields.

Note that fields like ``NAME``, ``VERSION``, and ``VARIANT`` are strings
suitable for presentation to users. Programs should use fields like
``ID``, ``ID_LIKE``, ``VERSION_ID``, or ``VARIANT_ID`` to identify
Linux distributions.

Example::

def get_like_distro():
info = platform.freedesktop_os_release()
ids = [info["ID"]]
if "ID_LIKE" in info:
# ids are space separated and ordered by precedence
ids.extend(info["ID_LIKE"].split())
return ids

.. versionadded:: 3.10
8 changes: 8 additions & 0 deletions Doc/whatsnew/3.10.rst
Original file line number Diff line number Diff line change
Expand Up @@ -254,6 +254,14 @@ Added negative indexing support to :attr:`PurePath.parents
<pathlib.PurePath.parents>`.
(Contributed by Yaroslav Pankovych in :issue:`21041`)

platform
--------

Added :func:`platform.freedesktop_os_release()` to retrieve operation system
identification from `freedesktop.org os-release
<https://www.freedesktop.org/software/systemd/man/os-release.html>`_ standard file.
(Contributed by Christian Heimes in :issue:`28468`)

py_compile
----------

Expand Down
57 changes: 57 additions & 0 deletions Lib/platform.py
Original file line number Diff line number Diff line change
Expand Up @@ -1230,6 +1230,63 @@ def platform(aliased=0, terse=0):
_platform_cache[(aliased, terse)] = platform
return platform

### freedesktop.org os-release standard
# https://www.freedesktop.org/software/systemd/man/os-release.html

# NAME=value with optional quotes (' or "). The regular expression is less
# strict than shell lexer, but that's ok.
_os_release_line = re.compile(
"^(?P<name>[a-zA-Z0-9_]+)=(?P<quote>[\"\']?)(?P<value>.*)(?P=quote)$"
)
# unescape five special characters mentioned in the standard
_os_release_unescape = re.compile(r"\\([\\\$\"\'`])")
# /etc takes precedence over /usr/lib
_os_release_candidates = ("/etc/os-release", "/usr/lib/os-relesase")

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
_os_release_candidates = ("/etc/os-release", "/usr/lib/os-relesase")
_os_release_candidates = ("/etc/os-release", "/usr/lib/os-release")

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ooops, nicely spotted :-) I created PR #23913 to fix it.

_os_release_cache = None


def _parse_os_release(lines):
# These fields are mandatory fields with well-known defaults
# in pratice all Linux distributions override NAME, ID, and PRETTY_NAME.
info = {
"NAME": "Linux",
"ID": "linux",
"PRETTY_NAME": "Linux",
}

for line in lines:
mo = _os_release_line.match(line)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The distro module uses shlex.shlex() to parse os-release.

lexer = shlex.shlex(lines, posix=True)
lexer.whitespace_split = True

The freedesktop specification says:

Shell special characters ("$", quotes, backslash, backtick) must be escaped with backslashes, following shell style.

It seems like your code and the distro module parse \" and \' differently for example.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please give an example.

Copy link
Member

Choose a reason for hiding this comment< 8000 /h3>

The reason will be displayed to describe this comment to others. Learn more.

Something like:

NAME='Fedora 19 (Schrödinger\'s Cat)'

https://fedoraproject.org/wiki/History_of_Fedora_release_names#Fedora_19_.28Schr.C3.B6dinger.27s_Cat.29

Note: I wrote the example manually, it doesn't come from a concrete os-release file.

if mo is not None:
info[mo.group('name')] = _os_release_unescape.sub(
r"\1", mo.group('value')
)

return info


def freedesktop_os_release():
"""Return operation system identification from freedesktop.org os-release
"""
global _os_release_cache

if _os_release_cache is None:
errno = None
for candidate in _os_release_candidates:
try:
with open(candidate, encoding="utf-8") as f:
_os_release_cache = _parse_os_release(f)
break
except OSError as e:
errno = e.errno
else:
raise OSError(
errno,
f"Unable to read files {', '.join(_os_release_candidates)}"
)

return _os_release_cache.copy()


### Command line interface

if __name__ == '__main__':
Expand Down
106 changes: 106 additions & 0 deletions Lib/test/test_platform.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,70 @@
from test import support
from test.support import os_helper

FEDORA_OS_RELEASE = """\
NAME=Fedora
VERSION="32 (Thirty Two)"
ID=fedora
VERSION_ID=32
VERSION_CODENAME=""
PLATFORM_ID="platform:f32"
PRETTY_NAME="Fedora 32 (Thirty Two)"
ANSI_COLOR="0;34"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:32"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f32/system-administrators-guide/"
SUPPORT_URL="https://fedoraproject.org/wiki/Communicating_and_getting_help"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=32
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=32
PRIVACY_POLICY_URL="https://fedoraproject.org/wiki/Legal:PrivacyPolicy"
"""

UBUNTU_OS_RELEASE = """\
NAME="Ubuntu"
VERSION="20.04.1 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.1 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
"""

TEST_OS_RELEASE = r"""
# test data
ID_LIKE="egg spam viking"
EMPTY=
# comments and empty lines are ignored

SINGLE_QUOTE='single'
EMPTY_SINGLE=''
DOUBLE_QUOTE="double"
EMPTY_DOUBLE=""
QUOTES="double\'s"
SPECIALS="\$\`\\\'\""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you mind to add a comment explaining that the format requires these 5 characters to be escaped with a blackslash? Maybe rename SPECIALS to ESCAPED.

# invalid lines
=invalid
=
INVALID
IN-VALID=value
IN VALID=value
"""


class PlatformTest(unittest.TestCase):
def clear_caches(self):
platform._platform_cache.clear()
platform._sys_version_cache.clear()
platform._uname_cache = N F438 one
platform._os_release_cache = None

def test_architecture(self):
res = platform.architecture()
Expand Down Expand Up @@ -382,6 +440,54 @@ def test_macos(self):
self.assertEqual(platform.platform(terse=1), expected_terse)
self.assertEqual(platform.platform(), expected)

def test_freedesktop_os_release(self):
self.addCleanup(self.clear_caches)
self.clear_caches()

if any(os.path.isfile(fn) for fn in platform._os_release_candidates):
info = platform.freedesktop_os_release()
self.assertIn("NAME", info)
self.assertIn("ID", info)

info["CPYTHON_TEST"] = "test"
self.assertNotIn(
"CPYTHON_TEST",
platform.freedesktop_os_release()
)
else:
with self.assertRaises(OSError):
platform.freedesktop_os_release()

def test_parse_os_release(self):
info = platform._parse_os_release(FEDORA_OS_RELEASE.splitlines())
self.assertEqual(info["NAME"], "Fedora")
self.assertEqual(info["ID"], "fedora")
self.assertNotIn("ID_LIKE", info)
self.assertEqual(info["VERSION_CODENAME"], "")

info = platform._parse_os_release(UBUNTU_OS_RELEASE.splitlines())
self.assertEqual(info["NAME"], "Ubuntu")
self.assertEqual(info["ID"], "ubuntu")
self.assertEqual(info["ID_LIKE"], "debian")
self.assertEqual(info["VERSION_CODENAME"], "focal")

info = platform._parse_os_release(TEST_OS_RELEASE.splitlines())
expected = {
"ID": "linux",
"NAME": "Linux",
"PRETTY_NAME": "Linux",
"ID_LIKE": "egg spam viking",
"EMPTY": "",
"DOUBLE_QUOTE": "double",
"EMPTY_DOUBLE": "",
"SINGLE_QUOTE": "single",
"EMPTY_SINGLE": "",
"QUOTES": "double's",
"SPECIALS": "$`\\'\"",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe write this specific test to make it more readable:

Suggested change
"SPECIALS": "$`\\'\"",
"SPECIALS": ''.join(['$', '`', '\\', "'", '"']),

So you can remove self.assertEqual(len(info["SPECIALS"]), 5).

}
self.assertEqual(info, expected)
self.assertEqual(len(info["SPECIALS"]), 5)


if __name__ == '__main__':
unittest.main()
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Add :func:`platform.freedesktop_os_release` function to parse freedesktop.org
``os-release`` files.
0