From c3d32bfdd232115943f8f1dd5fe8c25cae185d0f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?B=C3=A9n=C3=A9dikt=20Tran?= <10796600+picnixz@users.noreply.github.com> Date: Thu, 13 Jun 2024 15:01:02 +0200 Subject: [PATCH 01/18] improve documentation for private name mangling --- Doc/reference/expressions.rst | 32 +++++++++++++++++++++----------- 1 file changed, 21 insertions(+), 11 deletions(-) diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index 872773f4d28235..cc30bef3f8acf9 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -83,17 +83,27 @@ exception. pair: name; mangling pair: private; names -**Private name mangling:** When an identifier that textually occurs in a class -definition begins with two or more underscore characters and does not end in two -or more underscores, it is considered a :dfn:`private name` of that class. -Private names are transformed to a longer form before code is generated for -them. The transformation inserts the class name, with leading underscores -removed and a single underscore inserted, in front of the name. For example, -the identifier ``__spam`` occurring in a class named ``Ham`` will be transformed -to ``_Ham__spam``. This transformation is independent of the syntactical -context in which the identifier is used. If the transformed name is extremely -long (longer than 255 characters), implementation defined truncation may happen. -If the class name consists only of underscores, no transformation is done. +.. rubric:: Private name mangling + +When an identifier that textually occurs in a class definition begins with two +or more underscore characters and does not end in two or more underscores, it +is considered a :dfn:`private name` of that class. + +More precisely, private names are transformed to a longer form before code is +generated for them. The transformation rule is defined as follows: + +- If the class name consists only of underscores, no transformation is done, + e.g., the identifier ``__spam`` occurring in a class named ``_`` or ``__`` + is left as is. + +- Otherwise, the transformation inserts the class name, with leading + underscores removed and a single underscore inserted, in front of + the identifier, e.g., the identifier ``__spam`` occurring in a class + named ``Ham``, ``_Ham`` or ``__Ham`` is transformed to ``_Ham__spam``. + +The transformation is independent of the syntactical context in which the +identifier is used. Nonetheless, if the transformed name is extremely long +(longer than 255 characters), implementation-defined truncation may happen. .. _atom-literals: From ae8bead005d5a66da2dfcbab80e39cf0c43fddf5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?B=C3=A9n=C3=A9dikt=20Tran?= <10796600+picnixz@users.noreply.github.com> Date: Thu, 13 Jun 2024 15:41:09 +0200 Subject: [PATCH 02/18] add links --- Doc/faq/programming.rst | 3 ++- Doc/tutorial/classes.rst | 4 +++- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst index 0a88c5f6384f2b..1a77133326cc58 100644 --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -1739,7 +1739,8 @@ Variable names with double leading underscores are "mangled" to provide a simple but effective way to define class private variables. Any identifier of the form ``__spam`` (at least two leading underscores, at most one trailing underscore) is textually replaced with ``_classname__spam``, where ``classname`` is the -current class name with any leading underscores stripped. +current class name with any leading underscores stripped +(see :ref:`here ` for details and special cases). This doesn't guarantee privacy: an outside user can still deliberately access the "_classname__spam" attribute, and private values are visible in the object's diff --git a/Doc/tutorial/classes.rst b/Doc/tutorial/classes.rst index 1b64741c349ee9..e87f02160e1143 100644 --- a/Doc/tutorial/classes.rst +++ b/Doc/tutorial/classes.rst @@ -684,7 +684,7 @@ clashes of names with names defined by subclasses), there is limited support for such a mechanism, called :dfn:`name mangling`. Any identifier of the form ``__spam`` (at least two leading underscores, at most one trailing underscore) is textually replaced with ``_classname__spam``, where ``classname`` is the -current class name with leading underscore(s) stripped. This mangling is done +current class name with leading underscore(s) stripped. [#]_ This mangling is done without regard to the syntactic position of the identifier, as long as it occurs within the definition of a class. @@ -925,6 +925,8 @@ Examples:: .. rubric:: Footnotes +.. [#] See :ref:`here ` for details and special cases. + .. [#] Except for one thing. Module objects have a secret read-only attribute called :attr:`~object.__dict__` which returns the dictionary used to implement the module's namespace; the name :attr:`~object.__dict__` is an attribute but not a global name. From 8e138a49cbbb2bea4098bedfaed2bf63454a0761 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?B=C3=A9n=C3=A9dikt=20Tran?= <10796600+picnixz@users.noreply.github.com> Date: Thu, 13 Jun 2024 15:51:43 +0200 Subject: [PATCH 03/18] fix footnote --- Doc/tutorial/classes.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/tutorial/classes.rst b/Doc/tutorial/classes.rst index e87f02160e1143..d24ce6887c22c0 100644 --- a/Doc/tutorial/classes.rst +++ b/Doc/tutorial/classes.rst @@ -925,10 +925,10 @@ Examples:: .. rubric:: Footnotes -.. [#] See :ref:`here ` for details and special cases. - .. [#] Except for one thing. Module objects have a secret read-only attribute called :attr:`~object.__dict__` which returns the dictionary used to implement the module's namespace; the name :attr:`~object.__dict__` is an attribute but not a global name. Obviously, using this violates the abstraction of namespace implementation, and should be restricted to things like post-mortem debuggers. + +.. [#] See :ref:`here ` for details and special cases. From d8ea90a23a51c1adbb3871ac4e764cdc0a43928b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?B=C3=A9n=C3=A9dikt=20Tran?= <10796600+picnixz@users.noreply.github.com> Date: Fri, 14 Jun 2024 12:38:24 +0200 Subject: [PATCH 04/18] address rewiew comments --- Doc/faq/programming.rst | 7 ++-- Doc/reference/expressions.rst | 67 ++++++++++++++++++++++++++++++----- Doc/tutorial/classes.rst | 9 +++-- 3 files changed, 70 insertions(+), 13 deletions(-) diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst index 1a77133326cc58..05dabae78e6402 100644 --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -1739,14 +1739,17 @@ Variable names with double leading underscores are "mangled" to provide a simple but effective way to define class private variables. Any identifier of the form ``__spam`` (at least two leading underscores, at most one trailing underscore) is textually replaced with ``_classname__spam``, where ``classname`` is the -current class name with any leading underscores stripped -(see :ref:`here ` for details and special cases). +current class name with any leading underscores stripped. This doesn't guarantee privacy: an outside user can still deliberately access the "_classname__spam" attribute, and private values are visible in the object's ``__dict__``. Many Python programmers never bother to use private variable names at all. +.. seealso:: + + The :ref:`private name mangling specifications ` + for details and special cases. My class defines __del__ but it is not called when I delete the object. ----------------------------------------------------------------------- diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index cc30bef3f8acf9..54765f55ac2c67 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -90,20 +90,71 @@ or more underscore characters and does not end in two or more underscores, it is considered a :dfn:`private name` of that class. More precisely, private names are transformed to a longer form before code is -generated for them. The transformation rule is defined as follows: +generated for them. If the transformed name is longer than 255 characters, +implementation-defined truncation may happen. -- If the class name consists only of underscores, no transformation is done, - e.g., the identifier ``__spam`` occurring in a class named ``_`` or ``__`` - is left as is. +The transformation is independent of the syntactical context in which the +identifier is used and the transformation rule is defined as follows: + +- If the class name consists only of underscores, the transformation is the + identity, e.g., the identifier ``__spam`` occurring in a class named ``_`` + or ``__`` is left as is. - Otherwise, the transformation inserts the class name, with leading underscores removed and a single underscore inserted, in front of the identifier, e.g., the identifier ``__spam`` occurring in a class - named ``Ham``, ``_Ham`` or ``__Ham`` is transformed to ``_Ham__spam``. + named ``Foo``, ``_Foo`` or ``__Foo`` is transformed to ``_Foo__spam``. -The transformation is independent of the syntactical context in which the -identifier is used. Nonetheless, if the transformed name is extremely long -(longer than 255 characters), implementation-defined truncation may happen. + For identifiers declared using :keyword:`import` statements, this rule is + slightly different. Indeed, importing a module with a private name directly + in a class body raises a :exc:`ModuleNotFoundError` (unless the class name + only consists of underscores), as illustrated by the following example: + + .. code-block:: python + + class Foo: + import __spam # raises ModuleNotFoundError at runtime + + This restriction can be lifted by using the :func:`__import__` function, + in which case the transformation rule is applied normally: + + .. code-block:: python + + class Foo: + __spam = __import__("__spam") + + Foo._Foo__spam.do() + + .. note:: + + This restriction does not apply to modules imported as submodules of + private packages, e.g.: + + .. code-block:: python + + class Foo: + import __spam.util + + Foo._Foo__spam.util.do() + +The corresponding private member is accessed by its defining class using +its non-transformed name. On the other hand, the transformed name must be +used to access the private member *externally* (e.g., in a function or in +a subclass): + +.. code-block:: python + + class A: + def __one(self): + return 1 + def two(self): + return 2 * self.__one() + + class B(A): + def three(self): + return 3 * self._A__one() + + four = 4 * A()._A__one() .. _atom-literals: diff --git a/Doc/tutorial/classes.rst b/Doc/tutorial/classes.rst index d24ce6887c22c0..675faa8c52477d 100644 --- a/Doc/tutorial/classes.rst +++ b/Doc/tutorial/classes.rst @@ -684,10 +684,15 @@ clashes of names with names defined by subclasses), there is limited support for such a mechanism, called :dfn:`name mangling`. Any identifier of the form ``__spam`` (at least two leading underscores, at most one trailing underscore) is textually replaced with ``_classname__spam``, where ``classname`` is the -current class name with leading underscore(s) stripped. [#]_ This mangling is done +current class name with leading underscore(s) stripped. This mangling is done without regard to the syntactic position of the identifier, as long as it occurs within the definition of a class. +.. seealso:: + + The :ref:`private name mangling specifications ` + for details and special cases. + Name mangling is helpful for letting subclasses override methods without breaking intraclass method calls. For example:: @@ -930,5 +935,3 @@ Examples:: namespace; the name :attr:`~object.__dict__` is an attribute but not a global name. Obviously, using this violates the abstraction of namespace implementation, and should be restricted to things like post-mortem debuggers. - -.. [#] See :ref:`here ` for details and special cases. From 177e8aa6e93dab73fd9a43d159126cf4d6caf8f6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?B=C3=A9n=C3=A9dikt=20Tran?= <10796600+picnixz@users.noreply.github.com> Date: Fri, 14 Jun 2024 15:43:46 +0200 Subject: [PATCH 05/18] improve documentation with special cases --- Doc/reference/expressions.rst | 115 +++++++++++++++++++++++++--------- 1 file changed, 84 insertions(+), 31 deletions(-) diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index 54765f55ac2c67..09725dc5c36333 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -83,12 +83,17 @@ exception. pair: name; mangling pair: private; names -.. rubric:: Private name mangling +Private name mangling +^^^^^^^^^^^^^^^^^^^^^ When an identifier that textually occurs in a class definition begins with two or more underscore characters and does not end in two or more underscores, it is considered a :dfn:`private name` of that class. +.. seealso:: + + The :ref:`tutorial on classes <_tut-classdefinition>` for more details. + More precisely, private names are transformed to a longer form before code is generated for them. If the transformed name is longer than 255 characters, implementation-defined truncation may happen. @@ -105,37 +110,9 @@ identifier is used and the transformation rule is defined as follows: the identifier, e.g., the identifier ``__spam`` occurring in a class named ``Foo``, ``_Foo`` or ``__Foo`` is transformed to ``_Foo__spam``. - For identifiers declared using :keyword:`import` statements, this rule is - slightly different. Indeed, importing a module with a private name directly - in a class body raises a :exc:`ModuleNotFoundError` (unless the class name - only consists of underscores), as illustrated by the following example: - - .. code-block:: python - - class Foo: - import __spam # raises ModuleNotFoundError at runtime - - This restriction can be lifted by using the :func:`__import__` function, - in which case the transformation rule is applied normally: - - .. code-block:: python - - class Foo: - __spam = __import__("__spam") - - Foo._Foo__spam.do() - - .. note:: +.. _private-name-mangling-access: - This restriction does not apply to modules imported as submodules of - private packages, e.g.: - - .. code-block:: python - - class Foo: - import __spam.util - - Foo._Foo__spam.util.do() +.. rubric:: Accessing members with mangled names The corresponding private member is accessed by its defining class using its non-transformed name. On the other hand, the transformed name must be @@ -156,6 +133,82 @@ a subclass): four = 4 * A()._A__one() +.. _private-name-mangling-imports: + +.. rubric:: Mangled names in imports + +For identifiers declared using :keyword:`import` statements, the mangling +rule is slightly different. Throughout this paragraph, assume that we have +the following filesystem layout and, unless stated otherwise, the code snippets +are in the ``__main__.py`` file. + +.. code-block:: + + . + ├── __main__.py + ├── __pkg + │ ├── __init__.py + │ └── mod.py + ├── __bar.py + ├── __foo.py + └── _Bar__bar.py + +Importing a module with a private name directly in a class body may raise +a :exc:`ModuleNotFoundError`, as illustrated by the following example: + +.. code-block:: python + + class Bar: + # equivalent to import _Bar__bar + import __bar + print(Bar._Bar__bar) # + + class Foo: + # raises a ModuleNotFoundError at runtime since the interpreter + # tries to import '_Foo__foo' instead of '__foo' + import __foo + +If the ``__foo`` module is needed inside the ``Foo`` class, the import can +be performed via the :func:`__import__` function, in which case the usual +transformation rule is applied (a similar logic applies to :func:`getattr`, +:func:`setattr` and :func:`delattr`, see ): + +.. code-block:: python + + class Bar: + # explicitly import '__bar' instead of '_Bar__bar' + __bar = __import__("__bar") + print(Bar._Bar__bar) # + + class Foo: + # explicitly import '__foo' instead of '_Foo__foo' + __foo = __import__("__foo") + print(Foo._Foo__foo) # + +This restriction does not apply to modules imported as submodules of packages +with private names, e.g.: + +.. code-block:: python + + class Foo: + import __pkg.mod + + print(Foo._Foo__pkg) # + print(Foo._Foo__pkg.mod) # + +Note that a class whose name only consists of underscores does not +suffer from those restrictions on :keyword:`import` statements: + +.. code-block:: python + + class _: + import __bar # imports '__bar' + import __foo # imports '__foo' + import __pkg # imports '__pkg' + + print(_.__bar) # + print(_.__foo) # + print(_.__pkg) # .. _atom-literals: From 53384d2a71b94ef28b9b6e763832878b653a836e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?B=C3=A9n=C3=A9dikt=20Tran?= <10796600+picnixz@users.noreply.github.com> Date: Fri, 14 Jun 2024 15:47:47 +0200 Subject: [PATCH 06/18] fix RST code-block class --- Doc/reference/expressions.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index 09725dc5c36333..54ce269dd2e8b3 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -142,7 +142,7 @@ rule is slightly different. Throughout this paragraph, assume that we have the following filesystem layout and, unless stated otherwise, the code snippets are in the ``__main__.py`` file. -.. code-block:: +.. code-block:: text . ├── __main__.py From 4bbce14f1c69e6587907084751618fe4da04d3bd Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?B=C3=A9n=C3=A9dikt=20Tran?= <10796600+picnixz@users.noreply.github.com> Date: Fri, 14 Jun 2024 15:50:14 +0200 Subject: [PATCH 07/18] simplify a bit the text to avoid having too many future references --- Doc/reference/expressions.rst | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index 54ce269dd2e8b3..ec5c032d72a7c1 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -138,7 +138,7 @@ a subclass): .. rubric:: Mangled names in imports For identifiers declared using :keyword:`import` statements, the mangling -rule is slightly different. Throughout this paragraph, assume that we have +rule is slightly different. Throughout this paragraph, assume that we have the following filesystem layout and, unless stated otherwise, the code snippets are in the ``__main__.py`` file. @@ -170,8 +170,7 @@ a :exc:`ModuleNotFoundError`, as illustrated by the following example: If the ``__foo`` module is needed inside the ``Foo`` class, the import can be performed via the :func:`__import__` function, in which case the usual -transformation rule is applied (a similar logic applies to :func:`getattr`, -:func:`setattr` and :func:`delattr`, see ): +transformation rule is applied: .. code-block:: python From dd278733c7c8ee102e000453f12ef452a2a06a0a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?B=C3=A9n=C3=A9dikt=20Tran?= <10796600+picnixz@users.noreply.github.com> Date: Fri, 14 Jun 2024 15:55:05 +0200 Subject: [PATCH 08/18] fix sphinx ref --- Doc/reference/expressions.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index ec5c032d72a7c1..b1244760199e7f 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -92,7 +92,8 @@ is considered a :dfn:`private name` of that class. .. seealso:: - The :ref:`tutorial on classes <_tut-classdefinition>` for more details. + The :ref:`tutorial on classes ` for more details + on classes and their declaration. More precisely, private names are transformed to a longer form before code is generated for them. If the transformed name is longer than 255 characters, From ed85ecc9f218cf9a8b6413729242d55e05d4466e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?B=C3=A9n=C3=A9dikt=20Tran?= <10796600+picnixz@users.noreply.github.com> Date: Fri, 14 Jun 2024 15:57:41 +0200 Subject: [PATCH 09/18] add an additional reference --- Doc/reference/expressions.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index b1244760199e7f..90121b74c0561d 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -134,6 +134,11 @@ a subclass): four = 4 * A()._A__one() +.. seealso:: + + The tutorial on :ref:`private variables ` in classes for + a more complete example. + .. _private-name-mangling-imports: .. rubric:: Mangled names in imports From 20065685c21d8eb22c08cb25492e3555b684b26c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?B=C3=A9n=C3=A9dikt=20Tran?= <10796600+picnixz@users.noreply.github.com> Date: Fri, 14 Jun 2024 18:22:56 +0200 Subject: [PATCH 10/18] simplify wording --- Doc/reference/expressions.rst | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-) diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index 90121b74c0561d..5a82a82e8e649c 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -144,9 +144,8 @@ a subclass): .. rubric:: Mangled names in imports For identifiers declared using :keyword:`import` statements, the mangling -rule is slightly different. Throughout this paragraph, assume that we have -the following filesystem layout and, unless stated otherwise, the code snippets -are in the ``__main__.py`` file. +rule is slightly different. Throughout this paragraph, we consider the +following filesystem layout with code snippets in the ``__main__.py`` file. .. code-block:: text @@ -180,11 +179,6 @@ transformation rule is applied: .. code-block:: python - class Bar: - # explicitly import '__bar' instead of '_Bar__bar' - __bar = __import__("__bar") - print(Bar._Bar__bar) # - class Foo: # explicitly import '__foo' instead of '_Foo__foo' __foo = __import__("__foo") From 50c9adfa161f966592861ac281ae0375a7d794ea Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?B=C3=A9n=C3=A9dikt=20Tran?= <10796600+picnixz@users.noreply.github.com> Date: Sat, 15 Jun 2024 10:57:34 +0200 Subject: [PATCH 11/18] clarify some textual parts --- Doc/reference/expressions.rst | 18 ++---------------- 1 file changed, 2 insertions(+), 16 deletions(-) diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index 5a82a82e8e649c..636af9fe0870e1 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -107,8 +107,8 @@ identifier is used and the transformation rule is defined as follows: or ``__`` is left as is. - Otherwise, the transformation inserts the class name, with leading - underscores removed and a single underscore inserted, in front of - the identifier, e.g., the identifier ``__spam`` occurring in a class + underscores removed and a single leading underscore inserted, in front + of the identifier, e.g., the identifier ``__spam`` occurring in a class named ``Foo``, ``_Foo`` or ``__Foo`` is transformed to ``_Foo__spam``. .. _private-name-mangling-access: @@ -195,20 +195,6 @@ with private names, e.g.: print(Foo._Foo__pkg) # print(Foo._Foo__pkg.mod) # -Note that a class whose name only consists of underscores does not -suffer from those restrictions on :keyword:`import` statements: - -.. code-block:: python - - class _: - import __bar # imports '__bar' - import __foo # imports '__foo' - import __pkg # imports '__pkg' - - print(_.__bar) # - print(_.__foo) # - print(_.__pkg) # - .. _atom-literals: Literals From b3e24f6fc9daeb3a7c109a9369ebe51a78c0eac9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?B=C3=A9n=C3=A9dikt=20Tran?= <10796600+picnixz@users.noreply.github.com> Date: Sat, 15 Jun 2024 17:38:25 +0200 Subject: [PATCH 12/18] upgrade docs --- Doc/reference/expressions.rst | 104 ++++++---------------------------- 1 file changed, 17 insertions(+), 87 deletions(-) diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index 636af9fe0870e1..7980f08b57012d 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -92,15 +92,29 @@ is considered a :dfn:`private name` of that class. .. seealso:: - The :ref:`tutorial on classes ` for more details - on classes and their declaration. + The :ref:`class specifications `. More precisely, private names are transformed to a longer form before code is generated for them. If the transformed name is longer than 255 characters, implementation-defined truncation may happen. The transformation is independent of the syntactical context in which the -identifier is used and the transformation rule is defined as follows: +identifier is used but only the following private identifiers are mangled: + +- The name of imported modules, e.g., ``__spam`` in ``import __spam``. + If the module is part of a package (i.e., its name contains a dot), + the name is *not* mangled, e.g., the ``__foo`` in ``import __foo.bar`` + is not mangled. + +- The name of an imported member, e.g., ``__f`` in ``from spam import __f``. + +- Any name used as the name of a variable that is assigned or read or any + name of an attribute being accessed. + + The ``__name__`` attribute of nested functions, classes, and type aliases + is however not mangled. + +The transformation rule is defined as follows: - If the class name consists only of underscores, the transformation is the identity, e.g., the identifier ``__spam`` occurring in a class named ``_`` @@ -111,90 +125,6 @@ identifier is used and the transformation rule is defined as follows: of the identifier, e.g., the identifier ``__spam`` occurring in a class named ``Foo``, ``_Foo`` or ``__Foo`` is transformed to ``_Foo__spam``. -.. _private-name-mangling-access: - -.. rubric:: Accessing members with mangled names - -The corresponding private member is accessed by its defining class using -its non-transformed name. On the other hand, the transformed name must be -used to access the private member *externally* (e.g., in a function or in -a subclass): - -.. code-block:: python - - class A: - def __one(self): - return 1 - def two(self): - return 2 * self.__one() - - class B(A): - def three(self): - return 3 * self._A__one() - - four = 4 * A()._A__one() - -.. seealso:: - - The tutorial on :ref:`private variables ` in classes for - a more complete example. - -.. _private-name-mangling-imports: - -.. rubric:: Mangled names in imports - -For identifiers declared using :keyword:`import` statements, the mangling -rule is slightly different. Throughout this paragraph, we consider the -following filesystem layout with code snippets in the ``__main__.py`` file. - -.. code-block:: text - - . - ├── __main__.py - ├── __pkg - │ ├── __init__.py - │ └── mod.py - ├── __bar.py - ├── __foo.py - └── _Bar__bar.py - -Importing a module with a private name directly in a class body may raise -a :exc:`ModuleNotFoundError`, as illustrated by the following example: - -.. code-block:: python - - class Bar: - # equivalent to import _Bar__bar - import __bar - print(Bar._Bar__bar) # - - class Foo: - # raises a ModuleNotFoundError at runtime since the interpreter - # tries to import '_Foo__foo' instead of '__foo' - import __foo - -If the ``__foo`` module is needed inside the ``Foo`` class, the import can -be performed via the :func:`__import__` function, in which case the usual -transformation rule is applied: - -.. code-block:: python - - class Foo: - # explicitly import '__foo' instead of '_Foo__foo' - __foo = __import__("__foo") - print(Foo._Foo__foo) # - -This restriction does not apply to modules imported as submodules of packages -with private names, e.g.: - -.. code-block:: python - - class Foo: - import __pkg.mod - - print(Foo._Foo__pkg) # - print(Foo._Foo__pkg.mod) # - .. _atom-literals: Literals From cc25cc56d82961e37538907f1b87be7942a45391 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?B=C3=A9n=C3=A9dikt=20Tran?= <10796600+picnixz@users.noreply.github.com> Date: Sat, 15 Jun 2024 17:40:43 +0200 Subject: [PATCH 13/18] update FAQ --- Doc/faq/programming.rst | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst index 05dabae78e6402..c249b12f30712c 100644 --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -1741,6 +1741,25 @@ but effective way to define class private variables. Any identifier of the form is textually replaced with ``_classname__spam``, where ``classname`` is the current class name with any leading underscores stripped. +More generally, the corresponding private member is accessed by its defining +class using its non-transformed name. On the other hand, the transformed name +must be used to access the private member *externally* (e.g., in a function or +in a subclass): + +.. code-block:: python + + class A: + def __one(self): + return 1 + def two(self): + return 2 * self.__one() + + class B(A): + def three(self): + return 3 * self._A__one() + + four = 4 * A()._A__one() + This doesn't guarantee privacy: an outside user can still deliberately access the "_classname__spam" attribute, and private values are visible in the object's ``__dict__``. Many Python programmers never bother to use private variable From 035ed682aa3a853c69be28c239950f92bf8bc742 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?B=C3=A9n=C3=A9dikt=20Tran?= <10796600+picnixz@users.noreply.github.com> Date: Sat, 15 Jun 2024 17:49:56 +0200 Subject: [PATCH 14/18] address feedback review --- Doc/reference/expressions.rst | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index 7980f08b57012d..a7ac91a6539699 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -101,6 +101,12 @@ implementation-defined truncation may happen. The transformation is independent of the syntactical context in which the identifier is used but only the following private identifiers are mangled: +- Any name used as the name of a variable that is assigned or read or any + name of an attribute being accessed. + + The ``__name__`` attribute of nested functions, classes, and type aliases + is however not mangled. + - The name of imported modules, e.g., ``__spam`` in ``import __spam``. If the module is part of a package (i.e., its name contains a dot), the name is *not* mangled, e.g., the ``__foo`` in ``import __foo.bar`` @@ -108,12 +114,6 @@ identifier is used but only the following private identifiers are mangled: - The name of an imported member, e.g., ``__f`` in ``from spam import __f``. -- Any name used as the name of a variable that is assigned or read or any - name of an attribute being accessed. - - The ``__name__`` attribute of nested functions, classes, and type aliases - is however not mangled. - The transformation rule is defined as follows: - If the class name consists only of underscores, the transformation is the From d4a0431af9f2e9eb1eb2cd229fa6864f9f81f6ca Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?B=C3=A9n=C3=A9dikt=20Tran?= <10796600+picnixz@users.noreply.github.com> Date: Fri, 12 Jul 2024 17:08:33 +0200 Subject: [PATCH 15/18] Update Doc/reference/expressions.rst Co-authored-by: Jelle Zijlstra --- Doc/reference/expressions.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index a7ac91a6539699..dc5c59a62f5e62 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -116,15 +116,15 @@ identifier is used but only the following private identifiers are mangled: The transformation rule is defined as follows: +- The class name, with leading underscores removed and a single leading + underscore inserted, is inserted in front of the identifier, e.g., the + identifier ``__spam`` occurring in a class named ``Foo``, ``_Foo`` or + ``__Foo`` is transformed to ``_Foo__spam``. + - If the class name consists only of underscores, the transformation is the identity, e.g., the identifier ``__spam`` occurring in a class named ``_`` or ``__`` is left as is. -- Otherwise, the transformation inserts the class name, with leading - underscores removed and a single leading underscore inserted, in front - of the identifier, e.g., the identifier ``__spam`` occurring in a class - named ``Foo``, ``_Foo`` or ``__Foo`` is transformed to ``_Foo__spam``. - .. _atom-literals: Literals From 55e7044868d0f7518754c0c9f43bc54b9de44198 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?B=C3=A9n=C3=A9dikt=20Tran?= <10796600+picnixz@users.noreply.github.com> Date: Fri, 12 Jul 2024 17:12:10 +0200 Subject: [PATCH 16/18] fixup! whitespace --- Doc/reference/expressions.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index dc5c59a62f5e62..a64977b1ed8038 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -120,7 +120,7 @@ The transformation rule is defined as follows: underscore inserted, is inserted in front of the identifier, e.g., the identifier ``__spam`` occurring in a class named ``Foo``, ``_Foo`` or ``__Foo`` is transformed to ``_Foo__spam``. - + - If the class name consists only of underscores, the transformation is the identity, e.g., the identifier ``__spam`` occurring in a class named ``_`` or ``__`` is left as is. From 4a2c4a91974274f256240b509aff35f34bc30e88 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?B=C3=A9n=C3=A9dikt=20Tran?= <10796600+picnixz@users.noreply.github.com> Date: Fri, 12 Jul 2024 17:30:56 +0200 Subject: [PATCH 17/18] Update programming.rst --- Doc/faq/programming.rst | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst index c249b12f30712c..1ef4a263774cbc 100644 --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -1741,10 +1741,8 @@ but effective way to define class private variables. Any identifier of the form is textually replaced with ``_classname__spam``, where ``classname`` is the current class name with any leading underscores stripped. -More generally, the corresponding private member is accessed by its defining -class using its non-transformed name. On the other hand, the transformed name -must be used to access the private member *externally* (e.g., in a function or -in a subclass): +The identifier can be used unchanged within the class, but to access it outside +the class, the mangled name must be used: .. code-block:: python From 941dbe15f4197226891331118c26a6aa23a110f0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?B=C3=A9n=C3=A9dikt=20Tran?= <10796600+picnixz@users.noreply.github.com> Date: Sat, 13 Jul 2024 15:18:39 +0200 Subject: [PATCH 18/18] improve docs (again) --- Doc/faq/programming.rst | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst index 1ef4a263774cbc..61fbd1bb92ada6 100644 --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -1758,10 +1758,9 @@ the class, the mangled name must be used: four = 4 * A()._A__one() -This doesn't guarantee privacy: an outside user can still deliberately access -the "_classname__spam" attribute, and private values are visible in the object's -``__dict__``. Many Python programmers never bother to use private variable -names at all. +In particular, this does not guarantee privacy since an outside user can still +deliberately access the private attribute; many Python programmers never bother +to use private variable names at all. .. seealso::