10000 [3.13] gh-119933: Improve ``SyntaxError`` message for invalid type parameters expressions (GH-119976) by JelleZijlstra · Pull Request #120641 · python/cpython · GitHub
[go: up one dir, main page]

Skip to content

[3.13] gh-119933: Improve SyntaxError message for invalid type parameters expressions (GH-119976) #120641

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 57 additions & 4 deletions Doc/library/symtable.rst
Original file line number Diff line number Diff line change
Expand Up @@ -31,21 +31,74 @@ Generating Symbol Tables
Examining Symbol Tables
-----------------------

.. class:: SymbolTableType

An enumeration indicating the type of a :class:`SymbolTable` object.

.. attribute:: MODULE
:value: "module"

Used for the symbol table of a module.

.. attribute:: FUNCTION
:value: "function"

Used for the symbol table of a function.

.. attribute:: CLASS
:value: "class"

Used for the symbol table of a class.

The following members refer to different flavors of
:ref:`annotation scopes <annotation-scopes>`.

.. attribute:: ANNOTATION
:value: "annotation"

Used for annotations if ``from __future__ import annotations`` is active.

.. attribute:: TYPE_ALIAS
:value: "type alias"

Used for the symbol table of :keyword:`type` constructions.

.. attribute:: TYPE_PARAMETERS
:value: "type parameters"

Used for the symbol table of :ref:`generic functions <generic-functions>`
or :ref:`generic classes <generic-classes>`.

.. attribute:: TYPE_VARIABLE
:value: "type variable"

Used for the symbol table of the bound, the constraint tuple or the
default value of a single type variable in the formal sense, i.e.,
a TypeVar, a TypeVarTuple or a ParamSpec object (the latter two do
not support a bound or a constraint tuple).

.. versionadded:: 3.13

.. class:: SymbolTable

A namespace table for a block. The constructor is not public.

.. method:: get_type()

Return the type of the symbol table. Possible values are ``'class'``,
``'module'``, ``'function'``, ``'annotation'``, ``'TypeVar bound'``,
``'type alias'``, and ``'type parameter'``. The latter four refer to
different flavors of :ref:`annotation scopes <annotation-scopes>`.
Return the type of the symbol table. Possible values are members
of the :class:`SymbolTableType` enumeration.

.. versionchanged:: 3.12
Added ``'annotation'``, ``'TypeVar bound'``, ``'type alias'``,
and ``'type parameter'`` as possible return values.

.. versionchanged:: 3.13
Return values are members of the :class:`SymbolTableType` enumeration.

The exact values of the returned string may change in the future,
and thus, it is recommended to use :class:`SymbolTableType` members
instead of hard-coded strings.

.. method:: get_id()

Return the table's identifier.
Expand Down
31 changes: 26 additions & 5 deletions Include/internal/pycore_symtable.h
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,23 @@ typedef enum _block_type {
// Used for annotations if 'from __future__ import annotations' is active.
// Annotation blocks cannot bind names and are not evaluated.
AnnotationBlock,
// Used for generics and type aliases. These work mostly like functions
// (see PEP 695 for details). The three different blocks function identically;
// they are different enum entries only so that error messages can be more
// precise.
TypeVarBoundBlock, TypeAliasBlock, TypeParamBlock

// The following blocks are used for generics and type aliases. These work
// mostly like functions (see PEP 695 for details). The three different
// blocks function identically; they are different enum entries only so
// that error messages can be more precise.

// The block to enter when processing a "type" (PEP 695) construction,
// e.g., "type MyGeneric[T] = list[T]".
TypeAliasBlock,
// The block to enter when processing a "generic" (PEP 695) object,
// e.g., "def foo[T](): pass" or "class A[T]: pass".
TypeParametersBlock,
// The block to enter when processing the bound, the constraint tuple
// or the default value of a single "type variable" in the formal sense,
// i.e., a TypeVar, a TypeVarTuple or a ParamSpec object (the latter two
// do not support a bound or a constraint tuple).
TypeVariableBlock,
} _Py_block_ty;

typedef enum _comprehension_type {
Expand Down Expand Up @@ -82,7 +94,16 @@ typedef struct _symtable_entry {
PyObject *ste_children; /* list of child blocks */
PyObject *ste_directives;/* locations of global and nonlocal statements */
PyObject *ste_mangled_names; /* set of names for which mangling should be applied */

_Py_block_ty ste_type;
// Optional string set by symtable.c and used when reporting errors.
// The content of that string is a description of the current "context".
//
// For instance, if we are processing the default value of the type
// variable "T" in "def foo[T = int](): pass", `ste_scope_info` is
// set to "a TypeVar default".
const char *ste_scope_info;

int ste_nested; /* true if block is nested */
unsigned ste_free : 1; /* true if block has free variables */
unsigned ste_child_free : 1; /* true if a child block has free vars,
Expand Down
35 changes: 23 additions & 12 deletions Lib/symtable.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@
LOCAL, GLOBAL_IMPLICIT, GLOBAL_EXPLICIT, CELL)

import weakref
from enum import StrEnum

__all__ = ["symtable", "SymbolTable", "Class", "Function", "Symbol"]
__all__ = ["symtable", "SymbolTableType", "SymbolTable", "Class", "Function", "Symbol"]

def symtable(code, filename, compile_type):
""" Return the toplevel *SymbolTable* for the source code.
Expand Down Expand Up @@ -39,6 +40,16 @@ def __call__(self, table, filename):
_newSymbolTable = SymbolTableFactory()


class SymbolTableType(StrEnum):
MODULE = "module"
FUNCTION = "function"
CLASS = "class"
ANNOTATION = "annotation"
TYPE_ALIAS = "type alias"
TYPE_PARAMETERS = "type parameters"
TYPE_VARIABLE = "type variable"


class SymbolTable:

def __init__(self, raw_table, filename):
Expand All @@ -62,23 +73,23 @@ def __repr__(self):
def get_type(self):
"""Return the type of the symbol table.

The values returned are 'class', 'module', 'function',
'annotation', 'TypeVar bound', 'type alias', and 'type parameter'.
The value returned is one of the values in
the ``SymbolTableType`` enumeration.
"""
if self._table.type == _symtable.TYPE_MODULE:
return "module"
return SymbolTableType.MODULE
if self._table.type == _symtable.TYPE_FUNCTION:
return "function"
return SymbolTableType.FUNCTION
if self._table.type == _symtable.TYPE_CLASS:
return "class"
return SymbolTableType.CLASS
if self._table.type == _symtable.TYPE_ANNOTATION:
return "annotation"
if self._table.type == _symtable.TYPE_TYPE_VAR_BOUND:
return "TypeVar bound"
return SymbolTableType.ANNOTATION
if self._table.type == _symtable.TYPE_TYPE_ALIAS:
return "type alias"
if self._table.type == _symtable.TYPE_TYPE_PARAM:
return "type parameter"
return SymbolTableType.TYPE_ALIAS
if self._table.type == _symtable.TYPE_TYPE_PARAMETERS:
return SymbolTableType.TYPE_PARAMETERS
if self._table.type == _symtable.TYPE_TYPE_VARIABLE:
return SymbolTableType.TYPE_VARIABLE
assert False, f"unexpected type: {self._table.type}"

def get_id(self):
Expand Down
12 changes: 7 additions & 5 deletions Lib/test/test_symtable.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ def namespace_test(): pass
def generic_spam[T](a):
pass

class GenericMine[T: int]:
class GenericMine[T: int, U: (int, str) = int]:
pass
"""

Expand Down Expand Up @@ -78,6 +78,7 @@ class SymtableTest(unittest.TestCase):
GenericMine = find_block(top, "GenericMine")
GenericMine_inner = find_block(GenericMine, "GenericMine")
T = find_block(GenericMine, "T")
U = find_block(GenericMine, "U")

def test_type(self):
self.assertEqual(self.top.get_type(), "module")
Expand All @@ -87,13 +88,14 @@ def test_type(self):
self.assertEqual(self.internal.get_type(), "function")
self.assertEqual(self.foo.get_type(), "function")
self.assertEqual(self.Alias.get_type(), "type alias")
self.assertEqual(self.GenericAlias.get_type(), "type parameter")
self.assertEqual(self.GenericAlias.get_type(), "type parameters")
self.assertEqual(self.GenericAlias_inner.get_type(), "type alias")
self.assertEqual(self.generic_spam.get_type(), "type parameter")
self.assertEqual(self.generic_spam.get_type(), "type parameters")
self.assertEqual(self.generic_spam_inner.get_type(), "function")
self.assertEqual(self.GenericMine.get_type(), "type parameter")
self.assertEqual(self.GenericMine.get_type(), "type parameters")
self.assertEqual(self.GenericMine_inner.get_type(), "class")
self.assertEqual(self.T.get_type(), "TypeVar bound")
self.assertEqual(self.T.get_type(), "type variable")
self.assertEqual(self.U.get_type(), "type variable")

def test_id(self):
self.assertGreater(self.top.get_id(), 0)
Expand Down
100 changes: 100 additions & 0 deletions Lib/test/test_syntax.py
Original file line number Diff line number Diff line change
Expand Up @@ -2046,16 +2046,91 @@ def f(x: *b)
...
SyntaxError: Type parameter list cannot be empty

>>> def f[T: (x:=3)](): pass
Traceback (most recent call last):
...
SyntaxError: named expression cannot be used within a TypeVar bound

>>> def f[T: ((x:= 3), int)](): pass
Traceback (most recent call last):
...
SyntaxError: named expression cannot be used within a TypeVar constraint

>>> def f[T = ((x:=3))](): pass
Traceback (most recent call last):
...
SyntaxError: named expression cannot be used within a TypeVar default

>>> async def f[T: (x:=3)](): pass
Traceback (most recent call last):
...
SyntaxError: named expression cannot be used within a TypeVar bound

>>> async def f[T: ((x:= 3), int)](): pass
Traceback (most recent call last):
...
SyntaxError: named expression cannot be used within a TypeVar constraint

>>> async def f[T = ((x:=3))](): pass
Traceback (most recent call last):
...
SyntaxError: named expression cannot be used within a TypeVar default

>>> type A[T: (x:=3)] = int
Traceback (most recent call last):
...
SyntaxError: named expression cannot be used within a TypeVar bound

>>> type A[T: ((x:= 3), int)] = int
Traceback (most recent call last):
...
SyntaxError: named expression cannot be used within a TypeVar constraint

>>> type A[T = ((x:=3))] = int
Traceback (most recent call last):
...
SyntaxError: named expression cannot be used within a TypeVar default

>>> def f[T: (yield)](): pass
Traceback (most recent call last):
...
SyntaxError: yield expression cannot be used within a TypeVar bound

>>> def f[T: (int, (yield))](): pass
Traceback (most recent call last):
...
SyntaxError: yield expression cannot be used within a TypeVar constraint

>>> def f[T = (yield)](): pass
Traceback (most recent call last):
...
SyntaxError: yield expression cannot be used within a TypeVar default

>>> def f[*Ts = (yield)](): pass
Traceback (most recent call last):
...
SyntaxError: yield expression cannot be used within a TypeVarTuple default

>>> def f[**P = [(yield), int]](): pass
Traceback (most recent call last):
...
SyntaxError: yield expression cannot be used within a ParamSpec default

>>> type A[T: (yield 3)] = int
Traceback (most recent call last):
...
SyntaxError: yield expression cannot be used within a TypeVar bound

>>> type A[T: (int, (yield 3))] = int
Traceback (most recent call last):
...
SyntaxError: yield expression cannot be used within a TypeVar constraint

>>> type A[T = (yield 3)] = int
Traceback (most recent call last):
...
SyntaxError: yield expression cannot be used within a TypeVar default

>>> type A[T: (await 3)] = int
Traceback (most recent call last):
...
Expand All @@ -2066,6 +2141,31 @@ def f(x: *b)
...
SyntaxError: yield expression cannot be used within a TypeVar bound

>>> class A[T: (yield 3)]: pass
Traceback (most recent call last):
...
SyntaxError: yield expression cannot be used within a TypeVar bound

>>> class A[T: (int, (yield 3))]: pass
Traceback (most recent call last):
...
SyntaxError: yield expression cannot be used within a TypeVar constraint

>>> class A[T = (yield)]: pass
Traceback (most recent call last):
...
SyntaxError: yield expression cannot be used within a TypeVar default

>>> class A[*Ts = (yield)]: pass
Traceback (most recent call last):
...
SyntaxError: yield expression cannot be used within a TypeVarTuple default

>>> class A[**P = [(yield), int]]: pass
Traceback (most recent call last):
...
SyntaxError: yield expression cannot be used within a ParamSpec default

>>> type A = (x := 3)
Traceback (most recent call last):
...
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Improve :exc:`SyntaxError` messages for invalid expressions in a type
parameters bound, a type parameter constraint tuple or a default type
parameter.
Patch by Bénédikt Tran.
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Add the :class:`symtable.SymbolTableType` enumeration to represent the
possible outputs of the :class:`symtable.SymbolTable.get_type` method. Patch
by Bénédikt Tran.
6 changes: 3 additions & 3 deletions Modules/symtablemodule.c
Original file line number Diff line number Diff line change
Expand Up @@ -88,11 +88,11 @@ symtable_init_constants(PyObject *m)
return -1;
if (PyModule_AddIntConstant(m, "TYPE_ANNOTATION", AnnotationBlock) < 0)
return -1;
if (PyModule_AddIntConstant(m, "TYPE_TYPE_VAR_BOUND", TypeVarBoundBlock) < 0)
return -1;
if (PyModule_AddIntConstant(m, "TYPE_TYPE_ALIAS", TypeAliasBlock) < 0)
return -1;
if (PyModule_AddIntConstant(m, "TYPE_TYPE_PARAM", TypeParamBlock) < 0)
if (PyModule_AddIntConstant(m, "TYPE_TYPE_PARAMETERS", TypeParametersBlock) < 0)
return -1;
if (PyModule_AddIntConstant(m, "TYPE_TYPE_VARIABLE", TypeVariableBlock) < 0)
return -1;

if (PyModule_AddIntMacro(m, LOCAL) < 0) return -1;
Expand Down
Loading
Loading
0