10000 invalid syntax in builtins · Issue #83 · python/typeshed · GitHub
[go: up one dir, main page]

Skip to content

invalid syntax in builtins #83

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ddfisher opened this issue Feb 16, 2016 · 12 comments
Closed

invalid syntax in builtins #83

ddfisher opened this issue Feb 16, 2016 · 12 comments

Comments

@ddfisher
Copy link
Contributor

The PEP 484 section on stub files says that stub files have the same syntax as Python files. Typeshed conforms to this spec except in one place in builtins.pyi:

True = ...  # type: bool
False = ...  # type: bool
__debug__ = False

This is a "can't assign to keyword" syntax error.

I propose we remove these three lines from typeshed to make it PEP 484 compliant. This means that type checkers will have to hard code their knowledge that True, False, and __debug__ are bools, which seems reasonable.

@matthiaskramm
Copy link
Contributor

Since these are keywords in Python 3, not builtins, it seems fine to remove them from stdlib/3/builtins.pyi.
In Python 2, True and False are builtins, so for stdlib/2/builtins.pyi, we should only remove __debug__.

@gvanrossum
Copy link
Member

I presume you didn't mean to close this. :-) There's no PR yet and we also need to hack something in mypy before we can actually do this.

8000
@gvanrossum gvanrossum reopened this Feb 16, 2016
@ddfisher
Copy link
Contributor Author

@matthiaskramm, I think that's not quite right, actually. The PEP is actually vague on correct Python 2 behavior. Python 2 stub files aren't actually Python 2 syntax (at the moment), as they contain both function annotation and the ... expression. Also, my understanding is that typeshed wants to share stubs between Python 2 and 3 as much as possible, and it would be pretty unfortunate if all stubs needed to use the type comment notation for functions for compatibility instead of the nicer function annotations.

For this reason, I think it would be reasonable (and desirable) to interpret the PEP as saying that all stub files must have Python 3 syntax.

@gvanrossum
Copy link
Member

I think it's reasonable for the Python 2 stubs to use Python 3 syntax. After all they're just stubs.

@matthiaskramm
Copy link
Contributor

Yes, I didn't mean to close this :) - thanks for reopening.

I think we should adhere to Python 3 syntax as best we can. The "True / False" definition in stdlib/2/builtins.pyi is one example for where we would, however, allow Python 3 keywords as identifiers in *.pyi in order to complete define the (Python 2) builtins.
(I haven't gone looking for any Python 2 files that export a nonlocal attribute, but that would fall into the same category.)

I could see us, at some point in time, define a more rigorous and explicit syntax for *.pyi that pins down the allowed style in typeshed, and possibly even makes *.pyi a bit more powerful in some regards. (To e.g. get rid of the x = ... # type: Type syntax)
Then again, I might be biased because pytype has a dedicated *.pyi parser, whereas mypy, I believe, re-uses its normal *.py parser.

Btw. David, what's your use case? Why do you want builtins.pyi to be fully Python 3 compatible?

@ddfisher
Copy link
Contributor Author

My specific use case: I'm building a much faster parser for mypy (and for PRP 484 type comments in general) based on CPython's parser. Syntax errors in Python 3 are therefore also syntax errors in my parser.

I think that adhering strictly to Python 3 syntax is worthwhile, because that should be most compatible with existing tooling, and otherwise we end up defining a sort of ad-hoc spec.

@matthiaskramm
Copy link
Contributor

As far as pytype is concerned, ripping True / False out of builtins isn't going to affect us, since we have our own, slightly more powerful, definition of the builtins. It seems to be mostly mypy that needs to be extended.

However, I do feel that the Python 2 builtins.pyi should be complete, and if .pyi is lacking the power to properly describe some of its constructs, then we should think about how to extend it.

What would be a simple syntax to define certain identifiers even though they're keywords in a later Python version? Some suggestions:

# type: define True: bool
# define-identifier: True: bool
define[True] = ...  # type: bool  
True_magic_suffix = ...  # type: bool
True["redefine"] = ...  # type: bool

It's probably not worth it making the syntax overly pretty, since the instances of it are rare. But I do feel we need to put something into stdlib/2/builtins.pyi.

@gvanrossum
Copy link
Member

Honestly I think that changing the syntax to allow assignment to True, False, __debug__ in a stub is the least evil variant.

@ddfisher
Copy link
Contributor Author

I think it's better for tooling long-term for the syntax to conform strictly to Python 3. It is a little unfortunate that type checkers will have to record the types for True, False, and __debug__ manually, but I don't think it's that bad because: 1) it's only 3 additional small pieces of information about Python out of the vast number of things that will already need to be programmed in and 2) it's not likely to ever change.

If we're going to be changing the stub syntax from the current spec, can we update PEP 484 to make that clear? Also, if we're going to be modifying the syntax for stub file completeness, then we should add None as well.

@gvanrossum
Copy link
Member
gvanrossum commented Feb 20, 2016 via email

@matthiaskramm
Copy link
Contributor

It occurred to me that since you're using the same parser for .py and .pyi, reducing the set of possible identifiers (by removing the Python 3 keywords) will also limit your ability to parse some Python 2 code.
(E.g. Twisted uses nonlocal as a variable name in some places.)

However, all in all, this seems to be rare (I only found a handful of occurrences in our corporate codebase), so nothing worth losing sleep over. I'm fine with this as well.

@ddfisher
Copy link
Contributor Author

Actually, I'm planning on converting the Python 2.7 parser and using that to parse Python 2 .py files, but .pyi files will always be parsed with the Python 3 parser.

momandine pushed a commit to momandine/typeshed that referenced this issue Jul 5, 2016
hswong3i pushed a commit to alvistack/python-typeshed that referenced this issue May 25, 2025
Also run CI against pypy and update actions.

Closes: python#83
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants
0