8000 [WIP] bytes/str/unicode by gvanrossum · Pull Request #2203 · python/mypy · GitHub
[go: up one dir, main page]
Skip to content

[WIP] bytes/str/unicode #2203

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 8 commits into from
Closed

[WIP] bytes/str/unicode #2203

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
1974c66
Distinguish b'' and '' in PY2.
Sep 19, 2016
bfa0bac
Update the version of typed_ast we need (0.6.1).
Sep 21, 2016
44f5c95
Add the first bytes/str/unicode tests
Sep 30, 2016
71a9461
Fix python2eval.test
Sep 30, 2016
d5aea06
Sync typeshed again
Sep 30, 2016
86e7da9
Fix test some more
Sep 30, 2016
8c69076
Fix all python2eval tests
Oct 1, 2016
cd6ca21
Sync typeshed some more
Oct 1, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix all python2eval tests
  • Loading branch information
Guido van Rossum committed Oct 1, 2016
commit 8c690768386d60f755cbec7b06c54a7783caba7c
1 change: 1 addition & 0 deletions mypy/test/testpythoneval.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ def test_python_evaluation(testcase):
interpreter = python3_path
args = []
py2 = False
args.append('--fast-parser') # Some tests require this now.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit of a problem, because it means that these tests can't be run on Windows any more (until typed_ast is finally ported there -- I think @ddfisher mentioned he was making progress so hopefully I can stop worrying about this). The checker tests (testcheck.py) allow specifying this flag per testcase, but these eval tests don't have that feature.

args.append('--tb') # Show traceback on crash.
# Write the program to a file.
program = '_program.py'
Expand Down
27 changes: 16 additions & 11 deletions test-data/unit/python2eval.test
View file
Open in desktop
10000
Original file line number Diff line number Diff line change
Expand Up @@ -79,18 +79,18 @@ x = 1.5
[case testAnyStr_python2]
from typing import AnyStr
def f(x): # type: (AnyStr) -> AnyStr
if isinstance(x, str):
if isinstance(x, bytes):
Copy link
Member Author
@gvanrossum gvanrossum Oct 1, 2016 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is necessary because AnyStr now has three possible types (bytes, str, unicode). Note that I must return 'foo' for bytes and str, because at runtime they are still indistinguishable! (And these tests actually run the Python code!)

return b'foo'
elif isinstance(x, str):
return 'foo'
elif isinstance(x, bytes):
return b'qux'
else:
return u'zar'
print f('')
print f(b'')
print f(u'')
[out]
foo
qux
foo
zar

[case testGenericPatterns_python2]
Expand Down Expand Up @@ -156,13 +156,16 @@ f(**params)

[case testFromFutureImportUnicodeLiterals2_python2]
from __future__ import unicode_literals
def f(x: str) -> None: pass
def f(x):
# type: (str) -> None
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This (like a few others) was needed because we're now running these tests with --fast-parser, and the fast PY2 parser rejects PY3-style annotations!

pass
f(b'')
f(u'')
f('')
[out]
_program.py:4: error: Argument 1 to "f" has incompatible type "unicode"; expected "str"
_program.py:5: error: Argument 1 to "f" has incompatible type "unicode"; expected "str"
_program.py:5: error: Argument 1 to "f" has incompatible type "bytes"; expected "str"
_program.py:6: error: Argument 1 to "f" has incompatible type "unicode"; expected "str"
_program.py:7: error: Argument 1 to "f" has incompatible type "unicode"; expected "str"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that all three calls are now in error! bytes is not compatible with str, and '' becomes unicode because of the unicode_literals.


[case testStrUnicodeCompatibility_python2]
import typing
Expand Down Expand Up @@ -237,7 +240,7 @@ u'\x89'
import typing
import io
c = io.BytesIO()
c.write('\x89')
c.write(b'\x89')
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one feels reasonable.

print(repr(c.getvalue()))
[out]
'\x89'
Expand Down Expand Up @@ -400,11 +403,12 @@ def f(x: unicode) -> int: pass
def f(x: bytearray) -> int: pass
[out]
_program.py:2: error: No overload variant of "f" matches argument types [builtins.int]
_program.py:5: error: No overload variant of "f" matches argument types [builtins.bytes]
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because there's no overload for bytes, and there's no equivalency between bytes and bytearray. Not sure I like this.


[case testByteArrayStrCompatibility_python2]
def f(x): # type: (str) -> None
def f(x): # type: (bytes) -> None
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that bytearray() is compatible with bytes, but not with str, hence the change here.

pass
f(bytearray('foo'))
f(bytearray(b'foo'))
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The argument to the bytearray() constructor must be bytes. That's reasonable.


[case testAbstractProperty_python2]
from abc import abstractproperty, ABCMeta
Expand Down Expand Up @@ -469,7 +473,8 @@ re.subn(upat, lambda m: u'', u'')[0] + u''

[case testYieldRegressionTypingAwaitable_python2]
# Make sure we don't reference typing.Awaitable in Python 2 mode.
def g() -> int:
def g():
# type: () -> int
yield
[out]
_program.py: note: In function "g":
Expand Down
2 changes: 1 addition & 1 deletion typeshed
Open in desktop
Submodule typeshed updated 2 files
+4 −4 stdlib/2.7/__builtin__.pyi
+2 −2 stdlib/2.7/io.pyi
0