8000 problem finding data_dir on Windows when run using "python -m mypy" · Issue #1225 · python/mypy · GitHub
[go: up one dir, main page]
Skip to content

problem finding data_dir on Windows when run using "python -m mypy" #1225

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
allthedata opened this issue Feb 20, 2016 · 8 comments · Fixed by #1263
Closed

problem finding data_dir on Windows when run using "python -m mypy" #1225

allthedata opened this issue Feb 20, 2016 · 8 comments · Fixed by #1263

Comments

@allthedata
Copy link
Contributor

I ran into issues with mypy on a simple example. Running Windows 8.1, Python 3.5 using Anaconda. Installed mypy using pip install git+https://github.com/python/mypy.git
Saved the following as a.py:

import typing

def f():
    1 + 'x'  # No static type error (dynamically typed)

def g() -> None:
    1 + 'x'  # Type check error (statically typed)

Ran the following in the command prompt: python -m mypy a.py and got the following:

a.py:1: error: Could not find builtins
a.py:1: error: No library stub file for standard library module 'typing'
a.py:1: note: (Stub files are from https://github.com/python/typeshed)

I found that the files were installed in the following locations:

  • The .py files are in C:/Users/MyUserName/Anaconda3/Lib/site-packages/mypy
  • The typeshed and xml directories are in C:/Users/MyUserName/Anaconda3/Lib/mypy
  • The mypy script is C:/Users/MyUserName/Anaconda3/Scripts/mypy (Running mypy in the command prompt doesn't work in Windows, which is why I had to run it as a module.)

I ran pdb and found that when I call mypy as a module:

  1. __init__.py calls mypy.main.main(None)
  2. build.build is called with bin_dir=None
  3. build.default_data_dir is called with bin_dir=None
  4. build.default_data_dir returns C:/Users/MyUserName/Anaconda3/Lib/site-packages when it should return C:/Users/MyUserName/Anaconda3/Lib/mypy

In build.default_data_dir, for the case of not bin_dir I changed the line lib = os.path.dirname(os.path.dirname(parent)) to lib = os.path.dirname(parent) and now I can run mypy as a module. Not sure if it's the best solution but it works.

Possibly related issue: #653. Later realized that I can run the mypy script within git bash if I change #!/usr/bin/env python3 to #!/usr/bin/env python, and then I don't have to worry about the module stuff. But Windows users installing from pip may not have a bash shell.
@gvanrossum gvanrossum changed the title problem running as a module problem finding data_dir on Windows when run using "python -m mypy" Feb 20, 2016
@brettcannon
Copy link
Member

On Windows using CPython 3.5.1 in a venv I'm also running into the ni\__init__.py:1: error: Could not find builtins message. The package I want checked is in the same directory I'm running from using the command, .e.g. .\venv\Scripts\python.exe -m mypy ni.

@gvanrossum
< 8000 div class="ml-n3 timeline-comment unminimized-comment comment previewable-edit js-task-list-container js-comment timeline-comment--caret" data-body-version="243f6d900caef814562bf4cfabda4fc30b6488b3a754cdfa890f7e6e71da6658">
Copy link
Member

Maybe a fix similar to #1253 is needed, or a generalization thereof. I can't test on Windows but I'm happy to take PRs from people who are in a position to test them.

@ddfisher ddfisher added this to the 0.3.3 milestone Mar 1, 2016
@brettcannon
Copy link
Member

Starting to dig into this for CPython 3.5.1 on Windows 10 in a venv, it looks like

mypy/mypy/build.py

Line 767 in d4a555b

self.fail(self.path, 1, 'Could not find builtins')
is where the error message is being specified, so that suggests the builtins module can't be typed. OK, so that's a stdlib module, so I guess typeshed should be handling this? Anyway, when I print out

mypy/mypy/build.py

Line 1028 in d4a555b

key = (id, lib_path)
for builtins I get ('builtins', ('C:\\Users\\brcan\\Documents\\Repositories\\knights-who-say-ni', '.', '.', '.', '.', '. ', '.', '.', '.', '.', '.', '.')) which obviously contains no path to typeshed as installed in the venv.

Now looking at my venv, the location of typeshed in the venv is .\venv\Lib\mypy where there is a typeshed and xml directory (this is also where some tcl8.6 in .\venv\Lib so this seems to be where pip and setuptools puts your data files). When I looked at setup.py I see you specified the typeshed files as data_files in setup(), but then I also noticed you don't use pkg_resources or setuptools in your code to calculate the path to typeshed. So my suspicion is you're making some assumptions somewhere about where typeshed files are being installed that are not followed on Windows because you're hand-calculating the paths to data files instead of using pkg_resources and/or setuptools to do it for you.

@gvanrossum
Copy link
Member

Correct! The PR that I linked to actually shows the code where this determination is made. Sorry, I should have been more explicit. Here's the logic:

mypy/mypy/build.py

Line 215 in d4a555b

def default_data_dir(bin_dir: str) -> str:

@allthedata
Copy link
Contributor Author

Here's my proposed fix: master...allthedata:master. I tested it on Windows and Ubuntu and it seems to work. Side note: despite the comment in the function that says __file__ is __init__.py, on both platforms when I ran mypy as a module I found that __file__ was actually build.py. I didn't test it in a virtualenv so please let me know whether it works in that case.

@gvanrossum
Copy link
Member

Can you submit that as a PR? Also, you can fix that comment about __file__ -- it's outdated (I guess the code lived somewhere else long ago).

@brettcannon
Copy link
Member

Are you willing to take a dependency on pkg_resources and/or setuptools? I believe the former has a function you can simply use to do the calculation instead of having to reverse-engineer the logic of where setuptools installs data files.

@allthedata allthedata mentioned this issue Mar 2, 2016
Merged
@gvanrossum
Copy link
Member
gvanrossum commented Mar 2, 2016 via email

gvanrossum added a commit that referenced this issue Mar 3, 2016
@gvanrossum
Merge pull request #1263 from allthedata/master
5cd8319 
Fix for issue #1225
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix for issue #1225
4 participants
@brettcannon @ddfisher @gvanrossum @allthedata
0