8000 bpo-33351: (WIP) Patches to build on clang-cl by emmatyping · Pull Request #7680 · python/cpython · GitHub
[go: up one dir, main page]

Skip to content

bpo-33351: (WIP) Patches to build on clang-cl #7680

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from

Conversation

emmatyping
Copy link
Member
@emmatyping emmatyping commented Jun 13, 2018

I've been a bit busy so I haven't had time to prepare this, but I wanted to post something people can try, so here's something.

So far I have made changes to:

  • forward declare struct timeval in pytime (it should have been anyway)
  • move some struct packing in the correct place
  • Using a more portable stringize macro
  • defining compiler names

This builds and passes tests with MSVC, and clang-cl on x64.

If you would like to test this:

  • Grab a Windows snapshot build of clang trunk from here: http://llvm.org/builds/ or build clang from source (not recommended, this is quite time, processor, and memory intensive).
    then
  • Change project files to build with the LLVM-2014 toolset (I haven't figured out a good way to do this automatically yet).

Everything builds except:

  • _decimal (clang-cl doesn't like linking mpd it seems)
  • _distutils_findvs (unsurprisingly)
  • pyshellext (it doesn't like the idl file)

The good news is that clang generates MSVC ABI compatible binaries, so linking clang/msvc binaries together is fine.

The following tests fail under clang-cl:

test_compile test_compileall test_ctypes test_exceptions
test_fileio test_io test_json test_os test_pickle test_pickletools
test_richcmp test_runpy test_script_helper test_sys test_traceback
test_userdict test_userlist test_xml_etree

It seems many of them are failing with a stack overflow, so it seems clang is miscompiling something.

There is however a subset that can compile under clang while still passing the tests.

https://bugs.python.org/issue33351

This was mostly:
- forward declare struct timeval in pytime (it should have been anyway)
- moving struct packing in the correct place
- Using a more portable, standard garunteed stringize macro
- defining compiler names
@ahartikainen
Copy link

Should this PR also add clang-cl compiler option for distutils?
clang-cl could basically use a copy of the MSVCCompiler class in _msvccompiler.py

It would only need to change line 245: self.cc = _find_exe("cl.exe", paths) to self.cc = _find_exe("clang-cl.exe", paths). Also some of the compiler arguments are unnecessary.

Also line 960 compiler_class in ccompiler.py needs updating.

@emmatyping
Copy link
Member Author

Should this PR also add clang-cl compiler option for distutils?

@ahartikainen I think that is likely the next todo I have. I'll get back to this soon hopefully.

@zooba
Copy link
Member
zooba commented Apr 12, 2019

@ethanhs Are you still interested in pushing this forward? If so, can you merge against master and run CI again.

@emmatyping
Copy link
Member Author

@zooba, yes! I have been working on other things recently, but it would be helpful to know what a mergable PR should look like. Since clang-cl can fallback to cl, there may be a few modules I will need to use that for, as mentioned in the PR comment there seems to be an issue with how it compiles things (however I should test this with a newer clang to be sure).

@zooba
Copy link
Member
zooba commented Apr 12, 2019

Of the three modules that didn't build, distutils_findvs has been removed and pyshellext is probably optional (it's part of the py.exe launcher, not Python itself). So that will get you closer.

You should be able to pass /p:PlatformToolset="whatever" on the command line (or set it as an environment variable) to override the toolset when building outside of VS. The "right" way to do in within VS is to add a new configuration, but that's far more intrusive (going to have to touch every project file and test it extensively).

@zufuliu
Copy link
zufuliu commented May 30, 2019

Add clang-cl to distutils is just a few lines of change, I added a patch at https://github.com/zufuliu/llvm-utils/blob/master/clang/clang-cl-py3.diff, which works fine for 3.7.3.

@csabella
Copy link
Contributor
csabella commented Feb 6, 2020

This has been replaced by GH-18371.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants
0