8000 Question on building a hybrid C/C++ extension. · Issue #21 · pybind/python_example · GitHub
[go: up one dir, main page]

Skip to content

Question on building a hybrid C/C++ extension. #21

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
mreineck opened this issue Apr 12, 2017 · 13 comments
Closed

Question on building a hybrid C/C++ extension. #21

mreineck opened this issue Apr 12, 2017 · 13 comments

Comments

@mreineck
Copy link

While trying to use pybind11 for interfacing a hybrid C/C++ codebase, following your example project, I encountered the following problem:

If the source files are a mix of C and C++ files, some compilers (most notably clang) complain and stop if the "-std=c++14" flag is supplied when compiling a C file.

This is of course completely reasonable, but even after extensive searching on the web I did not find a way to specify separate C and C++ compilation flags to setuptools (in analogy to CFLAGS and CXXFLAGS use by autotools).

Do you happen to know a trick how to achieve this? If so, I think it would be great if this could be added to your test project, to illustrate how it is done!

@MathMagique
Copy link

I do not know if it is possible for your use case, but you could try to separate the C code into a separate extension module. So you would have a C extension module and a C++ extension module that links to the C module. Then you can use separate flags for each module.

In my project, I use multiple extension modules for other reasons, but also require linking between them: https://github.com/blue-yonder/turbodbc/blob/master/setup.py#L116 (get_extension_modules() function).

@mreineck
Copy link
Author
mreineck commented Sep 9, 2017

I'm not sure whether that works in my case. The problem is that the C library does not provide any Python interface, but is exclusively needed by the C++ code; so I would basically have to "misuse" the setup.py machinery to build/install a package with no Python interface at all. The problem will probably be the C header files, which the C++ library has to locate somehow.

I'll give it a try!

@MathMagique
Copy link

My project has a libturbodbc "extension" that does not expose Python stuff at all. The header files are easily found since you have to put it in the package anyways. I think you will be fine!

@mreineck
Copy link
Author

Thanks a lot! Looking at your setup.py has helped me write my own version, which works almost perfectly now.

The only remaining problem is that pybind11.get_include() reports incorrect paths ... I installed it via 'pip install --user', but it gives me a path of "/usr/local/include/python2.7", where there are no headers, of course. But that's a different issue...

@dean0x7d
Copy link
Member

Use pybind11.get_include(True) to get the --user install path.

@SylvainCorlay
Copy link
Member

@mreineck I should point out that user install are generally evil 👿, especially for python packages, jupyter extensions, and headers!

The reason is that any python installation would pick up these with a higher precedence than the ones of their prefix.

@dean0x7d
Copy link
Member

I should point out that user install are generally evil

Less evil than sudo installs...

@SylvainCorlay
Copy link
Member

Well, the right way to do things is a prefix install...

@mreineck
Copy link
Author

I see that I'll need a lot more time before I grasp all the intricacies of Python packages ... but for the moment I think I have all the ingredients I need! Thank you all for the help!

@benmaier
Copy link

short off-topic question concerning the --user install comment by @SylvainCorlay

I'm ususally compiling/installing my custom pybind11-packages on a sun grid engine compute cluster, where python is installed at root level using several specifically compiled libraries, e.g. LBLAS, etc. So I'm not going to install a new python environment. I found installing via --user extremely useful since I obviously don't have root permission, I only need to install it once and it works for every node (user directories are shared between compute nodes), and I can still use the packages installed at root level.
Is there an alternative, more canonic way to do what I need to do there? Otherwise I'd say --user is pretty useful in this case.

@SylvainCorlay
Copy link
Member

User installs only start being an issue when you use other environments than the root one, so you should be fine.

As soon as you have more than one, it start being nearly impossible to keep a consistent state with user installs.

@henryiii
Copy link
Collaborator

I think the final issue was fixed around pybind11 2.5.

@Schlaubischlump
Copy link
Schlaubischlump commented Apr 12, 2021

The problem still exists for pybind 2.6.2. Mixing C and C++ still fails, when using clang

error: invalid argument '-std=c++17' not allowed with 'C'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants
0