8000 Error Making Native Module Example · Issue #6264 · micropython/micropython · GitHub
[go: up one dir, main page]
Skip to content

Error Making Native Module Example #6264

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
adahl5 opened this issue Jul 21, 2020 · 18 comments
Closed

Error Making Native Module Example #6264

adahl5 opened this issue Jul 21, 2020 · 18 comments

Comments

@adahl5
Copy link
adahl5 commented Jul 21, 2020 •
edited
Loading

Hello, I'm running into an error trying to compile the native module example from here. I keep getting the error elftools.common.exceptions.ELFError: Magic number does not match make: *** [build/factorial.native.mpy] Error 1

This also happens when I try to make the examples in the examples/natmod directory as well.
@dpgeorge
Copy link
Member

I assume you refer to this docs page: https://docs.micropython.org/en/latest/develop/natmod.html#minimal-example

What host system are you compiling on: OS, make version, Python version, pyelftools version? Can you post the complete build output from make V=1 run in the examples/natmod/features0 directory?

@adahl5
Copy link
Author
adahl5 commented Jul 21, 2020 •
edited
Loading

macOS 10.15, Python 3.8.3, make 3.81, pyelftools version 0.26.
The output from make V=1 in features0 is as follows
/bin/mkdir -p build GEN build/features0.config.h python ../../../tools/mpy_ld.py '-vvv' --arch x64 --preprocess -o build/features0.config.h features0.c CC features0.c gcc -I. -I../../.. -std=c99 -Os -Wall -Werror -DNDEBUG -DNO_QSTR -DMICROPY_ENABLE_DYNRUNTIME -DMP_CONFIGFILE='<build/features0.config.h>' -fpic -fno-common -U _FORTIFY_SOURCE -fno-stack-protector -DMICROPY_FLOAT_IMPL=MICROPY_FLOAT_IMPL_DOUBLE -o build/features0.o -c features0.c LINK build/features0.o python ../../../tools/mpy_ld.py '-vvv' --arch x64 --qstrs build/features0.config.h -o build/features0.native.mpy build/features0.o qstr vals: factorial qstr objs: Traceback (most recent call last): File "../../../tools/mpy_ld.py", line 1081, in <module> main() File "../../../tools/mpy_ld.py", line 1077, in main do_link(args) File "../../../tools/mpy_ld.py", line 1047, in do_link load_object_file(env, file) File "../../../tools/mpy_ld.py", line 670, in load_object_file elf = elffile.ELFFile(f) File "/Users/adam/.pyenv/versions/3.8.3/lib/python3.8/site-packages/elftools/elf/elffile.py", line 67, in __init__ self._identify_file() File "/Users/adam/.pyenv/versions/3.8.3/lib/python3.8/site-packages/elftools/elf/elffile.py", line 421, in _identify_file elf_assert(magic == b'\x7fELF', 'Magic number does not match') File "/Users/adam/.pyenv/versions/3.8.3/lib/python3.8/site-packages/elftools/common/utils.py", line 77, in elf_assert _assert_with_exception(cond, msg, ELFError) File "/Users/adam/.pyenv/versions/3.8.3/lib/python3.8/site-packages/elftools/common/utils.py", line 109, in _assert_with_exception raise exception_type(msg) elftools.common.exceptions.ELFError: Magic number does not match make: *** [build/features0.native.mpy] Error 1

@dpgeorge
Copy link
Member

I think because you're on macOS it's producing DWARF not ELF files. Can you run file build/features0.o to see the file type?

@adahl5
Copy link
Author
adahl5 commented Jul 21, 2020

build/features0.o: Mach-O 64-bit object x86_64

@dpgeorge
Copy link
Member

Mach-O 64-bit object x86_64

Mach-O output files are not supported by this build process. At this stage there are 2 options:

  1. use a compiler that can produce ELF files
  2. use a Linux VM/container to build within

@adahl5
Copy link
Author
adahl5 commented Jul 21, 2020

Okay, is there a compiler that you know of that would produce ELF files? I'm currently trying to build in Ubuntu, I'm setting up my environment. If that works I'll close the issue.

@dpgeorge
Copy link
Member

is there a compiler that you know of that would produce ELF files?

I don't know, and I don't think it's easy (would need to build/install an elf-based toolchain). I'd suggest to use Ubuntu.

@adahl5
Copy link
Author
adahl5 commented Jul 21, 2020 •
edited
Loading

Okay, thank you. I found a converter for Mach-O to ELF and used it but now I'm getting the error LinkError: build/factorial.o: undefined symbol: _mp_fun_table
I'm getting this error on both macOS and Ubuntu.

@dpgeorge
Copy link
Member

I'm getting this error on both macOS and Ubuntu.

Do you mean that you get that undefined-symbol error when building everything under Ubuntu? Is Ubuntu running in a VM? Can you po 8000 st output of make V=1 when run in examples/natmod/features0 on Ubuntu?

@peterhinch
Copy link
Contributor

If I can chip in with my attempt, I get this outcome. My box runs native Ubuntu 18.04:

[adminpete@capybara]: /mnt/qnap2/data/Projects/MicroPython/micropython/examples/natmod/features0
$ make ARCH=armv7m
LINK build/features0.o
Traceback (most recent call last):
  File "../../../tools/mpy_ld.py", line 1081, in <module>
    main()
  File "../../../tools/mpy_ld.py", line 1077, in main
    do_link(args)
  File "../../../tools/mpy_ld.py", line 1047, in do_link
    load_object_file(env, file)
  File "../../../tools/mpy_ld.py", line 680, in load_object_file
    if s.data_size == 0:
AttributeError: 'Section' object has no attribute 'data_size'
../../../py/dynruntime.mk:139: recipe for target 'build/features0.native.mpy' failed
make: *** [build/features0.native.mpy] Error 1

The need to install pyelftools could do with being documented.

@adahl5
Copy link
Author
adahl5 commented Jul 23, 2020

Okay, when I run everything under Ubuntu it works. Thanks for your help.

@adahl5
Copy link
Author
adahl5 commented Jul 23, 2020 •
edited
Loading

Now that I've made a few changes I'm getting the error LinkError: build/test.o: .data non-empty
The output of make V=1 is LINK build/test.o python3 ../../../..//tools/mpy_ld.py '-vvv' --arch xtensawin --qstrs build/test.config.h -o build/test.native.mpy build/test.o qstr vals: capture, deinit, init qstr objs: LinkError: build/test.o: .data non-empty make: *** [build/test.native.mpy] Error 1

@dpgeorge
Copy link
Member

Now that I've made a few changes I'm getting the error LinkError: build/test.o: .data non-empty

Native .mpy files do not support a data section (eg int x = 123; in global scope). Instead use a bss variable and initialise the value explicitly (eg int x;, then x = 123; in a function).

@adahl5
Copy link
Author
adahl5 commented Jul 25, 2020 •
edited
Loading

I've moved my variable inside the function and am still getting the error. Could this be from "#define"s? I'm trying to adapt the esp32 camera library for MicroPython into a .mpy module.

Update: Got rid of all the #define's and am still having the issue. Here's a gist with the code gist

Update 2: Removed static keyword from the camera config and it fixed the issue. Now I'm having an issue where it says undefined symbol: esp_camera_init

@adahl5
Copy link
Author
adahl5 commented Jul 27, 2020

I don't know enough about C to fully understand what's going on but my guess is that the includes aren't working properly. As I had to manually define memcpy in the file even though it was in an include.

@jimmo
Copy link
Member
jimmo commented Jul 28, 2020

hi @adahl5 -- this discussion might be better suited to the forum. It would be useful if you can post the entire directory rather than just the one .c file.

Something that might be worth pointing out though -- you might find it easier to get this working as part of the main firmware first, and then later investigate turning it into a dynamic native modules. The dynamic native module system is quite complicated and there are a lot of ways to 8000 get unstuck, especially when trying to integrate with system functionality (i.e. anything that isn't in dynruntime.h). This is particularly true on ESP32 -- anything in the IDF. (See #5711 for some context there).

So instead if you start with just adding a new module to ports/esp32 (or maybe even start in ports/esp32/modesp32.c) and then go from there.

@jimmo jimmo closed this as completed Jul 28, 2020
@peterhinch
Copy link
Contributor

Comments on my above observation would be welcome: the provided example failed to build here.

@jimmo jimmo mentioned this issue Jul 29, 2020
Closed
@jimmo
Copy link
Member
jimmo commented Jul 29, 2020

Comments on my above observation would be welcome: the provided example failed to build here.

Sorry I missed that -- the pyelftools version in Ubuntu 18.04 pre-dates the required feature...

Raised #6296 to track this.

@jimmo jimmo mentioned this issue Aug 26, 2021
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@jimmo @dpgeorge @peterhinch @adahl5
0