8000 zephyr: Upgrade to Zephyr v3.7.0. by MaureenHelm · Pull Request #9335 · micropython/micropython · GitHub
[go: up one dir, main page]

Skip to content

zephyr: Upgrade to Zephyr v3.7.0. #9335

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Oct 3, 2024

Conversation

MaureenHelm
Copy link
Contributor
@MaureenHelm MaureenHelm commented Sep 16, 2022

Updates the Zephyr port build instructions and CI to use the latest
Zephyr release tag.

Tested on frdm_k64f.

In draft status until the Zephyr release is finalized (currently in release candidate / stabilization phase).
Zephyr v3.2.0 was released on Sep 30 2022.
Zephyr v3.3.0 was released on Feb 19 2023.
Zephyr v3.4.0 was released on Jun 16 2023.
Zephyr v3.5.0 was released on Oct 20 2023.
Zephyr v3.6.0 was released on Feb 23 2024.
Zephyr v3.7.0 was released on Jul 26 2024.

Fixes #9115

@codecov-commenter
Copy link
codecov-commenter commented Sep 16, 2022

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.57%. Comparing base (17d8234) to head (1291718).
Report is 11 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #9335   +/-   ##
=======================================
  Coverage   98.57%   98.57%           
=======================================
  Files         164      164           
  Lines       21336    21336           
=======================================
  Hits        21031    21031           
  Misses        305      305           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@MaureenHelm MaureenHelm marked this pull request as ready for review October 4, 2022 21:55
@MaureenHelm MaureenHelm changed the title zephyr: Upgrade to Zephyr v3.2.0-rc1. zephyr: Upgrade to Zephyr v3.2.0. Oct 4, 2022
@@ -36,7 +36,10 @@ Use the :ref:`machine.Pin <machine.Pin>` class::

from machine import Pin

pin = Pin(("GPIO_1", 21), Pin.IN) # create input pin on GPIO1
gpio1 = "gpio@400ff040" # GPIO1 device name
gpio2 = "gpio@400ff080" # GPIO2 device name
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems quite.... un-user-friendly, to know the address. Is there a better solution? How do other Zephyr projects handle this? Maybe in the device tree / overlay we can have an alias?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As @jimmo says below, he'll look into this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please have a look at #10859

@jimmo
Copy link
Member
jimmo commented Oct 5, 2022

Thanks @MaureenHelm !!

I think we're going to need to come up with a solution for the issue discussed in #9115 before merging this, i.e. we need to make it possible to instantiate pins and other peripherals without needing to know the register base addresses. I will investigate.

@GeorgeCGV GeorgeCGV mentioned this pull request Nov 29, 2022
@MaureenHelm MaureenHelm changed the title zephyr: Upgrade to Zephyr v3.2.0. zephyr: Upgrade to Zephyr v3.3.0. Feb 24, 2023
@github-actions
Copy link
github-actions bot commented Feb 24, 2023

Code size report:

   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:    +0 +0.000% standard
      stm32:    +0 +0.000% PYBV10
     mimxrt:    +0 +0.000% TEENSY40
        rp2:    +0 +0.000% RPI_PICO_W
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
  qemu rv32:    +0 +0.000% VIRT_RV32

@bogdanm
Copy link
bogdanm commented Feb 27, 2023

Adding a few thoughts to this discussion: we might be able to use some of the build files generated by Zephyr to solve this problem. For example, there is a build/zephyr/dts.cmake file that looks like this:

...
set_target_properties(devicetree_target PROPERTIES "DT_NODELABEL|gpio0" "/soc/peripheral@50000000/gpio@842500")
set_target_properties(devicetree_target PROPERTIES "DT_NODELABEL|gpio1" "/soc/peripheral@50000000/gpio@842800")
...

This can be parsed by a Python script to generate more friendly names (gpio0 or GPIO_0 instead of gpio@842500) for the various GPIO ports and possible other DT nodes (a cmake wizard might be able to do this directly from cmake, but a cmake wizard I am not). I'll check which Zephyr script(s) are used to generate this file, we might be able to use them.

There's also build/zephyr/zephyr.dts which can be used in the same way, not sure which one is easier (cmake parsing for the above file would be pretty easy to do in Python, but I'm guessing there are some Python packages that can parse .dts, so the latter option might be easier after all).

The point is that we probably need this additional processing step and it needs to be plugged in into Zephyr's cmake machinery. I'm not sure how to do this yet, but I'm going to look deeper into it, because I don't see a better way to fix this.

@MaureenHelm MaureenHelm changed the title zephyr: Upgrade to Zephyr v3.3.0. zephyr: Upgrade to Zephyr v3.4.0. Aug 31, 2023
@MaureenHelm
Copy link
Contributor Author

Rebased and updated for Zephyr v3.4.0. The issue discussed in #10859 and #9115 still remains.

@jimmo
Copy link
Member
jimmo commented Aug 31, 2023

Rebased and updated for Zephyr v3.4.0. The issue discussed in #10859 and #9115 still remains.

I have a resolution to both these working, based on the idea demonstrated by @bogdanm , just haven't found the time this week to get the PR finalised.

@MaureenHelm MaureenHelm changed the title zephyr: Upgrade to Zephyr v3.4.0. zephyr: Upgrade to Zephyr v3.5.0. Oct 27, 2023
@MaureenHelm
Copy link
Contributor Author

Rebased and updated for Zephyr v3.5.0

@MaureenHelm
Copy link
Contributor Author

Rebased and updated for Zephyr v3.6.0

@@ -36,7 +36,10 @@ Use the :ref:`machine.Pin <machine.Pin>` class::

from machine import Pin

pin = Pin(("GPIO_1", 21), Pin.IN) # create input pin on GPIO1
gpiob = "gpiob" # GPIO port B devicetree node label
gpioc = "gpioc" # GPIO port C devicetree node label
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it worth defining variables like this? The original example code had the strings written inline in the Pin constructor, and I think that's slightly better from an example point of view because it makes it easy to just cut-and-paste a single line and it should just work. Eg:

pin = Pin(("gpiob", 21), Pin.IN)

That's obvious at a glance just at that line how to specify a pin name.

dev = device_get_by_dt_nodelabel(dev_name);
}
#endif

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest making a general helper function for this bit:

const struct device *zephyr_device_find(mp_obj_t name) {
    const char *dev_name = mp_obj_str_get_str(name);
    const struct device *dev = device_get_binding(dev_name);

    #ifdef CONFIG_DEVICE_DT_METADATA
    if (dev == NULL) {
        dev = device_get_by_dt_nodelabel(dev_name);
    }
    #endif

    if (dev == NULL) {
        #if MICROPY_ERROR_REPORTING <= MICROPY_ERROR_REPORTING_TERSE
        mp_raise_ValueError(MP_ERROR_TEXT("device not found"));
        #else
        mp_raise_msg_varg(&mp_type_ValueError, MP_ERROR_TEXT("device %s not found"), dev_name);
        #endif
    }

    return dev;
}

That can be used in quite a few places in this PR.

@dpgeorge dpgeorge added this to the release-1.24.0 milestone Sep 9, 2024
@danicampora
Copy link
Member

Amazing! Looking forward for this to be merged :-)

@MaureenHelm
Copy link
Contributor Author

Anything else needed to get this merged?

Zephyr v3.1.0 moved all public headers to include/zephyr. Updates a few
Zephyr include paths that were missed in
4fd54a4.

Signed-off-by: Maureen Helm <maureen.helm@intel.com>
Zephyr v3.2.0 deprecated include/zephyr/zephyr.h in favor of
include/zephyr/kernel.h since it only included that header.

Signed-off-by: Maureen Helm <maureen.helm@intel.com>
@dpgeorge
Copy link
Member
dpgeorge commented Oct 1, 2024

@MaureenHelm If you don't mind, during merge I would squash some of the commits here, eg it probably needs only one commit to update the Docker image and SDK in tools/ci.sh, and then one commit to update to Zephyr v3.7.0.

@MaureenHelm
Copy link
Contributor Author

@MaureenHelm If you don't mind, during merge I would squash some of the commits here, eg it probably needs only one commit to update the Docker image and SDK in tools/ci.sh, and then one commit to update to Zephyr v3.7.0.

Ok

MaureenHelm and others added 6 commits October 2, 2024 07:49
Zephyr v3.2.0 deprecated the devicetree label property as a base
property, which had been used as the device name string for
device_get_binding(). The device name string is now the devicetree node
name appended with its unit-address. Update Zephyr port documentation
to reflect this change.

Signed-off-by: Maureen Helm <maureen.helm@intel.com>
Zephyr v3.2.0 deprecated FLASH_AREA macros in favor of FIXED_PARTITION
macros, using node labels instead of node label properties to reference
flash storage partitions.

Signed-off-by: Maureen Helm <maureen.helm@intel.com>
Zephyr v3.4.0 changed the declaration of the main function to return an
int to allow building Zephyr without the -ffreestanding compiler flag.

Signed-off-by: Maureen Helm <maureen.helm@analog.com>
Zephyr v3.4.0 changed the SPI chip select from a pointer to a struct
member to allow using the existing SPI dt-spec macros in C++.

Signed-off-by: Maureen Helm <maureen.helm@analog.com>
Upgrades CI to use the latest versions of the Zephyr docker image and
Zephyr SDK.

Signed-off-by: Maureen Helm <maureen.helm@analog.com>
Updates the Zephyr port build instructions and CI to use the latest
Zephyr release tag.

Tested on frdm_k64f.

Signed-off-by: Maureen Helm <maureen.helm@analog.com>
@dpgeorge
Copy link
Member
dpgeorge commented Oct 1, 2024

I've rebased this on latest master, and squashed some of the commits, and force-pushed to this branch.

I was ready to merge it but the CI is not happy:

/micropython/py/emitbc.c:926:1: fatal error: error writing to /tmp/ccCLRy5M.s: No space left on device
  926 | };
      | ^
compilation terminated.

Looks like the CI VM itself ran out of disk space?? That's only when building boards latter in the test, so maybe we can clean up previous builds to free up space?

There's also the following warning which I think needs to be fixed:

/micropython/ports/zephyr/zephyr_device.c: In function 'zephyr_device_find':
/micropython/ports/zephyr/zephyr_device.c:32:32: warning: implicit declaration of function 'device_get_binding' [-Wimplicit-function-declaration]
   32 |     const struct device *dev = device_get_binding(dev_name);
      |                                ^~~~~~~~~~~~~~~~~~
/micropython/ports/zephyr/zephyr_device.c:32:32: warning: initialization of 'const struct device *' from 'int' makes pointer from integer without a cast [-Wint-conversion]

Refactors Zephyr device lookup operations into a common helper function
to reduce boilerplate code that was repeated in multiple modules.

Suggested-by: Damien George <damien@micropython.org>
Signed-off-by: Maureen Helm <maureen.helm@analog.com>
Zephyr v3.7.0 added a new feature to allow getting devices by their
devicetree node labels. Use this feature in the MicroPython Zephyr port
to simplify constructing machine module objects, including Pin, SPI,
I2C, and UART. It's still possible to use the more verbose device names
(e.g., gpio@400ff040, i2c@40066000, spi@4002c000), but now we can also
use their devicetree node labels (e.g., gpiob, i2c0, spi0).

Node labels aren't standardized across all SoC families because they
generally try to follow their respective SoC hardware user manual naming
convention, however many boards define common labels for devices routed
to Arduino headers (e.g., arduino_i2c, arduino_serial, and arduino_spi).
That means I2C("arduino_i2c") will work on quite a few boards (>100 in
the main Zephyr tree).

Signed-off-by: Maureen Helm <maureen.helm@analog.com>
@dpgeorge
Copy link
Member
dpgeorge commented Oct 2, 2024

I fixed the CI out-of-space issue by using https://github.com/marketplace/actions/free-disk-space-ubuntu in the zephyr workflow.

The zephyr builds take up quite a lot of space.

Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge dpgeorge merged commit 1291718 into micropython:master Oct 3, 2024
67 checks passed
@dpgeorge
Copy link
Member
dpgeorge commented Oct 3, 2024

Merged!

Thanks again @MaureenHelm for keeping this PR up-to-date.

@MaureenHelm
Copy link
Contributor Author

There's also the following warning which I think needs to be fixed:

/micropython/ports/zephyr/zephyr_device.c: In function 'zephyr_device_find':
/micropython/ports/zephyr/zephyr_device.c:32:32: warning: implicit declaration of function 'device_get_binding' [-Wimplicit-function-declaration]
   32 |     const struct device *dev = device_get_binding(dev_name);
      |                                ^~~~~~~~~~~~~~~~~~
/micropython/ports/zephyr/zephyr_device.c:32:32: warning: initialization of 'const struct device *' from 'int' makes pointer from integer without a cast [-Wint-conversion]

Fixed

I fixed the CI out-of-space issue by using github.com/marketplace/actions/free-disk-space-ubuntu in the zephyr workflow.

Thanks!

@MaureenHelm MaureenHelm deleted the zephyr-v3.2.0 branch October 3, 2024 14:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

"ValueError: invalid port" on machine module due to devicetree property deprecation in zephyr (?)
8 participants
0