8000 shared/tinyusb: Add common cdc tx/rx functions. by andrewleech · Pull Request #14462 · micropython/micropython · GitHub
[go: up one dir, main page]

Skip to content

shared/tinyusb: Add common cdc tx/rx functions. #14462

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
May 31, 2024

Conversation

andrewleech
Copy link
Contributor

There are a few tinyusb CDC functions used for stdio that are currently replicated across a number of ports.
Not surprisingly in a couple of cases these have started to diverge slightly, with additional features added to one of them.

This PR consolidates a couple of key shared functions used directly by tinyusb based ports.

Copy link
github-actions bot commented May 10, 2024

Code size report:

   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:    +0 +0.000% standard
      stm32:    +0 +0.000% PYBV10
     mimxrt:   +16 +0.004% TEENSY40[incl -8(bss)]
        rp2:    +4 +0.001% RPI_PICO[incl -4(bss)]
       samd:   +28 +0.011% ADAFRUIT_ITSYBITSY_M4_EXPRESS

@andrewleech andrewleech force-pushed the tinyusb_shared_cdc branch 2 times, most recently from b4de5d4 to d606676 Compare May 14, 2024 01:11
Copy link
codecov bot commented May 14, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.39%. Comparing base (3613ad9) to head (c11efc7).

Additional details and impacted files
@@           Coverage Diff           @@
##           master   #14462   +/-   ##
=======================================
  Coverage   98.39%   98.39%           
=======================================
  Files         161      161           
  Lines       21204    21204           
=======================================
  Hits        20864    20864           
  Misses        340      340           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@andrewleech andrewleech force-pushed the tinyusb_shared_cdc branch from d606676 to 278ef2d Compare May 14, 2024 01:23
@dpgeorge dpgeorge added this to the release-1.24.0 milestone May 14, 2024
Copy link
Contributor
@projectgus projectgus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really useful, thanks @andrewleech! One small change suggestion, but I think we should merge it with or without the additional name change.

@andrewleech
Copy link
Contributor Author
andrewleech commented May 15, 2024 & 8000 #8226;

Note: I've only been able to test this on RP2, I don't own samd / mimxrt / renesas-ra boards.

edit: I've got my hands on samd, mimxrt and renesas-ra boards ( thanks @mattytrentini ) but haven't been able to figure out how to enable usb on renesas yet... renesas#2

So now rp2, samd21, samd51, mimxrt have been tested and appear to work as expected!

@andrewleech andrewleech force-pushed the tinyusb_shared_cdc branch 2 times, most recently from 52361d3 to 5bedc84 Compare May 16, 2024 06:13
@projectgus
Copy link
Contributor

Note: I've only been able to test this on RP2, I don't own samd / mimxrt / renesas-ra boards.

It may be redundant as I saw the follow-on PR has been tested on samd already, but I've tested this PR on samd and it seems to work as expected. 👍

@andrewleech
Copy link
Contributor Author
andrewleech commented May 22, 2024

Note: I've only been able to test this on RP2, I don't own samd / mimxrt / renesas-ra boards.

It may be redundant as I saw the follow-on PR has been tested on samd already, but I've tested this PR on samd and it seems to work as expected. 👍

Thanks I've just updated my previous post to reflect what I've now been able to test! Regardless, I certainly appreciate any additional testing as I'm not very familiar with some of these ports!

FYI thanks to this incredibly helpful tidbit of info, I'm also looking at bringing esp32s2 and s3 into using this same shared tinyusb implementation, as well as fixing the auto-bootloader jump there for S3 which is currently incredibly frustrating!
hathach/tinyusb#2647 (comment)
That should also make it easier / possible to get the runtime usb definition stuff working there too!

It's probably worth saving the esp stuff for a follow up PR though, it's a larger parcel of work though!

@projectgus
Copy link
Contributor

@andrewleech Oh wow, that's fantastic news! I had anticipated this being a much bigger task, very exciting.

@andrewleech
Copy link
Contributor Author

So now I've tested everything here except the renesas port changes.

Can anyone provide guidance on how to update the auto generated files in a renesas board (in particular the interrupt vector header) to enable USB on a evk board? @TakeoTakahashi2020 @dpgeorge

Alternatively @iabdalkader could you perhaps test this PR doesn't break anything on a Portenta C33 (I can really afford one myself for testing with).

@andrewleech andrewleech force-pushed the tinyusb_shared_cdc branch from 5bedc84 to 41a54fa Compare May 24, 2024 03:26
@iabdalkader
Copy link
Contributor

@andrewleech Yes I can test it, perhaps later today. What do I need to test ?

@andrewleech
Copy link
Contributor Author

What do I need to test ?

Thanks, on this PR there's no intended functional change so it's just a basic check that stdio / repl still seems to work.

If possible, the bigger change is in the follow on PR #14485
For that one is checking at startup whether the banner is buffered and shown on initial connection. The description there should show pretty clearly.

If you know, I'd love some pointers on how to create (or better, modify) the auto generated code in a renesas board so I can test / document it myself too.

@iabdalkader
Copy link
Contributor

it's just a basic check that stdio / repl still seems to work

REPL is still functional on the Portenta C33.

If you know, I'd love some pointers on how to create (or better, modify) the auto generated code in a renesas board so I can test / document it myself too.

You'll need to install the e2studio, load the configuration, edit and then generate the code. For this you need the configuration.xml, I think it's this one: configuration.xml.zip

@andrewleech
Copy link
Contributor Author

You'll need to install the e2studio, load the configuration, edit and then generate the code. For this you need the configuration.xml, I think it's this one: configuration.xml.zip

Ok thanks! I previously installed RA Smart Configurator which can open a "project", not sure if that will be the XML. I'll try tomorrow.

I really think this XML/project file needs to be included in the board folder as the source of the auto gen files.

@andrewleech
Copy link
Contributor Author

REPL is still functional on the Portenta C33.

That's great thanks!

@iabdalkader
Copy link
Contributor

not sure if that will be the XML. I'll try tomorrow.

There's a cfg.txt file too, those are the only two non-generated files.
ra_cfg.txt

@andrewleech
Copy link
Contributor Author

Thanks @iabdalkader that "configuration.xml" file opened just fine in the standalone "RA Smart Configurator" application!
Looking at how everything for your board was configured in that cleared up a lot for me.
I was then able to start with a basic new configuration for my ek board and enable the USB pins and interrupts in that, following the USB FS configuration of your project as a guide.
Running the "generate files" before and after making my USB config showed only changes to pin and vector files, I was then able to use these changes as a reference to modify the existing EK board generated files to enable USB there. I needed to disable a couple of existing interrupts as that chip only supports 32 user interrupts but that's fine for this test.

That got it going for me, with a cut up usb cable wired to the USB pins and a jumper from 3v to vbus to enable it I had cdc working (quicker than wiring resister divider from USB 5v to vbus pin).

More than just replicating your testing on this PR, this more importantly helped me finish off the updates in tinyusb to support my startup buffer feature.

Thanks again! I'll try to get a more succinct version of these notes into a readme update pr to help the next person. Would you mind me also including your configuration.xml in that pr (in the matching board folder) to reference as an example?

@iabdalkader
Copy link
Contributor

Would you mind me also including your configuration.xml in that pr (in the matching board folder) to reference as an example?

No please go ahead. The configuration should have been there with the board files.

@andrewleech andrewleech force-pushed the tinyusb_shared_cdc branch from 41a54fa to 2313625 Compare May 29, 2024 01:51
@andrewleech
Copy link
Contributor Author

I found a further consolidation that made sense in moving more of the cdc stdin handling into the mp_usbd_cdc_poll_interfaces() function, de-duplicating this code everywhere else.

With that change, I've restested on all platforms covered here and all appear to be working (basic read/write on repl and mpremote mount still works).

I previously had nrf also included here but it was a larger change which grew even larger as I tested and cleaned it up so moved it to its own PR.

With this, I consider this PR complete and ready for final review.

There are a few TinyUSB CDC functions used for stdio that are currently
replicated across a number of ports.  Not surprisingly in a couple of cases
these have started to diverge slightly, with additional features added to
one of them.

This commit consolidates a couple of key shared functions used directly by
TinyUSB based ports, and makes those functions available to all.

Signed-off-by: Andrew Leech <andrew@alelec.net>
Signed-off-by: Andrew Leech <andrew@alelec.net>
Signed-off-by: Andrew Leech <andrew@alelec.net>
Signed-off-by: Andrew Leech <andrew@alelec.net>
Signed-off-by: Andrew Leech <andrew@alelec.net>
@dpgeorge dpgeorge force-pushed the tinyusb_shared_cdc branch from e7b4332 to c11efc7 Compare May 31, 2024 09:54
@dpgeorge
Copy link
Member

Thanks, this looks really good. It was always on my list to eventually factor these functions, and I'm glad someone else got to it first :)

@dpgeorge dpgeorge merged commit c11efc7 into micropython:master May 31, 2024
62 checks passed
@ricksorensen
Copy link
Contributor
ricksorensen commented Jun 1, 2024

Using pull from 31 may 2024, and master branch, NRF52840 does not compile with these changes on my system.

It cannot find the ringbuf functions. Looked at the samd mods, and updated:

diff --git a/ports/nrf/mphalport.h b/ports/nrf/mphalport.h
index 7efe05a15..a51f84ef5 100644
--- a/ports/nrf/mphalport.h
+++ b/ports/nrf/mphalport.h
@@ -28,6 +28,7 @@
 #define __NRF52_HAL
 
 #include "py/mpconfig.h"
+#include "py/ringbuf.h"   // RJS
 #include <nrfx.h>
 #include "pin.h"
 #include "nrf_gpio.h"
@@ -42,6 +43,8 @@ typedef enum
 } HAL_StatusTypeDef;
 
 extern const unsigned char mp_hal_status_to_errno_table[4];
+extern int mp_interrupt_char;     //RJS
+extern ringbuf_t stdin_ringbuf;   //RJS
 
 NORETURN void mp_hal_raise(HAL_StatusTypeDef status);
 void mp_hal_set_interrupt_char(int c); // -1 to disable

and

diff --git a/ports/nrf/mphalport.c b/ports/nrf/mphalport.c
index 06c6ba5cc..682ceff80 100644
--- a/ports/nrf/mphalport.c
+++ b/ports/nrf/mphalport.c
@@ -40,6 +40,12 @@
 #include "nrf_clock.h"
 #endif
 
+// RJS 
+#ifndef MICROPY_HW_STDIN_BUFFER_LEN
+#define MICROPY_HW_STDIN_BUFFER_LEN 128
+#endif
+static uint8_t stdin_ringbuf_array[MICROPY_HW_STDIN_BUFFER_LEN];  //RJS
+ringbuf_t stdin_ringbuf = { stdin_ringbuf_array, sizeof(stdin_ringbuf_array), 0, 0 };   //RJS
 #if !defined(USE_WORKAROUND_FOR_ANOMALY_132) && \
     (defined(NRF52832_XXAA) || defined(NRF52832_XXAB))
 // ANOMALY 132 - LFCLK needs to avoid frame from 66us to 138us after LFCLK stop.

My code now compiles (BOARD=SEEED_XIAO_NRF52), and the REPL works with new build.

Note that the 1.23.0 tag compiles and runs without modification.

I do not think these are the optimal patches as I did them simply to resolve link/compile errors based on the samd code... and I have little knowledge of tinyusb. I am particularly concerned with the extern definitions.

@robert-hh
Copy link
Contributor
robert-hh commented Jun 1, 2024

Not surprisingly, the ARDUINO_NANO_33_BLE_SENSE build shows the same problem. Looking into the code the implementations seems somewhat confusing, because there in Makefile both drivers/usb/usb_cdc.c and lib/tinyusb/src/class/cdc/cdc_device.c are referred. The former uses properly ringbuffers.
So maybe the removal of the preliminary attempt to use the common USB functions failed.

@andrewleech
Edit: It's sufficient to remove the line (188):
tinyusb/mp_usbd_cdc.c \
from Makefile.

@andrewleech
Copy link
Contributor Author

Sorry for the breakage @ricksorensen and @robert-hh !
My nrf change was initially much larger, so I split them intoa separate pull request.
I thought I'd left just the minimum needed here for changed filename but must have got it wrong.

If either of you get a chance could you take a look at #15158 ?

In that follow up PR I remove the nrf specific USB CDC file and consolidate all this functionality.

But yes it makes sense to get the master build fixed quickly with or without that PR

@robert-hh
Copy link
Contributor
robert-hh commented Jun 2, 2024

If either of you get a chance could you take a look at #15158

Will do.
Edit: This branch does not even build for a Arduino BLE 33 Sense or a SEEED XIAO NRF52. Compile errors:

rivers/usb/usb_cdc.c: In function 'mp_usbd_port_get_serial_number':
drivers/usb/usb_cdc.c:108:46: error: 'MICROPY_HW_USB_DESC_STR_MAX' undeclared (first use in this function)
  108 |     MP_STATIC_ASSERT(sizeof(deviceid) * 2 <= MICROPY_HW_USB_DESC_STR_MAX);
      |                                              ^~~~~~~~~~~~~~~~~~~~~~~~~~~
../../py/misc.h:54:61: note: in definition of macro 'MP_STATIC_ASSERT'
   54 | #define MP_STATIC_ASSERT(cond) ((void)sizeof(char[1 - 2 * !(cond)]))
      |                                                             ^~~~
drivers/usb/usb_cdc.c:108:46: note: each undeclared identifier is reported only once for each function it appears in
  108 |     MP_STATIC_ASSERT(sizeof(deviceid) * 2 <= MICROPY_HW_USB_DESC_STR_MAX);
      |                                              ^~~~~~~~~~~~~~~~~~~~~~~~~~~
../../py/misc.h:54:61: note: in definition of macro 'MP_STATIC_ASSERT'
   54 | #define MP_STATIC_ASSERT(cond) ((void)sizeof(char[1 - 2 * !(cond)]))
      |                                                             ^~~~
CC ../../lib/tinyusb/src/portable/nordic/nrf5x/dcd_nrf5x.c
drivers/usb/usb_cdc.c:109:5: error: implicit declaration of function 'mp_usbd_hex_str'; did you mean 'mp_obj_new_str'? [-Werror=implicit-function-declaration]
  109 |     mp_usbd_hex_str(serial_buf, (uint8_t *)deviceid, sizeof(deviceid));
      |     ^~~~~~~~~~~~~~~
      |     mp_obj_new_str
drivers/usb/usb_cdc.c: In function 'usb_cdc_init':
drivers/usb/usb_cdc.c:124:5: error: implicit declaration of function 'mp_usbd_init'; did you mean 'mp_set_init'? [-Werror=implicit-function-declaration]
  124 |     mp_usbd_init();
      |     ^~~~~~~~~~~~
      |     mp_set_init
CC device/startup_nrf52840.c
drivers/usb/usb_cdc.c: In function 'USBD_IRQHandler':
drivers/usb/usb_cdc.c:143:5: error: implicit declaration of function 'tud_int_handler' [-Werror=implicit-function-declaration]
  143 |     tud_int_handler(0);
      |     ^~~~~~~~~~~~~~~

Rebasing to the current master branch fails as well due to conflicts in the make files of the other ports. Nothings difficult, just duplicating references to files, but it breaks rebase.

@ricksorensen
Copy link
Contributor

For me both boards compile okay with #15158. (Time on history is CDT (-5)

  980  2024/06/01 21:33:05 git clone https://github.com/andrewleech/micropython.git trynrf
  982  2024/06/01 21:33:57 cd trynrf/
  984  2024/06/01 21:34:17 make -C mpy-cross
  985  2024/06/01 21:34:47 cd ports/nrf/
  991  2024/06/01 21:35:56 ./drivers/bluetooth/download_ble_stack.sh 
  993  2024/06/01 21:36:23 make BOARD=SEEED_XIAO_NRF52 submodules
  995  2024/06/01 21:37:05 make BOARD=SEEED_XIAO_NRF52

I built yesterday but did not have access to my SEEED_XIAO_NRF52 board until this morning ... the REPL works. The only difference I see in the nrf port is in the ports/nrf/Makefile

diff  [thispr]/ports/nrf/Makefile [currupy]/ports/nrf/Makefile 
188c188
< 	tinyusb/mp_cdc_common.c \             # this PR
---
> 	tinyusb/mp_usbd_cdc.c \               # may 31 micropython

I also tried the pull request from this morning (#15189) which also worked. Same differences as above.

@dpgeorge
Copy link
Member
dpgeorge commented Jun 2, 2024

The esp32 and nrf build issues introduced by this PR should be fixed by #15189.

@iabdalkader
Copy link
Contributor

The esp32 and nrf build issues introduced by this PR should be fixed by #15189.

The Portenta C33 firmware has been broken after 1.23.0. It's likely that these changes and following ones, broke something. Also in 1.23.0 WiFi hangs. The last known good release was 1.22.1

@andrewleech
Copy link
Contributor Author
andrewleech commented Jul 25, 2024

The Portenta C33 firmware has been broken after 1.23.0

Can you give more details about the failures? Does repl not respond? Or is it related more to the 1200bps bootloader?

Wifi is a tricky one, maybe at least one of the boards using an esp-coprocessor based wifi could be added to the boards @dpgeorge or others involved in release testing has for running multitests on? Though I'm aware that's a laborious process already... hopefully the automated testing discussed in the meetup the other night can progress to help here!

oh, is the failure you're referring to the compile error?

../../lib/mbedtls_errors/mp_mbedtls_errors.c:275:13: error: 'MBEDTLS_ERR_PK_BUFFER_TOO_SMALL' undeclared here (not in a function); did you mean 'MBEDTLS_ERR_MPI_BUFFER_TOO_SMALL'?
  275 |         { -(MBEDTLS_ERR_PK_BUFFER_TOO_SMALL), "PK_BUFFER_TOO_SMALL" },
      |             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |             MBEDTLS_ERR_MPI_BUFFER_TOO_SMALL

Maybe this is a board that should be in CI, it's arguably the renesas board with the most features currently so covers the widest range of supported libraries for that port really?

@iabdalkader
Copy link
Contributor
iabdalkader commented Jul 25, 2024

oh, is the failure you're referring to the compile error?

No, the firmware builds fine but USB doesn't enumerate. When the firmware is loaded and the board exits DFU the device becomes dead.

Maybe this is a board that should be in CI, it's arguably the renesas board with the most features currently so covers the widest range of supported libraries for that port really?

I think it's already in CI.

@andrewleech
Copy link
Contributor Author

I think it's already in CI.

You're right, it is. It was a submodules issue on my end with the compile.

USB not enumerating at all? That sounds like either

  • tinyusb isn't being serviced correctly anymore; it's currently done in mpconfigport.h / MICROPY_HW_ENABLE_USBDEV / MICROPY_EVENT_POLL_HOOK which should possibly get moved to an interrupt wrap like most other ports now.
  • tinyusb update has broken the HS driver for this port?

I just retested master-ish on the "EK-RA4W1 with usb cable patched on" I've got , but it's just the HS peripheral which is a somewhat different driver to the HS one you're using iirc.

@andrewleech
Copy link
Contributor Author

@iabdalkader if you can it would be helpful to test #15550 to see if that changes anything for you?

Also you could try downgrading tinyusb submodule back to 1fdf2907 I think that's the commit it was for the previous release.

@iabdalkader
Copy link
Contributor

I did but it's still not working. I also reverted d144f06 (had to comment out SOF enable), and doesn't work either.

@iabdalkader
Copy link
Contributor
iabdalkader commented Jul 29, 2024

Reverting this 2d33071
Fixes it.

>>> 
MPY: sync filesystems
MPY: soft reboot
MicroPython v1.24.0-preview.143.ge1fe62f4f.dirty on 2024-07-29; Arduino Portenta C33 with RA6M5
Type "help()" for more information.
>>> 

FYI there's no issue with WiFi, it was just an old rev board with a broken crystal that made NTP/RTC get stuck.

@andrewleech
Copy link
Contributor Author

@iabdalkader that's really surprising that the shared code broke the USB support. There must be some difference between the FS and HS peripheral that's conflicting with that commit. If there's some way I could get my hands on a portenta I'd be happy to investigate the problem, though I don't know anyone local to me in Melbourne with one I could borrow.

@iabdalkader
Copy link
Contributor

@andrewleech Somehow it has something to do with the RTC/LSE issue on the old revision boards. Before switching to the shared TinyUSB code, this board used to work. If things are working fine on production boards then it's not a problem. I've asked someone to test on a different board to confirm.

@andrewleech andrewleech deleted the tinyusb_shared_cdc branch August 23, 2024 04:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants
0