stm32,rp2,esp32: Improve consistency of board and variant names. #12240

jimmo · 2023-08-15T15:03:08Z

Rename all nRF boards to use uppercase. I don't know the original history, but the nRF port has always had lower-case board names, and it's the only one that does this.
Remove the custom overrides of board names via the id field in board.json. This allowed e.g. PYBV11 to be built as pybv11 on the downloads page (and e.g. esp32/GENERIC as esp32). Because we now have a clash between esp8266 and esp32 GENERIC, these boards have been renamed to ESP8266_GENERIC and ESP32_GENERIC respectively.
Add board variant support for ESP8266 (and move the OTA generation into the Makefile rather than custom logic in autobuild/build-esp8266-latest.sh).
Change variant names to be uppercase. Originally these were lower-case to match the original names given to pyboard variants on the pre-boards.json downloads page.
Standardise on a convention for naming flash-size variants, e.g. FLASH_1M. Normally we would write e.g. 2MiB, but in uppercase, it's awkward to write 2MIB, so instead use 2M, 512K, etc for variant names, but use 2MiB when not constrained by case (e.g. a regular filename). This updates the WEACTSTUDIO board, and also applies to the new ESP8266_GENERIC board.
Append the board variant to the build directory. i.e. when building the DP_THREAD variant of PYBV11, the output directory will be build-PYBV11-DP_THREAD. This makes it clearer that you're building (or not building) the variant, and also allows you to switch between variants without clobbering the output directory.

This PR will require a bunch of renames to be applied to the files on the download server. I will add a comment tomorrow with the full list.

This work was funded through GitHub Sponsors.

github-actions · 2023-08-15T15:12:39Z

Code size report:

   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:    +0 +0.000% standard
      stm32:    +0 +0.000% PYBV10
     mimxrt:    +0 +0.000% TEENSY40
        rp2:    +0 +0.000% PICO
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS

codecov · 2023-08-15T15:35:48Z

Codecov Report

Merging #12240 (326dfd2) into master (2919a9f) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master   #12240   +/-   ##
=======================================
  Coverage   98.38%   98.38%           
=======================================
  Files         158      158           
  Lines       20900    20900           
=======================================
  Hits        20563    20563           
  Misses        337      337

jimmo · 2023-08-16T01:02:06Z

Updated esp32 and esp8266 to warn when trying to build with BOARD=GENERIC.

$ blaze BOARD=GENERIC_S3_SPIRAM
Makefile:18: The GENERIC* boards have been renamed to ESP32_GENERIC*
Makefile:20: *** Invalid BOARD specified: boards/GENERIC_S3_SPIRAM.  Stop.

jimmo · 2023-08-16T02:46:09Z

Here's a script that generates the renames for this PR and #12088

#!/bin/bash

# nRF port - rename boards from lower to upper (and - to _)
for board in actinius_icarus arduino_nano_33_ble_sense arduino_primo blueio_tag_evim dvk_bl652 evk_nina_b1 evk_nina_b3 feather52 ibk_blyst_nano idk_blyst_nano microbit nrf52840-mdk-usb-dongle particle_xenon pca10000 pca10001 pca10028 pca10031 pca10040 pca10056 pca10059 pca10090 seeed_xiao_nrf52 wt51822_s4at; do
    echo
    echo "# nRF:" $board
    board_upper=$(echo $board | tr a-z- A-Z_)
    for f in $board-20*; do echo mv $f ${f/$board/$board_upper}; done
done

# stm32, renesas, cc3200 - removed ID override from board.json. The previous
# overrides were exactly the same as the directory names except case and
# hyphen.
for board in MIKROE-QUAIL PYBD-SF2 PYBD-SF3 PYBD-SF6 EK-RA4M1 EK-RA4W1 EK-RA6M1 EK-RA6M2 RA4M1-CLICKER VK-RA6M5 wipy; do
    echo
    echo "# stm32, renesas, cc3200:" $board
    board_upper=$(echo $board | tr a-z- A-Z_)
    for f in $board-20*; do echo mv $f ${f/$board/$board_upper}; done
done

# pyboards - as above but also upper case the variant name
for board in pyblitev10 pybv10 pybv11; do
    for variant in dp dp-thread network thread; do
        echo
        echo "# stm32:" $board $variant
        board_upper=$(echo $board | tr a-z- A-Z_)
        variant_upper=$(echo $variant | tr a-z- A-Z_)
        for f in $board-$variant-20*; do
            nf=${f/$board/$board_upper}
            nf=${nf/-$variant-/-$variant_upper-}
            echo mv $f $nf
        done
    done
    for f in $board-20*; do
        echo mv $f ${f/$board/$board_upper}
    done
done

# archive pybv3
echo
echo "# pybv3"
for f in pybv3*; do echo mv $f archive/$f; done

# rp2 - as above, but also remove "rp2-" prefix and replace it with RPI_.
for board in rp2-pico rp2-pico-w; do
    echo
    echo "# rp2:" $board
    board_upper=$(echo $board | cut -c5- | tr a-z- A-Z_)
    for f in $board-20*; do echo mv $f ${f/$board/RPI_$board_upper}; done
done

# rp2 - rename WEACTSTUDIO variants
for f in WEACTSTUDIO-flash_2mb-*; do echo mv $f ${f/WEACTSTUDIO-flash_2mb/WEACTSTUDIO-FLASH_2M}; done
for f in WEACTSTUDIO-flash_4mb-*; do echo mv $f ${f/WEACTSTUDIO-flash_4mb/WEACTSTUDIO-FLASH_4M}; done
for f in WEACTSTUDIO-flash_8mb-*; do echo mv $f ${f/WEACTSTUDIO-flash_8mb/WEACTSTUDIO-FLASH_8M}; done

# esp32 - take idf4/5 (v1.12->v1.20) builds, and move the idf3 ones (v1.9.4->v1.14) to the "IDF3" variant.
echo
echo "# esp32"
for f in esp32-idf4-20*.*; do echo mv $f ${f/esp32-idf4/ESP32_GENERIC}; done
for f in esp32-idf3-20*.*; do echo mv $f ${f/esp32-idf3/ESP32_GENERIC-IDF3}; done
for f in esp32-20*.*; do echo mv $f ${f/esp32/ESP32_GENERIC}; done

# esp32spiram - take idf4/5 (v1.12->v1.20) builds only as the "SPIRAM" variant, and archive idf3 ones.
echo
echo "# esp32-spiram"
for f in esp32spiram-idf4-20*.*; do echo mv $f ${f/esp32spiram-idf4/ESP32_GENERIC-SPIRAM}; done
for f in esp32spiram-idf3-20*.*; do echo mv $f archive/$f; done
for f in esp32spiram-20*.*; do echo mv $f ${f/esp32spiram/ESP32_GENERIC-SPIRAM}; done
for f in esp32-spiram-20*.*; do echo mv $f ${f/esp32-spiram/ESP32_GENERIC-SPIRAM}; done

# esp32-variants: the esp32 variants are only idf4
for variant in d2wd ota unicore; do
    echo
    echo "# esp32" $variant
    variant_upper=$(echo $variant | tr a-z- A-Z_)

    for f in esp32-$variant-20*.*; do echo mv $f ${f/esp32-$variant/ESP32_GENERIC-$variant_upper}; done
done

# esp32c3 - keep the USB one, archive the old default
echo
echo "# esp32c3"
for f in esp32c3-20210902-v1.17.*; do echo mv $f ${f/esp32c3/ESP32_GENERIC_C3}; done
for f in esp32c3-usb-20*; do echo mv $f ${f/esp32c3-usb/ESP32_GENERIC_C3}; done
for f in esp32c3-20*-v1.18* esp32c3-20*-v1.19* esp32c3-20*-v1.20*; do echo mv $f archive/$f; done

# esp32s2 - keep the non-spiram old builds. archive the s2 wrover builds (as
# of now, the s2 builds support both spiram and non-spiram, but we're
# essentially removing historical access to generic spiram s2 builds)
echo
echo "# esp32s2"
for f in GENERIC_S2-20*; do echo mv $f ${f/GENERIC_S2/ESP32_GENERIC_S2}; done
for f in ESP32_S2_WROVER-20*; do echo mv $f archive/$f; done

# esp32s3 - spiram_oct to variant, use spiram as the default now, archive the non-spiram.
echo
echo "# esp32s3"
for f in GENERIC_S3_SPIRAM_OCT-20*; do echo mv $f ${f/GENERIC_S3_SPIRAM_OCT/ESP32_GENERIC_S3-SPIRAM_OCT}; done
for f in GENERIC_S3-spiram-oct-20*; do echo mv $f ${f/GENERIC_S3-spiram-oct/ESP32_GENERIC_S3-SPIRAM_OCT}; done
for f in GENERIC_S3_SPIRAM-20*; do echo mv $f ${f/GENERIC_S3_SPIRAM/ESP32_GENERIC_S3}; done
for f in GENERIC_S3-20*; do echo mv $f archive/$f; done

# all other esp32 - remove custom names
echo
echo "# esp32 other"
for f in wesp32-20*; do echo mv $f ${f/wesp32/SIL_WESP32}; done
for f in feathers2-20*; do echo mv $f ${f/feathers2/UM_FEATHERS2}; done
for f in featherS2-20*; do echo mv $f ${f/featherS2/UM_FEATHERS2}; done
for f in featherS2neo-20*; do echo mv $f ${f/featherS2neo/UM_FEATHERS2NEO}; done
for f in feathers3-20*; do echo mv $f ${f/feathers3/UM_FEATHERS3}; done
for f in pros3-20*; do echo mv $f ${f/pros3/UM_PROS3}; done
for f in tinypico-20*; do echo mv $f ${f/tinypico/UM_TINYPICO}; done
for f in tinypico-idf4-20*; do echo mv $f ${f/tinypico/UM_TINYPICO}; done
for f in tinypico-idf3-20*; do echo mv $f ${f/tinypico/UM_TINYPICO-IDF3}; done
for f in tinys2-20*; do echo mv $f ${f/tinys2/UM_TINYS2}; done
for f in tinys3-20*; do echo mv $f ${f/tinys3/UM_TINYS3}; done

# esp8266 - rename to ESP8266_GENERIC and fix variant names
echo
echo "# esp8266"
for f in esp8266-ota-20*; do echo mv $f ${f/esp8266-ota/ESP8266_GENERIC-OTA}; done
for f in esp8266-1m-20*; do echo mv $f ${f/esp8266-1m/ESP8266_GENERIC-FLASH_1M}; done
for f in esp8266-512k-20*; do echo mv $f ${f/esp8266-512k/ESP8266_GENERIC-FLASH_512K}; done
for f in esp8266-20*; do echo mv $f ${f/esp8266/ESP8266_GENERIC}; done

jimmo · 2023-08-16T03:16:49Z

Updated PICO --> RPI_PICO and PICO_W --> RPI_PICO_W.

PICO might not always be a unique name across all ports, and the convention generally for other boards is to do VENDOR_BOARD.

Like the esp32 and esp8266, I've added a warning to the makefile if someone tries to set BOARD=PICO.

(Updated the rename script above to include this).

jimmo · 2023-08-21T03:29:52Z

I have added a commit to add MICROPY_HW_ENABLE_UART_REPL to ESP32_GENERIC_C3 which should have been included in the previous PR. See https://github.com/orgs/micropython/discussions/12269#discussioncomment-6776280

dpgeorge · 2023-08-23T02:13:36Z

ports/nrf/README.md

+PCA10056             | s140                    | Peripheral and Central |                | [Segger](#segger-targets)
+PCA10059             | s140                    | Peripheral and Central | OpenBootloader | [nrfutil](#nrfutil-targets)
+PARTICLE_XENON       | s140                    | Peripheral and Central |                | [Black Magic Probe](#black-magic-probe-targets)
+NRF52840-MDK-USB-DONGLE | s140                 | Peripheral and Central | OpenBootloader | [nrfutil](#nrfutil-targets)


should these hyphens be underscores? (also below in this file)

dpgeorge · 2023-08-23T02:16:14Z

tools/ci.sh

-    make ${MAKEOPTS} -C ports/esp32 BOARD=GENERIC_S3
+    make ${MAKEOPTS} -C ports/esp32 BOARD=ESP32_GENERIC_C3
+    make ${MAKEOPTS} -C ports/esp32 BOARD=ESP32_GENERIC_S2
+    make ${MAKEOPTS} -C ports/esp32 BOARD=ESP32_GENERIC_S3


this change is in the wrong commit

This is to match all the other ports. This work was funded through GitHub Sponsors. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

This is to support a future change to add the variant name to the build directory and therefore should be the same style as the board name. This work was funded through GitHub Sponsors. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

This is to support a future change to add the variant name to the build directory and therefore should be the same style as the board name. This only affects the WEACTSTUDIO board. Also standardises on a convention for naming flash-size variants. Normally we would write e.g. 2MiB, but in uppercase, it's awkward to write 2MIB, so instead use 2M, 512K, etc for variant names, but use 2MiB when not constrained by case (e.g. a regular filename). This work was funded through GitHub Sponsors. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

This is to support a future change to add the variant name to the build directory and therefore should be the same style as the board name. This work was funded through GitHub Sponsors. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

This was used to override the firmware filename generated by the build server (to match the historical name before board definitions existed). Now we're making every E7F5 thing use the board definition name (i.e. the directory name). This work was funded through GitHub Sponsors. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

This allows switching between variants without clobbering the build output. This work was funded through GitHub Sponsors. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

Board names need to be unique across ports, and GENERIC clashes with the ESP8266 (which will be renamed to ESP8266_GENERIC). This work was funded through GitHub Sponsors. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

dpgeorge · 2023-08-23T05:13:19Z

ports/esp8266/README.md

@@ -23,6 +23,9 @@ Supported features include:

 Documentation is available at http://docs.micropython.org/en/latest/esp8266/quickref.html.

+The default build requires a 2MiB flash chip, but see below for support for
+1MiB and 512kB options.


There are quite a few inconsistent uses of kB vs KB vs KiB vs kiB in this commit. They should all be kiB.

dpgeorge · 2023-08-23T05:14:44Z

tools/autobuild/build-esp8266-latest.sh

-    echo "building $descr $board"
-    build_dir=/tmp/esp8266-build-$board
-    $MICROPY_AUTOBUILD_MAKE $@ BOARD=$board BUILD=$build_dir || exit 1
-    mv $build_dir/firmware-combined.bin $dest_dir/$descr$fw_tag.bin


The main firmware is firmware-combined.bin and I don't think the generic build function handles this filename. Probably FWBIN (in esp8266 Makefile) should be changed to just FWBIN = firmware.bin.

dpgeorge · 2023-08-23T05:15:55Z

The CI is failing due to the new filenames called manifest_xxx.py. Either they need to go in a subdir like 512kiB/manifest.py, or the Ruff exclusion pattern needs to be expanded to include these filenames.

ports/esp8266/boards/ESP8266_GENERIC/_boot.py

ports/esp8266/boards/ESP8266_GENERIC/board.md

This merges the existing GENERIC, GENERIC_1M, and GENERIC_512k boards into variants of the new ESP8266_GENERIC board (renamed from GENERIC so as not to clash with other ports). Also moves the generation of the "OTA" variant (previously generated by autobuild/build-esp8266-latest.sh) into the variant. Following the convention established for the WEACTSTUDIO rp2 board, the names of the variants are FLASH_1M and FLASH_512K (but rename the .ld files to use MiB and kiB). Updates autobuild to build esp8266 firmware the same way as other ports. This requires renaming the output from firmware-combined.bin to just firmware.bin. This work was funded through GitHub Sponsors. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

PICO might not always be a unique name across all ports, and the convention generally for other boards is to do VENDOR_BOARD. This work was funded through GitHub Sponsors. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

This should have been added in 4815af7 when the variants were combined. The original non-USB variant got this implicitly, and therefore was not in mpconfigvariant.h Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

jimmo · 2023-08-23T05:50:07Z

or the Ruff exclusion pattern needs to be expanded to include these filenames.

Done.

dpgeorge · 2023-08-23T23:47:55Z

Merged, thank you!

jimmo force-pushed the board-variant-caps branch from abe4a5a to 34f73bd Compare August 15, 2023 15:24

jimmo force-pushed the board-variant-caps branch 3 times, most recently from 4870c3a to 3554206 Compare August 16, 2023 01:01

jimmo force-pushed the board-variant-caps branch from 3554206 to a1db0a4 Compare August 16, 2023 03:14

jimmo force-pushed the board-variant-caps branch 2 times, most recently from e2d120f to 6f0cf5b Compare August 16, 2023 03:22

jimmo mentioned this pull request Aug 21, 2023

No more GENERIC_SPIRAM board? #12253

Closed

jimmo mentioned this pull request Aug 21, 2023

ESP32C3 no output on serial after flashing finished. #12270

Open

dpgeorge reviewed Aug 23, 2023

View reviewed changes

jimmo force-pushed the board-variant-caps branch from 04462c0 to c4f14c0 Compare August 23, 2023 03:53

jimmo added 7 commits August 23, 2023 13:54

nrf/boards: Rename all nRF boards to use uppercase.

f007d0a

This is to match all the other ports. This work was funded through GitHub Sponsors. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

esp32: Use uppercase variant names.

b5836e7

This is to support a future change to add the variant name to the build directory and therefore should be the same style as the board name. This work was funded through GitHub Sponsors. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

stm32: Use uppercase variant names.

c309308

This is to support a future change to add the variant name to the build directory and therefore should be the same style as the board name. This work was funded through GitHub Sponsors. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

{esp32,rp2,stm32}/Makefile: Append board variant to BUILD.

97ffc53

This allows switching between variants without clobbering the build output. This work was funded through GitHub Sponsors. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

esp32: Rename GENERIC* boards to ESP32_GENERIC*.

aa23698

Board names need to be unique across ports, and GENERIC clashes with the ESP8266 (which will be renamed to ESP8266_GENERIC). This work was funded through GitHub Sponsors. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

jimmo force-pushed the board-variant-caps branch from c4f14c0 to 51e47eb Compare August 23, 2023 03:54

dpgeorge reviewed Aug 23, 2023

View reviewed changes

jimmo force-pushed the board-variant-caps branch from 51e47eb to ed707e3 Compare August 23, 2023 05:32

dpgeorge reviewed Aug 23, 2023

View reviewed changes

ports/esp8266/boards/ESP8266_GENERIC/_boot.py Outdated Show resolved Hide resolved

ports/esp8266/boards/ESP8266_GENERIC/board.md Outdated Show resolved Hide resolved

jimmo added 3 commits August 23, 2023 15:49

rp2: Rename PICO, PICO_W to RPI_PICO, RPI_PICO_W.

873bd00

PICO might not always be a unique name across all ports, and the convention generally for other boards is to do VENDOR_BOARD. This work was funded through GitHub Sponsors. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

esp32/boards/ESP32_GENERIC_C3: Enable UART REPL.

326dfd2

This should have been added in 4815af7 when the variants were combined. The original non-USB variant got this implicitly, and therefore was not in mpconfigvariant.h Signed-off-by: Jim Mussared <jim.mussared@gmail.com>

jimmo force-pushed the board-variant-caps branch from ed707e3 to 326dfd2 Compare August 23, 2023 05:49

dpgeorge merged commit 326dfd2 into micropython:master Aug 23, 2023

dpgeorge added the ports Relates to multiple ports, or a new/proposed port label Aug 23, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

stm32,rp2,esp32: Improve consistency of board and variant names. #12240

stm32,rp2,esp32: Improve consistency of board and variant names. #12240

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

stm32,rp2,esp32: Improve consistency of board and variant names. #12240

stm32,rp2,esp32: Improve consistency of board and variant names. #12240

Uh oh!

Conversation

Uh oh!

Uh oh!

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!