8000 MQTT to AWS IoT Core fails with mbedtls_ssl_handshake_error · Issue #5929 · micropython/micropython · GitHub
[go: up one dir, main page]
Skip to content

MQTT to AWS IoT Core fails with mbedtls_ssl_handshake_error #5929

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
Aish-Git opened this issue Apr 16, 2020 · 25 comments
Open

MQTT to AWS IoT Core fails with mbedtls_ssl_handshake_error #5929

Aish-Git opened this issue Apr 16, 2020 · 25 comments
Labels
port-esp32

Comments

@Aish-Git
Copy link
Aish-Git commented Apr 16, 2020 •
edited
Loading

mbedtls_ssl_handshake error: -2880
I (14912) wifi:Traceback (most recent call last):
File "main.py", line 99, in
File "main.py", line 96, in
File "main.py", line 69, in mqtt
File "umqtt/simple.py", line 61, in connect
OSError: [Errno 5] EIO

Fails on the latest firmware but works on any of the 2019 builds - tried it with esp32-idf3-20191220-v1.12.bin and it works but needed BLE and hence upgraded to 2020 versions and fails on all the March/April releases.
@dpgeorge
Copy link
Member

mbedtls_ssl_handshake error: -2880

This is MBEDTLS_ERR_X509_ALLOC_FAILED, so it's running out of RTOS heap memory.

@Aish-Git
Copy link
Author

Thanks @dpgeorge - would adding a gc.collect() and checking for gc.free_men() prior to that socket connect help? I am reading certs into variables but assuming those are in the function stack.

@tve
Copy link
Contributor
tve commented Apr 16, 2020

Nope, it's the C heap, not the MP heap... Either use a module with PSRAM or reduce mp_task_heap_size by some factor (e.g. 80%) here https://github.com/micropython/micropython/blob/master/ports/esp32/main.c#L103-L104 and rebuild the fw and try that...

@Aish-Git
Copy link
Author

Hi @tve - noted. Curiously I tried the same code that worked on earlier firmware (20191220) doesn't work on the latest one 20200416. Getting mbedtls_ssl_handshake error: -10 error. Could it be an SSL wrap_socket issue?

@dpgeorge
Copy link
Member

uriously I tried the same code that worked on earlier firmware (20191220) doesn't work on the latest one 20200416

That's most likely due to an update to the ESP-IDF, which changes the heap layout and has less RTOS heap available.

@Carglglz
Copy link
Contributor

Hi @Aish-Git see the long explanation in #5543

@Aish-Git
Copy link
Author

Thanks @Carglglz - incredibly helpful and sorry didn't see that before I logged this one. Let me try some options listed there in terms of a 1096 bit key to keep the cert length small. I am also going to try the recommendation that @tve provided on increasing the C heap size. Will keep this thread posted...

@rahulsinh-jadeja
Copy link

I also received the ssl_handshake_status error. My error log is as the following

ssl_handshake_status: -256 Cannot connect MQTT: [Errno 5] EIO Traceback (most recent call last): File "main.py", line 236, in <module> File "main.py", line 217, in connect_mqtt File "umqtt/simple.py", line 62, in connect OSError: [Errno 5] EIO

I'm using AWS IoT with umqtt. Sometimes it get connected but sometime it gives me the above error. Can anyone help me with this?

@sactre
Copy link
sactre commented Sep 18, 2020

I make SSL connections and use AmazonIOT and my memory configuration is this.
`size_t mp_task_heap_size = 50 * 1024; //heap_caps_get_largest_free_block(MALLOC_CAP_8BIT);

void *mp_task_heap = malloc(mp_task_heap_size);`

@srcnet2
Copy link
srcnet2 commented Dec 4, 2020

Downgrading to 1.12 (esp32-idf3-20191220-v1.12.bin) fixed it for me. 1.13 must do something different.

@eliclement
Copy link

@Aish-Git did you had any success? I tried to increase the heap size using @tve's suggestions but no luck. I'm using a ESP32 WROVER with PSRAM. Did you managed to increase the C heap?

@Aish-Git
Copy link
Author

@eliclement I never did get to going down that path as I switched over to a ESP Pico board (TinyPico) that has an onboard SRAM so ended up working well for me.

@eliclement
Copy link

@Aish-Git OK thanks. I had to remove the BLE in the build and which freed some heap memory and it was sufficient.

@chtinnes
Copy link

I had similar issues when connecting to RabbitMQ (actually I think that AWS IoT is using RabbitMQ under the hood). I am not sure if you have control over the Ciphers. For me, I controlled it server side and using DHE-RSA-AES128-GCM-SHA256 i worked without any changes to heap. But anyway, more RAM would always be great =) .

Is it possible to restrict the Ciphers Mbedtls is using? Then, I think AWS IoT will also allow using DHE-RSA-AES128-GCM-SHA256 because it is still recommended and you can connect without issues.

@chtinnes chtinnes mentioned this issue Apr 7, 2021
Closed
@the-stanely
Copy link

Downgrading to 1.12 (esp32-idf3-20191220-v1.12.bin) fixed it for me. 1.13 must do something different.

Thank you for that, @srcnet2 . I've been working on this for a while and v1.12 idf3 binary is the only one that's able to reliably connect to AWS using a generic ESP32.

AWS for IoT is the 800 pound gorilla. Sure would be nice to use a generic ESP32 with the latest binary in an AWS application. Using idf4, "esp32-idf4-20210202-v1.14.bin" results in:

OSError: (-17168, 'MBEDTLS_ERR_RSA_PRIVATE_FAILED+MBEDTLS_ERR_MPI_ALLOC_FAILED')

@the-stanely
Copy link

@sactre said...

I make SSL connections and use AmazonIOT and my memory configuration is this.
`size_t mp_task_heap_size = 50 * 1024; //heap_caps_get_largest_free_block(MALLOC_CAP_8BIT);

void *mp_task_heap = malloc(mp_task_heap_size);`

If that's all it takes, it might be of value to make available updated binaries that are capable of connecting to AWS IoT Core using a generic ESP32.

@the-stanely
Copy link

I have good news and I have bad news! The new version of MicroPython, idf4 v1.15 -- esp32-20210418-v1.15.bin, works with AWS IoT! YAY!!!

v1.15 does not have mqtt built in so you have to install your own. That's OK because it's easy to do.

Now for the bad news. If I drop into REPL a few times while running my tests, eventually I get a WiFi error that looks like this:

Traceback (most recent call last):
  File "<stdin>", line 23, in <module>
OSError: Wifi Internal Error

I don't know why, and the message doesn't give me much to go on. I have to machine.reset() to get the WiFi to work again. A soft boot (^D) doesn't do it.

I will open another issue about this specific WiFi problem.

@the-stanely
Copy link

I can also force my WiFi station mode inactive in my case station.active(False), and that works to reset the WiFi, but then my software has to reconnect every time when coming back from REPL. Is it supposed to work like this?

@ironss
Copy link
ironss commented Aug 31, 2021 •
edited
Loading

My solution was to make Micropython leave some of the heap for the RTOS, instead of claiming all of free memory for itself. Experimentation showed that

  • 2K reserved never works
  • 4K reserved sometimes works
  • 8K reserved always works

But this probably depends on what other features are available.

    diff --git a/ports/esp32/main.c b/ports/esp32/main.c
    index ff6dd6957..0149e8a0b 100644
    --- a/ports/esp32/main.c
    +++ b/ports/esp32/main.c
    @@ -121,6 +121,7 @@ void mp_task(void *pvParameter) {
         #else
         // Allocate the uPy heap using malloc and get the largest available region
         size_t mp_task_heap_size = heap_caps_get_largest_free_block(MALLOC_CAP_8BIT);
    +    mp_task_heap_size -= 8192;  // Reserve 8K for C heap (mbedTLS needs it)
         void *mp_task_heap = malloc(mp_task_heap_size);
         #endif

Does it make sense to make this a configuration option?

My board has no SPIRAM, so I have no idea if this is necessary if there is plenty of RAM.

@pumelo
Copy link
pumelo commented Sep 1, 2021

By default mbedtls only uses internal ram on the esp32 (security concerns). However esp-idf provides a config to allow mbedtls to alloc also in the PSRAM space:
https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/kconfig.html#config-mbedtls-mem-alloc-mode

@nickboucart
Copy link
nickboucart commented Oct 13, 2022 •
edited
Loading

Solution proposed by @ironss above did the trick for me. After recompiling the latest trunk version of micropython (October 13th 2022), I can now connect to AWS IoT and send and receive messages.

Big thank you @ironss, spent about 3 days before I stumbled upon this solution.

EDIT: I compiled micropython against esp-idf 4.3.4 (latest I found)

@Tangerino
Copy link

The option described by @pumelo is implemented in latest MP? The "...alloc also in the PSRAM..."

@bneigher
Copy link

How would we go about debugging MBEDTLS_ERR_SSL_BAD_HS_CLIENT_HELLO errors? I have attached my code for reference... on ESP32-C3. I'm using the latest nightly of micropython for esp32-c3 (esp32c3-usb-20230411-unstable-v1.19.1-1014-gbde222ce8.bin), the latest version of mqqt_as

import gc
import time
import machine
import network
import json
gc.collect()

import ussl as ssl
import uasyncio as asyncio
from ubinascii import hexlify
gc.collect()

from net import connect # just a wrapper for Wifi connection
from lib.mqtt_as import config
from lib.mqtt_as import MQTTClient
gc.collect()

client_id = hexlify(machine.unique_id()).decode('utf-8')

# Set up the AWS IoT endpoint details
endpoint = "xxxxxxxxxxxxx-yyy.iot.us-west-2.amazonaws.com"
topic = "sdk/test/python"

# Set up the AWS IoT certificate and key files
# cert_file = "/flash/cert/esp32-c3.cert.pem"
cert_file = "/flash/cert/device.cert.der"

# key_file = "/flash/cert/esp32-c3.private.key"
key_file = "/flash/cert/device.private.der"

# root_ca_file = "/flash/cert/root-CA.crt"  # Path to the root CA certificate file
root_ca_file = "/flash/cert/root-CA.der"  # Path to the root CA certificate file

PVT_KEY = open(key_file,'rb').read()
CERT_KEY = open(cert_file, 'rb').read()
CA_KEY = open(root_ca_file, 'rb').read()

ssl_params = {
    'key': PVT_KEY,
    'cert': CA_KEY,
    'server_side': True,
    'server_hostname': endpoint,
    'do_handshake': True,
    'cert_reqs': ssl.CERT_REQUIRED,
    'cadata': CA_KEY
}
                    
# Connect to AWS IoT MQTT
def connectMQTT():
    internet_connection = connect()
    print("MQTT CONNECTING", internet_connection)
    config['client_id'] = client_id
    config['server'] = endpoint
    config['port'] = 8883
    config['keepalive'] = 5000
    config['ssl'] = True
    config['ssl_params'] = ssl_params
    config['ssid'] = internet_connection['ssid']
    config['wifi_pw'] = internet_connection['password']
    MQTTClient.DEBUG = True
    client = MQTTClient(config)
  
    try:
        asyncio.run(client.connect(quick=True))
        print("MQTT CONNECTED") #, client.isconnected())
    finally:
        print("MQTT CLOSED") #, client.isconnected())
        # client.close()
        # client.disconnect()
    
    return client

    
# Publish a message to the topic
def sendMQTT(client, message):
    print(f"Sending: {client.isconnected()} {message}")
    client.publish(topic, message)


connectMQTT()

Tried the stable micropython as well, doesn't seem to make any difference.

Connecting to broker.
Traceback (most recent call last):
  File "<stdin>", line 9, in <module>
  File "util.py", line 87, in <module>
  File "util.py", line 70, in connectMQTT
  File "uasyncio/core.py", line 1, in run
  File "uasyncio/core.py", line 1, in run_until_complete
  File "uasyncio/core.py", line 1, in run_until_complete
  File "lib/mqtt_as.py", line 637, in connect
  File "lib/mqtt_as.py", line 271, in _connect
  File "lib/ssl.py", line 30, in wrap_so
B41B
cket
OSError: (-30976, 'MBEDTLS_ERR_SSL_BAD_HS_CLIENT_HELLO')

@MKoushikYadav
Copy link

Hello, @ironss. How would you go about doing this on micropython. Should I edit the binary files or something? Its still 2023 and none of the solutions work so far. I am trying out different versions to test stuff.

@ironss
Copy link
ironss commented May 15, 2023

I can think of four options.

  • Use an ESP32 board that has SPIRAM. The uPy heap uses the SPIRAM, leaving the internal RAM for the C heap, for TLS, BT, file system file descriptors, etc. My project needed about 20 BLE GATT services, and about 100 GATT characteristics, which also use C heap. I eventually went for this option because if I made the C heap big enough for BLE, TLS, etc, there was not enough internal RAM left for the application to use.
  • Download the source code, edit it as described, then compile your own build and install it on your board. The instructions for ESP32 are detailed, and Just Work if you follow them accurately. I had to do this too, to increase the BLE settings (and to solve the original TLS problem).
  • Edit a binary image of uPy on a PC and install the edited version on your board. Good luck, you're on your own.
  • Write a uPy script to edit the flash directly on the board. You're even more out on a limb.
  • sgi

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
port-esp32
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
0