8000 Add ssl certificates on ESP8266 · Issue #2781 · micropython/micropython · GitHub
[go: up one dir, main page]
Skip to content

Add ssl certificates on ESP8266 #2781

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
Lir10 opened this issue Jan 11, 2017 · 50 comments
Closed

Add ssl certificates on ESP8266 #2781

Lir10 opened this issue Jan 11, 2017 · 50 comments
Labels
port-esp8266 proposed-close Suggest this issue should be closed

Comments

@Lir10
Copy link
Lir10 commented Jan 11, 2017 •
edited
Loading

Hi ,

I'm trying to connect to ssl server using this code :

>>>import usocket                                                                                                       
>>> import ussl                                                                                                          
>>> s = usocket.socket(usocket.AF_INET, usocket.SOCK_STREAM)                                                             
>>> ss = ussl.wrap_socket(s,  ca_certs='/flash/cert/ca.pem')

However , im getting the following traceback :

Traceback (most recent call last):                                                                                       
  File "<stdin>", line 1, in <module>                                                                                    
TypeError: extra keyword arguments given

Looks like

ussl.wrap_socket()

works only with one argument.

How can i add my own self signed certificates to the ssl socket object?
@Lir10 Lir10 changed the title Add ssl client certificate on ESP8266 Add ssl certificate on ESP8266 Jan 11, 2017
@Lir10 Lir10 changed the title Add ssl certificate on ESP8266 Add ssl certificates on ESP8266 Jan 11, 2017
@hiway
Copy link
hiway commented Jan 11, 2017

Warning: Currently, this function does NOT validate server certificates, which makes an SSL connection established prone to man-in-the-middle attacks.

https://docs.micropython.org/en/latest/esp8266/library/ussl.html

This will work:

>>> ss = ussl.wrap_socket(s)

But as the docs warn, it is not validated.

@Lir10
Copy link
Author
Lir10 commented Jan 11, 2017 •
edited
Loading

@hiway Thanks for the answer ,
I wonder what's the point to make ssl connection without certificates ?
So what's the difference between that and regular tcp connection ??

@briant-spindance
Copy link

Just wondering if there are any plans to resolve this issue?

@dpgeorge
Copy link
Member
dpgeorge commented Sep 6, 2017

Just wondering if there are any plans to resolve this issue?

The axtls library used on esp8266 does support certificates (although I'm not sure to what extent). So it's possible to add support for them but it's not planned at this stage. An alternative way to add certs (and at the same time getting more advanced crypto algorithms) would be to support mbedtls on the esp8266, but the esp8266 might not have enough RAM for this.

@pfalcon
Copy link
Contributor
pfalcon commented Sep 6, 2017

There're definitely plans to implement that, it's just plans for hundreds of other things have more priority, and implementing hundreds of things takes years, so nobody yet got to it. @Lir10 , @briant-spindance , you seem to need this more than other folks, so feel free to prepare patches for it. (High quality ones, which generally goes without saying. Please start at https://github.com/micropython/micropython/blob/master/CONTRIBUTING.md before doing any coding work.)

would be to support mbedtls on the esp8266, but the esp8266 might not have enough RAM for this.

axTLS was originally selected as a default TLS implementation (even before esp8266 port) because it has minimal code size and RAM requirements.

@Vrettel
Copy link
Vrettel commented Dec 8, 2017

Hello,

I recently tested SSL/TLS connection on my ESP8266 with micropython. The test was done with self-signed certificates from the server-broker that I passed to my ESP. Using the umqtt (simple.py) I found out as mentioned in an earlier post that the function ussl.wrap_socket() works only with one argument. With only one argument I can't include my certificate (ca_certs). By giving only the socket (first argument) the connections is successful. I tried to capture some packets and the connection seems to be encrypted.
My question is; If anyone can connect without certificate but the connection is encrypted, how secure is this connection ?

Thanks in advance

@peterhinch
Copy link
Contributor

See the docs here and here.

@dpgeorge
Copy link
Member

My question is; If anyone can connect without certificate but the connection is encrypted, how secure is this connection ?

You'll need to implement authentication yourself. Eg using a password with umqtt.

@Vrettel
Copy link
Vrettel commented Dec 13, 2017

@peter and @dpgeorge thank you for your replies. I 'll keep my connection secure with username/password and the current encryption method.

@laGapa laGapa mentioned this issue Feb 2, 2018
Open
@Sunrise17
Copy link
Sunrise17 commented Feb 9, 2018 •
edited
Loading

@dpgeorge , i am using simple umqtt for establishing SSL connection to AWS IoT with ESP32 board, but i have not succeeded to establish connection. I am getting following error from ussl.wrap_socket method;
Error in mqtt connect: [Exception] TypeError: extra keyword arguments given

client = MQTTClient(client_id=MQTT_CLIENT_ID, server=MQTT_HOST, port=MQTT_PORT, keepalive=10000, ssl=True, ssl_params={"keyfile":KEY_PATH, "certfile":CERT_PATH, "ca_certs":CACERT_PATH})

print(self.params) equals = {'keyfile': '/flash/cert/deviceCert.key', 'certfile': '/flash/cert/deviceCertAndCACert.pem', 'ca_certs': '/flash/cert/root.pem'}

Connect Method in umqtt is seen below, is it possible to establish full cert chain connection to AWS IoT?
def connect(self, clean_session=True): self.sock = socket.socket() addr = socket.getaddrinfo(self.server, self.port)[0][-1] self.sock.connect(addr) print(self.ssl_params) if self.ssl: import ussl self.sock = ussl.wrap_socket(self.sock, **self.ssl_params) ...

@Sunrise17
Copy link

In micropython version 1.9.4, has this issue been resolved?

8000
@peterhinch
Copy link
Contributor

It seems that it has. See here (scroll down to extmod section, fourth line reads "modussl_axtls: implement key and cert kw args to wrap_socket".

@Sunrise17
Copy link
Sunrise17 commented May 13, 2018 •
edited
Loading

I am trying to connect esp32 to AWS IoT broker (i have tried both ports 8883 and 443 regarding to latest update of AWS), In version 1.9.4, i have not seen ca_cert parameter may be that's why AWS rejects connection.

Connect function in mqtt.py;
ekran resmi 2018-05-13 19 54 26

but i am receiving following error;
ekran resmi 2018-05-13 19 54 11

@dpgeorge
Copy link
Member

@Sunrise17 the esp32 (and esp8266) supports the parameters key and cert when creating an SSL connection. Note that these are not the same as keyfile and certfile. The parameters key and cert need to be passed actual key/cert data, not a filename. You can load a file like this:

with open(keyfile, 'rb') as f:
    key = f.read()

That should work. Note that the ca_cert parameter is currently not implemented.

@Sunrise17
Copy link
Sunrise17 commented May 17, 2018 •
edited
Loading

Dear @dpgeorge , thanks for your response, i have tried your method to connect AWS IoT but still i am receiving same error. As you will see that ssl params passed as data in following below;

ekran resmi 2018-05-17 22 34 34

In AWS IoT documentation, format to publish mqtt message as follows;

$ mosquitto_pub --cafile root.cert --cert deviceCertAndCACert.crt --key deviceCert.key -h <prefix>.iot.us-east-1.amazonaws.com -p 8883 -q 1 -t  foo/bar -i  anyclientID --tls-version tlsv1.2 -m "Hello" -d

So, enabling cacert is necessary i think to establish direct connection from esp32 to AWS IoT?

@dpgeorge
Copy link
Member

@Sunrise17 you shouldn't post your private keys like that! I suggest you change them immediately.

It's hard to debug this problem without extra info. What is line 65 of mqtt.py?

@Sunrise17
Copy link
Sunrise17 commented May 18, 2018 •
edited
Loading

@dpgeorge , please find attacked mqtt.py file. Line 65 is :
self.sock.connect(addr)

mqtt.zip

Lines in main.py is:

CACERT_PATH = "/flash/cert/root.pem"
KEY_PATH = "/flash/cert/deviceCert.key"
CERT_PATH = "/flash/cert/deviceCertAndCACert.pem"

with open(KEY_PATH, 'rb') as f:
    key1 = f.read()

with open(CERT_PATH, 'rb') as f:
    cert1 = f.read()

client = MQTTClient(client_id=MQTT_CLIENT_ID, server=MQTT_HOST, port=MQTT_PORT, keepalive=10000, ssl=True, ssl_params={ "key":key1, "cert":cert1, "server_side":True })

@dpgeorge
Copy link
Member

please find attacked mqtt.py file. Line 65 is :
self.sock.connect(addr)

If it's failing on this line then it doesn't have anything to do with SSL (SSL handshake happens after this line succeeds). If it fails here then there is something wrong with the internet connection and/or the remote server (maybe it's domain name is wrong?).

@peterhinch
Copy link
Contributor

I think someone should write a FAQ on using SSL/TLS as this issue keeps cropping up. I'd offer but a) it's not my field and b) I never managed to get it to work either ;-)

@Sunrise17
Copy link
Sunrise17 commented May 31, 2018 •
edited
Loading

Dear @dpgeorge ,

I have tried three option for line 48 for path versions of key and cert file in wrap_socket method and i am receiving following errors in each try as below. (Note: key1 and cert1 returns bytes object, in option 3 that's why i have converted it to string and try) With same cert and key files, i have tried Python 3.5 from PC, it is well connecting to server but i could not succeed to connect from esp32 board.

KEY_PATH = "/flash/cert/client.key"
CERT_PATH = "/flash/cert/client.crt"
HOST, PORT = "45.255.66.124", 5800

with open(KEY_PATH, 'rb') as f:
    key1 = f.read()

with open(CERT_PATH, 'rb') as f:
    cert1 = f.read()

45 ::      s = usocket.socket(usocket.AF_INET, usocket.SOCK_STREAM)
46 ::      addr = usocket.getaddrinfo(HOST, PORT)[0][-1]
47 ::      s.connect(addr)

Option_1

48 ::      sock = ussl.wrap_socket(addr, key = KEY_PATH, cert = CERT_PATH)

Error: (after assertion "ret == 0" fail, continuously resetting the board)

assertion "ret == 0" failed: file "../../extmod/modussl_mbedtls.c", line 182, function: socket_new
abort() was called at PC 0x401368e7 on core 0

Backtrace: 0x4008c983:0x3ffb6530 0x4008c9af:0x3ffb6550 0x401368e7:0x3ffb6570     0x400ed05d:0x3ffb65a0 0x400dfb5a:0x3ffb65f0 0x400dbe49:0x3ffb6620 0x400dbeb1:0x3ffb6640 0x400e8f0d:0x3ffb6660 0x400dfbe8:0x3ffb6700 0x400dbe49:0x3ffb6780 0x400e8e81:0x3ffb67a0 0x400dfbe8:0x3ffb6840 0x400dbe49:0x3ffb68c0 0x400dbe76:0x3ffb68e0 0x400f872a:0x3ffb6900 0x400f8981:0x3ffb69a0 0x400ef907:0x3ffb69c0

Rebooting...
Guru Meditation Error: Core  0 panic'ed ()
Guru Meditation Error: Core  0 panic'ed (Guru Meditation Error: Core  0 panic'ed ()
Guru Meditation Error: Core  0 panic'ed ()
Guru Meditation Error: Core  0 panic'ed (Guru Meditation Error: Core  0 panic'ed (Guru Meditation Error: Core  0 panic'ed (Guru Meditation Error: Core  0 panic'ed (Unknown reason)
ets Jun  8 2016 00:22:57

Option_2

48 ::      sock = ussl.wrap_socket(addr, key = key1, cert = cert1)

Error:

File "main.py", line 48, in DownloadFile
OSError: stream operation not supported

Option_3

48 ::      sock = ussl.wrap_socket(addr, key = key1.decode("utf-8"), cert = cert1.decode("utf-8"))

Error:

File "main.py", line 48, in DownloadFile
OSError: stream operation not supported

@dpgeorge
Copy link
Member
dpgeorge commented Jun 1, 2018

@Sunrise17 you need to pass s to wrap_socket, not addr, like this:

sock = ussl.wrap_socket(s, key=key1, cert=cert1)

@Sunrise17
Copy link

@dpgeorge , thanks for your warning, it works well right now between SSL Python Server and ESP32 Client. In other side, to connect AWS IoT over mqtt with ESP32 board, i am still trying to solve it. Amazon is requiring ca_cert parameter in connection may be that's why its not connecting. After i have found some solution, i will write it here. Thanks again...

@manningt manningt mentioned this issue Oct 9, 2018
Open
@StanS-AWS
Copy link

I can connect to AWS MQTT with client cert using ESP32 WROOM-32 module. But the same exactly code and certs fail to connect when running on ESP8266 board. The error code is "invalid key". Both flashed micropython 1.9.4, the same exactly certs, but different results. Is there is a limit on cert length in 8266 build? What other factors may affect the connection?

@Swington
Copy link
Swington commented Jan 8, 2019

@StanS-AWS I guess the difference is in SDKs for ESP8266 and ESP32, cause there is actually a difference between ssl libs in C for those two platforms.
I have exactly the same problem and according to ESP8266 manual, it's SDK provides full ssl support. So I deduce that this functionality just isn't translated into Micropython API.

@StanS-AWS
Copy link

It turned-out ESP8266 certs need to be in .der format to work correctly. The code remains the same.

@StanS-AWS
Copy link
StanS-AWS commented Jan 13, 2019 •
edited
Loading

openssl x509 -in 8266-01.cert.pem -out 8266-01.cert.der -outform DER
openssl rsa -in 8266-01.private.key -out 8266-01.key.der -outform DER
https://awsiot.wordpress.com/2019/01/10/connect-8266-to-aws-mqtt-using-miropython/

@sebi5361 sebi5361 mentioned this issue Jan 26, 2020
Closed
@SlowBro904
Copy link

Does validation work properly on ESP32?

@jrichrd
Copy link
jrichrd commented Oct 6, 2020

Note that the ca_cert parameter is currently not implemented.

Does anyone know if the 'ca_cert' parameter is now implemented in the latest version of MicroPython? I'm not able to get it to work using the code samples above however I can connect if I remove the 'ca_cert' and use only the key and device cert.

@LMtx
Copy link
LMtx commented Oct 15, 2020

Note that the ca_cert parameter is currently not implemented.

Does anyone know if the 'ca_cert' parameter is now implemented in the latest version of MicroPython? I'm not able to get it to work using the code samples above however I can connect if I remove the 'ca_cert' and use only the key and device cert.

I would love to know this as well. But it looks like 'ca_cert' is still not implemented :( That is a big issue for any production-like usage of this library.

@jimmo
Copy link
Member
jimmo commented Oct 19, 2020

ca_certs is currently only implemented in the CC3200 port (WiPy). See #5998 for an ESP32 implementation.

@Donatussss
Copy link

Hello, I'm currently facing the invalid_key error on ESP8266 as well as Raspberry Pi Pico W even after converting the files to .der format using openssl. Anyone else facing this issue recently? Will really appreciate your help.

@Saranya-karan
Copy link

Hello, I'm currently facing the invalid_key error on ESP8266 as well as Raspberry Pi Pico W even after converting the files to .der format using openssl. Anyone else facing this issue recently? Will really appreciate your help.

hi iam also facing the error while using pico w..my error code is
Traceback (most recent call last):
File "", line 72, in
File "simple1.py", line 84, in connect
IndexError: bytes index out of range

do u find any solution?

@Donatussss
Copy link
Donatussss commented Aug 26, 2024 •
edited
Loading

Hi @Saranya-karan please view this comment I made with regards to the invalid_key. You'll tell me if it worked.
For the error you've mentioned I'd wish to see the simple1.py file

@Saranya-karan
Copy link
Saranya-karan commented Aug 26, 2024 •
edited
Loading

simple1.txt
i attached my simple1.py library here sir(in text file)

#My main program
import time
import machine
import network
import ujson
from simple1 import MQTTClient

SSID = b'xxxe'
PASS = b'xxx6'
CLIENT_ID = b'iotconsole-fb2cbb82-9022-4759-bd05-369994f6eb63'
AWS_ENDPOINT = b'xxxxot.us-east-1.amazonaws.com'

PUB_TOPIC = `b'temperature'`
SUB_TOPIC = b'temperature'

with open('/pri.der', 'rb') as f:
    DEV_KEY = f.read()

with open('/cert.der', 'rb') as f:
    DEV_CRT = f.read()
light = machine.Pin("LED", machine.Pin.OUT)
light.off()

def wifi_connect():
    print('Connecting to wifi...')
    wlan = network.WLAN(network.STA_IF)
    wlan.active(True)
    wlan.connect(SSID, PASS)
    while wlan.isconnected() == False:
        light.on()
        print('Waiting for connection...')
        time.sleep(0.5)
        light.off()
        time.sleep(0.5)
    print('Connection details: %s' % str(wlan.ifconfig()))

# Callback function for all subscriptions
def mqtt_subscribe_callback(topic, msg):
    print("Received topic: %s message: %s" % (topic, msg))
    if topic == SUB_TOPIC:
        mesg = ujson.loads(msg)
        if 'state' in mesg.keys():
            if mesg['state'] == 'on' or mesg['state'] == 'ON' or mesg['state'] == 'On':
                light.on()
                print('Light is ON')
            else:
                light.off()
                print('Light is OFF')

def get_rpi_temperature():
    sensor = machine.ADC(4)
    voltage = sensor.read_u16() * (3.3 / 65535)
    temperature = 27 - (voltage - 0.706) / 0.001721
    return temperature

wifi_connect()

mqtt = MQTTClient(
    client_id=CLIENT_ID,
    server=AWS_ENDPOINT,
    port=8883,
    keepalive=120,
    ssl=True,
    ssl_params={'key':DEV_KEY, 'cert':DEV_CRT, 'server_side':False})

mqtt.connect()
mqtt.set_callback(mqtt_subscribe_callback)

mqtt.subscribe(SUB_TOPIC)

while True:
    message = b'{"temperature":%s, "temperature_unit":"Degrees Celsius"}' % get_rpi_temperature()
    print('Publishing topic %s message %s' % (PUB_TOPIC, message))
    mqtt.publish(topic=PUB_TOPIC, msg=message, qos=1)
    mqtt.check_msg()
    time.sleep(5)

@Donatussss
Copy link

Okay, please send a screenshot of the error

@Saranya-karan
Copy link

image

@Donatussss
Copy link

Could be your connection details are invalid cause you're not getting a response from the broker

@Saranya-karan
Copy link

because of my network ?sir can you elaborate the issue.

@Saranya-karan
Copy link

Could be your connection details are invalid cause you're not getting a response from the broker

with the same certificates and end point i sent data to aws iot core in arduino ide(ESP32 board).but in micropython(picow board) it is making issues

@Donatussss
Copy link

I see, have you tried decoding the base64 text as I pointed out in that other comment

@Saranya-karan
Copy link

yes sir,i updated the files in der format..it throws error
Traceback (most recent call last):
File "", line 29, in
File "", line 24, in read_cert
TypeError: can't convert 'str' object to bytes implicitly

@Donatussss
Copy link

To do the base64 decoding you need the certificates in base64 format which is the certificate.pem.crt and private.pem.key files

@Saranya-karan
Copy link

I see, have you tried decoding the base64 text as I pointed out in that other comment

sir may i know which version of micropython u used for sending data to aws..
iam using 1.20.0

@Donatussss
Copy link
Donatussss commented Aug 26, 2024 •
edited
Loading

I am pretty much using the latest version

@Saranya-karan
Copy link

To do the base64 decoding you need the certificates in base64 format which is the certificate.pem.crt and private.pem.key files

image

def read_cert(filename):        
    with open(filename, 'rb') as f:
        text = f.read().strip()
        split_text = text.split(b'\n')  # Split using bytes newline
        base64_text = b''.join(split_text[1:-1])  # Join the byte strings
        return ubinascii.a2b_base64(base64_text)

DEV_KEY = read_cert('/d5126a8b49f0e6423dd6f5129d8a493feaf53fbf393e3355d3532ca988a51978
EED3
-private.pem.key')
DEV_CRT = read_cert('/d5126a8b49f0e6423dd6f5129d8a493feaf53fbf393e3355d3532ca988a51978-certificate.pem.crt')
yes i tried that way also.still finding the same issue

@Donatussss
Copy link
Donatussss commented Aug 26, 2024 •
edited
Loading

Okay, I'm not sure I can offer much help using comments, is it possible that you have your project in a repo then I can clone it and try on the pico w I have. You can have it private then add me as a collaborator. I'll work on it tomorrow

@Saranya-karan
Copy link
Saranya-karan commented Aug 26, 2024 •
edited
Loading

Okay, I'm not sure I can offer much help using comments, is it possible that you have your project in a repo then I can clone it and try on the pico w I have. You can have it private then add me as a collaborator. I'll work on it tomorrow

I will add the repo tomorrow sir. I provided my simple1. py library and main program. Nothing much there. thank you so much for the response

@Saranya-karan
Copy link

Okay, I'm not sure I can offer much help using comments, is it possible that you have your project in a repo then I can clone it and try on the pico w I have. You can have it private then add me as a collaborator. I'll work on it tomorrow

i added the files to my repo..it is in public view.pls let me know u can access the files..

@Donatussss
Copy link

Hi @Saranya-karan , I've made changes to the repo. The code seems to work on my end. image image.
You'll try it out and let me know. Also note the version of micropython.
I've also made it such that you don't have to hardcode your credentials everytime you create a new project needing the credentials. This will also be helpful when provisioning many devices and also when sharing your project

@jonnor
Copy link
Contributor
jonnor commented Sep 29, 2024

The original problem (as well as several other, semi-related) problems seems to have been fixed. Perhaps we can close this issue? For usage questions or debugging help, the recommended is to open a Github Discussion or come in the MicroPython Discord.

@jonnor jonnor added the proposed-close Suggest this issue should be closed label Sep 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
port-esp8266 proposed-close Suggest this issue should be closed
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
0