esp8266: Characters missing when reading from UART #4153
Comments
|
This is not the forum but a bug, feature tracker. |
|
Well, I have not found a way to read the 16th character. This is not expected behavior based on the current documentation 10000 and therefore I would define it as a "bug" in either code or docs. |
|
What kind of data are you reading at which speed? |
|
baudrate 9600 Using readline()-method Missing characters are shown by * First * is a space |
|
9600 Baud is pretty slow. if you are reading fast enough, then there should not be a problem. The ringbuffer for UART has a size of 16 bytes. That seems to indicate, that there is a problem. But looking at the code at py/ringbuf.h, it is not obvious to me. I have coded almost identical buffers in the past with no data loss at all, as long as I read out fast enough, |
|
To back up the comment from @adritium there is a limited audience here consisting of MicroPython maintainers and contributors. Your query will be seen by many more people, some of whom will have used the UART on ESP8266, in the forum. In my opinion it's unlikely to be a bug as the single bidirectional UART is widely used for the REPL and for uploading substantial programs. In that context it works fine. |
|
Hi all, thank you for you interest in this topic. I decided to do an extra check before following the suggestion to post in the forum instead. Below is a copy-paste from the webrepl where i first detach the default UART and then attach a new UART with the right baudrate for my incoming traffic. The incoming traffic is now posted directly to stdin and all characters are showing (nothing is missing): Doing the same but not directing to stdin: In the bottom example, every 16th character is missing. This leads me to believe that there has to be a difference in how the connections going to stdin and the ones going into another buffer is handled? I will be happy to post this question in the forum instead if you still think this belongs there. |
|
For me it looks more like an issue in combination with ser.readline() and the internal buffer mechanism. Could you try a combination of ser.any(()) and ser.read() instead? |
|
I would suggest that the rxbuf argument to the UART constructor should specify a buffer big enough to hold the longest line which might be transmitted. It's not obvious (to me) what readline() is supposed to do when the buffer fills up before a newline character is received. |
|
I do not see in the code that an rxuf argument would be supported. I have no ESP8266 here at this very moment, so I cannot try. The only buffer I could find is this small ringbuffer for incoming characters: |
|
Ah. My fault for reading the docs rather than studying the code :) The docs suggest that init() can be called with an rxbuf arg, but a quick try (and a look at the code) suggests it's not actually supported. It would therefore seem that readline() is only suitable for lines of <= 16 chars (including any end of line characters). So I'd agree with your suggestion of reading longer lines on a character by character basis. The approach I'd normally advocate would be to use uasyncio's StreamReader and StreamWriter objects, but I haven't tried this on ESP8266. I have serious doubt that a 16 character buffer is enough to cope with the latency implicit in using uasyncio (with other tasks competing for the CPU). So the issue to raise is the fact that the docs don't agree with reality. Either the docs or the code should be changed. Preferably the latter; a fixed size 16 byte buffer is of limited use, especially on a device with the amount of latency as the ESP8266. How does it reliably handle file uploads via rshell at 115200 baud? |
|
Hi Peter- It is still and issue, because stream readline() manages its own buffer of abritrary length. So it should work also for longer lines. its is only the ISR's circular buffer with a length of 16, being able to buffer 15 characters. |
|
Some further updates ser.any() in combination with ser.read() gives the same result as with ser.readline() FYI, I managed to solve my immediate problem by using sys.stdin as a workaround: |
|
Sadly, that was my suspicion. sys.stdin uses the larger buffer of 256 bytes. |
|
For the record, the lines I am reading are smaller than 256 bytes (~200 characters) and the same issue should therefore not be visible anyway when using sys.stdin |
|
It might be interesting if the issue reappears after 255 characters for longer lines. |
|
Indeed the issue here is with the UART RX buffer. When attached to dupterm as stdin/stdout the RX buffer that is used for incoming UART characters in the stdin buffer, which is 256 bytes, so 255 usable entries. When the UART is detached from the REPL the buffer is only 16 bytes, or 15 usable entries. So that explains all the behaviour seen/described above. In commit 52bec93 I implemented the uart.init(rxbuf=200)I think that should fully solve this issue. |
resolves micropython#4153: Fixed build issue when CIRCUITPY_USB is off
Hi,
First, thanks for micropython, its very handy!
But today I got stuck on something weird, it seem that when reading from a serial connection with the UART module it skips some characters. Actually it seems to skip every 16th character so that when calling either uart.readline() or uart.read(value>15) characters are missing.
Could potentially be a problem with an internal buffer that is not emptied/read properly?
For info,
Hardware is Weemos D1 mini pro (ESP8266ex)
Micropython version (latest): esp8266-20180907-v1.9.4-494-ge814db592
The text was updated successfully, but these errors were encountered: