checksum errors in ubx file #30

Stefal · 2021-06-02T10:05:42Z

Subject of the issue

I still have a lot of checksum errors when parsing the ubx file.

Your workbench

rtkFirmwareVersion=v1.3-May 31 2021

Steps to reproduce

Log ubx messages:

measurementRate=250
navigationRate=1
spiFrequency=36
enableLogging=1
message.nmea_dtm.msgRate=0
message.nmea_gbs.msgRate=0
message.nmea_gga.msgRate=0
message.nmea_gll.msgRate=0
message.nmea_gns.msgRate=0
message.nmea_grs.msgRate=0
message.nmea_gsa.msgRate=0
message.nmea_gst.msgRate=0
message.nmea_gsv.msgRate=0
message.nmea_rmc.msgRate=0
message.nmea_vlw.msgRate=0
message.nmea_vtg.msgRate=0
message.nmea_zda.msgRate=0
message.nav_clock.msgRate=0
message.nav_dop.msgRate=1
message.nav_eoe.msgRate=0
message.nav_geofence.msgRate=0
message.nav_hpposecef.msgRate=1
message.nav_hpposllh.msgRate=1
message.nav_odo.msgRate=0
message.nav_orb.msgRate=0
message.nav_posecef.msgRate=0
message.nav_posllh.msgRate=0
message.nav_pvt.msgRate=1
message.nav_relposned.msgRate=0
message.nav_sat.msgRate=1
message.nav_sig.msgRate=1
message.nav_status.msgRate=0
message.nav_svin.msgRate=0
message.nav_timebds.msgRate=0
message.nav_timegal.msgRate=0
message.nav_timeglo.msgRate=0
message.nav_timegps.msgRate=1
message.nav_timels.msgRate=0
message.nav_timeutc.msgRate=1
message.nav_velecef.msgRate=0
message.nav_velned.msgRate=0
message.rxm_measx.msgRate=0
message.rxm_rawx.msgRate=1
message.rxm_rlm.msgRate=0
message.rxm_rtcm.msgRate=0
message.rxm_sfrbx.msgRate=1

Start your log without any connection to a base, then after a few minutes, connect the RTK Surveyor to a local base station

Expected behavior

We should be able to convert the ubx file without seeing any checksum error

Actual behavior

I see no checksum error as long as there is no correction from a local base station, but many checksum errors as soon as the RTK Surveyor is connected to a local base station.

In the snapshot, the checksum errors started a few seconds before I got a fix (point are green)

The text was updated successfully, but these errors were encountered:

nseidle · 2021-06-11T20:59:52Z

My guess is the UART RX task buffer is getting overrun periodically.

I've done a few things to try to replicate:

There is now an option on the messages menu to turn on/off all 67 messages sent over the UART port.
There is now an option on the hidden test menu to turn on/off all 67 messages sent over the USB port.
Turned on logging in SW Maps.

Now I can log over USB what should be all the data generated by the ZED on the UART port and compare the three logs:

USB to UART
UART to SPP

Comparing the USB log (captured via u-center) and the log file from SD card I get the following diff:

I'm using Beyond Compare which is just an amazing tool. The highlighted red on the left shows how often and repeating the errors are.

The SW Map logs match the UART logs so SPP seems to be performing well. The F9PSerialReadTask() seems to be dropping characters.

nseidle · 2021-06-14T15:22:59Z

I did some tests where I removed the ESP32 from the equation and recorded the UART TX to a terminal and USB via u-center and the problem persists. This would seem to indicate that the UART is putting out slightly different data from USB which is a bigger u-blox firmware issue. I'm still determining if it is related to the number of messages, or fix rate, or a combination there of.

Stefal · 2021-06-15T09:29:03Z

Hmmm..interesting...
I should check if I have some checksum error on my other logger. It uses the @PaulZC work : https://github.com/PaulZC/F9P_RAWX_Logger

I don't know if I'll find some free time to do this soon.

pyrog · 2021-06-15T11:43:41Z

I recorded a track yesterday, without saving the F9P USB output 😞
I used Lefebure NTRIP Client over BlueTooth and serial terminal over USB.

Got 135 coordinates without fix

pyrog · 2021-06-15T14:27:04Z

Another test today NMEA only at 4 Hz (default settings)

F9P USB logged
And device used:
- without BT/NTRIP connection
- with BT/NTRIP connection (Lefebure NTRIP Client)

Analyse with JOSM

0 error without NTRIP (F9P USB output)

0 error without NTRIP (microSD)

1 error with NTRIP (F9P USB output)

22 errors with NTRIP (microSD)

Analyse with Beyond Compare

Myers O(ND) alignment
without NTRIP

with NTRIP

This analyse seem to show that the NMEA messages sent over USB and UART are a bit different.
So it's difficult to show i.e. CRC errors.

nseidle · 2021-06-15T20:35:01Z

Yep, you have to be really careful to configure the UART and USB messages to be identical so that the logs match. Additionally, if UBX data or RTCM is turned on Beyond Compare has problems aligning the logs. I often have to align the logs by going into the shortest log, finding a unique two sentences, and then removing all the garbage data above those points. It looks like this:

You can see on the left that the RMC messages appears on line 83905(!) vs line 1 on the right. Deleting everything above I get the following:

Then importing Beyond Compare has an easier time identifying where the two files align:

You can see in the above image we have perfect byte-wise recording of the NMEA data at 4Hz. The log on the left is 'teraterm'. It is the log of the UART data coming directly out of the ZED. I wanted to prove the ESP32 is correctly recording everything it receives and it appears to do a very good job.

Similarly if we compare the UART data to the USB data, we see (below) a perfect capture.

Side note: Turning on all 67 messages on both UART and USB works until you power cycle the ZED at which time it goes into a very bad 'slow PPS' death state. where the PSS LED blinks a slow 0.5Hz long blink. The ZED fails to enumerate over USB. It is recoverable by running a factory default reset over I2C but it's a huge pain programming and then reprogramming the ESP. I'm going to return to trying to replicate the message sets you two report and seem to regularly use.

nseidle · 2021-06-15T21:14:58Z

@Stefal - Please just drag/drop your settings file into the github comment window and it will upload it. Otherwise, I'm likely to miss an =1 in my setup. Here's my settings: SFE_Surveyor_Settings.txt

@pyrog - I added a 'Duplicate UART messages to USB' in the test menu just so I didn't have to use u-center to set all of Stefal's messages manually. It's in the release candidate branch and will be in the next RC.

Above: Ok, after alignment, I get error-free recording between USB and SD for a minute. I'll turn on NTRIP and run a bit longer.

Above: 1 minute no NTRIP then 3 minutes of NTRIP and RTK lock. Huh. It matches what you report. I'll see what I can do.

Thanks for all your work and help you two! Please keep it coming. I really appreciate it.

pyrog · 2021-06-15T21:22:07Z

Yep, you have to be really careful to configure the UART and USB messages to be identical so that the logs match.

How do you do that?
I thought that enabling or disabling a message in SFE_Surveyor_Settings.txt do that for UART and USB.

In practice, I need messages over BlueTooth or USB, and logging to SD card, but reliably.
Using i.e. USB and BT is for debugging purposes.

I often have to align the logs by going into the shortest log, finding a unique two sentences, and then removing all the garbage data above those points.

I do that directly in Beyond Compare 😃

I wanted to prove the ESP32 is correctly recording everything it receives and it appears to do a very good job.

Unfortunately, the "JOSM test" prove that data are corrupted.

EDIT: I just see quickly another post where you answer this question 😎
…and observe the data loss.

It's a bit late in the evening. I'll read carefully it tomorrow.

nseidle · 2021-06-15T21:33:36Z

I thought that enabling or disabling a message in SFE_Surveyor_Settings.txt do that for UART and USB.

No. Modifying the settings on RTK Surveyor changes only the UART1 on the ZED. The USB, UART2, and I2C ports are left as they were. Therefor, you have to use u-center (or the test menu feature) to setup the USB messages to be the same as the UART1 enabled messages.

I do that directly in Beyond Compare 😃

Hmm. How do you do that?

Unfortunately, the "JOSM test" prove that data are corrupted.

I disagree. I don't know or trust JSOM. It could be parsing sentences wrong. Similarly, if the ZED is the problem, there is nothing I will be able to do to make JOSM happy. My goal is to give you identical output on the SD as the ZED puts across the USB port. I suspect my code is the issue (and not u-blox's firmware on the ZED) but there is always the chance!

pyrog · 2021-06-15T21:49:34Z

Hmm. How do you do that?

Just search a NMEA sentence on the right pane, search the same on the left pane.
Then select the data and presse delete.

I don't know or trust JOSM.

As an advanced OpenStreetMap contributor, I know it pretty well and I could also check the source code if necessary.
(I checked manually corrupted NMEA files with U-CENTER. It report nothing 😢, JOSM yes )

JOSM report no errors with a NMEA record from the USB output, and some from the SD card.
And we "see" on the map that some coordinates are missing, other have missing fix.

My goal is to give you identical output on the SD as the ZED puts across the USB port

Mine too 😙

but there is always the chance!

Yes. The ZED firmware have at least one issue with RTCM over UART2

nseidle · 2021-06-15T22:06:10Z

That's better. Notes: Increased priority of serial read task from 0 to 1. Decreased TX SPP buffer to 64 bytes from 512.

nseidle · 2021-06-16T02:40:28Z

Ok, it's looking pretty good. I have seen a few errors but far fewer than before. Please give RTK_Surveyor_Firmware_v14RC_June15.zip a try.

Notes to my future self: The TX SPP buffer didn't matter it was the task priority.

pyrog · 2021-06-16T06:12:13Z

SparkFun RTK Surveyor v1.4-Jun 15 2021

Select hidden test menu

Menu: Test Menu
Bluetooth broadcasting as: Surveyor Rover-0CE2
Radio Port is now outputting RTCM
microSD card detected:
Qwiic port failed! No display detected.
Any character received over Blueooth connection will be displayed here
1) Display microSD contents
2) Turn on all messages on USB port
3) Reset USB Messages to Defaults (NMEAx6)
4) Duplicate UART messages to USB
x) Exit

Select 3 and 4

Test

SFE_Surveyor_Settings.txt
0 error detected by JOSM 😎

0 difference founded by Beyond Compare 😎

nseidle · 2021-06-17T15:24:42Z

Yay! That's good to hear.

@Stefal - Please give the latest RC a try. I'll close this issue for now but please feel free to open another if you see any more problems.

Stefal · 2021-06-17T15:57:09Z

I made 2 quick tests, and I still have some checksum errors:

Just to be sure, I made a ubx to kml convertion with u-center, and I have the same result as my script.

SFE_Surveyor_Settings.txt

I'll make more tests tomorrow with a unused SD card

Stefal · 2021-06-18T17:22:31Z

It seems better with my new µSD Card. Only 5 checksum errors for a 45mn log.

nseidle mentioned this issue Jun 14, 2021

Lost data #37

Closed

nseidle added the bug Something isn't working label Jun 14, 2021

nseidle closed this as completed Jun 17, 2021

nseidle mentioned this issue Sep 20, 2021

Unexpected reboot #54

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

checksum errors in ubx file #30

checksum errors in ubx file #30

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

checksum errors in ubx file #30

checksum errors in ubx file #30

Comments

Subject of the issue

Your workbench

Steps to reproduce

Expected behavior

Actual behavior

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Analyse with JOSM

Analyse with Beyond Compare

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Test

Uh oh!

Uh oh!

Uh oh!

Uh oh!