CN114265035B

CN114265035B - Laser radar data processing method and system

Info

Publication number: CN114265035B
Application number: CN202111475271.9A
Authority: CN
Inventors: 刘旭; 孙冬
Original assignee: Beijing Jingwei Hirain Tech Co Ltd
Current assignee: Beijing Jingwei Hirain Tech Co Ltd
Priority date: 2021-12-06
Filing date: 2021-12-06
Publication date: 2025-06-17
Anticipated expiration: 2041-12-06
Also published as: CN114265035A

Abstract

The application discloses a processing method and a processing system of laser radar data, wherein the method comprises the steps of obtaining a multi-frame two-dimensional depth map converted from laser radar original data and a filter determined according to the ratio of the transverse angle resolution and the longitudinal angle resolution of the laser radar; and for one two-dimensional depth map group, carrying out inter-frame compression on the two-dimensional depth map groups according to differences between the two-dimensional depth maps compressed in the second frame and the two-dimensional depth map compressed in the last frame in the two-dimensional depth map group and the two-dimensional depth map compressed in the first frame respectively, and carrying out inter-frame compression on all the two-dimensional depth map groups, and then carrying out integral compression on all the two-dimensional depth maps compressed in the inter-frame after carrying out inter-frame compression on all the two-dimensional depth map groups to obtain the integral compressed two-dimensional depth map. The application can realize data compression from multiple angles, thereby reducing the data volume.

Description

Laser radar data processing method and system

Technical Field

The application relates to the technical field of Internet, in particular to a laser radar data processing method and system.

Background

The lidar is a radar system that detects a characteristic quantity such as a position, a speed, etc. of a target by emitting a laser beam. The working principle is that a detection signal (namely a laser beam) is emitted to a target, then a received signal (namely a target echo) reflected from the target is compared with the emission signal, and after proper processing, related information of the target, such as parameters of the distance, the azimuth, the height, the speed, the gesture, even the shape and the like of the target, can be obtained, so that the detection, the tracking and the identification of the targets such as objects, planes, missiles and the like around a vehicle are realized.

The laser radar is used as one of the current mainstream vehicle-mounted sensors, has the advantages of high measurement accuracy, long measurement distance, high response speed, easy acquisition of three-dimensional information and the like, and is often used as a main sensor of vehicle-end, road side sensing or other intelligent equipment. At the same time, because of the characteristics of high precision and large information quantity of the laser point cloud, huge pressure is brought to transmission and storage links of a system by massive data in application scenes such as laser radar data transmission and storage. Therefore, how to reduce the data volume of storing or transmitting lidar data is an important issue to be resolved.

Disclosure of Invention

The application provides a processing method and a processing system for laser radar data, which can carry out intra-frame compression, inter-frame compression and integral compression on a two-dimensional depth map converted from laser radar original data, thereby greatly reducing the data volume.

The specific technical scheme is as follows:

In a first aspect, an embodiment of the present application provides a method for processing laser radar data, where the method includes:

Acquiring a multi-frame two-dimensional depth map converted from laser radar original data, and acquiring a filter determined according to the ratio of the transverse angle resolution to the longitudinal angle resolution of the laser radar;

Carrying out intra-frame compression on each frame of two-dimensional depth map according to the filter and a preset plane fitting algorithm to obtain a multi-frame intra-frame compressed two-dimensional depth map;

for a two-dimensional depth map group, performing inter-frame compression on the two-dimensional depth map group according to differences between a two-dimensional depth map compressed in a second frame and a two-dimensional depth map compressed in a last frame in the two-dimensional depth map group and a two-dimensional depth map compressed in a first frame, wherein each two-dimensional depth map group comprises two-dimensional depth maps compressed in continuous multi-frame frames with preset frames;

and after carrying out inter-frame compression on all the two-dimensional depth map groups, carrying out integral compression on all the two-dimensional depth maps subjected to inter-frame compression, and obtaining the two-dimensional depth map subjected to integral compression.

In one embodiment, the filter determined according to the ratio of the transverse angle resolution and the longitudinal angle resolution of the laser radar is obtained, and the filter comprises a filter for obtaining the ratio of the height to the width to be the ratio of the transverse angle resolution and the longitudinal angle resolution of the laser radar;

Before performing intra-frame compression on each frame of two-dimensional depth map according to the filter and a preset plane fitting algorithm to obtain a multi-frame intra-frame compressed two-dimensional depth map, the method further comprises the step of obtaining a final required filter when the ratio of the height to the width is the ratio of the transverse angular resolution to the longitudinal angular resolution of the laser radar and/or the compression ratio of the signal to noise ratio does not meet preset compression requirements, and adjusting the size of the filter according to the transverse distance resolution of the laser radar at the interested distance, the longitudinal distance resolution of the laser radar at the interested distance and the size of a target object at the interested distance until the adjusted compression ratio and the adjusted signal to noise ratio meet the preset compression requirements, wherein the interested distance is the distance corresponding to the original data of the laser radar, and the size comprises the height and the width.

In one embodiment, adjusting the size of the filter according to a lateral range resolution of the lidar over a range of interest, a longitudinal range resolution over the range of interest, and a size of a target object at the range of interest comprises:

Calculating a ratio of a height of a target object at the distance of interest to the lateral range resolution of the lidar over the distance of interest, a ratio of a width of the target object to the longitudinal range resolution, respectively;

If the ratio of the height of the target object to the transverse distance resolution is larger than the ratio of the width of the target object to the longitudinal distance resolution, the size ratio occupied by the height of the filter is increased;

And if the ratio of the height of the target object to the transverse distance resolution is smaller than the ratio of the width of the target object to the longitudinal distance resolution, the size occupied by the width of the filter is increased.

In one embodiment, the method includes performing intra-frame compression on each frame of two-dimensional depth map according to the filter and a preset plane fitting algorithm to obtain a multi-frame intra-frame compressed two-dimensional depth map, including:

Performing plane fitting on a first target area on the intra-frame compression two-dimensional depth map to be filtered by the filter according to the preset plane fitting algorithm to obtain a first fitting plane;

Calculating a first fitting error according to the difference between the coordinates of a plurality of data points on the first fitting plane and the coordinates of the same data point corresponding to the two-dimensional depth map to be intra-frame compressed;

If the first fitting error is smaller than a preset error threshold, a first fitting record is reserved, wherein the first fitting record comprises a fitting position of the first fitting plane and a fitting parameter of the first fitting plane, the fitting position of the first fitting plane is the position of the first target area on the compressed two-dimensional depth map in the frame to be detected, and the fitting parameter of the first fitting plane comprises a normal vector and a geometric equation of the first fitting plane;

If the first fitting error is greater than or equal to the preset error threshold, not reserving the first fitting record;

after the filter is slid to a second target area, performing plane fitting on the first target area and the second target area as an integral area according to the preset plane fitting algorithm to obtain a second fitting plane;

calculating a second fitting error according to the difference between the coordinates of a plurality of data points on the second fitting plane and the coordinates of the same data point corresponding to the intra-frame compression two-dimensional depth map before plane fitting;

If the second fitting error is smaller than the preset error threshold, replacing the first fitting record with a second fitting record, wherein the second fitting record comprises a fitting position of the second fitting plane and a fitting parameter of the second fitting plane, the fitting position of the second fitting plane comprises a position of the first target area and a position of the second target area, and the fitting parameter of the second fitting plane comprises a normal vector and a geometric equation of the second fitting plane;

If the second fitting error is greater than or equal to the preset error threshold, carrying out plane fitting on the second target area again until the last target area of the two-dimensional depth map to be intra-frame compressed is subjected to plane fitting treatment, and obtaining a two-dimensional depth map after the two-dimensional depth map to be intra-frame compressed is subjected to intra-frame compression, wherein the two-dimensional depth map after intra-frame compression comprises the last reserved fitting record and the original data of the target area without the reserved fitting record on the two-dimensional depth map to be intra-frame compressed;

And after finishing the intra-frame compression of the multi-frame two-dimensional depth map, obtaining a multi-frame intra-frame compressed two-dimensional depth map.

In one embodiment, for a two-dimensional depth map group, performing inter-frame compression on the two-dimensional depth map group according to differences between two-dimensional depth maps from a second frame of intra-frame compression to a last frame of intra-frame compression and a first frame of intra-frame compression, where the steps include:

For each frame of intra-frame compressed two-dimensional depth map except for the first frame of intra-frame compressed two-dimensional depth map in a two-dimensional depth map group, respectively calculating differences between a plurality of target areas in the current frame of intra-frame compressed two-dimensional depth map and target areas corresponding to the first frame of intra-frame compressed two-dimensional depth map, and obtaining inter-frame deviation of the current frame of intra-frame compressed two-dimensional depth map relative to the first frame of intra-frame compressed two-dimensional depth map, wherein the target areas are the sizes of areas subjected to primary filtering by the filter;

And reserving the compressed two-dimensional depth map in the first frame, and replacing the compressed two-dimensional depth map in the current frame with the position of the compressed two-dimensional depth map in the current frame and the corresponding inter-frame deviation.

In one embodiment, acquiring a multi-frame two-dimensional depth map converted from lidar raw data includes:

receiving laser radar original data sent by the laser radar;

Converting the laser radar original data into multi-frame laser radar point cloud data;

Sequentially adding each frame of laser radar point cloud data into a point cloud first-in first-out FIFO (first-out FIFO) queue according to the conversion sequence of the laser radar original data;

sequentially obtaining each frame of laser radar point cloud data from a point cloud FIFO queue according to a first-in first-out principle of the point cloud FIFO queue, and converting each frame of laser radar point cloud data into a two-dimensional depth map;

sequentially adding the converted two-dimensional depth map into a depth map FIFO queue according to the conversion sequence of the laser radar point cloud data;

and sequentially acquiring the two-dimensional depth map of each frame of laser radar in the depth map FIFO queue according to the principle of first-in first-out of the depth map FIFO queue.

In one embodiment, after performing inter-frame compression on all the two-dimensional depth map packets, performing overall compression on all the two-dimensional depth maps after inter-frame compression to obtain an overall compressed two-dimensional depth map, including:

sequentially adding each inter-frame compressed two-dimensional depth packet into a data transmission FIFO queue according to the inter-frame compression sequence of the two-dimensional depth map packet;

And after all the inter-frame compressed two-dimensional depth packets are added into the data transmission FIFO queue, carrying out integral compression on the data transmission FIFO queue, and obtaining an integral compressed two-dimensional depth map.

In one embodiment, after performing overall compression on all the two-dimensional depth maps after the inter-frame compression to obtain the overall compressed two-dimensional depth map, the method includes:

When the integrally compressed two-dimensional depth map is required to be decompressed, determining the integrally compressed two-dimensional depth map as a two-dimensional depth map to be decompressed;

The two-dimensional depth map to be decompressed is decompressed integrally, and a plurality of two-dimensional depth map groups to be decompressed among frames are obtained, wherein the two-dimensional depth map groups to be decompressed among frames are two-dimensional depth map groups after the two-dimensional depth maps compressed in continuous multi-frame frames with preset frames number;

According to the difference between the two-dimensional depth map compressed in the second frame in the two-dimensional depth map group to be decompressed in the last frame and the two-dimensional depth map compressed in the first frame, performing inter-frame decompression on the two-dimensional depth map group to be decompressed to obtain multi-frame two-dimensional depth maps to be decompressed in the frames, wherein the two-dimensional depth map to be decompressed in the frames comprises at least one fitting position, fitting parameters corresponding to the fitting position and original data of the two-dimensional depth map to be decompressed in the frames at a position which is not fit before the two-dimensional depth map is compressed in the frames, and the fitting parameters comprise normal vectors of fitting planes and geometric equations of the fitting planes;

And aiming at each frame of two-dimensional depth map to be decompressed in the frame, carrying out intra-frame decompression on the fitting position of the two-dimensional depth map to be decompressed in the frame according to the fitting position and fitting parameters corresponding to the fitting position, and obtaining original data of the fitting position on the two-dimensional depth map after intra-frame decompression, so that when all original data of the two-dimensional depth map to be decompressed in the frame on the two-dimensional depth map after intra-frame decompression are obtained, the two-dimensional depth map after intra-frame decompression of the two-dimensional depth map to be decompressed is obtained.

In one embodiment, when the two-dimensional depth map packet to be decompressed includes a two-dimensional depth map compressed in a first frame, a position of a two-dimensional depth map compressed in other frames except the two-dimensional depth map compressed in the first frame, and an inter-frame deviation of the position of the two-dimensional depth map compressed in other frames relative to the two-dimensional depth map compressed in the first frame, according to differences between the two-dimensional depth map compressed in a second frame to the two-dimensional depth map compressed in a last frame in the two-dimensional depth map packet to be decompressed in the first frame and the two-dimensional depth map compressed in the first frame, performing inter-frame decompression on the two-dimensional depth map packet to be decompressed in the second frame to obtain a multi-frame two-dimensional depth map to be decompressed, including:

For each frame of intra-frame compressed two-dimensional depth map except for a first frame of intra-frame compressed two-dimensional depth map in the two-dimensional depth map group to be decompressed, determining the current frame of intra-frame compressed two-dimensional depth map according to the position of the current frame of intra-frame compressed two-dimensional depth map, the inter-frame deviation of the current frame of intra-frame compressed two-dimensional depth map relative to the first frame of intra-frame compressed two-dimensional depth map and the first frame of intra-frame compressed two-dimensional depth map;

and after determining the compressed two-dimensional depth map in each frame in the two-dimensional depth map group to be decompressed, determining the compressed two-dimensional depth map in the frame as the decompressed two-dimensional depth map in the frame so as to obtain a plurality of frames of decompressed two-dimensional depth maps in the frame.

In a second aspect, an embodiment of the present application provides a processing apparatus for laser radar data, where the apparatus includes:

the depth map acquisition unit is used for acquiring a multi-frame two-dimensional depth map converted from laser radar original data;

A filter acquisition unit configured to acquire a filter determined from a ratio of a transverse angle resolution and a longitudinal angle resolution of the laser radar;

The intra-frame compression unit is used for carrying out intra-frame compression on each frame of two-dimensional depth map according to the filter and a preset plane fitting algorithm to obtain a multi-frame intra-frame compressed two-dimensional depth map;

The inter-frame compression unit is used for carrying out inter-frame compression on a two-dimensional depth map packet according to the difference between a two-dimensional depth map compressed in a second frame in the two-dimensional depth map packet and a two-dimensional depth map compressed in a last frame in the two-dimensional depth map packet and a two-dimensional depth map compressed in a first frame in the two-dimensional depth map packet, wherein each two-dimensional depth map packet comprises a plurality of frames of two-dimensional depth maps compressed in a continuous frame with preset frames;

and the integral compression unit is used for carrying out integral compression on all the two-dimensional depth maps after the inter-frame compression on all the two-dimensional depth map groups to obtain the two-dimensional depth map after integral compression.

In one embodiment, the depth map obtaining unit is configured to obtain a filter in which a ratio of a height to a width is a ratio of a lateral angular resolution to a longitudinal angular resolution of the laser radar;

the apparatus further comprises:

And the adjusting unit is used for carrying out intra-frame compression on each frame of two-dimensional depth map according to the filter and a preset plane fitting algorithm, and obtaining a final required filter before obtaining a multi-frame intra-frame compressed two-dimensional depth map, if the compression ratio and/or the signal to noise ratio of the filter with the ratio of the transverse angular resolution to the longitudinal angular resolution of the laser radar do not meet the preset compression requirement, adjusting the size of the filter according to the transverse distance resolution of the laser radar on the interested distance, the longitudinal distance resolution on the interested distance and the size of the target object at the interested distance until the compression ratio and the signal to noise ratio of the adjusted filter meet the preset compression requirement, wherein the interested distance is the distance corresponding to the original data of the laser radar, and the size comprises the height and the width.

In one embodiment, the adjustment unit includes:

The ratio calculation module is used for calculating the ratio of the height of the target object at the interested distance to the transverse distance resolution of the laser radar at the interested distance and the ratio of the width of the target object to the longitudinal distance resolution respectively;

And the adjusting module is used for adjusting the size proportion occupied by the height of the filter to be higher if the ratio of the height of the target object to the transverse distance resolution is larger than the ratio of the width of the target object to the longitudinal distance resolution, and adjusting the size proportion occupied by the width of the filter to be higher if the ratio of the height of the target object to the transverse distance resolution is smaller than the ratio of the width of the target object to the longitudinal distance resolution.

In one embodiment, an intra-frame compression unit includes:

the fitting module is used for carrying out plane fitting on a first target area on the intra-frame compression two-dimensional depth map to be filtered by the filter according to the preset plane fitting algorithm to obtain a first fitting plane;

The error calculation module is used for calculating a first fitting error according to the difference between the coordinates of a plurality of data points on the first fitting plane and the coordinates of the same data point corresponding to the two-dimensional depth map to be compressed in the frame;

The first preserving module is configured to preserve a first fitting record if the first fitting error is smaller than a preset error threshold, wherein the first fitting record includes a fitting position of the first fitting plane and a fitting parameter of the first fitting plane, the fitting position of the first fitting plane is a position of the first target region on the two-dimensional depth map to be compressed in the frame, the fitting parameter of the first fitting plane includes a normal vector and a geometric equation of the first fitting plane, and if the first fitting error is greater than or equal to the preset error threshold, the first fitting record is not preserved;

The fitting module is used for performing plane fitting on the first target area and the second target area as an integral area according to the preset plane fitting algorithm after the filter is slid to the second target area, so as to obtain a second fitting plane;

The error calculation module is used for calculating a second fitting error according to the difference between the coordinates of a plurality of data points on the second fitting plane and the coordinates of the same data point corresponding to the intra-frame compression two-dimensional depth map before plane fitting;

A first replacing module, configured to replace a second fit record with the first fit record if the second fit error is less than the preset error threshold, where the second fit record includes a fit position of the second fit plane and a fit parameter of the second fit plane, the fit position of the second fit plane includes a position of the first target region and a position of the second target region, and the fit parameter of the second fit plane includes a normal vector and a geometric equation of the second fit plane;

The fitting module is used for carrying out plane fitting on the second target area again if the second fitting error is greater than or equal to the preset error threshold value until the last target area of the two-dimensional depth map to be intra-frame compressed is subjected to plane fitting processing, and a two-dimensional depth map after intra-frame compression is obtained, wherein the two-dimensional depth map after intra-frame compression comprises the last reserved fitting record and the original data of the target area without the reserved fitting record on the two-dimensional depth map to be intra-frame compressed;

And the first acquisition module is used for acquiring the multi-frame two-dimensional depth map after the multi-frame two-dimensional depth map is subjected to intra-frame compression.

In one embodiment, an inter-frame compression unit includes:

The deviation calculation module is used for respectively calculating differences between a plurality of target areas in a two-dimensional depth map after the current frame and a target area corresponding to the two-dimensional depth map after the first frame in the two-dimensional depth map after the current frame in each frame of the two-dimensional depth map after the first frame in the two-dimensional depth map group except the two-dimensional depth map after the first frame in the two-dimensional depth map group, so as to obtain the inter-frame deviation of the two-dimensional depth map after the current frame in relation to the two-dimensional depth map after the first frame in the two-dimensional depth map group, wherein the target area is the area size of the filter for carrying out primary filtering;

the second reservation module is used for reserving the compressed two-dimensional depth map in the first frame;

And the second replacing module is used for replacing the compressed two-dimensional depth map in the current frame with the position of the compressed two-dimensional depth map in the current frame and the corresponding inter-frame deviation.

In one embodiment, the depth map acquisition unit includes:

The receiving module is used for receiving laser radar original data sent by the laser radar;

The first conversion module is used for converting the laser radar original data into multi-frame laser radar point cloud data;

The first adding module is used for sequentially adding the point cloud data of each frame of laser radar into a point cloud first-in first-out FIFO (first-out first-in first-out) queue according to the conversion sequence of the laser radar original data;

The second conversion module is used for sequentially acquiring each frame of laser radar point cloud data from the point cloud FIFO queue according to the principle of first-in first-out of the point cloud FIFO queue and converting each frame of laser radar point cloud data into a two-dimensional depth map;

The second adding module is used for sequentially adding the converted two-dimensional depth map into a depth map FIFO queue according to the conversion sequence of the laser radar point cloud data;

And the second acquisition module is used for sequentially acquiring each frame of laser radar two-dimensional depth map in the depth map FIFO queue according to the principle of first-in first-out of the depth map FIFO queue.

In one embodiment, an integral compression unit includes:

The third adding module is used for sequentially adding each inter-frame compressed two-dimensional depth packet into the data transmission FIFO queue according to the inter-frame compression sequence of the two-dimensional depth map packets;

And the integral compression module is used for integrally compressing the data transmission FIFO queue after adding all the two-dimensional depth packets compressed among frames into the data transmission FIFO queue, so as to obtain an integrally compressed two-dimensional depth map.

In one embodiment, the apparatus further comprises:

The determining unit is used for integrally compressing all the two-dimensional depth maps after the inter-frame compression to obtain an integrally compressed two-dimensional depth map, and determining the integrally compressed two-dimensional depth map as a two-dimensional depth map to be decompressed when the integrally compressed two-dimensional depth map is required to be decompressed;

The overall decompression unit is used for overall decompressing the two-dimensional depth map to be decompressed to obtain a plurality of two-dimensional depth map groups to be decompressed among frames, wherein the two-dimensional depth map groups to be decompressed among frames are two-dimensional depth map groups after the two-dimensional depth map groups are compressed among frames aiming at a two-dimensional depth map compressed in a plurality of frames with preset frames number;

The inter-frame decompression unit is used for performing inter-frame decompression on the two-dimensional depth map group to be subjected to inter-frame decompression according to differences between the two-dimensional depth map compressed in the second frame in the two-dimensional depth map group to be subjected to inter-frame decompression and the two-dimensional depth map compressed in the last frame in the two-dimensional depth map group to be subjected to inter-frame decompression, so as to obtain multi-frame two-dimensional depth maps to be subjected to intra-frame decompression, wherein the two-dimensional depth map to be subjected to intra-frame decompression comprises at least one fitting position, fitting parameters corresponding to the fitting position and original data of the two-dimensional depth map to be subjected to intra-frame decompression at a position which is not fitted before intra-frame compression, and the fitting parameters comprise normal vectors of a fitting plane and a geometric equation of the fitting plane;

And the intra-frame decompression unit is used for carrying out intra-frame decompression on the fitting position of the two-dimensional depth map to be intra-frame decompressed according to the fitting position and fitting parameters corresponding to the fitting position, so as to obtain the original data of the fitting position on the two-dimensional depth map after intra-frame decompression, and further obtain the two-dimensional depth map after intra-frame decompression of the two-dimensional depth map to be intra-frame decompressed when all the original data of the two-dimensional depth map after intra-frame decompression are obtained.

In one embodiment, an inter-frame decompression unit includes:

An inter-frame decompression module, configured to determine, when the two-dimensional depth map packet to be inter-frame decompressed includes a first intra-frame compressed two-dimensional depth map in the two-dimensional depth map packet to be inter-frame decompressed, positions of other intra-frame compressed two-dimensional depth maps except for the first intra-frame compressed two-dimensional depth map, and inter-frame deviations of the positions of the other intra-frame compressed two-dimensional depth maps with respect to the first intra-frame compressed two-dimensional depth map, for each intra-frame compressed two-dimensional depth map except for the first intra-frame compressed two-dimensional depth map in the two-dimensional depth map packet to be inter-frame decompressed, and for a position of a current intra-frame compressed two-dimensional depth map, an inter-frame deviation of a current intra-frame compressed two-dimensional depth map with respect to the first intra-frame compressed two-dimensional depth map, and the first intra-frame compressed two-dimensional depth map;

and the determining module is used for determining the two-dimensional depth map compressed in each frame in the two-dimensional depth map group to be decompressed in frames as the two-dimensional depth map to be decompressed in frames after determining the two-dimensional depth map compressed in frames in each frame, so as to obtain multi-frame two-dimensional depth maps to be decompressed in frames.

In a third aspect, an embodiment of the present application provides a processing system for laser radar data, where the system includes a laser radar, a terminal device, and a server;

the laser radar is used for acquiring laser radar original data and sending the laser radar original data to the terminal equipment;

The terminal device is configured to obtain an integrally compressed two-dimensional depth map by executing the method according to any one of the embodiments of the first aspect, and send the integrally compressed two-dimensional depth map to the server;

The server is configured to receive the integrally compressed two-dimensional depth map sent by the terminal device, determine the integrally compressed two-dimensional depth map as a two-dimensional depth map to be decompressed, decompress the two-dimensional depth map to be decompressed integrally to obtain a plurality of two-dimensional depth map packets to be decompressed between frames, wherein the two-dimensional depth map packets to be decompressed between frames are two-dimensional depth map packets obtained by performing inter-frame compression on the two-dimensional depth map compressed in continuous multi-frame frames with preset frames number, and perform inter-frame decompression on the two-dimensional depth map packets to be decompressed between frames according to differences between two-dimensional depth maps compressed in a second frame to a last frame in the two-dimensional depth map packets to be decompressed between frames and the two-dimensional depth map compressed in a first frame, obtaining a multi-frame two-dimensional depth map to be decompressed in a frame, wherein the two-dimensional depth map to be decompressed in the frame comprises at least one fitting position, fitting parameters corresponding to the fitting position and original data of the two-dimensional depth map to be decompressed in the frame at a position which is not fit before the two-dimensional depth map to be decompressed in the frame is compressed in the frame, the fitting parameters comprise normal vectors of fitting planes and geometric equations of the fitting planes, for each frame of the two-dimensional depth map to be decompressed in the frame, according to the fitting position and the fitting parameters corresponding to the fitting position, carrying out intra-frame decompression on the fitting position of the two-dimensional depth map to be decompressed in the frame to obtain original data of the fitting position on the two-dimensional depth map after the two-dimensional depth map to be decompressed in the frame is obtained, and obtaining the two-dimensional depth map after the two-dimensional depth map to be decompressed in the frame is decompressed in the frame.

In one embodiment, when the two-dimensional depth map packet to be decompressed includes a first intra-frame compressed two-dimensional depth map, a position of another intra-frame compressed two-dimensional depth map other than the first intra-frame compressed two-dimensional depth map, and an inter-frame deviation of the other intra-frame compressed two-dimensional depth map with respect to the first intra-frame compressed two-dimensional depth map, the server is configured to determine, for each intra-frame compressed two-dimensional depth map in the two-dimensional depth map packet to be decompressed except the first intra-frame compressed two-dimensional depth map, for the position of the current intra-frame compressed two-dimensional depth map, the inter-frame deviation of the current intra-frame compressed two-dimensional depth map with respect to the first intra-frame compressed two-dimensional depth map, and the inter-frame deviation of the other intra-frame compressed two-dimensional depth map, determine that the current intra-frame compressed two-dimensional depth map is the multi-frame compressed two-dimensional depth map, and determine that the two-frame depth map to be decompressed is the two-dimensional depth map to be decompressed after being decompressed.

In a fourth aspect, an embodiment of the present application provides a storage medium having stored thereon executable instructions that when executed by a processor cause the processor to implement a method according to any embodiment of the first aspect.

In a fifth aspect, an embodiment of the present application provides an electronic device, including:

One or more processors;

Storage means for storing one or more programs,

Wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any of the embodiments of the first aspect.

As can be seen from the foregoing, the processing method and system for laser radar data provided by the embodiments of the present application can obtain a multi-frame two-dimensional depth map converted from laser radar raw data, obtain a filter determined according to a ratio of a transverse angle resolution to a longitudinal angle resolution of the laser radar, then perform intra-frame compression on each frame of two-dimensional depth map according to the filter and a preset plane fitting algorithm to obtain a multi-frame intra-compressed two-dimensional depth map, perform inter-frame compression on a two-dimensional depth map packet composed of a continuous multi-frame intra-compressed two-dimensional depth map with a preset frame number, and finally perform overall compression on all the two-dimensional depth maps after the inter-frame compression is completed to obtain a final compression result. Therefore, the embodiment of the application not only can realize the intra-frame compression, the inter-frame compression and the integral compression of the two-dimensional depth map converted from the laser radar original data, greatly reduce the data volume of the two-dimensional depth map from the aspect of multi-angle compression, but also can filter the two-dimensional depth map according to the filter determined by the resolution of the horizontal angle and the longitudinal angle, and does not directly filter the two-dimensional depth map by using the filter with the same width and height, thereby improving the compression rate and the compression speed. When the laser radar data (namely the laser radar original data or the two-dimensional depth map) is required to be locally stored, the embodiment of the application can save the storage space, and when the laser radar data is required to be transmitted outwards, the transmission efficiency can be improved.

The technical effects that can be obtained by the embodiment of the application at least comprise:

1. If the compression ratio and/or the signal-to-noise ratio of the filter with the ratio of the transverse angle resolution to the longitudinal angle resolution of the laser radar do not meet the preset compression requirement, the size of the filter can be quickly adjusted through the transverse distance resolution, the longitudinal distance resolution and the size of the target object at the interested distance of the laser radar without blind adjustment, so that the adjustment efficiency can be improved.

2. In the data format conversion (including converting the laser radar original data into laser radar point cloud data and converting the laser radar point cloud data into a two-dimensional depth map) and data compression process, the frame rate of processing the laser radar data by an algorithm can be improved through simultaneous operation of multiple threads by adopting different FIFO (First Input First Output, first-in first-out) queues, the size of each frame of data is relatively stable under the condition that the type and environment of the laser radar are relatively stable, the memory occupied by each FIFO can be calculated in advance, and application is performed in the initialization stage of the algorithm operation, so that frequent memory application is avoided, and the calculation speed is accelerated.

Of course, it is not necessary for any one product or method of practicing the application to achieve all of the advantages set forth above at the same time.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It is apparent that the drawings in the following description are only some embodiments of the application. Other figures may be derived from these figures without inventive effort for a person of ordinary skill in the art.

Fig. 1 is a flowchart illustrating a method for processing lidar data according to an embodiment of the present application;

FIG. 2 is an exemplary diagram of laser radar data parallel processing according to an embodiment of the present application;

FIG. 3 is a flowchart illustrating another method for processing lidar data according to an embodiment of the present application;

fig. 4 is a schematic structural diagram of a laser radar data processing system according to an embodiment of the present application;

fig. 5 is a block diagram of a laser radar data processing device according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. It will be apparent that the described embodiments are only some, but not all, embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without any inventive effort, are intended to be within the scope of the application.

It should be noted that the terms "comprising" and "having" and any variations thereof in the embodiments of the present application and the accompanying drawings are intended to cover non-exclusive inclusions. A process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed but may alternatively include other steps or elements not listed or inherent to such process, method, article, or apparatus.

The application provides a laser radar data processing method and a laser radar data processing system, which can compress laser radar data from multiple angles, thereby greatly reducing the data volume of the laser radar data. The following describes embodiments of the present application in detail.

Fig. 1 is a flow chart of a laser radar data processing method according to an embodiment of the present application. The method may be applied to a terminal device, such as a vehicle, a mobile terminal, etc., and may comprise the steps of:

step S110, acquiring a multi-frame two-dimensional depth map converted from laser radar original data, and acquiring a filter determined according to the ratio of the transverse angle resolution and the longitudinal angle resolution of the laser radar.

As shown in FIG. 2, the specific implementation method for acquiring the multi-frame two-dimensional depth map converted from the laser radar original data comprises the steps of receiving the laser radar original data sent by the laser radar, converting the laser radar original data into multi-frame laser radar point cloud data, sequentially adding each frame of laser radar point cloud data into a point cloud FIFO queue according to the conversion sequence of the laser radar original data, sequentially acquiring each frame of laser radar point cloud data from the point cloud FIFO queue according to the first-in first-out principle of the point cloud FIFO queue, converting each frame of laser radar point cloud data into a two-dimensional depth map, sequentially adding the converted two-dimensional depth map into a depth map FIFO queue according to the conversion sequence of the laser radar point cloud data, and sequentially acquiring each frame of laser radar two-dimensional depth map in the depth map FIFO queue according to the first-in first-out principle of the depth map FIFO queue.

The method can be realized based on a laser radar driving program when the laser radar original data are converted into multi-frame laser radar point cloud data, and can be realized based on a conversion module when each frame of laser radar point cloud data are converted into a two-dimensional depth map. When each frame of laser radar two-dimensional depth map in the depth map FIFO queue is acquired, intra-frame compression and inter-frame compression can be achieved based on the compression module, and the inter-frame compressed two-dimensional depth map is added into the data transmission FIFO queue, so that after the data transmission FIFO queue is integrally compressed, the integrally compressed two-dimensional depth map is transmitted to a server.

Point cloud data refers to a set of vectors in a three-dimensional coordinate system. The scan data of the point cloud data is recorded in the form of points, each of which includes three-dimensional coordinates, and some of which may include color information or reflection intensity information. The maximum scanning range of the laser radar in the embodiment of the application can be 360 degrees, and can also be other ranges. And the laser radar point cloud data of one frame corresponds to laser radar original data scanned by the laser radar in a maximum scanning range. A two-dimensional depth map (referred to herein as raw data for an uncompressed two-dimensional depth map) includes a plurality of data points (which may also be referred to as pixel points), the coordinates of each data point including depth information representing the distance between a target object characterized by the data point and the lidar and intensity information representing the lidar pulse-echo intensity at which the data point was generated.

For each FIFO, the earliest enqueued data may be deleted to prevent memory overflow when the number reaches the upper length limit. By adopting the mechanism, the functions of the laser radar driving program, the conversion module and the compression module can be operated simultaneously through multiple threads, the frame rate of processing laser radar data by an algorithm can be improved, the size of each frame of data is relatively stable under the condition that the type and the environment of the laser radar are relatively stable, the memory occupied by each FIFO can be calculated in advance, and the application is carried out in the initialization stage of the operation of the algorithm, so that frequent memory application is avoided, and the calculation speed is accelerated. Meanwhile, the method can manually prescribe the length of each FIFO through applying the size of the memory, when a certain link is blocked, data can be accumulated in the corresponding FIFO, the earliest obtained data can be popped up after the maximum value is reached, the whole link can not be stopped, the robustness of a program is improved, the length of each FIFO can be monitored in real time during the running of the program, and the blocked link can be rapidly positioned.

In one embodiment, the specific implementation of obtaining the filter determined according to the ratio of the transverse angle resolution and the longitudinal angle resolution of the laser radar comprises obtaining a filter with the ratio of the height to the width being the ratio of the transverse angle resolution and the longitudinal angle resolution of the laser radar, for example, when the ratio of the transverse angle resolution to the longitudinal angle resolution is 1:5, the filter can be selected to be 1 in height and 5 in width, namely, the filter with the size being 1×5, and also can be selected to be 2×10. Angular resolution refers to the resolution capability of an imaging system or a component of the system, i.e., the ability of an imaging system or system element to differentially distinguish between the smallest spacing of two adjacent objects.

However, in practical applications, when the ratio of the transverse angle resolution to the longitudinal angle resolution of the lidar is directly used as the ratio of the height to the width of the filter, the compression ratio may be suitable, but the signal-to-noise ratio is low, so that the problem of higher distortion ratio of the decompressed lidar data may occur. In order to balance the compression ratio and the signal-to-noise ratio so that the compression ratio and the signal-to-noise ratio both meet preset compression requirements, the filter size can be adjusted based on the ratio of the transverse angular resolution to the longitudinal angular resolution of the lidar. In order to improve the adjustment efficiency, before executing step S120, if the compression ratio and/or the signal-to-noise ratio of the filter with the ratio of the height to the width being the ratio of the lateral angular resolution to the longitudinal angular resolution of the lidar do not meet the preset compression requirement, the size of the filter is adjusted according to the lateral distance resolution of the lidar at the distance of interest, the longitudinal distance resolution at the distance of interest, and the size of the target object at the distance of interest until the adjusted compression ratio and the adjusted signal-to-noise ratio of the filter meet the preset compression requirement, and the final required filter is obtained. The interested distance is a distance corresponding to the laser radar original data, and the size comprises a height and a width. The preset compression requirement includes a compression ratio less than or equal to a preset compression ratio threshold and a signal to noise ratio greater than or equal to a preset signal to noise ratio threshold.

The compression rate is a compression rate for calculating the two-dimensional depth map obtained by performing steps S120 and S130 on the multi-frame two-dimensional depth map, and the calculation method includes: Where η denotes a compression rate, c _key denotes a size of a two-dimensional depth map after the first inter-frame compression in one two-dimensional depth map packet mentioned below, c _i denotes a size of a two-dimensional depth map after the i+1st inter-frame compression in the two-dimensional depth map packet, Representing the size before intra-frame compression of each frame of the two-dimensional depth map, for scenes with little environmental variation,It can be approximately considered as a constant value, n represents the number of frames in a two-dimensional depth map packet, i.e., the preset number of frames mentioned below, 0+.i+.n-1, c ₀＝c_key, n is a positive integer.

In the laser radar image, when two targets are located at the same azimuth angle, but the distance between the targets and the laser radar is different, the minimum distance between the targets and the laser radar is the distance resolution. The method for calculating the distance resolution comprises the steps of calculating the transverse distance resolution of the laser radar on the interested distance according to the interested distance, the transverse angle resolution and the installation height of the laser radar, and calculating the longitudinal distance resolution of the laser radar on the interested distance according to the interested distance, the longitudinal angle resolution and the installation height of the laser radar.

The calculation process of the transverse distance resolution comprises substituting the interested distance l, the transverse angle resolution alpha _heng and the mounting height h of the laser radar into a first formula, and calculating the transverse distance resolution d _heng of the laser radar on the interested distance, wherein the first formula is that

And/or the calculation process of the longitudinal distance resolution comprises substituting the interested distance l, the longitudinal angle resolution alpha _zong and the mounting height h of the laser radar into a second formula, and calculating the longitudinal distance resolution d _zong of the laser radar on the interested distance, wherein the second formula is that

The method for adjusting the size of the filter according to the transverse distance resolution of the laser radar at the interested distance, the longitudinal distance resolution of the laser radar at the interested distance and the size of the target object at the interested distance comprises the steps of respectively calculating the ratio of the height of the target object at the interested distance to the transverse distance resolution of the laser radar at the interested distance and the ratio of the width of the target object to the longitudinal distance resolution, adjusting the size proportion occupied by the height of the filter to be higher if the ratio of the height of the target object to the transverse distance resolution is larger than the ratio of the width of the target object to the longitudinal distance resolution, and adjusting the size proportion occupied by the width of the filter to be higher if the ratio of the height of the target object to the transverse distance resolution is smaller than the ratio of the width of the target object to the longitudinal distance resolution. The specific size ratio of the height=height/(height+width), and the specific size ratio of the width=width/(height+width).

In one embodiment, if the laser radar has a lateral angular resolution and a longitudinal angular resolution of 0.1 degree and 0.5 degree, respectively, the distance of interest is 80m, the installation height is 6m, and the calculated lateral distance resolution and longitudinal distance resolution are 0.14m and 0.7m, respectively, then the ratio of the lateral distance resolution to the longitudinal distance resolution is 1:5, so that the filter may be set to a size of 1×5 first, but when the filter based on the size is compressed, the obtained signal-to-noise ratio does not meet the preset compression requirement, so that the adjustment may be performed based on the lateral distance resolution and the longitudinal distance resolution. Assuming that the height and width of the target object at the distance of interest are 1.5m and 5m, respectively, the ratio of the height of the target object to the lateral distance resolution is greater than the ratio of the width of the target object to the longitudinal distance resolution, whereby the dimensional specific gravity occupied by the height can be increased, and finally a2×6 filter can be obtained.

It should be noted that, since there is no dependency between "acquiring a multi-frame two-dimensional depth map converted from laser radar raw data" and "acquiring a filter determined according to a lateral angular resolution and a longitudinal angular resolution of the laser radar", the execution sequence of the two is not limited by the implementation of the present application.

And step S120, carrying out intra-frame compression on each frame of two-dimensional depth map according to the filter and a preset plane fitting algorithm to obtain a multi-frame intra-frame compressed two-dimensional depth map.

Carrying out plane fitting on a first target area on a two-dimensional depth map to be intra-frame filtered by the filter according to the preset plane fitting algorithm to obtain a first fitting plane, calculating a first fitting error according to the difference between the coordinates of a plurality of data points on the first fitting plane and the coordinates of the same data point corresponding to the two-dimensional depth map to be intra-frame compressed, if the first fitting error is smaller than a preset error threshold value, reserving a first fitting record, wherein the first fitting record comprises fitting positions of the first fitting plane and fitting parameters of the first fitting plane, the fitting positions of the first fitting plane are the positions of the first target area on the two-dimensional depth map to be intra-frame compressed, the fitting parameters of the first fitting plane comprise normal vectors and geometric equations of the first fitting plane, if the first fitting error is larger than or equal to the preset error threshold value, the first record is not reserved, sliding the filter to a second target area, and then taking the first fitting record and the second fitting plane as fitting parameters of the second fitting plane according to the preset plane fitting error, and calculating a plurality of coordinate values between the first fitting parameters and the second fitting plane as the second fitting parameters of the second fitting plane, and the second fitting plane is calculated according to the second fitting error, and the first fitting error is the second fitting error is calculated, and the first fitting error is the second fitting error is larger than the second fitting plane and the second fitting error is calculated and the second fitting error is the second fitting a second fitting plane is the second fitting plane. And if the second fitting error is greater than or equal to the preset error threshold, carrying out plane fitting on the second target area again until the last target area of the two-dimensional depth map to be compressed in the frame is subjected to plane fitting processing, and obtaining a two-dimensional depth map after carrying out intra-frame compression on the two-dimensional depth map to be compressed in the frame, wherein the two-dimensional depth map after being compressed in the frame comprises the last reserved fitting record and the original data of the target area which does not reserve the fitting record on the two-dimensional depth map to be compressed in the frame, and obtaining a multi-frame two-dimensional depth map after carrying out intra-frame compression on the multi-frame two-dimensional depth map.

The preset plane fitting algorithm can be a linear least square method or other plane fitting algorithms. The calculation method of the fitting error (including the first fitting error and the second fitting error) may include calculating a mean square error of differences for a plurality of data points based on differences between coordinates of the plurality of data points on a fitting plane (including the first fitting plane and the second fitting plane) and coordinates of the same data point corresponding to the two-dimensional depth map, and taking the mean square error as the fitting error.

The method comprises the steps of assuming that a two-dimensional depth map to be intra-frame compressed can be divided into target areas with the size of 3 filters, namely a first target area, a second target area and a third target area, if a first fitting error of a first fitting plane for the first target area is smaller than a preset error threshold, keeping a first fitting record, including fitting positions of the first fitting plane (positions of the first target area) and fitting parameters of the first fitting plane, sliding the filters to the second target area, taking the first target area and the second target area as an integral area to perform plane fitting to obtain a second fitting plane, if a second fitting error of the second fitting plane is smaller than the preset error threshold, replacing the first fitting record by the second fitting record, wherein the second fitting record comprises fitting positions of the second fitting plane (positions of the first target area and the second target area) and fitting parameters of the second plane, continuing sliding the filters to the third target area to perform plane fitting to obtain a third fitting plane as the integral area, and if the third fitting error of the second fitting plane is larger than the preset error threshold, keeping the third fitting plane to obtain a fourth fitting plane if the third fitting error is larger than the preset error of the fourth fitting plane or equal to the fourth fitting error of the preset error threshold, and keeping the fourth fitting error is larger than the preset to obtain the first fitting plane. The finally obtained intra-frame compressed two-dimensional depth map comprises the positions of the first target area and the second target area, fitting parameters of a second fitting plane for the whole area which is the first target area and the second target area, and original data of a third target area on the intra-frame pre-compression two-dimensional depth map.

And S130, for a two-dimensional depth map group, carrying out inter-frame compression on the two-dimensional depth map group according to differences between the two-dimensional depth map compressed in the second frame and the two-dimensional depth map compressed in the last frame in the two-dimensional depth map group and the two-dimensional depth map compressed in the first frame.

Each two-dimensional depth group comprises a two-dimensional depth map compressed in a plurality of continuous frames with the number of frames being a preset number. The two-dimensional depth map may be grouped at any time from the conversion of the lidar raw data into the two-dimensional depth map to the inter-frame compression. By the above formulaIt can be known that the larger the preset frame number n is, the smaller the compression ratio is, the better the compressibility is, but the better the compressibility is, the smaller the signal-to-noise ratio is, and the higher the distortion ratio of the two-dimensional depth map is, so that the n value needs to be determined according to practical experience, based on meeting the preset compression requirement set according to the compression ratio and the signal-to-noise ratio, for example, the n value may be 10.

For each frame of compressed two-dimensional depth map except for the first frame of compressed two-dimensional depth map in a two-dimensional depth map group, respectively calculating differences between a plurality of target areas in the current frame of compressed two-dimensional depth map and target areas corresponding to the first frame of compressed two-dimensional depth map to obtain inter-frame deviations corresponding to the current frame of compressed two-dimensional depth map, wherein the target areas are the sizes of the areas which are subjected to primary filtering by the filter, namely the first target area, the second target area and the like, reserving the first frame of compressed two-dimensional depth map, and replacing the current frame of compressed two-dimensional depth map with the corresponding inter-frame deviations.

When calculating the difference between a plurality of target areas in the two-dimensional depth map after the current frame and the target areas corresponding to the two-dimensional depth map after the first frame, if the data contained in the target area in the two-dimensional depth map after the current frame and the target area at the position corresponding to the two-dimensional depth map after the first frame are both fitting parameters, the difference between the two is the difference between normal vectors of two fitting planes and the difference between geometric equations of the two fitting planes, if the data contained in the target area in the two-dimensional depth map after the current frame and the target area at the position corresponding to the two-dimensional depth map after the first frame are all original data on the two-dimensional depth map before the frame compression, the difference between the two is the difference between the two original data (namely, the coordinate difference of data points), and if the data contained in the target area in the two-dimensional depth map after the current frame and the two-dimensional depth map after the first frame correspond to each other includes the fitting parameters and the original data, the two-dimensional depth map before the fitting parameters are converted, namely, the original data on the two-dimensional depth map before the frame compression can be calculated.

And step S140, after the inter-frame compression is carried out on all the two-dimensional depth map groups, carrying out integral compression on all the two-dimensional depth maps subjected to the inter-frame compression, and obtaining the two-dimensional depth map subjected to integral compression.

The whole compression may be a file compression method, or may be other compression methods, for example, a file compression method such as zip or rar. After the integrally compressed two-dimensional depth map is obtained, the integrally compressed two-dimensional depth map can be stored locally, and can be sent to a server, so that the server decompresses the integrally compressed two-dimensional depth map and then performs operations such as statistics and analysis on decompressed data.

When the integrally compressed two-dimensional depth map is required to be transmitted to a server, each inter-frame compressed two-dimensional depth map packet can be sequentially added to a data transmission FIFO queue according to the inter-frame compression sequence of the two-dimensional depth map packets, and after all inter-frame compressed two-dimensional depth packets are added to the data transmission FIFO queue, the data transmission FIFO queue is integrally compressed, so that the integrally compressed two-dimensional depth map is obtained. The whole data transmission FIFO queue can be compressed integrally, the data in the data transmission queue can be compressed in batches, and the integrally compressed two-dimensional depth map is sequentially sent to the server according to the first-in first-out principle.

The processing method of the laser radar data provided by the embodiment of the application can acquire the multi-frame two-dimensional depth map converted from the laser radar original data, acquire the filter determined according to the ratio of the transverse angle resolution to the longitudinal angle resolution of the laser radar, then perform intra-frame compression on each frame of two-dimensional depth map according to the filter and a preset plane fitting algorithm to obtain multi-frame intra-frame compressed two-dimensional depth maps, then perform inter-frame compression on two-dimensional depth map groups consisting of continuous multi-frame intra-frame compressed two-dimensional depth maps with preset frame numbers, and finally perform integral compression on all the two-dimensional depth maps after the inter-frame compression is completed to obtain a final compression result. Therefore, the embodiment of the application not only can realize the intra-frame compression, the inter-frame compression and the integral compression of the two-dimensional depth map converted from the laser radar original data, greatly reduce the data volume of the two-dimensional depth map from the aspect of multi-angle compression, but also can filter the two-dimensional depth map according to the filter determined by the resolution of the horizontal angle and the longitudinal angle, and does not directly filter the two-dimensional depth map by using the filter with the same width and height, thereby improving the compression rate and the compression speed. When the laser radar data (namely the laser radar original data or the two-dimensional depth map) is required to be locally stored, the embodiment of the application can save the storage space, and when the laser radar data is required to be transmitted outwards, the transmission efficiency can be improved.

Based on the embodiment of the method, the application also provides a laser radar data processing method, which can be applied to terminal equipment and a server. When the method is applied to the terminal equipment, the terminal equipment can store the compressed two-dimensional depth map (namely, the integrally compressed two-dimensional depth map finally obtained in the figure 1) to the local after being compressed based on the method shown in the figure 1, when the method is applied to the server, the integrally compressed two-dimensional depth map stored to the local can be determined to be the two-dimensional depth map to be decompressed by adopting the embodiment of the application, and when the method is applied to the server, the terminal equipment can send the compressed two-dimensional depth map (namely, the integrally compressed two-dimensional depth map finally obtained in the figure 1) to the server, and the server determines the received integrally compressed two-dimensional depth map to be the two-dimensional depth map to be decompressed by adopting the embodiment of the application. As shown in fig. 3, the method includes:

and S210, acquiring a two-dimensional depth map to be decompressed.

The two-dimensional depth map to be decompressed is data obtained by compressing a two-dimensional depth map obtained by converting laser radar original data, and the compression process is shown in the embodiment of fig. 1, which is not described again.

And S220, carrying out overall decompression on the two-dimensional depth map to be decompressed to obtain a plurality of two-dimensional depth map groups to be decompressed among frames.

The two-dimensional depth map grouping to be decompressed is performed by performing inter-frame compression on the two-dimensional depth map compressed in a plurality of continuous frames with the preset frame number. The overall decompression is an inverse of the algorithm used for the overall compression, for example, when the overall compression uses a zip compression algorithm, the overall decompression uses a zip decompression algorithm.

And step S230, carrying out inter-frame decompression on the two-dimensional depth map group to be decompressed according to the difference between the two-dimensional depth map compressed in the second frame in the two-dimensional depth map group to be decompressed to the two-dimensional depth map compressed in the last frame in the first frame and the two-dimensional depth map compressed in the first frame, so as to obtain multi-frame two-dimensional depth maps to be decompressed in the frames.

The two-dimensional depth map to be decompressed in the frame comprises at least one fitting position, fitting parameters corresponding to the fitting position and original data of the two-dimensional depth map to be decompressed in the frame at a position which is not fit before the compression in the frame, wherein the fitting parameters comprise normal vectors of a fitting plane and geometric equations of the fitting plane.

When the two-dimensional depth map group to be decompressed comprises a two-dimensional depth map compressed in a first frame in the two-dimensional depth map group to be decompressed, positions of other two-dimensional depth maps compressed in other frames except the first frame in the two-dimensional depth map group to be decompressed, and inter-frame deviations of the positions of the other two-dimensional depth maps compressed in the other frames relative to the two-dimensional depth map compressed in the first frame, the specific implementation process of the step comprises determining that the two-dimensional depth map compressed in each frame except the two-dimensional depth map compressed in the first frame in the two-dimensional depth map group to be decompressed is a multi-frame two-dimensional depth map for the two-dimensional depth map compressed in the current frame, the inter-frame deviations of the two-dimensional depth map compressed in the current frame relative to the two-dimensional depth map compressed in the first frame, and determining that the two-dimensional depth map compressed in the current frame is decompressed in each frame in the two-dimensional depth map group to be decompressed.

And adding the inter-frame deviation of the two-dimensional depth map after the first frame intra-frame compression and the two-dimensional depth map after the current frame intra-frame compression relative to the two-dimensional depth map after the first frame intra-frame compression to obtain data contained in the two-dimensional depth map after the current frame intra-frame compression, and determining the complete two-dimensional depth map after the current frame intra-frame compression by combining the positions of the two-dimensional depth maps after the current frame intra-frame compression to obtain the two-dimensional depth map after the intra-frame compression at which position is located.

Step S240, for each frame of two-dimensional depth map to be decompressed in the frame, according to the fitting position and the fitting parameters corresponding to the fitting position, performing intra-frame decompression on the fitting position of the two-dimensional depth map to be decompressed in the frame to obtain the original data of the fitting position on the two-dimensional depth map after intra-frame decompression, so as to obtain the two-dimensional depth map after intra-frame decompression on the two-dimensional depth map to be decompressed when all the original data of the two-dimensional depth map after intra-frame decompression on the two-dimensional depth map to be decompressed are obtained.

According to the fitting position and the fitting parameters corresponding to the fitting position, carrying out intra-frame decompression on the fitting position of the two-dimensional depth map to be intra-frame decompressed, and obtaining the original data of the fitting position on the two-dimensional depth map after intra-frame decompression. When selecting the data points on the fitting plane, the data points can be selected according to the distance between the data points on the two-dimensional depth map before intra-frame compression.

According to the processing method of the laser radar data, after the two-dimensional depth map to be decompressed is obtained, integral decompression, inter-frame decompression and intra-frame decompression can be sequentially carried out on the two-dimensional depth map to be decompressed, and finally the two-dimensional depth map before compression is obtained. Because the compression and decompression are reversible operations, the decompression process shows that the embodiment of the application can sequentially perform intra-frame compression, inter-frame compression and integral compression on the two-dimensional depth map converted from the laser radar original data, and greatly reduces the data volume of the two-dimensional depth map from the aspect of multi-angle compression.

Corresponding to the above method embodiment, the embodiment of the present application provides a processing system for laser radar data, as shown in fig. 4, where the system includes a laser radar, a terminal device and a server;

The terminal equipment is used for obtaining a two-dimensional depth map after integral compression by executing the method in the embodiment corresponding to the figure 1, and sending the two-dimensional depth map after integral compression to the server, wherein the compression process comprises intra-frame compression, inter-frame compression and integral compression;

Based on the above method embodiment, another embodiment of the present application provides a processing device for laser radar data, as shown in fig. 5, where the device includes:

a depth map acquisition unit 30 for acquiring a multi-frame two-dimensional depth map converted from laser radar raw data;

A filter acquisition unit 32 for acquiring a filter determined from a ratio of a lateral angle resolution and a longitudinal angle resolution of the laser radar;

An intra-frame compression unit 34, configured to perform intra-frame compression on each frame of two-dimensional depth map according to the filter and a preset plane fitting algorithm, so as to obtain a multi-frame intra-frame compressed two-dimensional depth map;

An inter-frame compression unit 36, configured to, for a two-dimensional depth map packet, perform inter-frame compression on the two-dimensional depth map packet according to differences between a two-dimensional depth map compressed in a second frame and a two-dimensional depth map compressed in a last frame in the two-dimensional depth map packet and a two-dimensional depth map compressed in a first frame, where each two-dimensional depth map packet includes two-dimensional depth maps compressed in a continuous multi-frame with a preset frame number;

And the overall compression unit 38 is configured to, after performing inter-frame compression on all the two-dimensional depth map packets, perform overall compression on all the two-dimensional depth maps after the inter-frame compression, and obtain an overall compressed two-dimensional depth map.

In one embodiment, the depth map obtaining unit 30 is configured to obtain a filter having a ratio of a height to a width that is a ratio of a lateral angular resolution and a longitudinal angular resolution of the laser radar;

the apparatus further comprises:

In one embodiment, the adjustment unit includes:

In one embodiment, intra-frame compression unit 34 includes:

In one embodiment, the inter-frame compression unit 36 includes:

In one embodiment, the depth map acquiring unit 30 includes:

In one embodiment, the integral compression unit 38 includes:

In one embodiment, the apparatus comprises:

In one embodiment, an inter-frame decompression unit includes:

Based on the above method embodiments, another embodiment of the present application provides a storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to implement the method as described above.

Based on the above method embodiment, another embodiment of the present application provides an electronic device, including:

One or more processors;

Storage means for storing one or more programs,

Wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the methods as described above.

The system and device embodiments correspond to the method embodiments, and have the same technical effects as the method embodiments, and specific description refers to the method embodiments. The apparatus embodiments are based on the method embodiments, and specific descriptions may be referred to in the method embodiment section, which is not repeated herein. Those of ordinary skill in the art will appreciate that the drawing is merely a schematic illustration of one embodiment and that modules or flow in the drawing are not necessarily required to practice the application.

It will be appreciated by those of ordinary skill in the art that modules in an apparatus of an embodiment may be distributed in an apparatus of an embodiment as described in the embodiments, and that corresponding changes may be located in one or more apparatuses different from the embodiment. The modules of the above embodiments may be combined into one module, or may be further split into a plurality of sub-modules.

It should be noted that the above-mentioned embodiments are merely for illustrating the technical solution of the present application, and not for limiting the same, and although the present application has been described in detail with reference to the above-mentioned embodiments, it should be understood by those skilled in the art that the technical solution described in the above-mentioned embodiments may be modified or some technical features may be equivalently replaced, and these modifications or substitutions do not make the essence of the corresponding technical solution deviate from the spirit and scope of the technical solution of the embodiment of the present application.

Claims

1. A method for processing laser radar data, characterized in that the method comprises:

Obtain a multi-frame two-dimensional depth map converted from laser radar raw data, and obtain a filter determined according to the ratio of the horizontal angle resolution and the vertical angle resolution of the laser radar;

Performing intra-frame compression on each frame of the two-dimensional depth map according to the filter and the preset plane fitting algorithm to obtain two-dimensional depth maps after intra-frame compression of multiple frames;

For a two-dimensional depth map group, inter-frame compression is performed on the two-dimensional depth map group according to the differences between the two-dimensional depth maps compressed from the second frame to the last frame in the two-dimensional depth map group and the two-dimensional depth map compressed from the first frame, including: for each frame of the two-dimensional depth map compressed from the first frame in the two-dimensional depth map group except for the two-dimensional depth map compressed from the first frame, respectively calculating the differences between a plurality of target areas in the two-dimensional depth map compressed from the current frame and the target areas corresponding to the two-dimensional depth map compressed from the first frame, and obtaining the inter-frame deviation of the two-dimensional depth map compressed from the current frame relative to the two-dimensional depth map compressed from the first frame, wherein the target area is the size of the area where the filter performs one filtering; retaining the two-dimensional depth map compressed from the first frame, and replacing the two-dimensional depth map compressed from the current frame with the position and the corresponding inter-frame deviation of the two-dimensional depth map compressed from the current frame; wherein each two-dimensional depth group includes a plurality of consecutive frames of two-dimensional depth maps compressed from the intra-frame with a preset number of frames;

After inter-frame compression is performed on all the two-dimensional depth map groups, all the inter-frame compressed two-dimensional depth maps are compressed as a whole to obtain an overall compressed two-dimensional depth map.

2. The method according to claim 1, characterized in that obtaining a filter determined according to the ratio of the lateral angular resolution to the longitudinal angular resolution of the laser radar comprises: obtaining a filter whose height-to-width ratio is the ratio of the lateral angular resolution to the longitudinal angular resolution of the laser radar;

Before intra-frame compression is performed on each frame of the two-dimensional depth map according to the filter and the preset plane fitting algorithm to obtain multiple frames of intra-frame compressed two-dimensional depth maps, the method also includes: if the compression rate and/or signal-to-noise ratio of the filter whose ratio of height to width is the ratio of the lateral angular resolution to the longitudinal angular resolution of the laser radar does not meet the preset compression requirements, the size of the filter is adjusted according to the lateral distance resolution of the laser radar at the distance of interest, the longitudinal distance resolution at the distance of interest, and the size of the target object at the distance of interest, until the compression rate and signal-to-noise ratio of the adjusted filter meet the preset compression requirements, thereby obtaining the final required filter, wherein the distance of interest is the distance corresponding to the original data of the laser radar, and the size includes height and width.

3. The method according to claim 2, characterized in that the size of the filter is adjusted according to the lateral distance resolution of the laser radar at the distance of interest, the longitudinal distance resolution at the distance of interest and the size of the target object at the distance of interest, comprising:

Calculating respectively the ratio of the height of the target object at the distance of interest to the lateral distance resolution of the laser radar at the distance of interest, and the ratio of the width of the target object to the longitudinal distance resolution;

If the ratio of the height of the target object to the lateral distance resolution is greater than the ratio of the width of the target object to the longitudinal distance resolution, the proportion of the height of the filter is increased;

If the ratio of the height of the target object to the lateral distance resolution is smaller than the ratio of the width of the target object to the longitudinal distance resolution, the size proportion of the width of the filter is increased.

4. The method according to claim 1, characterized in that, according to the filter and the preset plane fitting algorithm, each frame of the two-dimensional depth map is intra-frame compressed to obtain a plurality of frames of intra-frame compressed two-dimensional depth maps, comprising:

Performing plane fitting on a first target area on the two-dimensional depth map to be compressed within the frame filtered by the filter according to the preset plane fitting algorithm to obtain a first fitting plane;

Calculating a first fitting error according to a difference between coordinates of a plurality of data points on the first fitting plane and coordinates of a corresponding same data point on the two-dimensional depth map to be compressed within the frame;

If the first fitting error is less than a preset error threshold, retaining a first fitting record, wherein the first fitting record includes a fitting position of the first fitting plane and fitting parameters of the first fitting plane, the fitting position of the first fitting plane is a position of the first target area on the compressed two-dimensional depth map in the frame to be framed, and the fitting parameters of the first fitting plane include a normal vector and a geometric equation of the first fitting plane;

If the first fitting error is greater than or equal to the preset error threshold, the first fitting record is not retained;

After sliding the filter to the second target area, performing plane fitting on the first target area and the second target area as a whole area according to the preset plane fitting algorithm to obtain a second fitting plane;

Calculating a second fitting error according to a difference between coordinates of a plurality of data points on the second fitting plane and coordinates of a corresponding same data point on the two-dimensional depth map to be compressed within the frame before plane fitting;

If the second fitting error is less than the preset error threshold, replacing the first fitting record with a second fitting record, wherein the second fitting record includes a fitting position of the second fitting plane and fitting parameters of the second fitting plane, the fitting position of the second fitting plane includes a position of the first target area and a position of the second target area, and the fitting parameters of the second fitting plane include a normal vector and a geometric equation of the second fitting plane;

If the second fitting error is greater than or equal to the preset error threshold, plane fitting is performed again on the second target area until the last target area of the two-dimensional depth map to be compressed in the frame is subjected to plane fitting processing, so as to obtain a two-dimensional depth map after intra-frame compression of the two-dimensional depth map to be compressed in the frame, wherein the two-dimensional depth map after intra-frame compression includes the last retained fitting record and the original data of the target area without retained fitting record on the two-dimensional depth map to be compressed in the frame;

After completing the intra-frame compression of the multiple frames of two-dimensional depth maps, the multiple frames of intra-frame compressed two-dimensional depth maps are obtained.

5. The method according to claim 1, characterized in that obtaining a multi-frame two-dimensional depth map converted from the laser radar raw data comprises:

Receiving laser radar raw data sent by the laser radar;

Converting the laser radar raw data into multi-frame laser radar point cloud data;

According to the conversion order of the laser radar raw data, each frame of laser radar point cloud data is added to the point cloud first-in-first-out FIFO queue in turn;

According to the first-in-first-out principle of the point cloud FIFO queue, each frame of laser radar point cloud data is obtained from the point cloud FIFO queue in sequence, and each frame of laser radar point cloud data is converted into a two-dimensional depth map;

According to the conversion order of the LiDAR point cloud data, the converted two-dimensional depth maps are added to the depth map FIFO queue in sequence;

According to the first-in-first-out principle of the depth map FIFO queue, each frame of the laser radar two-dimensional depth map in the depth map FIFO queue is obtained in sequence.

6. The method according to claim 5, characterized in that after performing inter-frame compression on all two-dimensional depth map groups, all two-dimensional depth maps after inter-frame compression are compressed as a whole to obtain the two-dimensional depth map after overall compression, comprising:

According to the inter-frame compression order of the two-dimensional depth map grouping, each inter-frame compressed two-dimensional depth grouping is added to the data transmission FIFO queue in turn;

After all the inter-frame compressed two-dimensional depth packets are added to the data transmission FIFO queue, the data transmission FIFO queue is compressed as a whole to obtain a two-dimensional depth map after overall compression.

7. The method according to any one of claims 1 to 6, characterized in that after overall compression of all two-dimensional depth maps after inter-frame compression to obtain the overall compressed two-dimensional depth map, the method comprises:

When the overall compressed two-dimensional depth map needs to be decompressed, determining the overall compressed two-dimensional depth map as the two-dimensional depth map to be decompressed;

Decompressing the two-dimensional depth map to be decompressed as a whole to obtain a plurality of two-dimensional depth map groups to be decompressed between frames, wherein the two-dimensional depth map groups to be decompressed between frames are two-dimensional depth map groups after inter-frame compression of two-dimensional depth maps after intra-frame compression of a plurality of consecutive frames having a preset number of frames;

According to the differences between the two-dimensional depth map after intra-frame compression from the second frame to the last frame in the two-dimensional depth map group to be decompressed between frames and the two-dimensional depth map after intra-frame compression of the first frame, the two-dimensional depth map group to be decompressed between frames is inter-frame decompressed to obtain multiple frames of two-dimensional depth maps to be decompressed within the frame, wherein the two-dimensional depth map to be decompressed within the frame includes at least one fitting position, fitting parameters corresponding to the fitting position, and original data of the two-dimensional depth map to be decompressed within the frame at an unfitted position before intra-frame compression, and the fitting parameters include a normal vector of a fitting plane and a geometric equation of the fitting plane;

For each frame of the two-dimensional depth map to be decompressed within the frame, the fitting position of the two-dimensional depth map to be decompressed within the frame is decompressed within the frame according to the fitting position and the fitting parameters corresponding to the fitting position, and the original data of the fitting position on the two-dimensional depth map after intra-frame decompression is obtained, so that when all the original data on the two-dimensional depth map after intra-frame decompression of the two-dimensional depth map to be decompressed within the frame are obtained, the two-dimensional depth map after intra-frame decompression of the two-dimensional depth map to be decompressed within the frame is obtained.

8. The method according to claim 7 is characterized in that when the two-dimensional depth map group to be decompressed between frames includes the first frame intra-compressed two-dimensional depth map in the two-dimensional depth map group to be decompressed between frames, the positions of other intra-compressed two-dimensional depth maps except the first frame intra-compressed two-dimensional depth map, and the inter-frame deviations of the positions of the other intra-compressed two-dimensional depth maps relative to the first frame intra-compressed two-dimensional depth map, the two-dimensional depth map group to be decompressed between frames is decompressed between frames according to the differences between the two-dimensional depth maps compressed between the second frame to the last frame intra-compressed two-dimensional depth map in the two-dimensional depth map group to be decompressed between frames and the first frame intra-compressed two-dimensional depth map, to obtain multiple frames of two-dimensional depth maps to be decompressed between frames, comprising:

For each intra-frame compressed two-dimensional depth map in the two-dimensional depth map group to be inter-frame decompressed except the first intra-frame compressed two-dimensional depth map, determine the current intra-frame compressed two-dimensional depth map according to the position of the current intra-frame compressed two-dimensional depth map, the inter-frame deviation of the current intra-frame compressed two-dimensional depth map relative to the first intra-frame compressed two-dimensional depth map, and the first intra-frame compressed two-dimensional depth map;

After determining the intra-frame compressed two-dimensional depth map of each frame in the two-dimensional depth map group to be decompressed between frames, the intra-frame compressed two-dimensional depth map is determined as the two-dimensional depth map to be decompressed within the frame, so as to obtain multiple frames of two-dimensional depth maps to be decompressed within the frame.

9. A laser radar data processing system, characterized in that the system includes a laser radar, a terminal device and a server;

The laser radar is used to obtain raw laser radar data and send the raw laser radar data to the terminal device;

The terminal device is used to obtain a two-dimensional depth map after overall compression by executing the method according to any one of claims 1 to 6, and send the two-dimensional depth map after overall compression to the server;

The server is used to receive the overall compressed two-dimensional depth map sent by the terminal device, determine the overall compressed two-dimensional depth map as the two-dimensional depth map to be decompressed, decompress the two-dimensional depth map to be decompressed as a whole, and obtain multiple two-dimensional depth map groups to be decompressed between frames, wherein the two-dimensional depth map group to be decompressed between frames is a two-dimensional depth map group after inter-frame compression for a plurality of consecutive intra-frame compressed two-dimensional depth maps having a preset number of frames, and according to the differences between the two-dimensional depth maps compressed between the second frame to the last frame in the two-dimensional depth map group to be decompressed between frames and the two-dimensional depth map compressed between the first frame, decompress the two-dimensional depth map group to be decompressed between frames to obtain a plurality of two-dimensional depth maps to be decompressed within the frame , wherein the two-dimensional depth map to be decompressed within the frame includes at least one fitting position, a fitting parameter corresponding to the fitting position, and original data of the two-dimensional depth map to be decompressed within the frame at an unfitted position before intra-frame compression, the fitting parameter includes a normal vector of a fitting plane and a geometric equation of the fitting plane, and for each frame of the two-dimensional depth map to be decompressed within the frame, the fitting position of the two-dimensional depth map to be decompressed within the frame is decompressed within the frame according to the fitting position and the fitting parameter corresponding to the fitting position, and the original data of the fitting position on the two-dimensional depth map after intra-frame decompression is obtained, so that when all the original data of the two-dimensional depth map to be decompressed within the frame on the two-dimensional depth map after intra-frame decompression are obtained, the two-dimensional depth map after intra-frame decompression of the two-dimensional depth map to be decompressed within the frame is obtained.