CN119941550B - Dynamic obstacle point cloud filtering method and device - Google Patents

Abstract

The present application discloses a method for filtering out dynamic obstacle point clouds, comprising: obtaining a frame image and inertial measurement data containing a target area; obtaining second point cloud data and the pose information of the frame image using an iterative error Kalman filter algorithm; obtaining third point cloud data in a global coordinate system using a registration algorithm; dividing the spatial range covered by the third point cloud data into grid cells, and counting the number of point clouds and the point cloud height variance in each grid cell; determining a dynamic grid if the number of point clouds exceeds a first threshold and the absolute value of the variance difference between the number of point clouds and the number of historical frame point clouds exceeds a second threshold; re-marking the dynamic grid as a static grid if the dynamic confidence is lower than a set threshold; obtaining a point cloud of dynamic obstacles using a spatial region growing algorithm, and filtering out the point cloud of dynamic obstacles to obtain static point cloud data of the target area. The present application achieves effective filtering of dynamic obstacle point clouds.

Description

Dynamic obstacle point cloud filtering method and device

Technical Field

The present application relates to the field of computer vision technology, and in particular to a method and device for extracting dynamic obstacle point clouds.

Background Art

In autonomous driving and robotic navigation, LiDAR (LiDAR) is often used to acquire real-time 3D point cloud data of the surrounding environment. This point cloud data contains both static obstacles (such as buildings, roads, and fixed facilities) and dynamic obstacles (such as pedestrians, vehicles, and animals). In dynamic environments, efficiently and accurately removing dynamic obstacles becomes a critical issue in point cloud processing.

Existing methods typically rely on the separation of static environment modeling and dynamic obstacle detection, but still have difficulties in real-time processing and application in complex environments. Therefore, how to accurately and in real time remove dynamic obstacles from point cloud data is an important technical problem that needs to be solved.

Summary of the Invention

The present application provides a method and device for filtering dynamic obstacle point clouds, which can effectively filter dynamic obstacle point clouds.

This application provides the following solutions:

According to a first aspect, a method for filtering out a dynamic obstacle point cloud is provided, the method comprising: acquiring one or more frame images containing a target area and inertial measurement data corresponding to each of the frame images; generating first point cloud data for each of the frame images; utilizing an iterative error Kalman filter algorithm to fuse the first point cloud data with the inertial measurement data to obtain second point cloud data and the pose information of the frame image; utilizing a registration algorithm to determine a pose transformation matrix of the frame image relative to a reference frame based on the pose information; utilizing the pose transformation matrix to transform the second point cloud data from a local coordinate system to a global coordinate system for representation, and obtaining third point cloud data in the global coordinate system; in the global coordinate system, dividing the spatial range covered by the third point cloud data into one or more grid cells, and counting the number of point clouds and the point cloud height variance in each of the grid cells; utilizing a plane fitting algorithm to determine whether the grid cell is If it is not a ground grid unit, the third point cloud data judged as a ground grid unit is filtered out; if the number of point clouds in the current frame exceeds a first threshold and the absolute value of the variance difference between the number of point clouds in the current frame and the number of point clouds in the historical frame exceeds a second threshold, the grid unit is marked as a dynamic grid; historical frame information of the dynamic grid is obtained, and the historical frame information is the number of point clouds and the point cloud height variance of one or more frame images before the current frame image corresponding to the current dynamic grid in the dynamic grid; the historical frame information is processed by a time series analysis algorithm to calculate the dynamic confidence of the dynamic grid; if the dynamic confidence is lower than the set threshold, the mark of the dynamic grid is modified to a static grid; for the grid unit marked as a dynamic grid, a point cloud of a dynamic obstacle is obtained by a spatial region growing algorithm, and the point cloud of the dynamic obstacle is filtered out from the third point cloud data to obtain static point cloud data of the target area.

According to an implementable method in an embodiment of the present application, the use of a registration algorithm to determine the pose transformation matrix of the frame image relative to the reference frame based on the pose information includes: obtaining point cloud data of the reference frame; using a point-to-point iterative nearest point algorithm to generate an initial pose transformation matrix of the second point cloud data and the reference frame point cloud data; and using a gradient descent algorithm to optimize the initial pose transformation matrix to obtain the pose transformation matrix.

According to an achievable method in an embodiment of the present application, dividing the spatial range covered by the third point cloud data into more than one grid units in the global coordinate system includes: generating a three-dimensional bounding box for the third point cloud data in the global coordinate system, non-uniformly dividing the bounding box along three mutually perpendicular coordinate axis directions in the three-dimensional coordinate system, and adaptively adjusting the grid resolution according to the distribution characteristics of the third point cloud data; assigning a multidimensional index to each of the grid units, and assigning the third point cloud data to the corresponding grid unit.

According to an implementable method in an embodiment of the present application, the use of time series analysis to process the historical frame information and calculate the dynamic confidence of the dynamic grid includes: constructing a point cloud quantity time series and a point cloud height variance time series based on the point cloud quantity and the point cloud height variance in the dynamic grid corresponding to the historical frame information; calculating the rate of change, stability and change trend of the point cloud quantity and height variance in the dynamic grid based on the point cloud quantity time series and the point cloud height variance time series; defining a dynamic confidence formula based on the change rate, stability and change trend, and calculating the dynamic confidence based on the dynamic confidence formula.

According to an implementable method in an embodiment of the present application, the plane fitting algorithm is used to determine whether the grid unit is a ground grid unit based on the point cloud height variance, and the third point cloud data determined to be a ground grid unit is filtered out, including: based on the point cloud data in each of the grid cells, a random sampling consistency algorithm is used to perform plane model fitting to obtain a fitting plane; the vertical distance between each point in the third point cloud data and the fitting plane is calculated, and whether it is a ground point is determined based on a distance threshold, and the points that meet the distance threshold condition are removed from the point cloud data; wherein, the distance threshold is dynamically adjusted based on the point cloud height variance.

According to an achievable method in an embodiment of the present application, for the grid unit marked as a dynamic grid, obtaining a point cloud of a dynamic obstacle through a spatial region growing algorithm, and filtering out the point cloud of the dynamic obstacle from the third point cloud data includes: selecting an initial seed point cloud from the dynamic grid according to preset indicators, performing regional expansion from the initial seed point cloud according to a distance metric and a point cloud density function, incorporating adjacent point clouds into the expanded area to form a continuous point cloud area; and using a clustering algorithm to divide the continuous point cloud area into multiple independent areas, and filtering out the point cloud of the dynamic obstacle from the independent areas.

According to an achievable method in an embodiment of the present application, defining a dynamic confidence formula based on the change rate, stability and change trend includes: dividing a plurality of different time intervals; obtaining the timestamp information of the historical frame information, and dividing the historical frame information into corresponding time intervals based on the timestamp information; setting different weights for the change rate, stability and change trend in different time intervals based on the importance differences of the number of point clouds and height variance in different time intervals, and determining a dynamic confidence formula based on the weights.

According to a second aspect, a dynamic obstacle point cloud filtering device is provided, the device comprising: a data acquisition unit, configured to acquire one or more frame images containing a target area and inertial measurement data corresponding to each of the frame images; a data fusion unit, configured to generate first point cloud data for each of the frame images; using an iterative error Kalman filter algorithm, the first point cloud data is fused with the inertial measurement data to obtain second point cloud data and the pose information of the frame image; a coordinate conversion unit, configured to use a registration algorithm to determine the pose transformation matrix of the frame image relative to a reference frame according to the pose information; using the pose transformation matrix, the second point cloud data is converted from a local coordinate system to a global coordinate system for representation, and third point cloud data in the global coordinate system is obtained; a grid data statistics unit, configured to divide the spatial range covered by the third point cloud data into one or more grid cells in the global coordinate system, and count the number of point clouds and the point cloud height variance in each of the grid cells; a ground point cloud filtering unit, configured to use the point cloud height variance to obtain the third point cloud data. A plane fitting algorithm is used to determine whether the grid cell is a ground grid cell, and the third point cloud data in the grid cell determined to be a ground grid cell is filtered out. A dynamic grid determination unit is configured to mark the grid cell as a dynamic grid if the number of point clouds in the current frame exceeds a first threshold and the absolute value of the variance difference between the number of point clouds in the current frame and the number of point clouds in the historical frame exceeds a second threshold. Historical frame information of the dynamic grid is obtained, where the historical frame information is the number of point clouds and the point cloud height variance in the dynamic grid in one or more frame images preceding the frame image corresponding to the current dynamic grid. The historical frame information is processed using a time series analysis algorithm to calculate the dynamic confidence of the dynamic grid. If the dynamic confidence is lower than a set threshold, the label of the dynamic grid is changed to a static grid. A dynamic point cloud filtering unit is configured to obtain a point cloud of a dynamic obstacle using a spatial region growing algorithm for the grid cell marked as a dynamic grid, and filter the point cloud of the dynamic obstacle from the third point cloud data to obtain static point cloud data of the target area.

According to a third aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the steps of any one of the methods according to the first aspect are implemented.

According to a fourth aspect, an electronic device is provided, comprising: one or more processors; and a memory associated with the one or more processors, the memory being used to store program instructions, which, when read and executed by the one or more processors, execute the steps of the method described in any one of the above-mentioned first aspects.

According to the specific embodiments provided in this application, this application discloses the following technical effects:

(1) This method generates the first point cloud data by acquiring multiple frame images containing the target area and their corresponding inertial measurement data, and then fuses the inertial measurement data with the iterative error Kalman filter algorithm to obtain the second point cloud data and the pose information of the frame image. The registration algorithm is used to further determine the pose transformation matrix and convert the data into a global coordinate system. Through grid processing and statistical analysis, dynamic grids are identified, and the dynamic confidence is calculated in combination with time series analysis, so as to accurately identify and filter out the point cloud of dynamic obstacles. The advantage of this method is that it improves the detection accuracy of dynamic obstacles and reduces interference in environmental modeling.

(2) This application uses a point-to-point iterative closest point algorithm and a gradient descent optimization algorithm to calculate and optimize the pose transformation matrix, further enhancing the spatial consistency and accuracy of point cloud data, significantly reducing errors in the data registration process, and improving the stability and accuracy of subsequent processing.

(3) This application utilizes three-dimensional bounding boxes and non-uniform resolution adjustment technology when performing grid division in a global coordinate system, which can more accurately reflect the distribution characteristics of point cloud data. This method enables more detailed analysis of each grid cell, improves the detection and resolution of dynamic grids, and further enhances the differentiation between static and dynamic obstacles.

(4) This application calculates the dynamic confidence of the dynamic grid through time series analysis. This method can effectively evaluate the dynamic behavior of each grid cell and reduce the false positive rate. The introduction of dynamic confidence makes the judgment of the dynamic grid more scientific and avoids the problem of false positives or missed detections caused by environmental changes.

(5) The plane fitting method based on the random sampling consistency algorithm in this application can accurately identify the ground grid and remove ground points. This method ensures the accurate removal of ground points by dynamically adjusting the distance threshold to adapt to different environmental conditions, thereby improving the filtering efficiency and point cloud data quality.

(6) This application uses a spatial region growing algorithm to filter out point clouds from dynamic grids. This method can effectively separate dynamic obstacles from complex point clouds. A clustering algorithm is used to subdivide the point cloud regions, ensuring that dynamic obstacles can be accurately detected and filtered out. This technology can significantly improve the real-time performance and reliability of environmental perception systems.

(7) This application improves the sensitivity to dynamic changes in dynamic grids by defining a dynamic confidence formula and adjusting the weight of time intervals. By combining analysis with the timestamp information of historical data, the dynamic confidence is more closely aligned with actual environmental changes, facilitating more accurate identification and processing of dynamic grids and improving the overall performance of the system.

Of course, any product implementing the present application does not necessarily need to achieve all of the advantages described above at the same time.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

FIG1 is a diagram of a system architecture applicable to an embodiment of the present application;

FIG2 is a flow chart of a method for filtering a dynamic obstacle point cloud according to an embodiment of the present application;

FIG3 is a schematic diagram of the process of a dynamic obstacle point cloud filtering method provided by an embodiment of the present application;

FIG4 is a structural block diagram of a dynamic obstacle point cloud filtering device provided in an embodiment of the present application;

FIG5 is a schematic block diagram of an electronic device provided in an embodiment of the present application.

DETAILED DESCRIPTION

The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The singular forms "a", "an", "the" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise.

It should be understood that the term "and/or" as used herein is merely a description of the relationship between associated objects, indicating that three possible relationships exist. For example, "A and/or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character "/" in this document generally indicates that the associated objects are in an "or" relationship.

The word "if," as used herein, may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to the determination" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)," depending on the context.

Currently, there are some technical methods for filtering dynamic obstacle point clouds, most of which rely on motion detection, model prediction, and other technologies. However, these methods require additional computing resources and complex algorithm design, and the accuracy of point cloud filtering in complex dynamic scenes is low.

In view of this, the present application provides a new approach. To facilitate understanding of the present application, the system architecture on which the present application is based is first described. Figure 1 shows an exemplary system architecture to which embodiments of the present application can be applied. As shown in Figure 1, the system architecture may include: a user device and a dynamic obstacle point cloud filtering device located on the server side.

The user can input frame images and inertial measurement data through the user device, and send them to the dynamic obstacle point cloud filtering device on the server side through the user device. The dynamic obstacle point cloud filtering device can use the method provided in the embodiments of the present application to perform dynamic obstacle point cloud filtering on the point cloud data in the frame image to obtain static point cloud data of the target area. In response to the request of the user terminal, the server side can send the static point cloud data of the target area to the user terminal, and the user terminal can render it using the three-dimensional real scene model to obtain a two-dimensional image, a three-dimensional image, a VR (virtual reality) scene, or an AR (augmented reality) scene, etc.

User devices may include, but are not limited to, smart mobile terminals, smart home devices, wearable devices, and personal computers (PCs). Smart mobile devices may include mobile phones, tablets, laptops, PDAs (Personal Digital Assistants), and internet-connected cars. Smart home devices may include smart TVs and smart refrigerators. Wearable devices may include smart watches, smart glasses, virtual reality devices, augmented reality devices, and mixed reality devices (i.e., devices that support both virtual reality and augmented reality).

The dynamic obstacle point cloud filtering device can be deployed as a standalone server, in a server cluster, or even on a cloud server. A cloud server, also known as a cloud computing server or cloud host, is a host product within the cloud computing service ecosystem. It addresses the management difficulties and scalability limitations of traditional physical hosts and virtual private servers (VPS). In addition to the architecture shown in Figure 1, the dynamic obstacle point cloud filtering device can also be deployed on a computer terminal with high computing power.

It should be understood that the user equipment and the dynamic obstacle point cloud filtering device in FIG1 are merely illustrative and any number of user equipment and dynamic obstacle point cloud filtering devices may be provided according to implementation requirements.

FIG2 is a flow chart of a method for filtering a dynamic obstacle point cloud provided by an embodiment of the present application. The method can be executed by the dynamic obstacle point cloud filtering device in the system shown in FIG1. As shown in FIG2, the method can include the following steps:

Step 201: Acquire one or more frame images containing a target area and inertial measurement data corresponding to each frame image.

Step 202: For each of the frame images, generate first point cloud data; use an iterative error Kalman filter algorithm to fuse the first point cloud data with the inertial measurement data to obtain second point cloud data and the pose information of the frame image.

Step 203: Using a registration algorithm, determine the pose transformation matrix of the frame image relative to the reference frame based on the pose information; using the pose transformation matrix, transform the second point cloud data from the local coordinate system to the global coordinate system for representation, and obtain the third point cloud data in the global coordinate system.

Step 204: In the global coordinate system, the spatial range covered by the third point cloud data is divided into one or more grid units, and the number of point clouds and the point cloud height variance in each grid unit are counted.

Step 205: Based on the point cloud height variance, a plane fitting algorithm is used to determine whether the grid unit is a ground grid unit, and the third point cloud data determined to be a ground grid unit is filtered out.

Step 206: If the number of point clouds in the current frame exceeds a first threshold and the absolute value of the variance difference between the number of point clouds in the current frame and the number of point clouds in the historical frame exceeds a second threshold, the grid unit is marked as a dynamic grid; historical frame information of the dynamic grid is obtained, where the historical frame information is the number of point clouds and the point cloud height variance in the dynamic grid of one or more frame images before the frame image corresponding to the current dynamic grid; the historical frame information is processed using a time series analysis algorithm to calculate the dynamic confidence of the dynamic grid; if the dynamic confidence is lower than a set threshold, the mark of the dynamic grid is changed to a static grid.

Step 207: For the grid cells marked as dynamic grids, a point cloud of a dynamic obstacle is obtained by using a spatial region growing algorithm, and the point cloud of the dynamic obstacle is filtered out from the third point cloud data to obtain static point cloud data of the target area.

As can be seen from the above process, this method generates first point cloud data by acquiring multiple frame images containing the target area and their corresponding inertial measurement data. This data is then fused with the inertial measurement data through an iterative error Kalman filter algorithm to obtain the second point cloud data and the pose information of the frame images. A registration algorithm is used to further determine the pose transformation matrix and convert the data to a global coordinate system. Dynamic grids are identified through gridding and statistical analysis, and dynamic confidence is calculated using time series analysis to accurately identify and filter out point clouds of dynamic obstacles. The advantage of this method is that it improves the detection accuracy of dynamic obstacles and reduces interference in environmental modeling. By fusing multiple frames of images and inertial measurement data, combined with time series analysis and a spatial region growing algorithm, the accuracy of dynamic obstacle detection is improved.

The following describes in detail each step of the above process and the effects that can be produced, in conjunction with the embodiments. It should be noted that the terms "first" and "second" in this disclosure do not restrict the size, order, or quantity, but are merely used to distinguish between the two sets of point cloud data. For example, "first point cloud data" and "second point cloud data" are used to distinguish between the two sets of point cloud data.

First, the above step 201 , namely “obtaining one or more frame images including a target area and inertial measurement data corresponding to each frame image”, is described in detail with reference to an embodiment.

In this application, the target area is the area where dynamic point cloud filtering needs to be performed, and the frame image contains the target area. The frame image is a continuous image frame, which can represent the perspective information in a specific area or scene. Preferably, a laser radar can be used to obtain the frame image. The laser radar device completes a scan within a fixed time interval to generate a frame of data. Different laser radar devices have different scanning speeds, and the scanning frequency (the number of frames scanned per second) can range from tens of frames to hundreds of frames.

Inertial measurement data comes from an inertial measurement unit (IMU), an electronic device used to measure and report the motion state of an object in three-dimensional space. It usually includes an accelerometer, a gyroscope, and sometimes a magnetometer. Inertial measurement data contains the acceleration, angular velocity, and other motion parameters of the device at a specific point in time. By obtaining this data, the frame image can be matched with its corresponding motion information, thereby achieving higher-precision point cloud data processing. After obtaining the inertial measurement data, the inertial measurement data can be preprocessed, such as denoising, low-pass filtering, and time synchronization, to improve the accuracy of the inertial measurement data.

The above-mentioned step 202, namely "generating first point cloud data for each of the frame images; fusing the first point cloud data with the inertial measurement data using an iterative error Kalman filter algorithm to obtain second point cloud data and the pose information of the frame image" is described in detail below in conjunction with an embodiment.

In this application, the first point cloud data is raw point cloud data collected by a LiDAR device, which is generated by the LiDAR device by scanning a specific environment area. Each frame image represents the spatial data acquired by the LiDAR device at a specific point in time, including the distance and direction of the laser beam reflected from the target area.

The Iterative Error Kalman Filter (IERKF) algorithm is a recursive filtering method for fusing multi-source data, particularly well-suited for handling noise and errors in dynamic systems. It reduces estimation errors by predicting and updating the system state, gradually approaching the true value over multiple iterations. In this application, it is used to fuse the first point cloud data generated by a LiDAR with the inertial measurement data from an IMU to improve the spatial accuracy of the point cloud data and the estimated accuracy of the device's position and pose.

Figure 3 is a schematic diagram of the process of the dynamic obstacle point cloud filtering method provided by an embodiment of the present application. Using an iterative error Kalman filter algorithm, the first point cloud data is fused with the IMU data to generate an improved second point cloud data set. This data is more spatially accurate and consistent with the device's actual position and attitude. Simultaneously, the state information output by the filtering algorithm provides pose information for the frame image, including the device's position and attitude. Specifically, the IMU's acceleration and angular velocity data can be first used to predict the device's position and attitude at the current time point. The predicted pose is then compared with the first point cloud data. Error calculation and filter update steps are used to correct the pose estimate, ensuring that the output pose information is more consistent with the actual situation. Multiple iterations are then performed to optimize the filter state, improving the accuracy of the pose estimate and point cloud data. Finally, the second point cloud data output by the filter contains the corrected point cloud information, reflecting the fused environment space. The pose information generated simultaneously includes the device's position and attitude during the frame image acquisition process, such as position coordinates and orientation angle.

The following is a detailed description of step 203, namely, "using a registration algorithm to determine the pose transformation matrix of the frame image relative to the reference frame based on the pose information; using the pose transformation matrix to transform the second point cloud data from the local coordinate system to the global coordinate system for representation, and obtaining the third point cloud data in the global coordinate system" in conjunction with an embodiment.

The registration algorithm is a method for spatially aligning multi-frame point cloud data. Based on the pose information of the frame image, the registration algorithm can calculate the pose transformation matrix between the current frame image and the reference frame. The reference frame refers to a specific frame point cloud or coordinate system used as a benchmark in the process of processing a series of point cloud data. The data of other frames are aligned with the reference frame through pose estimation to ensure the uniformity of the point cloud data in space. The reference frame can be a selected static frame or point cloud data at a certain moment in a dynamic environment, usually represented in the form of a global coordinate system. The present application can be implemented by a variety of registration algorithms, such as the Iterative Closest Point (ICP) algorithm and the Normalized Distribution Transform (NDT) algorithm, which can calculate the relative displacement and rotation relationship between the two frames of data by matching similar feature points in the point cloud data. Specifically, feature points (such as edge points and plane points) can be extracted from the second point cloud data and the reference frame point cloud for matching calculations. Using the ICP algorithm, feature points are matched through nearest neighbor search to calculate the initial transformation matrix. Further, through error optimization (such as the least squares method), a high-precision pose transformation matrix is obtained. Finally, the pose transformation matrix describing the frame image relative to the reference frame is obtained.

A pose transformation matrix describes the motion of a rigid body. It contains a rotation matrix and a translation vector, representing the change in position and orientation of a frame of data in space relative to a reference frame. The pose transformation matrix is typically a 4×4 homogeneous matrix, where the rotation defines the change in pose and the translation defines the change in position.

As an implementable approach, the present application can employ a point-to-point iterative closest point algorithm to obtain a pose transformation matrix. Specifically, the point cloud data of the reference frame is obtained; an initial pose transformation matrix of the second point cloud data and the reference frame point cloud data is calculated using a point-to-point iterative closest point algorithm; and the initial pose transformation matrix is optimized using a gradient descent algorithm to obtain the pose transformation matrix.

First, the iterative closest point (ICP) algorithm is used to match the closest point pairs in the two point clouds. An error function is constructed to describe the geometric deviation between the two point clouds. The error function is then iteratively optimized to obtain an initial pose transformation matrix. This initial pose transformation matrix, consisting of a translation matrix and a planar rotation matrix, is used to initially align the two point clouds. Next, a gradient descent algorithm is used to optimize this initial pose transformation matrix. Specifically, an optimization objective function is constructed based on the output of the initial pose transformation matrix. The objective function is typically the sum of the squared point-to-point distances between the two point clouds. The gradient descent algorithm continuously adjusts the parameters of the pose transformation matrix (including rotation angles and translations) and iteratively optimizes the objective function until a preset error threshold or a limit on the number of iterations is reached. Ultimately, the optimized pose transformation matrix is obtained. The optimized pose transformation matrix accurately describes the pose relationship of the current frame's point cloud data relative to the reference frame's point cloud data and serves as the core parameter for subsequent point cloud data transformation to the global coordinate system.

The local coordinate system is a reference coordinate system with the LiDAR sensor as its origin, while the global coordinate system is a unified coordinate system that describes the entire environment. Through the pose transformation matrix, the point cloud data in the local coordinate system can be converted to the global coordinate system for representation, ensuring the consistency of the spatial position between each frame of data. Specifically, each point of the second point cloud data is transformed using the pose transformation matrix. The formula is:

P _global = T·P _local (1)

Among them, P _local is the point cloud coordinate in the local coordinate system, T is the pose transformation matrix, and P _global is the point cloud coordinate in the global coordinate system.

The transformed point cloud data is uniformly represented in the global coordinate system, and the data from all frames can be seamlessly fused to form a third point cloud data set that meets the requirements of environmental modeling in terms of spatial consistency and accuracy.

The above step 204, namely "dividing the spatial range covered by the third point cloud data into one or more grid units in the global coordinate system, and counting the number of point clouds and the point cloud height variance in each of the grid units" is described in detail below in conjunction with an embodiment.

During dynamic obstacle detection, accurate analysis of point cloud data in a global coordinate system requires segmenting the spatial range of the point cloud data. Specifically, the point cloud data in a global coordinate system covers a specific three-dimensional spatial range, which typically encompasses the entire point cloud in the target area. By dividing the covered spatial range into multiple grid cells, the point cloud data can be locally grouped for more efficient analysis of the distribution characteristics of the point cloud in different spatial regions. Each grid cell is a three-dimensional cube region that contains the point cloud data within that region.

The grid cells can be divided in a variety of ways, for example, uniformly divided along three mutually perpendicular coordinate axes in a three-dimensional coordinate system according to a fixed resolution. Preferably, the present application can perform non-uniform division, generate a three-dimensional bounding box for the third point cloud data in the global coordinate system, divide the bounding box non-uniformly along three mutually perpendicular coordinate axes in the three-dimensional coordinate system, and adaptively adjust the grid resolution according to the distribution characteristics of the third point cloud data; assign a multi-dimensional index to each of the grid cells, and assign the third point cloud data to the corresponding grid cell.

Specifically, a 3D bounding box is generated for the third point cloud data in the global coordinate system. This allows the entire point cloud's spatial range to be enclosed in a minimal 3D rectangular box to ensure that all point cloud data is contained within the bounding box. The bounding box's boundaries are determined by the maximum and minimum coordinate values of the point cloud data, extending along the X, Y, and Z axes of the 3D coordinate system, respectively, thereby defining the spatial distribution range of the point cloud in the global coordinate system.

Based on the 3D bounding box, the bounding box can be divided into multiple grid cells by performing non-uniform partitioning along the three mutually perpendicular coordinate axes (X, Y, and Z). This non-uniform partitioning process is based on the distribution characteristics of the point cloud data. For example, in areas with high point cloud density, smaller grid cells are used to improve resolution; in areas with sparse point clouds, larger grid cells can be used to reduce computational complexity. By analyzing the density and distribution characteristics of the point cloud, the grid resolution can be adaptively adjusted, making the partitioning more targeted and efficient.

Each generated grid cell is assigned a unique multi-dimensional index, which can represent the position of the grid cell in the three-dimensional coordinate system. For example, a triplet (i, j, k) is used to represent a grid cell, where i, j, and k are the serial numbers of the grid cell in the X, Y, and Z axis directions, respectively. Then, each point in the third point cloud data is assigned to the corresponding grid cell according to its coordinate value. Specifically, by judging the coordinates of the point, the specific grid cell in which it falls within the bounding box is determined, thereby mapping the point cloud data to a three-dimensional grid structure.

After the division is complete, the point cloud within each grid cell is statistically analyzed. Statistical metrics include the number of points within each grid cell and the point cloud height variance. The point cloud number represents the number of points contained in that grid cell, reflecting the local density of the point cloud. The point cloud height variance indicates the degree of dispersion of the point cloud within the grid in the height direction, and is used to determine the uniformity and hierarchy of the point cloud distribution. This statistical information lays the foundation for subsequent dynamic obstacle extraction.

Among them, the point cloud height variance The calculation formula is:

Where h(p _j ) is the height of point p _j , N( _gi ) is the number of point clouds, is the mean height of all points in grid _gi , The calculation formula is:

The above step 205, i.e., "determining whether the grid unit is a ground grid unit using a plane fitting algorithm based on the point cloud height variance, and filtering out the third point cloud data determined to be a ground grid unit" is described in detail below in conjunction with an embodiment.

First, within the global coordinate system space covered by the third point cloud data, potential ground grid cells are initially screened based on the point cloud height variance. This variance reflects the uniformity of the point cloud's vertical distribution (Z-axis) within the grid cell. A low height variance indicates that the point cloud exhibits minimal variation in height and is flat, close to the ground. A height variance threshold is set, and grid cells with a height variance below this threshold are marked as candidate ground grid cells.

Then, a plane fitting algorithm is applied to the candidate ground grid to further determine whether it is a ground grid cell. Specifically, the plane fitting algorithm fits the spatial distribution of the point cloud data in the grid cell, builds a plane model, and calculates the distance from the point cloud to the plane. The fitted plane can be solved using the least squares method, and its formula is:

ax+by+cz+d＝0 (4)

Where a, b, and c are the plane normal parameters, and d is the offset. The least squares method is used to fit the point cloud data in the grid cell, and the perpendicular distance from each point to the fitted plane is calculated. If the mean or maximum distance from all point clouds to the fitted plane is below a preset threshold, the grid cell is considered to conform to the ground characteristics.

Point cloud data identified as ground grid cells is further filtered out to prevent them from interfering with subsequent processing (such as obstacle identification or environment modeling). Specifically, these ground point clouds are removed from the third point cloud dataset, retaining the non-ground point clouds for higher-precision spatial analysis and processing.

As an implementable method, the present application can use a random sampling consistency algorithm to fit a plane model based on the point cloud data in each of the grid cells to obtain a fitting plane; calculate the vertical distance between each point in the third point cloud data and the fitting plane, determine whether it is a ground point based on a distance threshold, and remove points that meet the distance threshold condition from the point cloud data; wherein, the distance threshold is dynamically adjusted based on the point cloud height variance.

Specifically, the point cloud data in each grid cell is first evaluated based on its vertical distribution characteristics (Z axis) based on its height variance. When the height variance is below a preset standard, grid cells with potential ground features are initially selected as candidates for subsequent processing.

A random sampling consensus algorithm is applied to fit a plane model to the point cloud data within the candidate grid cells. This algorithm iteratively constructs possible plane models by randomly selecting subsets of the point cloud data as fitting samples. The algorithm then calculates the number of inliers within the model—the number of point clouds that satisfy the plane model within a certain tolerance. By maximizing the number of inliers, the optimal plane model is determined to describe the overall planar characteristics of the grid cell.

After obtaining the fitted plane, calculate the perpendicular distance between each point and the plane. The perpendicular distance is defined as the shortest distance from the point to the plane, and its formula is:

Among them, a, b, c are the normal vector parameters of the fitted plane, x, y, z are the point cloud coordinates, and d is the offset of the plane.

The distance threshold is dynamically adjusted based on the point cloud height variance to identify ground points. Specifically, when the height variance is small, indicating that the point cloud has a small range of height variation, the distance threshold can be set to a lower value to improve the accuracy of ground point identification. When the height variance is large, indicating that the point cloud has a more dispersed height distribution, the distance threshold can be appropriately relaxed to include more points that may belong to the ground.

Finally, point clouds that meet the distance threshold condition, that is, point clouds with a vertical distance less than the dynamically adjusted threshold, are judged to be ground points and removed from the third point cloud data. The non-ground point cloud data is retained for further analysis, such as dynamic obstacle detection or environment modeling.

The following describes in detail step 206 in conjunction with an embodiment, namely, "if the number of point clouds in the current frame exceeds a first threshold and the absolute value of the variance difference between the number of point clouds in the current frame and the number of point clouds in the historical frame exceeds a second threshold, marking the grid unit as a dynamic grid; obtaining historical frame information of the dynamic grid, the historical frame information being the number of point clouds and the point cloud height variance in the dynamic grid of one or more frame images before the frame image corresponding to the current dynamic grid; processing the historical frame information using a time series analysis algorithm to calculate the dynamic confidence of the dynamic grid; if the dynamic confidence is lower than the set threshold, changing the mark of the dynamic grid to a static grid."

When determining whether a grid unit is a dynamic grid, the present application first makes a preliminary judgment on whether the grid unit is a dynamic grid using a first threshold and a second threshold, and then further judges the dynamic grid by calculating the dynamic confidence level.

Among them, if the number of point clouds in the current frame exceeds the first threshold and the absolute value of the variance difference between the number of point clouds in the current frame and the number of point clouds in the historical frame exceeds the second threshold, then the grid unit may contain dynamic objects, such as moving vehicles or pedestrians. At this time, the grid unit is marked as a dynamic grid to indicate that it has greater variability or instability. The historical frame of this application may refer to the previous frame data of the current frame. When calculating the absolute value of the variance difference between the number of point clouds in the current frame and the number of point clouds in the historical frame, it can be judged whether the absolute value of the variance difference between the number of point clouds in the current frame and the previous frame exceeds the second threshold; the historical frame of this application may also refer to the previous multiple frames of data of the current frame, and the absolute value of the variance difference between the number of point clouds in the current frame and the previous multiple frames is calculated respectively. When the multiple absolute values obtained all exceed the second threshold, it is determined that the absolute value of the variance difference between the number of point clouds in the current frame and the number of point clouds in the historical frame exceeds the second threshold.

To further evaluate the dynamic grid's actual dynamics, we extract historical frame information from the grid's history. This information refers to the number of point clouds and the point cloud height variance in the previous frame or frames of the current dynamic grid. This historical data allows us to analyze the dynamic grid's temporal trends, providing a basis for subsequent dynamic assessments.

Using time series analysis methods to process historical frames of dynamic grid data can capture patterns of temporal change. Time series analysis can include algorithms such as moving averages, exponentially weighted averages, or autoregressive analysis. These analyses help calculate the dynamic confidence level of a dynamic grid, specifically the likelihood that it possesses dynamic characteristics. Dynamic confidence is typically a quantitative indicator that indicates the degree of dynamicity of a grid cell.

As an implementable method, the present application uses time series analysis to process historical frame information, and calculates the dynamic confidence of the dynamic grid, including: constructing a point cloud quantity time series and a point cloud height variance time series based on the point cloud quantity and the point cloud height variance in the dynamic grid corresponding to the historical frame information; calculating the change rate, stability and change trend of the point cloud quantity and height variance in the dynamic grid based on the point cloud quantity time series and the point cloud height variance time series; defining a dynamic confidence formula based on the change rate, stability and change trend, and calculating the dynamic confidence based on the dynamic confidence formula.

Specifically, we first construct two time series based on the number of point clouds and the point cloud height variance recorded in the historical frame information corresponding to the dynamic grid: the point cloud number time series and the point cloud height variance time series. These two time series represent the changes in the number of point clouds and the point cloud height variance in the dynamic grid within consecutive time frames.

Next, based on the time series of point cloud number and point cloud height variance, we calculated the rate of change, stability, and trend of the dynamic grid's point cloud number and height variance, respectively. The rate of change represents the magnitude of change between adjacent frames in the time series, quantifying the degree of fluctuation in the number of point clouds or height variance. Stability describes whether the number of point clouds or height variance fluctuates within a certain range by analyzing the variance or standard deviation of the time series. Trend analysis methods (such as linear regression or polynomial fitting) are used to determine whether the time series has a significant upward or downward trend.

Based on these calculation results, a dynamic confidence formula is defined. The dynamic confidence formula can be constructed by comprehensively considering the rate of change, stability, and change trend through a weighted approach. For example, the dynamic confidence can be expressed as:

D＝w ₁ ·R+w ₂ ·(1-S)+w ₃ ·T (6)

Where R represents the rate of change, S represents stability, T represents the trend of change, and _w1 , _w2 , and _w3 are weight coefficients used to balance the impact of different factors on the dynamic confidence. Based on this formula, the dynamic confidence of the dynamic grid is calculated to obtain a quantitative value.

Furthermore, the present application can also divide multiple different time intervals; obtain the timestamp information of the historical frame information, and divide the historical frame information into corresponding time intervals according to the timestamp information; according to the importance difference of the number of point clouds and height variance in different time intervals, set different weights for the change rate, stability and change trend in different time intervals, and determine the dynamic confidence formula based on the weights.

Specifically, timestamp information is first extracted from the historical frame information of the dynamic grid, and the historical frame data is divided into multiple time intervals according to the time dimension. The time interval division can be flexibly set according to the specific application scenario, for example, evenly divided into the nearest short time interval, medium time interval, and distant time interval. The grouping rules can also be dynamically adjusted according to task requirements to balance the impact of real-time and historical data on the dynamic confidence calculation.

Within each time interval, three key characteristics are calculated based on the historical changes in the number of point clouds and height variance: rate of change, stability, and trend. Because different time intervals may contribute differently to dynamics assessment, it is necessary to assign weights to the characteristics of each interval based on their importance. For example, data from the most recent time interval has a stronger impact on dynamics assessment and can be given a higher weight; data from more distant time intervals may provide auxiliary information for trend assessment and thus be given a relatively lower weight. Weights can be set empirically or optimized based on actual scenarios.

Based on the above analysis, a dynamic confidence formula is defined, which comprehensively calculates the dynamic confidence by combining the weighted values of the rate of change, stability, and change trend of each time interval. The formula can be expressed as:

Where _Cd is the dynamic confidence; i represents the time interval; n is the total number of time intervals; _ωi is the weight of the time interval; _Ri , _Si , and _Ti represent the rate of change, stability, and change trend in the i-th time interval, respectively; α, β, and γ are the weight coefficients of the characteristic items, which are set according to specific needs.

If the dynamic confidence level derived from time series analysis falls below a pre-set threshold, the dynamic grid is no longer dynamic enough to be considered dynamic. This could be due to sensor noise or other anomalies, for example. In this case, the dynamic grid can be relabeled as static, reducing false positives for dynamic scenes and optimizing point cloud data processing accuracy.

Preferably, when performing dynamic grid determination, the present application can also determine whether the grid is a newly added scene. If the grid's historical frame information shows that the grid has never been occupied, the grid is considered to belong to a new scene, not a dynamic obstacle, and the grid is then marked as a newly added grid. Whether the grid is occupied can be determined by determining the number of point clouds of the grid in the historical data. If the number of point clouds is zero or is below a preset threshold, the grid is considered unoccupied.

The above step 207, i.e., "for the grid cells marked as dynamic grids, obtaining a point cloud of dynamic obstacles through a spatial region growing algorithm, and filtering the point cloud of the dynamic obstacles from the third point cloud data to obtain static point cloud data of the target area," is described in detail below in conjunction with an embodiment.

In this application, first, based on the position of the dynamic grid, the point cloud within it is used as the initial seed point cloud to execute the spatial region growing algorithm. The region growing algorithm is a commonly used clustering method, which mainly adds adjacent points with similar characteristics to the same set in a recursive or iterative manner. In a specific implementation, it is possible to determine whether the point clouds belong to the same region based on the spatial distance between the point clouds and the similarity of the normal vectors. For example, a spatial distance threshold and a normal vector angle threshold are set, and only when both are met will the point be included in the current region. This method can divide the point cloud data within the dynamic grid into multiple connected regions, which are further used to extract a complete dynamic obstacle point cloud.

Secondly, based on the dynamic characteristics of the dynamic grid (such as areas with high dynamic confidence), the identified connected regions are marked as dynamic obstacle point cloud data. Dynamic obstacles often have significant spatial variation characteristics, such as vehicles, people, and animals. These point cloud data represent dynamic objects that change over time in the target area and need to be filtered out when modeling the environment.

The extracted dynamic obstacle point cloud is then removed from the third point cloud data in the global coordinate system to eliminate the interference of dynamic objects on the environment modeling. In specific implementations, point cloud data marked as dynamic obstacles can be removed or masked point by point through point cloud indexing or coordinate matching, thereby retaining the static point cloud data.

Finally, static point cloud data of the target area is obtained. This static point cloud data is not disturbed by dynamic objects and can more accurately reflect the fixed environmental characteristics of the target area, providing a basis for further applications such as 3D environment modeling, path planning, or object detection.

As an implementable method, the present application filters out the point cloud of dynamic obstacles for the dynamic grid through a spatial region growing algorithm, including: selecting an initial seed point cloud from the dynamic grid according to preset indicators, expanding the region from the initial seed point cloud according to a distance metric and a point cloud density function, incorporating adjacent point clouds into the expanded region to form a continuous point cloud region; using a clustering algorithm to divide the continuous point cloud region into multiple independent regions, and filtering out the point cloud of dynamic obstacles from the independent regions.

Specifically, the selection of the initial seed point cloud is based on preset indicators, which may include characteristics such as the spatial distribution, local density, curvature, or height of the point cloud. For example, a point cloud with high point density and a certain degree of spatial connectivity is selected as the initial seed point cloud to ensure the stability and accuracy of the region expansion.

Next, a spatial region growing algorithm is used to expand the region starting from the initial seed point cloud. This region expansion process is based on two key criteria: a distance metric and a point cloud density function. The distance metric, typically calculated using Euclidean distance, is used to assess the spatial distance between the seed point cloud and its neighboring point clouds, ensuring sufficient spatial similarity between the expanded point cloud and the seed point cloud. The point cloud density function is used to determine the local density characteristics of neighboring point clouds, avoiding the inclusion of isolated point clouds or discrete noise points in the expanded region. Through gradual expansion, neighboring point clouds that meet the criteria are incorporated into the expanded region, forming a continuous point cloud region.

After a continuous point cloud region is generated, a clustering algorithm is applied to partition it into regions. Clustering algorithms analyze the spatial distribution characteristics of points within a point cloud region and divide it into multiple independent regions. Common clustering algorithms include the Density-Based Spatial Clustering of Applications with Noise (DBSCAN) algorithm and the K-Means algorithm. Taking the DBSCAN algorithm as an example, by setting a density threshold and a minimum point count threshold, the point cloud is divided into several independent regions with good connectivity, while automatically filtering out isolated point clouds.

Finally, the point cloud of dynamic obstacles is filtered out based on the segmentation results. Combining the spatial characteristics of dynamic obstacles (such as height range, density distribution, or motion trajectory) with the region segmentation results, the identified dynamic obstacle point cloud is removed from the continuous point cloud area, retaining the static point cloud data for subsequent processing.

The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

According to another embodiment, a dynamic obstacle point cloud filtering device is provided. FIG4 shows a schematic block diagram of the dynamic obstacle point cloud filtering device according to one embodiment, and the device is provided on the server side of the architecture shown in FIG1 . As shown in FIG4 , the device 400 includes:

The data acquisition unit 401 is configured to acquire one or more frame images containing a target area and inertial measurement data corresponding to each frame image.

The data fusion unit 402 is configured to generate first point cloud data for each of the frame images; and fuse the first point cloud data with the inertial measurement data using an iterative error Kalman filter algorithm to obtain second point cloud data and the pose information of the frame image.

The coordinate conversion unit 403 is configured to use a registration algorithm to determine the posture transformation matrix of the frame image relative to the reference frame based on the posture information; use the posture transformation matrix to convert the second point cloud data from the local coordinate system to the global coordinate system for representation, and obtain third point cloud data in the global coordinate system.

The grid data statistics unit 404 is configured to divide the spatial range covered by the third point cloud data into one or more grid units in the global coordinate system, and count the number of point clouds and the point cloud height variance in each grid unit.

The ground point cloud filtering unit 405 is configured to determine whether the grid unit is a ground grid unit by using a plane fitting algorithm according to the point cloud height variance, and to filter the third point cloud data in the ground grid unit.

The dynamic grid judgment unit 406 is configured to mark the grid unit as a dynamic grid if the number of point clouds in the current frame exceeds a first threshold and the absolute value of the variance difference between the number of point clouds in the current frame and the number of point clouds in the historical frame exceeds a second threshold; obtain historical frame information of the dynamic grid, the historical frame information being the number of point clouds and the point cloud height variance in the dynamic grid of one or more frame images before the frame image corresponding to the current dynamic grid; process the historical frame information using a time series analysis algorithm to calculate the dynamic confidence of the dynamic grid; and change the mark of the dynamic grid to a static grid if the dynamic confidence is lower than a set threshold.

The dynamic point cloud filtering unit 407 is configured to obtain a point cloud of a dynamic obstacle by using a spatial region growing algorithm for the grid cell marked as a dynamic grid, and filter the point cloud of the dynamic obstacle from the third point cloud data to obtain static point cloud data of the target area.

As one of the feasible ways, when the coordinate conversion unit 403 uses the registration algorithm to determine the pose transformation matrix of the frame image relative to the reference frame according to the pose information, it can be configured as follows: obtaining the point cloud data of the reference frame; using the point-to-point iterative nearest point algorithm to generate the initial pose transformation matrix of the second point cloud data and the reference frame point cloud data; and using the gradient descent algorithm to optimize the initial pose transformation matrix to obtain the pose transformation matrix.

As one of the feasible ways, the grid data statistics unit 404 can be configured to divide the spatial range covered by the third point cloud data into more than one grid units in the global coordinate system as follows: 3. Generate a three-dimensional bounding box for the third point cloud data in the global coordinate system, perform non-uniform division on the bounding box along three mutually perpendicular coordinate axis directions in the three-dimensional coordinate system, and adaptively adjust the grid resolution according to the distribution characteristics of the third point cloud data; assign a multi-dimensional index to each of the grid units, and assign the third point cloud data to the corresponding grid unit.

As one of the feasible ways, the dynamic grid judgment unit 406 can be configured to process the historical frame information using time series analysis and calculate the dynamic confidence of the dynamic grid as follows: construct a point cloud quantity time series and a point cloud height variance time series based on the point cloud quantity and the point cloud height variance in the dynamic grid corresponding to the historical frame information; calculate the rate of change, stability and change trend of the point cloud quantity and height variance in the dynamic grid based on the point cloud quantity time series and the point cloud height variance time series; define a dynamic confidence formula based on the change rate, stability and change trend, and calculate the dynamic confidence based on the dynamic confidence formula.

As one of the feasible ways, the ground point cloud filtering unit 405 can be configured as follows when using a plane fitting algorithm to determine whether the grid unit is a ground grid unit based on the point cloud height variance, and filtering the third point cloud data determined to be a ground grid unit: based on the point cloud data in each of the grid cells, a random sampling consistency algorithm is used to perform plane model fitting to obtain a fitting plane; the vertical distance between each point in the third point cloud data and the fitting plane is calculated, and whether it is a ground point is determined according to a distance threshold, and the points that meet the distance threshold condition are removed from the point cloud data; wherein the distance threshold is dynamically adjusted based on the point cloud height variance.

As one of the feasible ways, the dynamic point cloud filtering unit 407 obtains the point cloud of the dynamic obstacle by using the spatial region growing algorithm for the grid unit marked as the dynamic grid, and filters the point cloud of the dynamic obstacle from the third point cloud data. It can be configured as follows: from the dynamic grid, an initial seed point cloud is selected according to a preset indicator; based on the distance metric and the point cloud density function, the region is expanded from the initial seed point cloud, and the adjacent point clouds are included in the expanded region to form a continuous point cloud region; the continuous point cloud region is divided into multiple independent regions by using a clustering algorithm, and the point cloud of the dynamic obstacle is filtered out from the independent regions.

As one of the feasible ways, the dynamic grid judgment unit 406 can be configured to: divide a plurality of different time intervals when defining the dynamic confidence formula according to the change rate, stability and change trend; obtain the timestamp information of the historical frame information, and divide the historical frame information into corresponding time intervals according to the timestamp information; set different weights for the change rate, stability and change trend in different time intervals according to the importance difference of the number of point clouds and height variance in different time intervals, and determine the dynamic confidence formula according to the weights.

Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple. For the relevant parts, refer to the partial description of the method embodiment. The device embodiment described above is merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without expending creative work.

It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

In addition, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of any one of the methods in the aforementioned method embodiments are implemented.

And an electronic device comprising: one or more processors; and a memory associated with the one or more processors, the memory being used to store program instructions, which, when read and executed by the one or more processors, execute the steps of the method described in any one of the aforementioned method embodiments.

The present application also provides a computer program product, comprising a computer program, which implements the steps of any one of the methods described in the aforementioned method embodiments when executed by a processor.

5 exemplarily illustrates the architecture of an electronic device, which may include a processor 510, a video display adapter 511, a disk drive 512, an input/output interface 513, a network interface 514, and a memory 520. The processor 510, the video display adapter 511, the disk drive 512, the input/output interface 513, the network interface 514, and the memory 520 may be communicatively connected via a communication bus 530.

The processor 510 may be implemented as a general-purpose CPU, a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and may be used to execute relevant programs to implement the technical solutions provided in this application.

The memory 520 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 520 can store an operating system 521 for controlling the operation of the electronic device 500, and a basic input and output system (BIOS) 522 for controlling the low-level operations of the electronic device 500. In addition, a web browser 523, a data storage management system 524, and a dynamic obstacle point cloud filtering device 525, etc. can also be stored. The above-mentioned dynamic obstacle point cloud filtering device 525 can be an application program that specifically implements the operations of the aforementioned steps in the embodiment of the present application. In short, when the technical solution provided by the present application is implemented by software or firmware, the relevant program code is stored in the memory 520 and is called and executed by the processor 510.

The input/output interface 513 is used to connect input/output modules to implement information input and output. The input/output modules can be configured as components in the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and output devices may include a display, speaker, vibrator, indicator light, etc.

The network interface 514 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via a wired method (such as USB, network cable, etc.) or a wireless method (such as mobile network, WIFI, Bluetooth, etc.).

The bus 530 comprises a pathway for transmitting information between the various components of the device (eg, the processor 510 , the video display adapter 511 , the disk drive 512 , the input/output interface 513 , the network interface 514 , and the memory 520 ).

It should be noted that although the above device only shows a processor 510, a video display adapter 511, a disk drive 512, an input/output interface 513, a network interface 514, a memory 520, a bus 530, etc., in a specific implementation, the device may also include other components necessary for normal operation. In addition, it will be understood by those skilled in the art that the above device may also include only the components necessary to implement the solution of the present application, and does not necessarily include all the components shown in the figure.

Through the description of the above embodiments, it can be seen that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer program product, which can be stored in a storage medium such as ROM/RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments of the present application or certain parts of the embodiments.

The above is a detailed introduction to the technical solutions provided by this application. Specific examples are used herein to illustrate the principles and implementation methods of this application. The description of the above embodiments is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the contents of this specification should not be understood as limiting this application.

Claims (10)

1. A method for filtering a dynamic obstacle point cloud, the method comprising:

acquiring more than one frame image containing a target area and inertial measurement data corresponding to each frame image;

fusing the first point cloud data with the inertial measurement data by utilizing an iterative error Kalman filtering algorithm to obtain second point cloud data and pose information of the frame image;

The second point cloud data is converted from a local coordinate system to a global coordinate system to be represented by the pose transformation matrix, and third point cloud data in the global coordinate system is obtained;

Dividing the space range covered by the third point cloud data into more than one grid unit under the global coordinate system, and counting the number of point clouds and the point cloud height variance in each grid unit;

Judging whether the grid unit is a ground grid unit or not by using a plane fitting algorithm according to the point cloud height variance, and filtering the third point cloud data in the ground grid unit;

if the number of the point clouds in the current frame exceeds a first threshold value and the absolute value of the variance difference between the number of the point clouds in the current frame and the number of the point clouds in the historical frame exceeds a second threshold value, marking the grid unit as a dynamic grid, acquiring historical frame information of the dynamic grid, wherein the historical frame information is the number of the point clouds and the variance of the point clouds in the dynamic grid of one or more frame images of the current frame image corresponding to the current dynamic grid;

and aiming at the grid unit marked as the dynamic grid, obtaining the point cloud of the dynamic obstacle through a space region growing algorithm, and filtering the point cloud of the dynamic obstacle from the third point cloud data to obtain the static point cloud data of the target region.

2. The method of claim 1, the determining a pose transformation matrix of the frame image relative to a reference frame from the pose information using a registration algorithm comprising:

Acquiring point cloud data of the reference frame;

Generating an initial pose transformation matrix of the second point cloud data and the reference frame point cloud data by using a point-to-point iterative nearest point algorithm;

and optimizing the initial pose transformation matrix by using a gradient descent algorithm to obtain the pose transformation matrix.

3. The method of claim 1, the partitioning the spatial range covered by the third point cloud data into more than one grid cell under the global coordinate system comprising:

Generating a three-dimensional bounding box aiming at the third point cloud data of the global coordinate system, carrying out non-uniform division on the bounding box along three mutually perpendicular coordinate axis directions in the three-dimensional coordinate system, and self-adaptively adjusting the grid resolution according to the distribution characteristics of the third point cloud data;

and distributing a multidimensional index to each grid cell, and distributing the third point cloud data to the corresponding grid cell.

4. The method of claim 1, the processing the historical frame information using time series analysis, calculating a dynamic confidence of a dynamic grid comprising:

constructing a point cloud quantity time sequence and a point cloud height variance time sequence according to the point cloud quantity and the point cloud height variance in the dynamic grid corresponding to the historical frame information;

Calculating the change rate, stability and change trend of the point cloud quantity and the height variance of the dynamic grid according to the point cloud quantity time sequence and the point cloud height variance time sequence;

And defining a dynamic confidence coefficient formula according to the change rate, the stability and the change trend, and calculating the dynamic confidence coefficient according to the dynamic confidence coefficient formula.

5. The method of claim 1, wherein determining whether the grid cell is a ground grid cell using a plane fitting algorithm according to the point cloud height variance, and filtering the third point cloud data determined to be in the ground grid cell comprises:

based on the point cloud data in each grid unit, carrying out plane model fitting by adopting a random sampling consistency algorithm to obtain a fitting plane;

And calculating the vertical distance between each point in the third point cloud data and the fitting plane, judging whether the point is a ground point according to a distance threshold, and removing the points meeting the distance threshold condition from the point cloud data, wherein the distance threshold is dynamically adjusted based on the point cloud height variance.

6. The method of claim 1, wherein the obtaining, for the grid cells marked as dynamic grids, a point cloud of a dynamic obstacle through a spatial region growing algorithm, and filtering the point cloud of the dynamic obstacle from the third point cloud data comprises:

Selecting initial seed point clouds from the dynamic grids according to preset indexes, performing region expansion from the initial seed point clouds according to distance measurement and a point cloud density function, and incorporating adjacent point clouds into an expansion region to form a continuous point cloud region;

and dividing the continuous point cloud area into a plurality of independent areas by using a clustering algorithm, and filtering the point clouds of the dynamic obstacles from the independent areas.

7. The method of claim 4, wherein defining a dynamic confidence formula based on the rate of change, stability, and trend of change comprises:

Acquiring time stamp information of the historical frame information, and dividing the historical frame information into corresponding time intervals according to the time stamp information;

According to the importance difference of the point cloud quantity and the height variance in different time intervals, different weights are set for the change rate, the stability and the change trend in different time intervals, and a dynamic confidence coefficient formula is determined according to the weights.

8. A dynamic obstacle point cloud filtering device, the device comprising:

A data acquisition unit configured to acquire one or more frame images including a target area and inertial measurement data corresponding to each of the frame images;

The data fusion unit is configured to generate first point cloud data for each frame image, and fuse the first point cloud data with the inertial measurement data by utilizing an iterative error Kalman filtering algorithm to obtain second point cloud data and pose information of the frame image;

A coordinate conversion unit configured to determine a pose conversion matrix of the frame image relative to a reference frame according to the pose information by using a registration algorithm, convert the second point cloud data from a local coordinate system to a global coordinate system by using the pose conversion matrix, and obtain third point cloud data in the global coordinate system;

The grid data statistics unit is configured to divide the space range covered by the third point cloud data into more than one grid unit under the global coordinate system, and count the number of point clouds and the point cloud height variance in each grid unit;

The ground point cloud filtering unit is configured to judge whether the grid unit is a ground grid unit or not by utilizing a plane fitting algorithm according to the point cloud height variance, and filter the third point cloud data in the ground grid unit;

The dynamic grid judging unit is configured to mark the grid unit as a dynamic grid if the number of the point clouds of the current frame exceeds a first threshold value and the absolute value of the variance difference between the number of the point clouds of the current frame and the number of the point clouds of the historical frame exceeds a second threshold value; the method comprises the steps of obtaining historical frame information of a dynamic grid, processing the historical frame information by using a time sequence analysis algorithm, and calculating dynamic confidence coefficient of the dynamic grid, wherein the historical frame information is the number of point clouds and the height variance of the point clouds of one or more frame images, which are the previous to the current frame image corresponding to the dynamic grid, in the dynamic grid;

The dynamic point cloud filtering unit is configured to obtain the point cloud of the dynamic obstacle through a space region growing algorithm aiming at the grid unit marked as the dynamic grid, and filter the point cloud of the dynamic obstacle from the third point cloud data to obtain the static point cloud data of the target region.

9. An electronic device comprising one or more processors and a memory associated with the one or more processors for storing program instructions that, when read and executed by the one or more processors, perform the steps of the method of any of claims 1 to 7.

10. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the steps of the method according to any one of claims 1 to 7.