[go: up one dir, main page]
Next Article in Journal
Optimized Landslide Susceptibility Mapping and Modelling Using the SBAS-InSAR Coupling Model
Previous Article in Journal
Oceanic Precipitation Nowcasting Using a UNet-Based Residual and Attention Network and Real-Time Himawari-8 Images
Previous Article in Special Issue
Spatial Information Enhancement with Multi-Scale Feature Aggregation for Long-Range Object and Small Reflective Area Object Detection from Point Cloud
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Remote Sensing
Volume 16
Issue 16
10.3390/rs16162872
remotesensing-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Evaluation of Traffic Sign Occlusion Rate Based on a 3D Point Cloud Space

by
Ziqi Wang
1,
Xiaofei Wang
1,*,
Jun Li
2 and
Jiangbei Yao
3
1
School of Civil Engineering and Transportation, South China University of Technology, Guangzhou 510640, China
2
Guangzhou Municipal Engineering Design & Research Institute Co., Ltd., Guangzhou 510060, China
3
Guangdong Communication Planning & Design Institute Group Co., Ltd., Guangzhou 510507, China
*
Author to whom correspondence should be addressed.
Remote Sens. 2024, 16(16), 2872; https://doi.org/10.3390/rs16162872
Submission received: 6 June 2024 / Revised: 1 August 2024 / Accepted: 3 August 2024 / Published: 6 August 2024
(This article belongs to the Special Issue LiDAR and Point Cloud Processing for Digital Surface Modelling and 3D Scene Reconstruction)
Browse Figures
Figure 1
<p>Technical scheme.</p> "> Figure 2
<p>Installation of LiDAR and camera.</p> "> Figure 3
<p>Intensity information and ring information of LiDAR.</p> "> Figure 4
<p>Point cloud denoising effect.</p> "> Figure 5
<p>Real photo and point cloud shape of signs.</p> "> Figure 6
<p>Sign and plane point cloud recognized by the RANSAC algorithm.</p> "> Figure 7
<p>Flowchart of the RANSAC algorithm.</p> "> Figure 8
<p>Direction of surface normal vectors.</p> "> Figure 9
<p>Mesh reconstruction process.</p> "> Figure 10
<p>Scanning consistency display.</p> "> Figure 11
<p>Road conditions as shown in Google maps.</p> "> Figure 12
<p>Rosbag information.</p> "> Figure 13
<p>Point cloud map after the slope calculation.</p> "> Figure 14
<p>Point cloud model before and after slope segmentation.</p> "> Figure 15
<p>Point cloud model before and after the intensity segmentation of the non-ground point cloud.</p> "> Figure 16
<p>Three planar point clouds separated by RANSAC.</p> "> Figure 17
<p>Test points corresponding to the three signs.</p> "> Figure 17 Cont.
<p>Test points corresponding to the three signs.</p> "> Figure 18
<p>Additional test points corresponding to sign 1.</p> "> Figure 19
<p>Heat map of the occlusion rate for sign 1.</p> "> Figure 20
<p>Location of sign 1 and the tree point cloud.</p> "> Figure 21
<p>Field situation of sign 1.</p> "> Figure 22
<p>Field situations of sign 2 and sign 3.</p> "> Figure 23
<p>Sign 2, virtual sign 3, and added test points.</p> "> Figure 24
<p>Heat map of the occlusion rate for sign 2.</p> ">
Versions Notes

Abstract

:
The effectiveness of road signs is hindered by obstructions, such as vegetation, mutual obstruction of signs, or the road alignment itself. The traditional evaluation of road sign recognition effectiveness is conducted through in-vehicle field surveys. However, this method has several drawbacks, including discontinuous identification, unclear positioning, incomplete coverage, and being time-consuming. Consequently, it is unable to effectively assess the recognition status of road signs at any arbitrary point within the road space. Therefore, this study employed laser scanning to construct a point-surface model, which was based on a point cloud algorithm and SLAM (Simultaneous Localization and Mapping), integrated LiDAR and inertial navigation system data, and optimized the point model after processing steps such as denoising, resampling, and three-dimensional model construction. Furthermore, a method for assessing the highway sign occlusion rate based on the picking algorithm was proposed. The algorithm was applied to an actual road environment, and the occlusion by other items was simulated. The results demonstrated the effectiveness of the method. This new method provides support for the fast and accurate calculation of road sign occlusion rates, which is of great importance for ensuring the safe traveling of vehicles.

1. Introduction

Road signs encompass a variety of signs, including warning, prohibition, instruction, and auxiliary signs, which can effectively convey driving information to drivers. However, due to vegetation growth on the roads, the design of signs cannot fully account for mutual shading or the alignment of the road itself. Consequently, some parts of signs are blocked, resulting in drivers being unable to recognize the information presented on them [1]. It is therefore necessary for road operators to investigate whether road signs are blocked and to carry out the necessary maintenance according to the blocking situation. The traditional investigation methods are manual inspection and driver reporting, but these methods of identification are not continuous, the specific location is often not clear, the coverage is incomplete, and it is a time-consuming process. Consequently, it is not an effective means of assessing the identification of signs at any point in the road space. Therefore, there is a need for a more efficient and accurate means of analyzing sign occlusion. Currently, more research is being conducted using video information for identification purposes. Soheilian et al. employed multiple-view color images to detect and reconstruct 3D traffic signs. Their experimental findings demonstrated that, in terms of 3D errors, the average in-plane error of the constrained solution reached 36 mm, while that of the unconstrained solution reached only 48 mm. Furthermore, for directional errors, the improvement was even more pronounced, with the constrained solution achieving an average error of 4.55°, compared to the unconstrained solution’s 9.12° average error [2]. However, this method is limited in that it can only identify the experimental vehicle’s driving trajectory in terms of the occlusion rate. To obtain a more comprehensive understanding of the occlusion rate across the entire road range, it is necessary to conduct repeated driving tests at various locations, which is a time-consuming process. Furthermore, it is not possible to determine the occlusion of the other lanes through a single acquisition [3,4,5].
Point cloud maps can accurately describe the real spatial information of road areas. Each discrete point in the space contains an XYZ coordinate, as well as other additional information, such as the intensity, RGB color, and so on. The construction of point clouds typically occurs using two distinct methods: the direct acquisition of the point cloud information via LiDAR, or the back-computing of the visual information to determine the location of the point cloud and its associated information. As the point cloud in a single time frame is usually too sparse, the point cloud data, IMU (inertial measurement unit), GPS (global positioning system), and other information in a certain time frame are often collected to construct a complete point cloud map, which is known as a SLAM (Simultaneous Localization and Mapping) algorithm. The most common SLAM algorithms that utilize LiDAR include LOAM, LEGO-LOAM, LIOSAM, and FASTLIO, among others [6,7,8,9]. SLAM algorithms that employ vision sensors include LSD-SLAM, the ORB-SLAM series, and others [10,11,12,13]. Additionally, there are SLAM algorithms that utilize both LiDAR and vision sensors, such as R3LIVE and FAST-LIVO [14,15].
Accordingly, Li et al. integrated the Adaptive Voxel Grid DBSCAN (AVG-DBSCAN) algorithm and the Region Growing (RG) algorithm to identify obstacles and track objects based on a multi-LiDAR fused point cloud model and a vehicle dynamics model, achieving an average accuracy of 97.53% for detection and 95.1% for tracking. The average duration of the entire process was just 30 ms [16]. Xu et al. constructed a model, designated BtcDet, which learns object shapes a priori and estimates the complete object shapes that are partially occluded in the point cloud [17]. Oliveira et al. subsequently employed LiDAR data to perform occlusion detection using the height gradient, which, in turn, led to the generation of orthoimages. They also proposed that occlusion can be checked using height gradients [18]. Habib et al. identified the presence of sudden elevation changes and the resulting occlusion by sequentially checking the angle of deviation from the nadir of the line of sight connecting the DSM cell and a set of predefined synthetic projection centers [19]. Xiang et al. utilized 3D voxel patterns to detect occluded objects from a single image and to estimate multiple objects’ 3D attributes from a single image, as well as to estimate the 3D attributes of multiple objects [20].
In the field of road engineering, SLAM techniques are frequently employed to obtain information about the road itself in order to monitor lesions. For instance, Feng et al. utilized graph convolutional neural networks (GCNs) to detect cracks on roads [21], while Deng et al. conducted an evaluation using the YOLO5 model [22]. In a similar vein, Yamaguchi et al. employed iPhone LiDAR to utilize the U-Net architecture to calculate the depth and thickness of asphalt pavement cracks [23]. Additionally, other studies have focused on the extraction of road alignment and boundary parameters to form a high-precision map product [24]. Ma et al. conducted a three-dimensional reconstruction of the point cloud within the sight distance cone to calculate the specified sight distance of the road, although the method still employs the point-to-point computation rule [25]. Huang et al. employed mobile laser scan data and the hidden point removal (HPR) operator to assess the occlusion rate of road structures. However, this method necessitates the processing of points for occlusion calculation, which can result in errors [26]. Moreover, the study merely assessed whether drivers could discern the signage, without offering a quantitative occlusion rate calculation.
In conclusion, contemporary research on occluded traffic signs is mainly concentrated within the domain of machine learning, with the objective of elucidating the recognition of signage when it is occluded. However, these studies devoted less attention to the actual impact of traffic signs on drivers. Meanwhile, mainstream algorithms typically process point cloud data for deep learning, overlooking the structural properties of the actual point cloud data. This results in prolonged training times and diminished operational efficiency. Future research should prioritize the calculation of the quantitative occlusion rate and the preservation of the structural properties of the point cloud to enhance the comprehension of and responsiveness to the impact of traffic sign occlusion.
In light of the existing research foundation, this study built upon the current research foundation to investigate the accurate and effective construction of point cloud maps and sign occlusion assessment algorithms. It also proposed a method for evaluating the occlusion rate of signs based on the three-dimensional point cloud of a road. First, a point cloud model is constructed using a LiDAR and inertial navigation system. The sign and environment point clouds are then separated. A mesh reconstruction is then carried out on the environment’s point cloud model. Finally, the occlusion rate of the sign point cloud in the mesh model is calculated and evaluated. The technical scheme is shown in Figure 1.

2. Materials and Methods

2.1. Point Cloud Model

In order to fulfill the requirements of the SLAM algorithm, it is necessary to utilize LiDAR and an inertial navigation system for the acquisition of road information. The data transmitted by the devices can be read and recorded in real time using the ROS platform of the Ubuntu system [27], as well as the SDK of each hardware manufacturer. The specific device information and installation methods used in this study are shown in Table 1 and Figure 2.
The LiDAR emits a laser beam and receives the reflected laser beam information to measure the distance and direction of the target object. The UDP protocol is employed to transmit data between the LiDAR and the computer, and the LiDAR data can be obtained in real time using Wireshark software (Version number: 4.0.16). Additionally, the point cloud information can be observed in Rviz based on the Ros platform and SDK code [28] in the Ubuntu system. The result of the recorded LiDAR scan consists of the 3D coordinates, reflection intensity, ring information, and reflection time of each point, as illustrated in Figure 3. This figure depicts the difference in the various information elements of the LiDAR point cloud data at the same position.
An inertial measurement unit (IMU) typically comprises accelerometers and gyroscopes, with some more sophisticated models incorporating magnetometers as an additional sensor. The output of the system comprises acceleration in three dimensions, the angle of rotation, and the direction of deflection relative to the Earth’s magnetic poles. These can be utilized to estimate the device’s orientation, specifically its position and attitude. Similarly, changes in the orientation of the IMU device can be observed in RVIZ [29]. In this study, the computer was connected to the Helios 1615 Lidar via the RJ45 interface and to the D435i via the USB interface. The data from these devices were read and recorded in the ROS platform of Ubuntu system through the SDKs of the two devices. This recording method can be used to temporally align the multiple devices and the data using the computer timestamps in the absence of GPS timing devices.
Furthermore, open-source algorithms are typically tailored to a limited number of devices or datasets, necessitating device-specific modifications to the source code. These modifications may include the radar type, model, and harness, as well as the error of the IMU and the bit-position transformation matrix between the radar and the IMU. In this study, the LIOSAM algorithm was employed for the construction of point cloud maps, which utilize four factors for map optimization. The first factor is the IMU pre-integration factor, which is obtained by integrating the IMU measurements between two neighboring keyframes. The second factor is the laser odometry factor, which is derived from the results of the frame map matching between each keyframe and the preceding n keyframes. The third factor is the GPS factor, which is obtained from the GPS measurement of each keyframe. The fourth factor is the loopback factor, which is obtained by frame map matching between each keyframe and the two keyframes that are two minutes and one keyframe later than the timing of the candidate loopback keyframe. Due to the limitations of the equipment, in this study, the source code of LIOSAM was modified to adapt to the 6-axis IMU of D435i, with no GPS information being input. Additionally, it should be noted that loopback recording was not possible in a road environment due to the limitations of data recording. Consequently, the actual LIO-odom branch under LIOSAM was employed in this study [8].
The angular velocity ω ∧ t and acceleration a ∧ t output from the IMU at time t can be expressed by Equations (1) and (2), where b t and n t represent the zero bias and white noise, respectively:
ω ∧ t = ω t + b t ω + n t ω
a ∧ t = R t BW a t − g + b t a + n t a
Thus, the velocity v, position P, and rotation R of the device at time t + Δt can be calculated using Equations (3)–(5):
v t + Δ t = v t + g Δ t + R t a ∧ t − b t a − n t a Δ t
p t + Δ t = p t + v t Δ t + 1 2 g Δ t 2 + 1 2 R t a ∧ t − b t a − n t a Δ t 2
R t + Δ t = R t exp ⁡ ω ∧ t − b t ω − n t ω Δ t
Thus, the relative motion of the device between two frames can be calculated from the pre-integrated values of Equations (6)–(8):
Δ v i j = R i ⊤ v j − v i − g Δ t i j
Δ p i j = R i ⊤ p j − p i − v i Δ t i j − 1 2 g Δ t i j 2
Δ R i j = R i ⊤ R j
Radar odometry, on the other hand, calculates the distance between a feature and its corresponding plane or edge using Equations (9) and (10), where k, u, v, and w represent the indices of various features, including planes and edges, and p represents the coordinates of the corresponding point:
d e k = p i + 1 , k e − p i , u e × p i + 1 , k e − p i , v e p i , u e − p i , v e
d p k = p i + 1 , k p − p i , u p p i , u p − p i , v p × p i , u p − p i , w p p i , u p − p i , v p × p i , u p − p i , w p
The radar odometry factor between the two bit positions is then obtained by solving the corresponding distance minima, i.e., Equation (11):
min T i + 1 ⁡ ∑ p i + 1 , k e ∈ ′ F i + 1 e d e k + ∑ p i + 1 , k p ∈ ′ F i + 1 p d p k
In addition, when data acquisition is performed in an unenclosed area, it is inevitable that there will be other moving objects in the environment, including pedestrians and vehicles, and these moving objects will produce colored noise points in the constructed point cloud model, as shown in Figure 4a. Therefore, to obtain a finer model, the point cloud needs to be denoised, as shown in Figure 4b. Common point cloud denoising methods include statistical methods, distance-based methods, shape-based methods, and machine-learning-based methods. In this instance, the statistical SOR filtering method was employed. First, for each point, the average distance d from it to all points in its K-neighborhood is computed. This process is completed for each point in the input point cloud, whereby each point is found to have its corresponding d, thus resulting in the generation of an array containing the values of d at each point, denoted as distance. Second, for all points in the input point cloud, it is assumed that each element in the distance array constitutes a Gaussian distribution, whereby the array is a sample, and the sample capacity is the number of points contained in the point cloud. The shape of the Gaussian distribution curve is determined by the mean and standard deviation of the samples. Points with d-values outside the standard range (defined by the mean and variance of the samples) can be defined as outliers and removed from the dataset. However, the denoising process can also incorrectly handle the non-noise, which in turn affects the accuracy of the point cloud model.

2.2. Point Cloud Model Separation Algorithm

Before constructing the 3D model, the point cloud model needs to be pre-cut and categorized. Since our goal was to study the evaluation of the occlusion rate of road signs, the point cloud belonging to the road signs needs to be separated from the road environment. In this study, the 3D features of sign shape, gantry, and other structures were used to identify and separate the road signs from the road environment, as shown in Figure 5.

2.2.1. Ground Point Cloud and Non-Ground Point Cloud Separation Algorithms

Since both road signs and ground markings have high intensity, as well as planar features, the ground point cloud and non-ground point cloud need to be separated first. In this study, the cloth simulation filter (CSF) ground filter algorithm and slope method ground filter algorithm were used and compared. After conducting a comparative analysis, we found that the slope method ground filter algorithm performed better in the research environment of this study. Specifically, in the area investigated in this study, the vegetation is denser and the terrain is more significant. The slope method ground filter algorithm could separate the ground point cloud more accurately, thus avoiding the misjudgment and errors that the CSF algorithm may produce in this environment. Based on these observations and analytical results, the slope method ground filtering algorithm was finally selected as the main method with which to separate the ground point cloud from the non-ground point cloud, in order to ensure the accuracy and reliability of the data processing.
In this study, the CSF ground filter was used to flip the point cloud, and then a piece of cloth was simulated to fall from the top under gravity via Equation (12), where X represents the position of the particles in the “cloth” at time t, F e x t X , t represents the external driving factors (gravity, collision, etc.), and F i n t X , t represents the internal driving factors. Then, the tissue that finally falls and comes to rest can represent the current terrain [30]:
m ∂ X t ∂ t 2 = F e x t X , t + F i n t X , t
The slope method of ground filtering separates the ground point cloud by calculating the slope between the point clouds and using the slope values [31]. The Z value of the point cloud is reassigned to the point cloud as a special value to retain the height information; this special value represents the height of each point in the point cloud, so the gradient of the point cloud can be calculated to obtain the slope of each point using Equation (13). The slope is calculated based on the Z value and therefore directly reflects the slope of the terrain:
d f x d x = lim h → 0 ⁡ f x + h − f x h

2.2.2. Extraction of Sign Planes

After distinguishing which points belong to the ground and non-ground categories, the non-ground point cloud needs to be further processed for the planar extraction of the sign. The study employed the RANSAC algorithm to process the high-intensity point cloud model. In comparison to other algorithms, RANSAC can identify the correct model parameters when confronted with a considerable number of outliers. Furthermore, its operational complexity and efficiency are relatively straightforward. Moreover, the RANSAC algorithm can select suitable parameters in accordance with the attributes of the point cloud data, thereby enhancing the precision of the extracted shapes. The RANSAC algorithm extracts the shape by randomly extracting the minimal point set and reconstructing the corresponding shape primitives from the point cloud data [32]. The minimal point set is the smallest number of points required to uniquely define a given geometric primitive. These generated candidate shapes are tested over all points in the dataset to determine how many points can be well approximated by the primitives. After a certain number of tests, the shape with the highest number of approximated points is extracted, and the algorithm continues the computation among the remaining points until a certain threshold is reached; finally, it outputs the recognized result, e.g., the yellow rectangle in Figure 6 is the result of fitting a plane to the point cloud of the sign, and the red dots are the point cloud corresponding to that plane. The overall flow of the RANSAC algorithm is shown in Figure 7.

2.3. D Model Construction and Optimization Processing

2.3.1. Point Cloud Normalization

Because point cloud models are generally disordered, it is difficult to extract their features; this often results in low-quality point cloud models due to the loss of detail or the generation of misaligned structures in the reconstruction. Normals can provide local orientation information for each point, helping to determine the orientation of the surface and ensuring that the 3D model is accurately oriented. In addition, using normal information in 3D model reconstruction can smooth out noisy points that appear on the surface. Therefore, 3D model reconstruction from point clouds usually requires the input of the normal information of the point cloud.
As shown in Figure 8, for a 3D surface, the normal vector of the tangent plane at a point (u,v) is the normal vector of the surface at point (u,v). Since the point cloud is a point sample of the surface, the normal vector of the fitted sampled surface is the normal vector of the point cloud [33].
The study employed the principal component analysis (PCA) method, which is highly efficacious in addressing large-scale point cloud data on roads. Furthermore, the method demonstrates high accuracy even when the point cloud is noisy or the surface shape is complex. The implementation steps for this method can be realized using linear algebra libraries and toolkits, which are relatively simple. For each point, the K nearest neighbors (or neighbors within a radius) are calculated, and the covariance matrix Cov of the PCA is subsequently calculated using Equation (14):
C o v = ∑ N i × N i − C
where Ni is the neighborhood point and C is the center point.
Then, the eigenvector corresponding to the smallest eigenvalue in the covariance matrix Cov is found, which is the normal direction of the point.

2.3.2. Mesh Reconstruction

The mesh is reconstructed from the point cloud with the normal. Unlike the point cloud, the mesh has the concept of surface, which is more convenient for judging the intersection. There are two main types of mesh reconstruction methods, namely, triangulation and surface fitting. The triangulation methods refer to the triangulation of the points of the point cloud as fixed points of the mesh. Delaunay triangulation can preserve the geometric details and convey the attribute information of the point cloud, as shown in Figure 9a; it also exhibits better adaptability to point clouds with different features [34]. Surface fitting methods, on the other hand, use surfaces to fit the point cloud data, and then discretize the surfaces to the mesh. This method is more noise-resistant while coping better with a smaller range of missing data. Considering that the geometric features of the road environment are more prominent and distinctive, we chose to use the triangulation method, which can better preserve the details of the point cloud. Due to the working principle of LiDAR, there are some missing data in every two frames of point cloud data, forming a hole, as shown in Figure 9b. We adopted the triangulation method to fill the hole in the 3D model, and the completed model of the hole is shown in Figure 9c.

2.4. Single-Point Occlusion Rate

In this study, the single-point occlusion rate is defined as the proportion of points in the point cloud to which the connecting lines in the point cloud that the sign belongs to are occluded by the 3D model to all the points in the point cloud model of the sign at a certain fixed point. This ratio is denoted as μ, as shown in Equation (15):
μ = j N O
where N O is the total number of points in the sign point cloud and j is the number of occluded points in the sign point cloud.
This ratio provides a metric for evaluating the number of points in a sign point cloud that cannot be directly observed at a given observation point or viewpoint due to occlusion by the rest of the 3D model. To illustrate this idea, consider a fixed observation point in 3D space from which lines of sight must reach the point cloud of the sign. If the lines of sight are obstructed by other elements of the 3D model before reaching specific points in the point cloud of the sign, those points are considered to be occluded. The proportion of all occluded points in the sign point cloud is calculated as the single-point occlusion rate for that fixed point μ. Due to the limitations of the operating mechanism of LiDAR, as illustrated in Figure 10, the point cloud density of objects in closer proximity to the LiDAR is higher, while the point cloud density of objects at greater distances is lower. This affects the subsequent evaluation of the occlusion rate, necessitating the resampling of the point cloud model. The process of resampling typically entails a reduction in the number of points while simultaneously enhancing the efficiency of the algorithm. This approach ensures that the fundamental characteristics of the point cloud remain unaltered.
The most common resampling methods include voxel downsampling, uniform sampling, geometric sampling, and random downsampling. Voxel grid downsampling is accomplished by dividing the point cloud into a series of voxels (cubic grids) and utilizing the center of gravity or average of the points within each voxel to represent all the points within that voxel. This approach markedly reduces the number of points in the point cloud while maintaining its overall geometry and structure. This method effectively reduces the density of the point cloud, simplifies computation, improves processing efficiency, and preserves the global structure of the point cloud, rendering it suitable for real-time 3D scanning, point cloud alignment, and map construction scenarios that require fast processing. In contrast, uniform sampling ensures that the sampled points are distributed evenly in space by selecting a specific proportion or number of points in the point cloud in a uniform manner. Uniform sampling is well-suited to tasks requiring the maintenance of a uniform distribution of point cloud data, such as 3D reconstruction and visualization. Geometric sampling is a method of sampling a point cloud based on its geometric features. It typically selects points with significant geometric features, such as high curvature or edge points. This approach is achieved by analyzing the curvature or surface normals of the point cloud; this preserves important geometric features in the point cloud and helps to improve accuracy in subsequent processing tasks such as feature matching, shape analysis, and object recognition. Geometric sampling is well-suited to applications where geometric features are extracted and preserved, such as edge detection, shape analysis, and feature point extraction. In contrast, random downsampling employs a random selection of points from the point cloud; it is a straightforward approach that is nevertheless associated with poor uniformity and feature retention. Random downsampling is a straightforward method of implementation, with minimal computational overheads. It is well-suited to rapid preprocessing and data simplification, as well as preliminary data checking and large-scale data preprocessing. Consequently, in this study, voxel sampling was employed for resampling with the objective of minimizing the impact of building locations.
Following the previous step, the original 3D point cloud data are decomposed into a point cloud set that includes only the sign point cloud and a 3D model that does not include the sign. The point cloud set is defined as an array V O , as indicated in Equation (16). This equation specifies that the point v i is a point within the sign range ( R O 3 ), where the sign consists of N O points
V O = v i ∈ R O 3 i = 1 ⋯ N O
The 3D model is defined as the group (V, T, K), expressed by Equation (17), where V is the set of vertices v m of the model, T is the set of edges t m n of the model, and K is the set of faces k m n p of the model:
V = v m ∈ R 3 m = 1 ⋯ N m T = t 01 = v 0 , v 1 , t 12 = v 1 , v 2 , t 02 = v 0 , v 2 ⋯ K = k 012 = v 0 , v 1 , v 2 , k 123 = v 1 , v 2 , v 3 , ⋯
For a particular point X, X is constructed to be connected one by one to each point in the array V O to form the array shown in Equation (18):
T x = ( t x 0 = v x , v 0 , t x 1 = v x , v 1 , t x 2 = v x , v 2 ⋯ } )
Subsequently, the picking algorithm [35] in DirectX SDK is employed to ascertain whether each line in T x intersects with each face in K. The number of lines with intersections is recorded and denoted as j. The ray equation is as follows: the ray equation is set as O + D t , where O is the starting point of the ray and D is the direction of the ray. The parametric equation of the triangle is given by Equation (19):
1 − u − v V 0 + u V 1 + v V 2
where V 0 , V 1 , a n d V 2 are the three points of the triangle; u and v are the weights of V 1 and V 2 , respectively; and u ≥ 0 ,   v ≥ 0 ,   1 − u − v ≥ 0 . Thus, the intersection of the ray with the plane can be found by solving Equation (20):
O + D t = 1 − u − v V 0 + u V 1 + v V 2
Let E 1 = V 1 − V 0 , E 2 = V 2 − V 0 , T = 0 − V 0 . Then, Equation (21) can be solved:
t u v = 1 D × E 2 · E 1 T × E 1 · E 2 D × E 2 · T T × E 1 · D
Finally, the ratio of j and N O is calculated as the occlusion rate μ of the point, as defined in Equation (15). Since N O is the number of all points in the sign point cloud and j is the occluded point in the sign point cloud, the occlusion rate μ takes a value of [0,1].

2.5. Sign Occlusion Rate

In practice, the same sign may have different occlusion rates at different locations on the road and from different viewpoints. This variation is attributable to the presence of obstacles in the environment, such as buildings, trees, and vehicles, which obstruct the line of sight to varying degrees, thereby reducing the visibility of the sign at specific locations. The µ values at different locations were averaged to obtain an average occlusion rate, μ a v g , which provides an overall overview of the occlusion situation. However, this does not accurately reflect the degree of occlusion of the sign under the most unfavorable conditions. For example, it is assumed that the sign will be fully obscured from some critical viewpoints, while it may be only partially obscured or fully visible from most other viewpoints. The use of the average occlusion rate μ a v g for the evaluation may result in the concealment of the existence of the most unfavorable conditions, leading to an overall evaluation result that is overly optimistic and does not accurately reflect the visibility of the sign in all situations. Such misjudgments and potential risks can affect the design of traffic management systems, the planning of camera locations, and the performance of visual navigation.
Consequently, in order to comprehensively and accurately assess the occlusion of signs, two evaluation parameters are introduced in this study: the average occlusion rate μ a v g and the maximum occlusion rate μ m a x . These are defined by Equations (22) and (23).
The maximum occlusion rate is defined as the maximum value of the occlusion rates of the individual points, as illustrated in Equation (23):
μ a v g = ∑ 1 n μ i n
μ m a x = max ⁡ μ 1 , μ 2 , … μ n
where μ i is occlusion rate of the ith point, and n is the number of observation locations.
The average occlusion rate, μ a v g , represents the overall occlusion of the sign at all possible observation locations. It is defined as the arithmetic mean of the occlusion rate, μ, at each location. It provides an overall understanding of the extent to which the sign is obscured, accounting for visibility in general. However, to complement this overall assessment and to ensure that correct judgments can be made even in the most unfavorable situations, it is also necessary to consider the maximum occlusion rate, μ m a x . μ m a x is the maximum single-point occlusion rate encountered by the sign across all viewing positions, reflecting the worst possible degree to which the sign can be occluded at the worst possible viewing angle. By considering μ m a x , the most unfavorable occlusion scenarios can be identified and prevented, thereby ensuring that the visibility of the sign is adequately assessed even under the most adverse environmental conditions.
For real roads, the road markings exhibit high point cloud intensity values during the LiDAR scanning process due to their reflective properties. Conversely, as road markings possess a thickness, the precise location of each lane line can be discerned in the three-dimensional model. The length of the imaginary and real markings of a given road is essentially fixed, and thus the length of the road can be calculated by multiplying the length and number of markings. This specific method entails the separation of the road marking according to the actual length of the marking and the coordinate information alterations in the point cloud data. Thereafter, the relationship between the coordinates and the actual length is calculated. Once the actual length of the road is determined, the information regarding the markings and the road structure can be inverted to obtain the road’s three-dimensional coordinates.
The recognition distance of the sign was calculated based on the running speed of the road section and the position on the road where the distance from the sign is the recognition distance. This was determined using the 3D model. The two most unfavorable positions in the recognition range were considered here: the left wheel of the vehicle fitting the leftmost side of the road lane line and the right wheel of the vehicle fitting the rightmost side of the road lane line. The driver’s position was set to 0.6 m from the left side of the vehicle, 1.2 m from the right side, and 1.2 m high, considering the driver’s position and apparent height. Previous studies have demonstrated that the recognition distance of a sign varies according to the speed and the environment. For instance, in front of a tunnel, the recognition distance is typically 200 m [36], while it can be reduced on an ordinary road [37]. Accordingly, regarding the requisite limit value for visual recognition distance, a minimum distance of 100 m was established herein.

3. Results

The operational highway in Guangdong Province, China, depicted in Figure 11, served as the subject of this study.
Figure 12 presents the results of the recorded rosbag. The point cloud information of LiDAR is represented using /rslidar_points, the imu information recorded by D435i is represented using /camera/imu, and the video information recorded by D435i for later inspection is represented as /camera/color/image_raw.
The outer parameters of the IMU and LiDAR were calibrated using the LiDAR-align algorithm [38], and the calibration result is expressed in Equation (24). In the preceding methodology, the LIO-odom algorithm was employed to construct the point cloud map model, resulting in a model comprising 5,542,352 points. Following the application of a SOR filter for denoising, the number of points was reduced to 5,140,501. In this instance, the number of points utilized for the mean distance estimation was set to 6, while the standard deviation multiplier threshold was fixed to 1.00.
R = 0.99998 0.00132129 0.00119364 0.063821 − 0.0013214 0.999999 8.82581 e − 05 0.0461159 − 0.00119352 − 8.98352 e − 05 0.999999 0.000772845 0 0 0 1
The resulting point cloud map, which depicts the calculated gradient, is illustrated in Figure 13. The green regions indicate areas with a higher gradient, while the blue regions represent lower gradients, which correspond to the road segment. The original point cloud was segmented according to the slope into two distinct parts: a non-ground point cloud and a ground point cloud. The gradient slope separation was set to 0.1, resulting in a total of 1,309,066 points in the ground point cloud and 3,831,435 points in the non-ground point cloud.
The point cloud should be separated based on intensity. The high-intensity portion was then processed using the RANSAC algorithm to separate the planar point cloud from the remaining parts. As illustrated in Figure 14 and Figure 15, the points with an intensity value exceeding 200 were designated as the high-intensity point cloud. In the RANSAC algorithm, the minimum number of support points per primitive was set to 500, the maximum distance to primitive is established as having a value of 7.575, the sampling resolution was set to 15.15, the maximum normal deviation was set to 25°, and the overlooking probability was set to 0.01.
Three signboards were successfully separated and recognized in the example road section, as shown in Figure 16. These were labeled as signs 1~3. Sign 1 is the hanging roadside sharp curve deceleration warning signboard, sign 2 is the gantry road direction indication signboard, and sign 3 is the gantry lane-splitting speed limit signboard. For each signboard, a recognition distance of 100 to 200 m in the direction of oncoming traffic was detected, and a test point was set up every 20 m, resulting in a total of 12 points for visibility detection. Sign 1 had 1313 points, and, after re-sampling, it dropped to 1000 points. Sign 2 had 5524 points, and, after re-sampling, it dropped to 1000 points. Sign 3 had 5797 points, and, after re-sampling, it dropped to 2000 points. The test points for each sign were selected as illustrated in Figure 17, and the occlusion rate calculation results are presented in Table 2.
It can be observed that the 12 test points of sign 2 were not blocked in the road model, indicating that the road area can be fully visualized within the range of 200 in the oncoming direction. The 12 test points of sign 3 exhibited some obscured regions, with the most notable being point 6, which represents the first lane on the left side of the sign, 200 m in front of the sign in the oncoming direction. While some of the points were obscured, the obscuring rate was relatively low, with the maximum observed rate being 3 ‰ and the average rate being 0.42‰. This indicates that sign 2 and sign 3 are unlikely to experience occlusion.
In sign 1, the average occlusion rate was 7.33%, with a maximum of 34.8% occurring at point 2 in lane 3 on the left side of the signboard, 120 m ahead in the oncoming direction. Consequently, 32 test points were encrypted for sign 1, totaling 44 points, which were renumbered for calculation purposes. The test points were selected as shown in Figure 18. The result of the occlusion rate calculation is presented in Table 3.
In Figure 19, the white area represents a situation in which vehicles should not drive under normal circumstances. The yellow dotted line represents the location of the road lane line. The color change in the heat map represents the magnitude of the occlusion rate, with blue indicating an absence of occlusion and red signifying a more severe occlusion situation. An optimal road environment would be free of any occlusion, indicated by a pure blue color. The results of the analysis indicate that, when a vehicle moves to the left side, the occlusion of a sign occurs at a distance of approximately 100 to 120 m from the sign. As the vehicle’s position shifts to the right, the location of the occlusion retreats until a more extensive range of occlusion occurs at approximately 180 m when the vehicle reaches the far right. Furthermore, at approximately 240 to 280 m from the sign, there is some degree of occlusion in all lanes. Upon examination of the point cloud data, it was determined that there were tree-like point clouds in front of sign 1 which obstruct it, as illustrated in Figure 20.
A field investigation revealed the presence of a tree in front of sign 1 that had overgrown and invaded the area above the road, resulting in the occlusion of sign 1 (Figure 21). Furthermore, the investigation revealed that signs 2 and 3 were not obstructed, as illustrated in Figure 22.
The process of designing road and traffic engineering facilities ensures that each meets the requirements of the specification. However, the current specification lacks a requirement for the inspection of interactions. On some roads, a front sign, columns, or piers cover the sign. In the example road section in this study, the point cloud of the gantry sign 3 and its affiliated structures was moved to 50 m in front of sign 2 by translational rotation. This method was used to simulate the situation of mutual occlusion between signboards. Additionally, 44 points were re-selected and re-calculated. The test points are shown in Figure 23. The results are shown in Table 4.
Figure 24 illustrates that the occlusion was more pronounced at a distance of 120 to 140 m from the signboard. Given that sign 3 is situated to the right of the road, it can be postulated that greater occlusion may occur at these distances from the centerline of the road. Conversely, at more distant locations, the occlusion was comparatively low due to the influence of curves and height settings.

4. Discussion

This study presents a novel approach to calculating the occlusion rate of signs on highways based on point cloud maps. The objective of this new method is to provide a more rapid and precise means of calculating the occlusion rate of signs on highways. The combination of point cloud maps and field surveys allows for a more comprehensive assessment of the occlusion situation in the roadway environment, thereby providing crucial support for the safe operation of autonomous vehicles.
This method circumvents the constraints of conventional sign occlusion detection methods and is capable of expeditiously and meticulously determining the occlusion rate at disparate locations on the roadway. This method enables the point cloud map to be employed in the identification of potential road occlusions, with subsequent analyses and judgements conducted in conjunction with field investigations. Concurrently, the method employs the mesh processing of the point cloud, which addresses the issue of the overlapping of point cloud data, which otherwise renders the shape contours inconspicuous.
To assess the efficacy of this methodology, an experiment was conducted using real-world data from Guangdong Province, China. The results demonstrated that the occlusion rate of signs calculated using this method was highly consistent with the findings of the field survey. Additionally, a situation in which signboards overlap with each other was also simulated based on real data. The results demonstrated that effective operations were still possible in an environment where signboards overlap. The feasibility and accuracy of the method were confirmed in the context of existing operating highways.
While the algorithm has certain advantages, it also exhibits certain shortcomings. Firstly, the algorithm was only tested on a limited number of roadways and requires further validation on a wider range of roads. Theoretically, the algorithm can be applied to point cloud data collected by disparate devices and point cloud maps constructed using varying algorithms. However, LiDAR is susceptible to degradation and the generation of noise under the conditions of precipitation, which may limit the applicability of the algorithm to data collected in such environments. Furthermore, the presence of a considerable number of vehicles in the road environment may potentially impact the precision of the map construction, consequently affecting the accuracy of the algorithm. Given the higher traffic volume during daytime hours and the correspondingly low number of vehicles on the road at night, a dataset compiled at night may prove more suitable for the algorithm presented in this study. In this study, a daytime dataset was employed with the objective of simultaneously acquiring visual information about the road for validation purposes.
Future research will focus on the significance of developing accurate three-dimensional point cloud maps to identify various types of information about signs. This will further enhance our comprehension and perception of the road environment, thereby facilitating more precise driving guidance for autonomous vehicles. Furthermore, the potential of machine learning techniques to automate the recognition and processing of signs will be investigated. This would eliminate the need for manual recognition processes.

Author Contributions

Conceptualization, Z.W. and X.W.; methodology, Z.W. and X.W.; software, J.L.; supervision, X.W.; validation, J.Y.; visualization, J.L. and J.Y.; writing—original draft, Z.W.; writing—review and editing, Z.W. and X.W. All authors have read and agreed to the published version of the manuscript.

Funding

This work was jointly supported by the Natural Science Foundation of Guangdong Province [No. 2024A1515011177] and Guangdong Transportation Planning and Design Institute Group Co. Ltd. [No. 20221615].

Data Availability Statement

The data utilized in the experiment are not available due to confidentiality reasons. The LIO-SAM and other algorithm codes used in the experiment can be downloaded from GitHub.

Conflicts of Interest

Author Jun Li was employed by the Guangzhou Municipal Engineering Design & Research Institute Co., Ltd. Author Jiangbei Yao was employed by the Guangdong Communication Planning & Design Institute Group Co., Ltd. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

  1. Ben-Bassat, T.; Shinar, D.; Caird, J.K.; Dewar, R.E.; Lehtonen, E.; Sinclair, M.; Zakowska, L.; Simmons, S.; Liberman, G.; Pronin, M. Ergonomic Design Improves Cross-Cultural Road Sign Comprehension. Transp. Res. Part F Traffic Psychol. Behav. 2021, 78, 267–279. [Google Scholar] [CrossRef]
  2. Soheilian, B.; Paparoditis, N.; Vallet, B. Detection and 3D Reconstruction of Traffic Signs from Multiple View Color Images. ISPRS J. Photogramm. Remote Sens. 2013, 77, 1–20. [Google Scholar] [CrossRef]
  3. Ruta, A.; Li, Y.; Liu, X. Real-Time Traffic Sign Recognition from Video by Class-Specific Discriminative Features. Pattern Recognit. 2010, 43, 416–430. [Google Scholar] [CrossRef]
  4. Wu, W.; Chen, X.; Yang, J. Detection of Text on Road Signs from Video. IEEE Trans. Intell. Transp. Syst. 2005, 6, 378–390. [Google Scholar] [CrossRef]
  5. Xu, J.; Huang, Y.; Ying, D. Traffic Sign Detection and Recognition Using Multi-Frame Embedding of Video-Log Images. Remote Sens. 2023, 15, 2959. [Google Scholar] [CrossRef]
  6. Zhang, J.; Singh, S. LOAM: Lidar Odometry and Mapping in Real-time. In Proceedings of the Robotics: Science and Systems X, Berkeley, CA, USA, 12–16 July 2014; Fox, D., Kavraki, L.E., Kurniawati, H., Eds.; IEEE: Piscataway, NJ, USA, 2014. [Google Scholar]
  7. Shan, T.; Englot, B. Lego-Loam: Lightweight and Ground-Optimized Lidar Odometry and Mapping on Variable Terrain. In Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain, 1–5 October 2018; pp. 4758–4765. [Google Scholar]
  8. Shan, T.; Englot, B.; Meyers, D.; Wang, W.; Ratti, C.; Rus, D. Lio-Sam: Tightly-Coupled Lidar Inertial Odometry via Smoothing and Mapping. In Proceedings of the 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA, 24 October 2020–24 January 2021; pp. 5135–5142. [Google Scholar]
  9. Xu, W.; Zhang, F. Fast-Lio: A Fast, Robust Lidar-Inertial Odometry Package by Tightly-Coupled Iterated Kalman Filter. IEEE Robot. Autom. Lett. 2021, 6, 3317–3324. [Google Scholar] [CrossRef]
  10. Engel, J.; Schöps, T.; Cremers, D. LSD-SLAM: Large-Scale Direct Monocular SLAM. In Computer Vision–ECCV 2014; Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T., Eds.; Lecture Notes in Computer Science; Springer International Publishing: Cham, Switzerland, 2014; Volume 8690, pp. 834–849. ISBN 978-3-319-10604-5. [Google Scholar]
  11. Mur-Artal, R.; Montiel, J.M.M.; Tardos, J.D. ORB-SLAM: A Versatile and Accurate Monocular SLAM System. IEEE Trans. Robot. 2015, 31, 1147–1163. [Google Scholar] [CrossRef]
  12. Mur-Artal, R.; Tardós, J.D. Orb-Slam2: An Open-Source Slam System for Monocular, Stereo, and Rgb-d Cameras. IEEE Trans. Robot. 2017, 33, 1255–1262. [Google Scholar] [CrossRef]
  13. Campos, C.; Elvira, R.; Rodríguez, J.J.G.; Montiel, J.M.; Tardós, J.D. Orb-Slam3: An Accurate Open-Source Library for Visual, Visual–Inertial, and Multimap Slam. IEEE Trans. Robot. 2021, 37, 1874–1890. [Google Scholar] [CrossRef]
  14. Lin, J.; Zhang, F. R 3 LIVE: A Robust, Real-Time, RGB-Colored, LiDAR-Inertial-Visual Tightly-Coupled State Estimation and Mapping Package. In Proceedings of the 2022 International Conference on Robotics and Automation (ICRA), Philadelphia, PA, USA, 23–27 May 2022; pp. 10672–10678. [Google Scholar]
  15. Zheng, C.; Zhu, Q.; Xu, W.; Liu, X.; Guo, Q.; Zhang, F. Fast-Livo: Fast and Tightly-Coupled Sparse-Direct Lidar-Inertial-Visual Odometry. In Proceedings of the 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Kyoto, Japan, 23–27 October 2022; pp. 4003–4009. [Google Scholar]
  16. Li, J.; Zhang, Y.; Liu, X.; Zhang, X.; Bai, R. Obstacle Detection and Tracking Algorithm Based on Multi-Lidar Fusion in Urban Environment. IET Intell. Transp. Syst. 2021, 15, 1372–1387. [Google Scholar] [CrossRef]
  17. Xu, Q.; Zhong, Y.; Neumann, U. Behind the Curtain: Learning Occluded Shapes for 3D Object Detection. Proc. AAAI Conf. Artif. Intell. 2022, 36, 2893–2901. [Google Scholar] [CrossRef]
  18. Oliveira, H.C.; Galo, M. OCCLUSION DETECTION BY HEIGHT GRADIENT FOR TRUE ORTHOPHOTO GENERATION, USING LIDAR DATA. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2013, XL-1-W1, 275–280. [Google Scholar] [CrossRef]
  19. Habib, A.F.; Chang, Y.-C.; Lee, D.C. Occlusion-Based Methodology for the Classification of Lidar Data. Photogramm. Eng. Remote Sens. 2009, 75, 703–712. [Google Scholar] [CrossRef]
  20. Xiang, Y.; Choi, W.; Lin, Y.; Savarese, S. Data-Driven 3d Voxel Patterns for Object Category Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; pp. 1903–1911. [Google Scholar]
  21. Feng, H.; Li, W.; Luo, Z.; Chen, Y.; Fatholahi, S.N.; Cheng, M.; Wang, C.; Junior, J.M.; Li, J. GCN-Based Pavement Crack Detection Using Mobile LiDAR Point Clouds. IEEE Trans. Intell. Transp. Syst. 2021, 23, 11052–11061. [Google Scholar] [CrossRef]
  22. Deng, L.; Zhang, A.; Guo, J.; Liu, Y. An Integrated Method for Road Crack Segmentation and Surface Feature Quantification under Complex Backgrounds. Remote Sens. 2023, 15, 1530. [Google Scholar] [CrossRef]
  23. Yamaguchi, T.; Mizutani, T. Quantitative Road Crack Evaluation by a U-Net Architecture Using Smartphone Images and Lidar Data. Comput.-Aided Civ. Infrastruct. Eng. 2024, 39, 963–982. [Google Scholar] [CrossRef]
  24. Zhang, W. Lidar-Based Road and Road-Edge Detection. In Proceedings of the 2010 IEEE Intelligent Vehicles Symposium, La Jolla, CA, USA, 21–24 June 2010; pp. 845–848. [Google Scholar]
  25. Ma, Y.; Zheng, Y.; Cheng, J.; Easa, S. Real-Time Visualization Method for Estimating 3D Highway Sight Distance Using LiDAR Data. J. Transp. Eng. Part Syst. 2019, 145, 04019006. [Google Scholar] [CrossRef]
  26. Huang, P.; Cheng, M.; Chen, Y.; Luo, H.; Wang, C.; Li, J. Traffic Sign Occlusion Detection Using Mobile Laser Scanning Point Clouds. IEEE Trans. Intell. Transp. Syst. 2017, 18, 2364–2376. [Google Scholar] [CrossRef]
  27. Koubaa, A. (Ed.) Robot Operating System (ROS); Studies in Computational Intelligence; Springer International Publishing: Cham, Switzerland, 2016; Volume 625, ISBN 978-3-319-26052-5. [Google Scholar]
  28. GitHub-RoboSense-LiDAR/Rslidar_sdk: RoboSense LiDAR SDK for ROS & ROS2. Available online: https://github.com/RoboSense-LiDAR/rslidar_sdk (accessed on 30 May 2024).
  29. IntelRealSense/Librealsense. Available online: https://github.com/IntelRealSense/librealsense (accessed on 30 May 2024).
  30. Zhang, W.; Qi, J.; Wan, P.; Wang, H.; Xie, D.; Wang, X.; Yan, G. An Easy-to-Use Airborne LiDAR Data Filtering Method Based on Cloth Simulation. Remote Sens. 2016, 8, 501. [Google Scholar] [CrossRef]
  31. Himmelsbach, M.; Hundelshausen, F.V.; Wuensche, H.-J. Fast Segmentation of 3D Point Clouds for Ground Vehicles. In Proceedings of the 2010 IEEE Intelligent Vehicles Symposium, La Jolla, CA, USA, 21–24 June 2010; pp. 560–565. [Google Scholar]
  32. Fischler, M.A.; Bolles, R.C. Random Sample Consensus: A Paradigm for Model Fitting with Applications To Image Analysis and Automated Cartography. Commun. ACM 1981, 24, 381–395. [Google Scholar] [CrossRef]
  33. Hoppe, H.; DeRose, T.; Duchamp, T.; McDonald, J.; Stuetzle, W. Surface Reconstruction from Unorganized Points. In Proceedings of the 19th Annual Conference on Computer Graphics and Interactive Techniques, Chicago, IL, USA, 27–31 July 1992; Association for Computing Machinery: New York, NY, USA, 1992; pp. 71–78. [Google Scholar]
  34. Lee, D.T.; Schachter, B.J. Two Algorithms for Constructing a Delaunay Triangulation. Int. J. Comput. Inf. Sci. 1980, 9, 219–242. [Google Scholar] [CrossRef]
  35. Kim, S.-H. Development of 3D Terrain Tools Which Improves a Picking Speed Using Cross Detection. J. Korea Contents Assoc. 2012, 12, 78–85. [Google Scholar] [CrossRef]
  36. Bai, H.; Yu, L.; Zhang, T.; Xie, D.; Cui, N. Locating Speed Limit Signs for Freeway Tunnel Entrance and Exit. In Proceedings of the International Conference on Transportation and Development 2018, Pittsburgh, PA, USA, 15–18 July 2018; American Society of Civil Engineers: Pittsburgh, PA, USA, 2018; pp. 377–386. [Google Scholar]
  37. Zhao, W.; Wang, L.; Ye, F.; Liu, H. Impacts of Environmental Luminosity around Driver on Distance of Visual Cognition. J. Transp. Inf. Saf. 2013, 6, 11–16. [Google Scholar]
  38. Ethz-Asl/Lidar_align. Available online: https://github.com/ethz-asl/lidar_align (accessed on 7 May 2024).
Remotesensing 16 02872 g001
Figure 1. Technical scheme.
Figure 1. Technical scheme.
Remotesensing 16 02872 g001
Remotesensing 16 02872 g002
Figure 2. Installation of LiDAR and camera.
Figure 2. Installation of LiDAR and camera.
Remotesensing 16 02872 g002
Remotesensing 16 02872 g003
Figure 3. Intensity information and ring information of LiDAR.
Figure 3. Intensity information and ring information of LiDAR.
Remotesensing 16 02872 g003
Remotesensing 16 02872 g004
Figure 4. Point cloud denoising effect.
Figure 4. Point cloud denoising effect.
Remotesensing 16 02872 g004
Remotesensing 16 02872 g005
Figure 5. Real photo and point cloud shape of signs.
Figure 5. Real photo and point cloud shape of signs.
Remotesensing 16 02872 g005
Remotesensing 16 02872 g006
Figure 6. Sign and plane point cloud recognized by the RANSAC algorithm.
Figure 6. Sign and plane point cloud recognized by the RANSAC algorithm.
Remotesensing 16 02872 g006
Remotesensing 16 02872 g007
Figure 7. Flowchart of the RANSAC algorithm.
Figure 7. Flowchart of the RANSAC algorithm.
Remotesensing 16 02872 g007
Remotesensing 16 02872 g008
Figure 8. Direction of surface normal vectors.
Figure 8. Direction of surface normal vectors.
Remotesensing 16 02872 g008
Remotesensing 16 02872 g009
Figure 9. Mesh reconstruction process.
Figure 9. Mesh reconstruction process.
Remotesensing 16 02872 g009
Remotesensing 16 02872 g010
Figure 10. Scanning consistency display.
Figure 10. Scanning consistency display.
Remotesensing 16 02872 g010
Remotesensing 16 02872 g011
Figure 11. Road conditions as shown in Google maps.
Figure 11. Road conditions as shown in Google maps.
Remotesensing 16 02872 g011
Remotesensing 16 02872 g012
Figure 12. Rosbag information.
Figure 12. Rosbag information.
Remotesensing 16 02872 g012
Remotesensing 16 02872 g013
Figure 13. Point cloud map after the slope calculation.
Figure 13. Point cloud map after the slope calculation.
Remotesensing 16 02872 g013
Remotesensing 16 02872 g014
Figure 14. Point cloud model before and after slope segmentation.
Figure 14. Point cloud model before and after slope segmentation.
Remotesensing 16 02872 g014
Remotesensing 16 02872 g015
Figure 15. Point cloud model before and after the intensity segmentation of the non-ground point cloud.
Figure 15. Point cloud model before and after the intensity segmentation of the non-ground point cloud.
Remotesensing 16 02872 g015
Remotesensing 16 02872 g016
Figure 16. Three planar point clouds separated by RANSAC.
Figure 16. Three planar point clouds separated by RANSAC.
Remotesensing 16 02872 g016
Remotesensing 16 02872 g017aRemotesensing 16 02872 g017b
Figure 17. Test points corresponding to the three signs.
Figure 17. Test points corresponding to the three signs.
Remotesensing 16 02872 g017aRemotesensing 16 02872 g017b
Remotesensing 16 02872 g018
Figure 18. Additional test points corresponding to sign 1.
Figure 18. Additional test points corresponding to sign 1.
Remotesensing 16 02872 g018
Remotesensing 16 02872 g019
Figure 19. Heat map of the occlusion rate for sign 1.
Figure 19. Heat map of the occlusion rate for sign 1.
Remotesensing 16 02872 g019
Remotesensing 16 02872 g020
Figure 20. Location of sign 1 and the tree point cloud.
Figure 20. Location of sign 1 and the tree point cloud.
Remotesensing 16 02872 g020
Remotesensing 16 02872 g021
Figure 21. Field situation of sign 1.
Figure 21. Field situation of sign 1.
Remotesensing 16 02872 g021
Remotesensing 16 02872 g022
Figure 22. Field situations of sign 2 and sign 3.
Figure 22. Field situations of sign 2 and sign 3.
Remotesensing 16 02872 g022
Remotesensing 16 02872 g023
Figure 23. Sign 2, virtual sign 3, and added test points.
Figure 23. Sign 2, virtual sign 3, and added test points.
Remotesensing 16 02872 g023
Remotesensing 16 02872 g024
Figure 24. Heat map of the occlusion rate for sign 2.
Figure 24. Heat map of the occlusion rate for sign 2.
Remotesensing 16 02872 g024
Table 1. Device parameter information.
Table 1. Device parameter information.
Equipment NameEquipment Parameters
Robosense Helios 1615
(RoboSense Technology Co., Ltd., Shenzhen, China)		# of lines: 32
Horizontal FOV: 360°
Vertical FOV: 70° (−55°~+15°)
Points per second: 576,000 pts/s (single return)/1,152,000 pts/s (dual return)
Intel® RealSense™ Depth Camera D435i
(Intel Corporation, Bangkok, Thailand)		Inertial sensor: Bosch BMI055 6-axis
IMU rate: 190 fps
RGB frame resolution: 1920 × 1080
RGB frame rate:3 0 fps
RGB sensor FOV (H × V): 69° × 42°
Table 2. Test results for the three signs.
Table 2. Test results for the three signs.
Sign 1Sign 2Sign 3
Point 131100
Point 234800
Point 3400
Point 4400
Point 5301
Point 6306
Point 7602
Point 8201
Point 9500
Point 10400
Point 1117500
Point 121500
Average73.3300.83
Table 3. Test results for sign 1.
Table 3. Test results for sign 1.
Number of
Obstructions		 Number of
Obstructions		 Number of
Obstructions		 Number of
Obstructions
Point 1311Point 126Point 233Point 346
Point 2348Point 133Point 245Point 352
Point 34Point 14389Point 254Point 365
Point 44Point 153Point 26415Point 374
Point 53Point 164Point 274Point 38175
Point 63Point 176Point 28161Point 3915
Point 75Point 185Point 294Point 4013
Point 84Point 195Point 30344Point 4114
Point 9340Point 20348Point 31345Point 42348
Point 10325Point 21348Point 3231Point 43340
Point 11326Point 22341Point 3328Point 4428
Table 4. Test results for sign 2.
Table 4. Test results for sign 2.
Number of
Obstructions		 Number of
Obstructions		 Number of
Obstructions		 Number of
Obstructions
Point 1114Point 1212Point 2319Point 3417
Point 25Point 13371Point 243Point 35402
Point 35Point 14430Point 25435Point 36275
Point 4298Point 1516Point 26286Point 375
Point 52Point 16416Point 2725Point 38447
Point 6117Point 173Point 287Point 3919
Point 7122Point 188Point 293Point 40306
Point 8301Point 192Point 303Point 413
Point 93Point 204Point 313Point 422
Point 104Point 215Point 32223Point 4311
Point 116Point 226Point 337Point 444
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Wang, Z.; Wang, X.; Li, J.; Yao, J. Evaluation of Traffic Sign Occlusion Rate Based on a 3D Point Cloud Space. Remote Sens. 2024, 16, 2872. https://doi.org/10.3390/rs16162872

AMA Style

Wang Z, Wang X, Li J, Yao J. Evaluation of Traffic Sign Occlusion Rate Based on a 3D Point Cloud Space. Remote Sensing. 2024; 16(16):2872. https://doi.org/10.3390/rs16162872

Chicago/Turabian Style

Wang, Ziqi, Xiaofei Wang, Jun Li, and Jiangbei Yao. 2024. "Evaluation of Traffic Sign Occlusion Rate Based on a 3D Point Cloud Space" Remote Sensing 16, no. 16: 2872. https://doi.org/10.3390/rs16162872

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop