[go: up one dir, main page]

 
 
Sign in to use this feature.

Years

Between: -

Subjects

remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline

Journals

remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline

Article Types

Countries / Regions

remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline

Search Results (5,471)

Search Parameters:
Keywords = point cloud

Order results
Result details
Results per page
Select all
Export citation of selected articles as:
17 pages, 17602 KiB  
Article
Enhancing Detection of Pedestrians in Low-Light Conditions by Accentuating Gaussian–Sobel Edge Features from Depth Maps
by Minyoung Jung and Jeongho Cho
Appl. Sci. 2024, 14(18), 8326; https://doi.org/10.3390/app14188326 (registering DOI) - 15 Sep 2024
Abstract
Owing to the low detection accuracy of camera-based object detection models, various fusion techniques with Light Detection and Ranging (LiDAR) have been attempted. This has resulted in improved detection of objects that are difficult to detect due to partial occlusion by obstacles or [...] Read more.
Owing to the low detection accuracy of camera-based object detection models, various fusion techniques with Light Detection and Ranging (LiDAR) have been attempted. This has resulted in improved detection of objects that are difficult to detect due to partial occlusion by obstacles or unclear silhouettes. However, the detection performance remains limited in low-light environments where small pedestrians are located far from the sensor or pedestrians have difficult-to-estimate shapes. This study proposes an object detection model that employs a Gaussian–Sobel filter. This filter combines Gaussian blurring, which suppresses the effects of noise, and a Sobel mask, which accentuates object features, to effectively utilize depth maps generated by LiDAR for object detection. The model performs independent pedestrian detection using the real-time object detection model You Only Look Once v4, based on RGB images obtained using a camera and depth maps preprocessed by the Gaussian–Sobel filter, and estimates the optimal pedestrian location using non-maximum suppression. This enables accurate pedestrian detection while maintaining a high detection accuracy even in low-light or external-noise environments, where object features and contours are not well defined. The test evaluation results demonstrated that the proposed method achieved at least 1–7% higher average precision than the state-of-the-art models under various environments. Full article
(This article belongs to the Special Issue Object Detection and Image Classification)
Show Figures

Figure 1

Figure 1
<p>Block diagram of the proposed multi-sensor-based detection model.</p>
Full article ">Figure 2
<p>Process for generating a depth map for image registration: (<b>a</b>) RGB image; (<b>b</b>) PCD projected on RGB image; (<b>c</b>) depth map.</p>
Full article ">Figure 3
<p>Preprocessing of depth maps using the Gaussian–Sobel filter: (<b>a</b>) depth map; (<b>b</b>) depth map after Gaussian filtering; (<b>c</b>) depth map after Gaussian–Sobel filtering; (<b>d</b>) depth map after Canny edge filtering.</p>
Full article ">Figure 3 Cont.
<p>Preprocessing of depth maps using the Gaussian–Sobel filter: (<b>a</b>) depth map; (<b>b</b>) depth map after Gaussian filtering; (<b>c</b>) depth map after Gaussian–Sobel filtering; (<b>d</b>) depth map after Canny edge filtering.</p>
Full article ">Figure 4
<p>Flowchart for non-maximum suppression (NMS).</p>
Full article ">Figure 5
<p>Comparison of pedestrian detection performance of the proposed model and similar models at 100% brightness: (<b>a</b>) depth map; (<b>b</b>) RGB + depth map; (<b>c</b>) Maragos and Pessoa [<a href="#B12-applsci-14-08326" class="html-bibr">12</a>]; (<b>d</b>) Deng [<a href="#B13-applsci-14-08326" class="html-bibr">13</a>]; (<b>e</b>) Ali and Clausi [<a href="#B14-applsci-14-08326" class="html-bibr">14</a>]; (<b>f</b>) proposed model.</p>
Full article ">Figure 5 Cont.
<p>Comparison of pedestrian detection performance of the proposed model and similar models at 100% brightness: (<b>a</b>) depth map; (<b>b</b>) RGB + depth map; (<b>c</b>) Maragos and Pessoa [<a href="#B12-applsci-14-08326" class="html-bibr">12</a>]; (<b>d</b>) Deng [<a href="#B13-applsci-14-08326" class="html-bibr">13</a>]; (<b>e</b>) Ali and Clausi [<a href="#B14-applsci-14-08326" class="html-bibr">14</a>]; (<b>f</b>) proposed model.</p>
Full article ">Figure 6
<p>Comparison of pedestrian detection performance of the proposed model and similar models at 40% brightness level: (<b>a</b>) depth map; (<b>b</b>) RGB + depth map; (<b>c</b>) Maragos and Pessoa [<a href="#B12-applsci-14-08326" class="html-bibr">12</a>]; (<b>d</b>) Deng [<a href="#B13-applsci-14-08326" class="html-bibr">13</a>]; (<b>e</b>) Ali and Clausi [<a href="#B14-applsci-14-08326" class="html-bibr">14</a>]; (<b>f</b>) proposed model.</p>
Full article ">Figure 6 Cont.
<p>Comparison of pedestrian detection performance of the proposed model and similar models at 40% brightness level: (<b>a</b>) depth map; (<b>b</b>) RGB + depth map; (<b>c</b>) Maragos and Pessoa [<a href="#B12-applsci-14-08326" class="html-bibr">12</a>]; (<b>d</b>) Deng [<a href="#B13-applsci-14-08326" class="html-bibr">13</a>]; (<b>e</b>) Ali and Clausi [<a href="#B14-applsci-14-08326" class="html-bibr">14</a>]; (<b>f</b>) proposed model.</p>
Full article ">Figure 7
<p>Comparison of the pedestrian detection performance of the proposed model and similar models at 40% brightness and 0.5% noise level: (<b>a</b>) depth map; (<b>b</b>) RGB + depth map; (<b>c</b>) Maragos and Pessoa [<a href="#B12-applsci-14-08326" class="html-bibr">12</a>]; (<b>d</b>) Deng [<a href="#B13-applsci-14-08326" class="html-bibr">13</a>]; (<b>e</b>) Ali and Clausi [<a href="#B14-applsci-14-08326" class="html-bibr">14</a>]; (<b>f</b>) proposed model.</p>
Full article ">Figure 7 Cont.
<p>Comparison of the pedestrian detection performance of the proposed model and similar models at 40% brightness and 0.5% noise level: (<b>a</b>) depth map; (<b>b</b>) RGB + depth map; (<b>c</b>) Maragos and Pessoa [<a href="#B12-applsci-14-08326" class="html-bibr">12</a>]; (<b>d</b>) Deng [<a href="#B13-applsci-14-08326" class="html-bibr">13</a>]; (<b>e</b>) Ali and Clausi [<a href="#B14-applsci-14-08326" class="html-bibr">14</a>]; (<b>f</b>) proposed model.</p>
Full article ">
20 pages, 3178 KiB  
Article
Comparative Studies of the Measurement Accuracy of Basic Gear Wheel Parameters
by Agata Świerek, Paweł Nowakowski, Lidia Marciniak-Podsadna and Piotr Góral
Metrology 2024, 4(3), 469-488; https://doi.org/10.3390/metrology4030029 (registering DOI) - 15 Sep 2024
Abstract
This article presents the results of comparative tests of gear wheels based on the contactless and contact measurement methods. Measurements of gear wheels in accuracy classes containing deviations within the range of measurement capabilities of the GOM ATOS II optical scanner are proposed. [...] Read more.
This article presents the results of comparative tests of gear wheels based on the contactless and contact measurement methods. Measurements of gear wheels in accuracy classes containing deviations within the range of measurement capabilities of the GOM ATOS II optical scanner are proposed. Elementary deviations of teeth related to the involute profile were analyzed. In undertaking a non-contact gear measurement using the GOM ATOS II scanner, a new method was developed to extract parameters from the point cloud, which were then used to determine the total deviation of the profile. The results of the measurements obtained using the non-contact method were compared with the results obtained with the contact method using the Wenzel WGT 600 four-axis machine specialized for measuring gear wheels. Measurement uncertainty was also compared. The result of the conducted tests is the comparability of results for gear wheels made in accuracy class 10 according to DIN 3961/62. The proposed non-contact method shows the possibility of using it to measure gear wheels commonly used in agricultural and construction machines. The information obtained from comparing the measurement model and the nominal wheel model provides additional information about surface defects of the part which result from the production and operation process. Full article
Show Figures

Figure 1

Figure 1
<p>Gears: (<b>a</b>) brand-new teeth, (<b>b</b>) used teeth. Ring 1: number of teeth 25; module 4; pressure angle 20°; tooth width 19.6 mm. Ring 2: number of teeth 30; module 4; pressure angle 20°; tooth width 21.5 mm.</p>
Full article ">Figure 2
<p>(<b>a</b>) GOM ATOS II optical coordinate scanner, (<b>b</b>) 3D surface model of the gear wheel obtained after polygonization.</p>
Full article ">Figure 3
<p>(<b>a</b>) View of the nominal CAD model of the gear in CATIA, (<b>b</b>) view of the map of deviations of the surface model from the nominal values.</p>
Full article ">Figure 4
<p>Measuring machine: (<b>a</b>) test stand, (<b>b</b>) with a gear wheel prepared for contact measurements.</p>
Full article ">Figure 5
<p>View of the scale of deviations of the actual model from the nominal model as an example of a non-contact measurement.</p>
Full article ">Figure 6
<p>View of the scale of deviations of the real model from the nominal model.</p>
Full article ">Figure 7
<p>A view of a fragment of the protocol from contact measurements with the total profile deviation for selected teeth.</p>
Full article ">
21 pages, 13544 KiB  
Article
Three-Dimensional Reconstruction of Forest Scenes with Tree–Shrub–Grass Structure Using Airborne LiDAR Point Cloud
by Duo Xu, Xuebo Yang, Cheng Wang, Xiaohuan Xi and Gaofeng Fan
Forests 2024, 15(9), 1627; https://doi.org/10.3390/f15091627 (registering DOI) - 15 Sep 2024
Viewed by 133
Abstract
Fine three-dimensional (3D) reconstruction of real forest scenes can provide a reference for forestry digitization and forestry resource management applications. Airborne LiDAR technology can provide valuable data for large-area forest scene reconstruction. This paper proposes a 3D reconstruction method for complex forest scenes [...] Read more.
Fine three-dimensional (3D) reconstruction of real forest scenes can provide a reference for forestry digitization and forestry resource management applications. Airborne LiDAR technology can provide valuable data for large-area forest scene reconstruction. This paper proposes a 3D reconstruction method for complex forest scenes with trees, shrubs, and grass, based on airborne LiDAR point clouds. First, forest vertical distribution characteristics are used to segment tree, shrub, and ground–grass points from an airborne LiDAR point cloud. For ground–grass points, a ground–grass grid model is constructed. For tree points, a method based on hierarchical canopy point fitting is proposed to construct a trunk model, and a crown model is constructed with the 3D α-shape algorithm. For shrub points, a shrub model is directly constructed based on the 3D α-shape algorithm. Finally, tree, shrub, and ground–grass models are spatially combined to achieve the reconstruction of real forest scenes. Taking six forest plots located in Hebei, Yunnan, and Guangxi provinces in China and Baden-Württemberg in Germany as study areas, experimental results show that the accuracy of individual tree segmentation reaches 87.32%, the accuracy of shrub segmentation reaches 60.00%, the height accuracy of the grass model is evaluated with an RMSE < 0.15 m, the volume accuracy of shrub and tree models is assessed with R2 > 0.848 and R2 > 0.904, respectively. Furthermore, we compared the model constructed in this study with simplified point cloud and voxel models. The results demonstrate that the proposed modeling approach can meet the demand for the high-accuracy and lightweight modeling of large-area forest scenes. Full article
Show Figures

Figure 1

Figure 1
<p>Study area location and ALS point cloud.</p>
Full article ">Figure 2
<p>ALS and TLS tree and shrub point cloud. (<b>a</b>) Automatically extracted ALS tree point cloud. (<b>b</b>) Manually annotated TLS tree point cloud. (<b>c</b>) Automatically extracted ALS shrub point cloud. (<b>d</b>) Manually annotated TLS shrub point cloud.</p>
Full article ">Figure 3
<p>Technical route for 3D forest scene modeling based on ALS point cloud.</p>
Full article ">Figure 4
<p>Ground–grass model.</p>
Full article ">Figure 5
<p>Diagram of separation of crown trunk points. (<b>a</b>) Individual tree point cloud. (<b>b</b>) Vertical distribution histogram of point cloud count. (<b>c</b>) Vertical distribution histogram of point cloud dispersion.</p>
Full article ">Figure 6
<p>Modeling of trees with trunk points.</p>
Full article ">Figure 7
<p>Modeling of trees without trunk points.</p>
Full article ">Figure 8
<p>Shrub model construction.</p>
Full article ">Figure 9
<p>Height validation results of grass models.</p>
Full article ">Figure 10
<p>Volume validation results of shrub models.</p>
Full article ">Figure 11
<p>Height validation results of tree models.</p>
Full article ">Figure 12
<p>Volume validation results of tree crown models.</p>
Full article ">Figure 13
<p>Three-dimensional forest scene models.</p>
Full article ">
20 pages, 2929 KiB  
Article
R-LVIO: Resilient LiDAR-Visual-Inertial Odometry for UAVs in GNSS-denied Environment
by Bing Zhang, Xiangyu Shao, Yankun Wang, Guanghui Sun and Weiran Yao
Drones 2024, 8(9), 487; https://doi.org/10.3390/drones8090487 (registering DOI) - 14 Sep 2024
Viewed by 179
Abstract
In low-altitude, GNSS-denied scenarios, Unmanned aerial vehicles (UAVs) rely on sensor fusion for self-localization. This article presents a resilient multi-sensor fusion localization system that integrates light detection and ranging (LiDAR), cameras, and inertial measurement units (IMUs) to achieve state estimation for UAVs. To [...] Read more.
In low-altitude, GNSS-denied scenarios, Unmanned aerial vehicles (UAVs) rely on sensor fusion for self-localization. This article presents a resilient multi-sensor fusion localization system that integrates light detection and ranging (LiDAR), cameras, and inertial measurement units (IMUs) to achieve state estimation for UAVs. To address challenging environments, especially unstructured ones, IMU predictions are used to compensate for pose estimation in the visual and LiDAR components. Specifically, the accuracy of IMU predictions is enhanced by increasing the correction frequency of IMU bias through data integration from the LiDAR and visual modules. To reduce the impact of random errors and measurement noise in LiDAR points on visual depth measurement, cross-validation of visual feature depth is performed using reprojection error to eliminate outliers. Additionally, a structure monitor is introduced to switch operation modes in hybrid point cloud registration, ensuring accurate state estimation in both structured and unstructured environments. In unstructured scenes, a geometric primitive capable of representing irregular planes is employed for point-to-surface registration, along with a novel pose-solving method to estimate the UAV’s pose. Both private and public datasets collected by UAVs validate the proposed system, proving that it outperforms state-of-the-art algorithms by at least 12.6%. Full article
18 pages, 1260 KiB  
Article
Technical Evaluation of a Stand-Alone Photovoltaic Heat Pump Dryer without Batteries
by Antonio Quijano, Celena Lorenzo, Antonio Berlanga and Luis Narvarte
Energies 2024, 17(18), 4612; https://doi.org/10.3390/en17184612 (registering DOI) - 14 Sep 2024
Viewed by 178
Abstract
This paper presents the results of the technical validation of an innovative prototype for drying alfalfa bales. It is based on a 4.1 kW Heat Pump (HP) that uses an advanced technology (optimized for extracting the humidity from the air) and is directly [...] Read more.
This paper presents the results of the technical validation of an innovative prototype for drying alfalfa bales. It is based on a 4.1 kW Heat Pump (HP) that uses an advanced technology (optimized for extracting the humidity from the air) and is directly powered by a 6.6 kWp PV generator without grid or batteries support. The main technical challenges of this work were managing solar irradiance fluctuations due to cloud-passing and achieving good drying efficiency. The prototype has been validated for two consecutive drying campaigns in La Rioja (Spain). There were no abrupt stops generated by cloud-passing. The PRPV, which evaluates the performance of the PV system only during the periods when the PV energy can be used by the HP unit, presented values of 0.82 and 0.85, comparable to a well-performing grid-connected PV system. Although the bales’ initial relative humidity (RH) ranged from 18 to 30%, all but one of them presented a final RH below 16%, which is the limit point to avoid fermentation. The drying times ranged from 1 to 5 h, and the specific energy consumption per liter of water extracted, from 0.7 to 1.46 kWh/l. These values are comparable to traditional diesel and grid-powered systems. It is worth noting that the agricultural drying market represented USD 1.7 billion in 2023. Full article
(This article belongs to the Collection Featured Papers in Solar Energy and Photovoltaic Systems Section)
20 pages, 21023 KiB  
Article
Deformation-Adapted Spatial Domain Filtering Algorithm for UAV Mining Subsidence Monitoring
by Jianfeng Zha, Penglong Miao, Hukai Ling, Minghui Yu, Bo Sun, Chongwu Zhong and Guowei Hao
Sustainability 2024, 16(18), 8039; https://doi.org/10.3390/su16188039 (registering DOI) - 14 Sep 2024
Viewed by 235
Abstract
Underground coal mining induces surface subsidence, leading to disasters such as damage to buildings and infrastructure, landslides, and surface water accumulation. Preventing and controlling disasters in subsidence areas and reutilizing land depend on understanding subsidence regularity and obtaining surface subsidence monitoring data. These [...] Read more.
Underground coal mining induces surface subsidence, leading to disasters such as damage to buildings and infrastructure, landslides, and surface water accumulation. Preventing and controlling disasters in subsidence areas and reutilizing land depend on understanding subsidence regularity and obtaining surface subsidence monitoring data. These data are crucial for the reutilization of regional land resources and disaster prevention and control. Subsidence hazards are also a key constraint to mine development. Recently, with the rapid advancement of UAV technology, the use of UAV photogrammetry for surface subsidence monitoring has become a significant trend in this field. The periodic imagery data quickly acquired by UAV are used to construct DEM through point cloud filtering. Then, surface subsidence information is obtained by differencing DEM from different periods. However, due to the accuracy limitations inherent in UAV photogrammetry, the subsidence data obtained through this method are characterized by errors, making it challenging to achieve high-precision ground surface subsidence monitoring. Therefore, this paper proposes a spatial domain filtering algorithm for UAV photogrammetry combined with surface deformation caused by coal mining based on the surface subsidence induced by coal mining and combined with the characteristics of the surface change. This algorithm significantly reduces random error in the differential DEM, achieving high-precision ground subsidence monitoring using UAV. Simulation and field test results show that the surface subsidence elevation errors obtained in the simulation tests are reduced by more than 50% compared to conventional methods. In field tests, this method reduced surface subsidence elevation errors by 39%. The monitoring error for surface subsidence was as low as 8 mm compared to leveling survey data. This method offers a new technical pathway for high-precision surface subsidence monitoring in mining areas using UAV photogrammetry. Full article
(This article belongs to the Section Pollution Prevention, Mitigation and Sustainability)
Show Figures

Figure 1

Figure 1
<p>Location of the study area, working face, and observation line.</p>
Full article ">Figure 2
<p>Histogram of error distribution of conventional methods.</p>
Full article ">Figure 3
<p>Schematic diagram of surface inclination calculation. The arrows in the figure refer to the direction in which the sampling point is calculated along the x or y direction.</p>
Full article ">Figure 4
<p>Schematic diagram of mean filtering in spatial domain. The blue dots represent the four corners of the grid corresponding to each sampling point; the blue arrows point to the images representing the results after processing using the algorithm in this paper.</p>
Full article ">Figure 5
<p>Algorithm flow chart. The red dots in the graph represent the direction of the calculation along the x or y direction.</p>
Full article ">Figure 6
<p>Distribution of error intervals of sampling points after different grid treatments.</p>
Full article ">Figure 7
<p>Error proportion diagram under different grid intervals.</p>
Full article ">Figure 8
<p>Mean error of sampling points under different grid intervals.</p>
Full article ">Figure 9
<p>Road settlement map measured by UAV based on original data.</p>
Full article ">Figure 10
<p>Road settlement map obtained by spatial domain filtering.</p>
Full article ">Figure 11
<p>Comparison of path changes for different grid sizes.</p>
Full article ">Figure 12
<p>Surface settlement maps obtained through the proposed algorithm. (<b>a</b>) Phase II. (<b>b</b>) Phase III. (<b>c</b>) Phase IV. (<b>d</b>) Phase V. (<b>e</b>) Phase VI. (<b>f</b>) Phase VII.</p>
Full article ">Figure 12 Cont.
<p>Surface settlement maps obtained through the proposed algorithm. (<b>a</b>) Phase II. (<b>b</b>) Phase III. (<b>c</b>) Phase IV. (<b>d</b>) Phase V. (<b>e</b>) Phase VI. (<b>f</b>) Phase VII.</p>
Full article ">Figure 13
<p>Extracting the isoline map of surface subsidence by UAV. (<b>a</b>) Phase II isoline map of surface subsidence. (<b>b</b>) Phase III isoline map of surface subsidence. (<b>c</b>) Phase IV isoline map of surface subsidence. (<b>d</b>) Phase V isoline map of surface subsidence. (<b>e</b>) Phase VI isoline map of surface subsidence. (<b>f</b>) Phase VII isoline map of surface subsidence.</p>
Full article ">Figure 13 Cont.
<p>Extracting the isoline map of surface subsidence by UAV. (<b>a</b>) Phase II isoline map of surface subsidence. (<b>b</b>) Phase III isoline map of surface subsidence. (<b>c</b>) Phase IV isoline map of surface subsidence. (<b>d</b>) Phase V isoline map of surface subsidence. (<b>e</b>) Phase VI isoline map of surface subsidence. (<b>f</b>) Phase VII isoline map of surface subsidence.</p>
Full article ">Figure 14
<p>Accuracy assessment for different mining phases. (<b>a</b>) phase IV direction E-W. (<b>b</b>) phase IV direction N-S. (<b>c</b>) phase V direction E-W. (<b>d</b>) phase V direction N-S. (<b>e</b>) phase VI direction E-W. (<b>f</b>) phase VI direction N-S.</p>
Full article ">
15 pages, 2064 KiB  
Article
Research on the Depth Image Reconstruction Algorithm Using the Two-Dimensional Kaniadakis Entropy Threshold
by Xianhui Yang, Jianfeng Sun, Le Ma, Xin Zhou, Wei Lu and Sining Li
Sensors 2024, 24(18), 5950; https://doi.org/10.3390/s24185950 - 13 Sep 2024
Viewed by 250
Abstract
The photon-counting light laser detection and ranging (LiDAR), especially the Geiger mode avalanche photon diode (Gm-APD) LiDAR, can obtain three-dimensional images of the scene, with the characteristics of single-photon sensitivity, but the background noise limits the imaging quality of the laser radar. In [...] Read more.
The photon-counting light laser detection and ranging (LiDAR), especially the Geiger mode avalanche photon diode (Gm-APD) LiDAR, can obtain three-dimensional images of the scene, with the characteristics of single-photon sensitivity, but the background noise limits the imaging quality of the laser radar. In order to solve this problem, a depth image estimation method based on a two-dimensional (2D) Kaniadakis entropy thresholding method is proposed which transforms a weak signal extraction problem into a denoising problem for point cloud data. The characteristics of signal peak aggregation in the data and the spatio-temporal correlation features between target image elements in the point cloud-intensity data are exploited. Through adequate simulations and outdoor target-imaging experiments under different signal-to-background ratios (SBRs), the effectiveness of the method under low signal-to-background ratio conditions is demonstrated. When the SBR is 0.025, the proposed method reaches a target recovery rate of 91.7%, which is better than the existing typical methods, such as the Peak-picking method, Cross-Correlation method, and the sparse Poisson intensity reconstruction algorithm (SPIRAL), which achieve a target recovery rate of 15.7%, 7.0%, and 18.4%, respectively. Additionally, comparing with the SPIRAL, the reconstruction recovery ratio is improved by 73.3%. The proposed method greatly improves the integrity of the target under high-background-noise environments and finally provides a basis for feature extraction and target recognition. Full article
(This article belongs to the Special Issue Application of LiDAR Remote Sensing and Mapping)
Show Figures

Figure 1

Figure 1
<p>Flowchart of the proposed algorithm.</p>
Full article ">Figure 2
<p>(<b>a</b>) Distribution of the probability density of the signal and noise when the signal position is at the 300th time bin. (<b>b</b>) Counting histogram of 0.1 s data with signal and noise.</p>
Full article ">Figure 3
<p>Signal detection rate for extracting different numbers of wave peaks under different SBRs.</p>
Full article ">Figure 4
<p>(<b>a</b>) The standard depth image. (<b>b</b>) The standard intensity image.</p>
Full article ">Figure 5
<p>Depth images obtained using different methods of reconstruction. (<b>a</b>–<b>d</b>) SBR is 0.01; (<b>e</b>–<b>h</b>) SBR is 0.02; (<b>i</b>–<b>l</b>) SBR is 0.04; (<b>m</b>–<b>p</b>) SBR is 0.06; (<b>q</b>–<b>t</b>) SBR is 0.08; (<b>a</b>,<b>e</b>,<b>i</b>,<b>m</b>,<b>q</b>) Peak-picking method; (<b>b</b>,<b>f</b>,<b>j</b>,<b>n</b>,<b>r</b>) Cross-Correlation method; (<b>c</b>,<b>g</b>,<b>k</b>,<b>o</b>,<b>s</b>) SPIRAL method; (<b>d</b>,<b>h</b>,<b>l</b>,<b>p</b>,<b>t</b>) proposed method.</p>
Full article ">Figure 6
<p>The relationship between the SBR of the scene and the SNR of the reconstructed image using different methods.</p>
Full article ">Figure 7
<p>The photos of the experimental targets.</p>
Full article ">Figure 8
<p>The number of echo photons/pixel of the scenes. (<b>a</b>) Building at a distance of 730 m with an SBR of 0.078. (<b>b</b>) Building at a distance of 730 m with an SBR of 0.053. (<b>c</b>) Building at a distance of 730 m with an SBR of 0.031. (<b>d</b>) Building at a distance of 1400 m with an SBR of 0.031. (<b>e</b>) Building at a distance of 1400 m with an SBR of 0.025.</p>
Full article ">Figure 9
<p>Truth depth images and depth images obtained using different methods of the 730 m building. (<b>a</b>–<b>e</b>) SBR is 0.078; (<b>f</b>–<b>j</b>) SBR is 0.053; (<b>k</b>–<b>o</b>) SBR is 0.031; (<b>a</b>,<b>f</b>,<b>k</b>) truth depth image; (<b>b</b>,<b>g</b>,<b>l</b>) Peak-picking method; (<b>c</b>,<b>h</b>,<b>m</b>) Cross-Correlation method; (<b>d</b>,<b>i</b>,<b>n</b>) SPIRAL method; (<b>e</b>,<b>j</b>,<b>o</b>) proposed method.</p>
Full article ">Figure 10
<p>Truth depth images and depth images obtained using different methods of the 1400 m building. (<b>a</b>–<b>e</b>) SBR is 0.031; (<b>f</b>–<b>j</b>) SBR is 0.025; (<b>a</b>,<b>f</b>) truth depth image; (<b>b</b>,<b>g</b>) Peak-picking method; (<b>c</b>,<b>h</b>) Cross-Correlation method; (<b>d</b>,<b>i</b>) SPIRAL method; (<b>e</b>,<b>j</b>) proposed method.</p>
Full article ">Figure 11
<p>Intensity images with different signal-to-background ratios. (<b>a</b>) A 730 m building intensity image with an SBR of 0.078; (<b>b</b>) a 730 m building intensity image with an SBR of 0.053; (<b>c</b>) a 730 m building intensity image with an SBR of 0.031; (<b>d</b>) a 1400 m building intensity image with an SBR of 0.031; (<b>e</b>) a 1400 m building intensity image with an SBR of 0.025.</p>
Full article ">
31 pages, 73552 KiB  
Article
Enhancing 3D Rock Localization in Mining Environments Using Bird’s-Eye View Images from the Time-of-Flight Blaze 101 Camera
by John Kern, Reinier Rodriguez-Guillen, Claudio Urrea and Yainet Garcia-Garcia
Technologies 2024, 12(9), 162; https://doi.org/10.3390/technologies12090162 - 12 Sep 2024
Viewed by 312
Abstract
The mining industry faces significant challenges in production costs, environmental protection, and worker safety, necessitating the development of autonomous systems. This study presents the design and implementation of a robust rock centroid localization system for mining robotic applications, particularly rock-breaking hammers. The system [...] Read more.
The mining industry faces significant challenges in production costs, environmental protection, and worker safety, necessitating the development of autonomous systems. This study presents the design and implementation of a robust rock centroid localization system for mining robotic applications, particularly rock-breaking hammers. The system comprises three phases: assembly, data acquisition, and data processing. Environmental sensing was accomplished using a Basler Blaze 101 three-dimensional (3D) Time-of-Flight (ToF) camera. The data processing phase incorporated advanced algorithms, including Bird’s-Eye View (BEV) image conversion and You Only Look Once (YOLO) v8x-Seg instance segmentation. The system’s performance was evaluated using a comprehensive dataset of 627 point clouds, including samples from real mining environments. The system achieved efficient processing times of approximately 5 s. Segmentation accuracy was evaluated using the Intersection over Union (IoU), reaching 95.10%. Localization precision was measured by the Euclidean distance in the XY plane (EDXY), achieving 0.0128 m. The normalized error (enorm) on the X and Y axes did not exceed 2.3%. Additionally, the system demonstrated high reliability with R2 values close to 1 for the X and Y axes, and maintained performance under various lighting conditions and in the presence of suspended particles. The Mean Absolute Error (MAE) in the Z axis was 0.0333 m, addressing challenges in depth estimation. A sensitivity analysis was conducted to assess the model’s robustness, revealing consistent performance across brightness and contrast variations, with an IoU ranging from 92.88% to 96.10%, while showing greater sensitivity to rotations. Full article
Show Figures

Figure 1

Figure 1
<p>Rock-breaker hammers.</p>
Full article ">Figure 2
<p>YOLO v8-Seg architecture [<a href="#B39-technologies-12-00162" class="html-bibr">39</a>].</p>
Full article ">Figure 3
<p>System architecture.</p>
Full article ">Figure 4
<p>Point clouds based on sensor placement. (<b>a</b>) Angle between sensors less than 30°. (<b>b</b>) Angle between sensors approximately between 120° and 190°.</p>
Full article ">Figure 5
<p>Mineralogical and morphological characteristics. (<b>a</b>) “La Patagua” mine. (<b>b</b>) Rock fragment displaying fractures and calcite veinlets.</p>
Full article ">Figure 6
<p>Created database. (<b>a</b>) Without overlap. (<b>b</b>) With overlap. (<b>c</b>) High lighting. (<b>d</b>) Suspended particles.</p>
Full article ">Figure 7
<p>Labeling distribution.</p>
Full article ">Figure 8
<p>Data augmentation. (<b>a</b>) Blur to 2 pixels. (<b>b</b>) Brightness to 15%. (<b>c</b>) Exposure to −5%.</p>
Full article ">Figure 9
<p>Centroid localization algorithm.</p>
Full article ">Figure 10
<p>Point cloud preprocessing.</p>
Full article ">Figure 11
<p>Point cloud registration.</p>
Full article ">Figure 12
<p>RANSAC.</p>
Full article ">Figure 13
<p>BEV images converted from point clouds.</p>
Full article ">Figure 14
<p>Results from training the YOLO v8x-Seg model.</p>
Full article ">Figure 15
<p>Postprocessing. (<b>a</b>) Var 1. (<b>b</b>) Var 2.</p>
Full article ">Figure 16
<p>IoU metric results by image. (<b>a</b>) Without overlap. (<b>b</b>) With overlap.</p>
Full article ">Figure 17
<p><math display="inline"><semantics> <msup> <mi>R</mi> <mn>2</mn> </msup> </semantics></math> metrics and <math display="inline"><semantics> <mrow> <mi>m</mi> <mi>a</mi> <mi>x</mi> <mo>_</mo> <mi>e</mi> <mi>r</mi> <mi>r</mi> <mi>o</mi> <mi>r</mi> </mrow> </semantics></math> without overlap. (<b>a</b>) N_S_N_O_V1. (<b>b</b>) N_S_N_O_V2. (<b>c</b>) S_N_O_V1. (<b>d</b>) S_N_O_V2.</p>
Full article ">Figure 18
<p><math display="inline"><semantics> <msup> <mi>R</mi> <mn>2</mn> </msup> </semantics></math> metrics and <math display="inline"><semantics> <mrow> <mi>m</mi> <mi>a</mi> <mi>x</mi> <mo>_</mo> <mi>e</mi> <mi>r</mi> <mi>r</mi> <mi>o</mi> <mi>r</mi> </mrow> </semantics></math> with overlap. (<b>a</b>) N_S_O_V1. (<b>b</b>) N_S_O_V2. (<b>c</b>) S_O_V1. (<b>d</b>) S_O_V2.</p>
Full article ">Figure 19
<p>Metrics used to assess the location of the rock centroid by image. (<b>a</b>) <math display="inline"><semantics> <mrow> <mi>M</mi> <mi>A</mi> <mi>E</mi> </mrow> </semantics></math> without overlap. (<b>b</b>) <math display="inline"><semantics> <mrow> <mi>M</mi> <mi>A</mi> <mi>E</mi> </mrow> </semantics></math> with overlap. (<b>c</b>) <math display="inline"><semantics> <msub> <mi>e</mi> <mi>norm</mi> </msub> </semantics></math> without overlap. (<b>d</b>) <math display="inline"><semantics> <msub> <mi>e</mi> <mi>norm</mi> </msub> </semantics></math> with overlap. (<b>e</b>) <math display="inline"><semantics> <mrow> <mi>E</mi> <msub> <mi>D</mi> <mi>XY</mi> </msub> </mrow> </semantics></math> without overlap. (<b>f</b>) <math display="inline"><semantics> <mrow> <mi>E</mi> <msub> <mi>D</mi> <mi>XY</mi> </msub> </mrow> </semantics></math> with overlap.</p>
Full article ">Figure 20
<p>Examples of rock center localization in the image and centroid in the point cloud. Blue dots represent the ground truth, and red crosses represent the prediction. (<b>a</b>) Point cloud representation in the CloudCompare software. (<b>b</b>) Instance segmentation in a BEV image using YOLO v8x-Seg. (<b>c</b>) Localization in a BEV image. (<b>d</b>) Localization in the point cloud using the Open3D library.</p>
Full article ">
29 pages, 9403 KiB  
Article
DIO-SLAM: A Dynamic RGB-D SLAM Method Combining Instance Segmentation and Optical Flow
by Lang He, Shiyun Li, Junting Qiu and Chenhaomin Zhang
Sensors 2024, 24(18), 5929; https://doi.org/10.3390/s24185929 - 12 Sep 2024
Viewed by 316
Abstract
Feature points from moving objects can negatively impact the accuracy of Visual Simultaneous Localization and Mapping (VSLAM) algorithms, while detection or semantic segmentation-based VSLAM approaches often fail to accurately determine the true motion state of objects. To address this challenge, this paper introduces [...] Read more.
Feature points from moving objects can negatively impact the accuracy of Visual Simultaneous Localization and Mapping (VSLAM) algorithms, while detection or semantic segmentation-based VSLAM approaches often fail to accurately determine the true motion state of objects. To address this challenge, this paper introduces DIO-SLAM: Dynamic Instance Optical Flow SLAM, a VSLAM system specifically designed for dynamic environments. Initially, the detection thread employs YOLACT (You Only Look At CoefficienTs) to distinguish between rigid and non-rigid objects within the scene. Subsequently, the optical flow thread estimates optical flow and introduces a novel approach to capture the optical flow of moving objects by leveraging optical flow residuals. Following this, an optical flow consistency method is implemented to assess the dynamic nature of rigid object mask regions, classifying them as either moving or stationary rigid objects. To mitigate errors caused by missed detections or motion blur, a motion frame propagation method is employed. Lastly, a dense mapping thread is incorporated to filter out non-rigid objects using semantic information, track the point clouds of rigid objects, reconstruct the static background, and store the resulting map in an octree format. Experimental results demonstrate that the proposed method surpasses current mainstream dynamic VSLAM techniques in both localization accuracy and real-time performance. Full article
(This article belongs to the Special Issue Sensors and Algorithms for 3D Visual Analysis and SLAM)
Show Figures

Figure 1

Figure 1
<p>Performance of the traditional ORB-SLAM3 algorithm in highly dynamic environments. (<b>a</b>) Image of the highly dynamic scene. (<b>b</b>) Feature point extraction in the highly dynamic scene, where yellow boxes indicate moving objects and dynamic feature points are marked in red. (<b>c</b>) Comparison between the estimated camera pose and the ground truth camera pose. (<b>d</b>) Reconstruction results of dense mapping.</p>
Full article ">Figure 2
<p>The overall system framework of DIO-SLAM. Key innovations are highlighted in red font, while the original ORB-SLAM3 framework is represented by unfilled boxes. (<b>a</b>) Detection thread, represented by green boxes. (<b>b</b>) Optical flow thread, represented by blue boxes. (<b>c</b>) Dynamic feature point filtering module, which is composed of both the detection and optical flow threads. (<b>d</b>) Independent dense mapping thread.</p>
Full article ">Figure 3
<p>Instance segmentation results and non-rigid object mask extraction. (<b>a</b>) RGB frame used for segmentation. (<b>b</b>) Instance segmentation output.</p>
Full article ">Figure 4
<p>Separation of non-rigid and rigid object masks based on semantic information.</p>
Full article ">Figure 5
<p>Optical flow network inputs and output. (<b>a</b>) Frame <math display="inline"><semantics> <mrow> <mi>n</mi> <mo>−</mo> <mn>1</mn> </mrow> </semantics></math>. (<b>b</b>) Frame <math display="inline"><semantics> <mi>n</mi> </semantics></math>. (<b>c</b>) Dense optical flow.</p>
Full article ">Figure 6
<p>Optical flow changes between adjacent frames.</p>
Full article ">Figure 7
<p>Iterative removal of camera self-motion flow using optical flow residuals. (<b>a</b>) Original dense optical flow. (<b>b</b>) Number of iterations = 5. (<b>c</b>) Number of iterations = 7. (<b>d</b>) Number of iterations = 9.</p>
Full article ">Figure 8
<p>Optical flow consistency for determining the moving rigid object region.</p>
Full article ">Figure 9
<p>Motion frame propagation.</p>
Full article ">Figure 10
<p>Effect of dynamic feature point removal. The colored areas in the figure depict the optical flow of moving rigid objects, while the green areas indicate the final extracted feature points. The feature points of non-rigid objects, such as the human body, are removed in all scenes. (<b>a</b>,<b>b</b>) A chair is being dragged, with its feature points being removed. (<b>c</b>,<b>d</b>) Hitting a balloon, where the feature points on the balloon are removed. (<b>e</b>) The box is stationary, and the feature points are normally extracted. (<b>f</b>) The box is being moved, with its feature points removed. (<b>g</b>,<b>h</b>) The box is put down, and its feature points are restored.</p>
Full article ">Figure 11
<p>Absolute trajectory error and relative pose error of fr3_walkingx_xyz.</p>
Full article ">Figure 12
<p>Absolute trajectory error and relative pose error of fr3_walking_static.</p>
Full article ">Figure 13
<p>Absolute trajectory error and relative pose error of fr3_walking_rpy.</p>
Full article ">Figure 14
<p>Absolute trajectory error and relative pose error of fr3_walking_halfsphere.</p>
Full article ">Figure 15
<p>Absolute trajectory error and relative pose error of fr3_sitting_static.</p>
Full article ">Figure 16
<p>Dense point cloud reconstruction. (<b>a</b>) RGB frame, dense point cloud, and octree map of the fr3_walking_xyz sequence. (<b>b</b>) RGB frame, dense point cloud, and octree map of the moving_nonobstructing_box sequence.</p>
Full article ">Figure 16 Cont.
<p>Dense point cloud reconstruction. (<b>a</b>) RGB frame, dense point cloud, and octree map of the fr3_walking_xyz sequence. (<b>b</b>) RGB frame, dense point cloud, and octree map of the moving_nonobstructing_box sequence.</p>
Full article ">Figure 17
<p>Point cloud error heatmaps. (<b>a</b>) kt0 sequence. (<b>b</b>) kt1 sequence. (<b>c</b>) kt2 sequence. (<b>d</b>) kt3 sequence.</p>
Full article ">Figure 18
<p>Real-world scenario test results. (<b>a</b>) Color images. (<b>b</b>) Depth images. (<b>c</b>) Optical flow of moving objects. (<b>d</b>) Moving rigid object masks. (<b>e</b>) Feature points in traditional ORB-SLAM3. (<b>f</b>) Feature points in DIO-SLAM.</p>
Full article ">
25 pages, 4874 KiB  
Article
Monitoring and Predicting Air Quality with IoT Devices
by Claudia Banciu, Adrian Florea and Razvan Bogdan
Processes 2024, 12(9), 1961; https://doi.org/10.3390/pr12091961 (registering DOI) - 12 Sep 2024
Viewed by 378
Abstract
The growing concern about air quality and its influence on human health has prompted the development of sophisticated monitoring and forecast systems. This article gives a thorough investigation into forecasting the air quality index (AQI) with an Internet of Things (IoT) device that [...] Read more.
The growing concern about air quality and its influence on human health has prompted the development of sophisticated monitoring and forecast systems. This article gives a thorough investigation into forecasting the air quality index (AQI) with an Internet of Things (IoT) device that analyzes temperature, humidity, PM10, and PM2.5 levels. The dataset used for this analysis comprises 5869 data points across six critical parameters essential for accurate air quality prediction. The data from these sensors is sent to the ThingSpeak cloud platform for storage and preliminary analysis. The system forecasts AQI using a TensorFlow-based regression model, delivering real-time insights. The combination of IoT technology and machine learning improves the accuracy and responsiveness of air quality monitoring systems, making it a useful tool for environmental management and public health protection. This work presents comparatively the effectiveness of feedforward neural network models trained with the ‘adam’ and ‘RMSprop’ optimizers over different epochs, as well as the machine learning algorithm random forest with varying numbers of estimators to forecast AQI. The models were trained using both types of regression analysis: linear regression and random forest regression. The findings show that the model achieves a high degree of accuracy, with the predictions closely aligning with the actual AQI values, thus having the potential to significantly reduce the negative health impact associated with poor air quality, protecting public health and alerting users when pollution levels are higher than allowed. Specifically, the random forest model with 100 estimators delivers the best overall performance for both AQI 10 and AQI 2.5, achieving the lowest Mean Absolute Error (MAE) of 0.2785 for AQI 10 and 0.2483 for AQI 2.5. This integration of IoT technology and advanced predictive analysis addresses the significant worldwide issue of air pollution by identifying the pollution hotspots and allowing decision-makers for quick reactions, and the development of effective strategies to reduce pollution sources. Full article
Show Figures

Figure 1

Figure 1
<p>System architecture.</p>
Full article ">Figure 2
<p>Hardware architecture.</p>
Full article ">Figure 3
<p>Interconnecting components.</p>
Full article ">Figure 4
<p>Electrical diagram.</p>
Full article ">Figure 5
<p>Flowchart RPI application.</p>
Full article ">Figure 6
<p>Communication between device and ThingSpeak.</p>
Full article ">Figure 7
<p>Software architecture.</p>
Full article ">Figure 8
<p>Graph generated in ThingSpeak web application for AQI 10.</p>
Full article ">Figure 9
<p>Graph generated in ThingSpeak web application for AQI 2.5.</p>
Full article ">Figure 10
<p>Heatmap of correlation between variables.</p>
Full article ">Figure 11
<p>Predicted vs. actual AQI 10—prediction model with ‘adam’ optimizer.</p>
Full article ">Figure 12
<p>Predicted vs. actual AQI 10—prediction model with ‘RMSprop’ optimizer.</p>
Full article ">Figure 13
<p>Predicted vs. actual AQI 2.5—prediction model with ‘adam’ optimizer.</p>
Full article ">Figure 14
<p>Predicted vs. actual AQI 2.5—prediction model with ‘RMSprop optimizer.</p>
Full article ">Figure 15
<p>Predicted vs. actual AQI 10—random forest regression.</p>
Full article ">Figure 16
<p>Predicted vs. actual AQI 2.5—random forest regression.</p>
Full article ">
19 pages, 20386 KiB  
Article
YOD-SLAM: An Indoor Dynamic VSLAM Algorithm Based on the YOLOv8 Model and Depth Information
by Yiming Li, Yize Wang, Liuwei Lu and Qi An
Electronics 2024, 13(18), 3633; https://doi.org/10.3390/electronics13183633 - 12 Sep 2024
Viewed by 249
Abstract
Aiming at the problems of low positioning accuracy and poor mapping effect of the visual SLAM system caused by the poor quality of the dynamic object mask in an indoor dynamic environment, an indoor dynamic VSLAM algorithm based on the YOLOv8 model and [...] Read more.
Aiming at the problems of low positioning accuracy and poor mapping effect of the visual SLAM system caused by the poor quality of the dynamic object mask in an indoor dynamic environment, an indoor dynamic VSLAM algorithm based on the YOLOv8 model and depth information (YOD-SLAM) is proposed based on the ORB-SLAM3 system. Firstly, the YOLOv8 model obtains the original mask of a priori dynamic objects, and the depth information is used to modify the mask. Secondly, the mask’s depth information and center point are used to a priori determine if the dynamic object has missed detection and if the mask needs to be redrawn. Then, the mask edge distance and depth information are used to judge the movement state of non-prior dynamic objects. Finally, all dynamic object information is removed, and the remaining static objects are used for posing estimation and dense point cloud mapping. The accuracy of camera positioning and the construction effect of dense point cloud maps are verified using the TUM RGB-D dataset and real environment data. The results show that YOD-SLAM has a higher positioning accuracy and dense point cloud mapping effect in dynamic scenes than other advanced SLAM systems such as DS-SLAM and DynaSLAM. Full article
(This article belongs to the Section Computer Science & Engineering)
Show Figures

Figure 1

Figure 1
<p>Overview of YOD-SLAM.</p>
Full article ">Figure 2
<p>The process of modifying prior dynamic object masks using depth information. Figure (<b>a</b>) shows the depth image corresponding to the current frame. Through the algorithm presented in this article, the background area that is excessively covered in (<b>b</b>) is removed in (<b>c</b>). The expanded edges of the human body have achieved better coverage in (<b>d</b>).</p>
Full article ">Figure 3
<p>The process of redrawing the mask of previously missed dynamic objects. Figure (<b>a</b>) shows the depth image corresponding to the current frame. In (<b>b</b>), we can see that people in the distance were not covered by the original mask, resulting in missed detections. The mask in (<b>c</b>) can be obtained by filling in the mask with the depth information specific to that location in (<b>a</b>).</p>
Full article ">Figure 4
<p>The process of excluding prior static objects in motion.</p>
Full article ">Figure 5
<p>Comparison of estimated trajectories and real trajectories of four systems.</p>
Full article ">Figure 6
<p>The results of mask modification on dynamic objects. The three graphs of each column come from the same time as their respective datasets. The first line is the depth image corresponding to the current frame; the second is the original mask obtained by YOLOv8; and the third is the final mask after our modification.</p>
Full article ">Figure 7
<p>Comparison of point cloud maps between ORB-SLAM3 and YOD-SLAM in two sets of highly dynamic sequences.</p>
Full article ">Figure 7 Cont.
<p>Comparison of point cloud maps between ORB-SLAM3 and YOD-SLAM in two sets of highly dynamic sequences.</p>
Full article ">Figure 8
<p>Comparison of point cloud maps between ORB-SLAM3 and YOD-SLAM in low- and static dynamic sequences, where fr2/desk/p is a low-dynamic scene, while fr2/rpy is a static scene.</p>
Full article ">Figure 9
<p>Intel RealSense Depth Camera D455.</p>
Full article ">Figure 10
<p>Mask processing and ORB feature point extraction in real laboratory environments. Several non English exhibition boards are leaning against the wall to simulate typical indoor environments. The facial features of the characters have been treated with confidentiality.</p>
Full article ">Figure 11
<p>Comparison of dense point cloud mapping between ORB-SLAM3 and YOD-SLAM in real laboratory environments. We marked the map areas affected by dynamic objects with red circles.</p>
Full article ">
15 pages, 3271 KiB  
Article
Spiking PointCNN: An Efficient Converted Spiking Neural Network under a Flexible Framework
by Yingzhi Tao and Qiaoyun Wu
Electronics 2024, 13(18), 3626; https://doi.org/10.3390/electronics13183626 - 12 Sep 2024
Viewed by 261
Abstract
Spiking neural networks (SNNs) are generating wide attention due to their brain-like simulation capabilities and low energy consumption. Converting artificial neural networks (ANNs) to SNNs provides great advantages, combining the high accuracy of ANNs with the robustness and energy efficiency of SNNs. Existing [...] Read more.
Spiking neural networks (SNNs) are generating wide attention due to their brain-like simulation capabilities and low energy consumption. Converting artificial neural networks (ANNs) to SNNs provides great advantages, combining the high accuracy of ANNs with the robustness and energy efficiency of SNNs. Existing point clouds processing SNNs have two issues to be solved: first, they lack a specialized surrogate gradient function; second, they are not robust enough to process a real-world dataset. In this work, we present a high-accuracy converted SNN for 3D point cloud processing. Specifically, we first revise and redesign the Spiking X-Convolution module based on the X-transformation. To address the problem of non-differentiable activation function arising from the binary signal from spiking neurons, we propose an effective adjustable surrogate gradient function, which can fit various models well by tuning the parameters. Additionally, we introduce a versatile ANN-to-SNN conversion framework enabling modular transformations. Based on this framework and the spiking X-Convolution module, we design the Spiking PointCNN, a highly efficient converted SNN for processing 3D point clouds. We conduct experiments on the public 3D point cloud datasets ModelNet40 and ScanObjectNN, on which our proposed model achieves excellent accuracy. Code will be available on GitHub. Full article
(This article belongs to the Special Issue Artificial Intelligence in Image Processing and Computer Vision)
Show Figures

Figure 1

Figure 1
<p>Structure of X-transformation convolution.</p>
Full article ">Figure 2
<p>Structure of spiking X-transformation convolution module.</p>
Full article ">Figure 3
<p>Comparison of spiking signals and sigmoid and surrogate functions with different k. Left picture shows the functions while the right one shows their gradients.</p>
Full article ">Figure 4
<p>Comparison of proposed adjustable surrogate gradient function with traditional activation functions.</p>
Full article ">Figure 5
<p>The process of a numeric matrix being converted to spiking matrix sets.</p>
Full article ">Figure 6
<p>The complete structure of Spiking PointCNN.</p>
Full article ">
18 pages, 9000 KiB  
Article
Multilevel Geometric Feature Embedding in Transformer Network for ALS Point Cloud Semantic Segmentation
by Zhuanxin Liang and Xudong Lai
Remote Sens. 2024, 16(18), 3386; https://doi.org/10.3390/rs16183386 - 12 Sep 2024
Viewed by 283
Abstract
Effective semantic segmentation of Airborne Laser Scanning (ALS) point clouds is a crucial field of study and influences subsequent point cloud application tasks. Transformer networks have made significant progress in 2D/3D computer vision tasks, exhibiting superior performance. We propose a multilevel geometric feature [...] Read more.
Effective semantic segmentation of Airborne Laser Scanning (ALS) point clouds is a crucial field of study and influences subsequent point cloud application tasks. Transformer networks have made significant progress in 2D/3D computer vision tasks, exhibiting superior performance. We propose a multilevel geometric feature embedding transformer network (MGFE-T), which aims to fully utilize the three-dimensional structural information carried by point clouds and enhance transformer performance in ALS point cloud semantic segmentation. In the encoding stage, compute the geometric features surrounding tee sampling points at each layer and embed them into the transformer workflow. To ensure that the receptive field of the self-attention mechanism and the geometric computation domain can maintain a consistent scale at each layer, we propose a fixed-radius dilated KNN (FR-DKNN) search method to address the limitation of traditional KNN search methods in considering domain radius. In the decoding stage, we aggregate prediction deviations at each level into a unified loss value, enabling multilevel supervision to improve the network’s feature learning ability at different levels. The MGFE-T network can predict the class label of each point in an end-to-end manner. Experiments were conducted on three widely used benchmark datasets. The results indicate that the MGFE-T network achieves superior OA and mF1 scores on the LASDU and DFC2019 datasets and performs well on the ISPRS dataset with imbalanced classes. Full article
Show Figures

Figure 1

Figure 1
<p>MGFE-T Semantic Segmentation Network Architecture.</p>
Full article ">Figure 2
<p>GFE-T/Transformer Block with Residual Connection.</p>
Full article ">Figure 3
<p>GFE-T Module Architecture.</p>
Full article ">Figure 4
<p>Comparison of FR-DKNN with other methods (<span class="html-italic">k</span> = 4, <span class="html-italic">d</span> = 2).</p>
Full article ">Figure 5
<p>Preview of the LASDU dataset.</p>
Full article ">Figure 6
<p>Preview of the DFC2019 dataset (3 of 110 files).</p>
Full article ">Figure 7
<p>Preview of the ISPRS dataset.</p>
Full article ">Figure 8
<p>Visualization of semantic segmentation results for some regions of the LASDU dataset (the first, second, and third columns are the ground truth, the results of the baseline, and the results of MGFE-T, respectively).</p>
Full article ">Figure 9
<p>Visualization of semantic segmentation results for some regions of the DFC2019 dataset (the first, second, and third columns are the ground truth, the results of the baseline, and the results of MGFE-T, respectively).</p>
Full article ">Figure 10
<p>Visualization of semantic segmentation results for some regions of the ISPRS dataset (the first, second, and third columns are the ground truth, the results of the baseline, and the results of MGFE-T, respectively).</p>
Full article ">Figure 11
<p>Comparison of experimental results for different radius percentiles.</p>
Full article ">
19 pages, 6287 KiB  
Article
Research on Multiscale Atmospheric Chaos Based on Infrared Remote-Sensing and Reanalysis Data
by Zhong Wang, Shengli Sun, Wenjun Xu, Rui Chen, Yijun Ma and Gaorui Liu
Remote Sens. 2024, 16(18), 3376; https://doi.org/10.3390/rs16183376 - 11 Sep 2024
Viewed by 315
Abstract
The atmosphere is a complex nonlinear system, with the information of its temperature, water vapor, pressure, and cloud being crucial aspects of remote-sensing data analysis. There exist intricate interactions among these internal components, such as convection, radiation, and humidity exchange. Atmospheric phenomena span [...] Read more.
The atmosphere is a complex nonlinear system, with the information of its temperature, water vapor, pressure, and cloud being crucial aspects of remote-sensing data analysis. There exist intricate interactions among these internal components, such as convection, radiation, and humidity exchange. Atmospheric phenomena span multiple spatial and temporal scales, from small-scale thunderstorms to large-scale events like El Niño. The dynamic interactions across different scales, along with external disturbances to the atmospheric system, such as variations in solar radiation and Earth surface conditions, contribute to the chaotic nature of the atmosphere, making long-term predictions challenging. Grasping the intrinsic chaotic dynamics is essential for advancing atmospheric analysis, which holds profound implications for enhancing meteorological forecasts, mitigating disaster risks, and safeguarding ecological systems. To validate the chaotic nature of the atmosphere, this paper reviewed the definitions and main features of chaotic systems, elucidated the method of phase space reconstruction centered on Takens’ theorem, and categorized the qualitative and quantitative methods for determining the chaotic nature of time series data. Among quantitative methods, the Wolf method is used to calculate the Largest Lyapunov Exponents, while the G–P method is used to calculate the correlation dimensions. A new method named Improved Saturated Correlation Dimension method was proposed to address the subjectivity and noise sensitivity inherent in the traditional G–P method. Subsequently, the Largest Lyapunov Exponents and saturated correlation dimensions were utilized to conduct a quantitative analysis of FY-4A and Himawari-8 remote-sensing infrared observation data, and ERA5 reanalysis data. For both short-term remote-sensing data and long-term reanalysis data, the results showed that more than 99.91% of the regional points have corresponding sequences with positive Largest Lyapunov exponents and all the regional points have correlation dimensions that tended to saturate at values greater than 1 with increasing embedding dimensions, thereby proving that the atmospheric system exhibits chaotic properties on both short and long temporal scales, with extreme sensitivity to initial conditions. This conclusion provided a theoretical foundation for the short-term prediction of atmospheric infrared radiation field variables and the detection of weak, time-sensitive signals in complex atmospheric environments. Full article
(This article belongs to the Topic Atmospheric Chemistry, Aging, and Dynamics)
Show Figures

Figure 1

Figure 1
<p>Analysis process of chaotic nature.</p>
Full article ">Figure 2
<p>(<b>a</b>) 60 × 0 LLE diagram of FY-4A AGRI China Area 4KM L1 data for Channel 1; (<b>b</b>) 60 × 60 LLE diagram of FY-4A AGRI China Area 4KM L1 data for Channel 4; (<b>c</b>) 60 × 60 LLE diagram of FY-4A AGRI China Area 4KM L1 data for Channel 10; (<b>d</b>) 60 × 60 LLE diagram of FY-4A AGRI China Area 4KM L1 data for Channel 13.</p>
Full article ">Figure 3
<p>(<b>a</b>) 40 × 40 LLE diagram of Himawari-8 AHI Full Disk 2KM L1 data for Channel 2; (<b>b</b>) 40 × 40 LLE diagram of Himawari-8 AHI Full Disk 2KM L1 data for Channel 12.</p>
Full article ">Figure 4
<p>(<b>a</b>) 41 × 41 LLE diagram of ERA5 hourly data for Z500; (<b>b</b>) 41 × 41 LLE diagram of ERA5 hourly data for T850.</p>
Full article ">Figure 5
<p>(<b>a</b>) Log–log plots of the correlation integral <math display="inline"><semantics> <mrow> <mi>C</mi> <mfenced> <mi>r</mi> </mfenced> </mrow> </semantics></math> versus the distance <math display="inline"><semantics> <mi>r</mi> </semantics></math> for different embedding dimensions <span class="html-italic">m</span> for channel 1 of FY-4A AGRI China area 4KM L1 data; (<b>b</b>) Log–log plots of the correlation integral <math display="inline"><semantics> <mrow> <mi>C</mi> <mfenced> <mi>r</mi> </mfenced> </mrow> </semantics></math> versus the distance <math display="inline"><semantics> <mi>r</mi> </semantics></math> for different embedding dimensions <span class="html-italic">m</span> for channel 4 of FY-4A AGRI China area 4KM L1 data; (<b>c</b>) Log–log plots of the correlation integral <math display="inline"><semantics> <mrow> <mi>C</mi> <mfenced> <mi>r</mi> </mfenced> </mrow> </semantics></math> versus the distance <math display="inline"><semantics> <mi>r</mi> </semantics></math> for different embedding dimensions <span class="html-italic">m</span> for channel 10 of FY-4A AGRI China area 4KM L1 data; (<b>d</b>) Log–log plots of the correlation integral <math display="inline"><semantics> <mrow> <mi>C</mi> <mfenced> <mi>r</mi> </mfenced> </mrow> </semantics></math> versus the distance <math display="inline"><semantics> <mi>r</mi> </semantics></math> for different embedding dimensions <span class="html-italic">m</span> for channel 13 of FY-4A AGRI China area 4KM L1 data of FY-4A AGRI China area 4KM L1 data.</p>
Full article ">Figure 6
<p>Correlation Dimension vs. Embedding Dimension Curve for channel 1, channel 4, channel 10 and channel 13 of FY-4A AGRI China area 4KM L1 data.</p>
Full article ">Figure 7
<p>(<b>a</b>) Log–log plots of the correlation integral <math display="inline"><semantics> <mrow> <mi>C</mi> <mfenced> <mi>r</mi> </mfenced> </mrow> </semantics></math> versus the distance <math display="inline"><semantics> <mi>r</mi> </semantics></math> for different embedding dimensions <span class="html-italic">m</span> for channel 2 of Himawari-8 AHI full-disk 2KM L1 data; (<b>b</b>) Log–log plots of the correlation integral <math display="inline"><semantics> <mrow> <mi>C</mi> <mfenced> <mi>r</mi> </mfenced> </mrow> </semantics></math> versus the distance <math display="inline"><semantics> <mi>r</mi> </semantics></math> for different embedding dimensions <span class="html-italic">m</span> for channel 12 of Himawari-8 AHI full-disk 2KM L1 data.</p>
Full article ">Figure 8
<p>Correlation Dimension vs. Embedding Dimension Curve for channel 2 and channel 12 of Himawari-8 AHI Full disk 2KM L1 data at location (4050, 5250).</p>
Full article ">Figure 9
<p>(<b>a</b>) Log–log plots of the correlation integral <math display="inline"><semantics> <mrow> <mi>C</mi> <mfenced> <mi>r</mi> </mfenced> </mrow> </semantics></math> versus the distance <math display="inline"><semantics> <mi>r</mi> </semantics></math> for different embedding dimensions <span class="html-italic">m</span> for Z500 hourly data of ERA5 at 20.00°S, 175.00°W; (<b>b</b>) Log–log plots of the correlation integral <math display="inline"><semantics> <mrow> <mi>C</mi> <mfenced> <mi>r</mi> </mfenced> </mrow> </semantics></math> versus the distance <math display="inline"><semantics> <mi>r</mi> </semantics></math> for different embedding dimensions <span class="html-italic">m</span> for T850 hourly data of ERA5 at 20.00°S, 175.00°W.</p>
Full article ">Figure 10
<p>(<b>a</b>) Correlation Dimension vs. Embedding Dimension Curve for Z500 hourly data of ERA5 at 20.00°S, 175.00°W; (<b>b</b>) Correlation Dimension vs. Embedding Dimension Curve for T850 hourly data of ERA5 at 20.00°S, 175.00°W.</p>
Full article ">
20 pages, 661 KiB  
Article
Novel Green Strategy to Recover Bioactive Compounds with Different Polarities from Horned Melon Peel
by Teodora Cvanić, Mirjana Sulejmanović, Milica Perović, Jelena Vulić, Lato Pezo, Gordana Ćetković and Vanja Travičić
Foods 2024, 13(18), 2880; https://doi.org/10.3390/foods13182880 - 11 Sep 2024
Viewed by 307
Abstract
Around 20–30% of the horned melon’s weight is peel. This peel is often discarded or underutilized despite containing valuable bioactive compounds. Conventional methods for extracting polyphenols and carotenoids from horned melon peel are typically inefficient, environmentally harmful, or require significant time and energy. [...] Read more.
Around 20–30% of the horned melon’s weight is peel. This peel is often discarded or underutilized despite containing valuable bioactive compounds. Conventional methods for extracting polyphenols and carotenoids from horned melon peel are typically inefficient, environmentally harmful, or require significant time and energy. The potential of green cloud point extraction (CPE) or green surfactant-based extraction for recovering bioactives with different polarities from this kind of by-product has not been thoroughly investigated. Therefore, this study focused on optimizing CPE process parameters using a one-variable-at-a-time (OVAT) approach. Optimal CPE demonstrated superior yields compared to conventional, ultrasound, microwave, ultrasound-assisted CPE, and microwave-assisted CPE methods. Further, a Plackett–Burman design identified key factors influencing optimal CPE conditions, while artificial neural network (ANN) analysis assessed each input variable’s impact on outcomes. Maximum extraction efficiency for total phenolics (352.49 mg GAE/100 g), total carotenoids (16.59 mg β-carotene/100 g), and antioxidant activity (989.02 μmol TE/100 g) was achieved under conditions of: surfactant type = Tween 80, surfactant concentration = 2%; solid:liquid ratio = 1:100; pH = 6612; equilibration temperature = 35 °C; equilibration time = 60 min; salt type = NaCl; salt concentration = 16.4%; centrifugation speed = 7906× g ; centrifugation time = 13.358 min; and No. of CPE steps = Step 1. This comprehensive approach aimed to enhance the understanding and optimization of CPE for maximizing the recovery of bioactives from the horned melon peel, addressing the inefficiencies of traditional extraction methods. Full article
Show Figures

Figure 1

Figure 1
<p>Relative influence of variables X1–X11 on (<b>a</b>) TP, (<b>b</b>) TC, and (<b>c</b>) AA. X1—surfactant type; X2—surfactant concentration; X3—solid:liquid ratio; X4—pH; X5—equilibration temperature; X6—equilibration time; X7—salt type; X8—salt concentration; X9—centrifugation speed; X10—centrifugation time; and X11—the number of CPE steps.</p>
Full article ">
Back to TopTop