A Real-Time Tree Crown Detection Approach for Large-Scale Remote Sensing Images on FPGAs
"> Figure 1
<p>The overall process of the original tree crown detection algorithm. (<b>a</b>) The input image; (<b>b</b>) Image pre-processing and non-overlapping local maximum filtering; (<b>c</b>) Transect sampling; (<b>d</b>) Circular-window-based local maximum filtering; (<b>e</b>) Minimum distance filtering; (<b>f</b>) The output results of the tree crown detection algorithm.</p> "> Figure 2
<p>The Quickbird satellite image of the whole study area (<b>left</b>) and a local area of accuracy evaluation regions (<b>right</b>). The red squares denote the selected regions for evaluating the detection accuracy.</p> "> Figure 3
<p>The overall framework of the tree crown detection for large-scale satellite images on FPGAs.</p> "> Figure 4
<p>The basic process of Kernel 1.</p> "> Figure 5
<p>The basic process of Kernel 2.</p> "> Figure 6
<p>The proposed strategy of removing loop dependencies using extra BRAMs.</p> "> Figure 7
<p>The detection image results of region 1, 2 and 3 obtained from our proposed PF-TCD algorithm (<b>left</b>) and the original algorithm (<b>right</b>).</p> "> Figure 8
<p>The detection image results of region 4, 5 and 6 obtained from our proposed PF-TCD algorithm (<b>left</b>) and the original algorithm (<b>right</b>).</p> "> Figure 9
<p>The running time for processing images in different sizes.</p> ">
Abstract
:1. Introduction
- The direct mapping of the original tree crown detection algorithm to FPGAs results in high resource utilization and each pixel requires to be streamed for multiple times. Through reconstructing and modifying the original workflow into three computational kernels on FPGAs, we design a pipelined-friendly tree crown detection approach (PF-TCD) so that each pixel of the image can be streamed for only once.
- Through optimizing and adjusting the local maximum filtering, transact sampling, and minimum distance filtering algorithms of the tree crown detection approach, the utilization of different resources (look-up tables (LUTs), flip-flops (FFs), digital signal processor blocks (DSPs), and block random access memory (BRAMs)) is reduced significantly and well balanced, avoiding any of them becoming the performance bottleneck.
- We propose a complete FPGA-based framework for processing the large-scale remote sensing image in real time, which provides a high performance solution for tree crown detection from the raw remote sensing images to the final detection results. The proposed framework can process the satellite image of 12,188 × 12,576 pixels in 0.33 s, achieving 18.7-times speedup over the well-optimized software implementation on 12-core CPU.
2. Background
2.1. Image Pre-Processing and Non-Overlapping Local Maximum Filtering
2.2. Transect Sampling and Circular-Window-Based Local Maximum Filtering
Algorithm 1 Transect sampling |
|
2.3. Minimum Distance Filtering
Algorithm 2 Minimum distance filtering |
|
3. Data
4. Methods
4.1. Overall Framework of Tree Crown Detection for Large-Scale Remote Sensing Images on FPGAs
4.2. Kernel 1: Transect Sampling Radius Calculation and Local Maximum Filtering
4.3. Kernel 2: Transect Sampling Radius Based Local Maximum Filtering
4.4. Kernel 3: Minimum Distance Filtering
Algorithm 3 Minimum distance filtering on FPGA |
|
5. Experimental Results Analysis
5.1. Performance Analysis of Our Proposed Approach
5.2. Resource Utilization of Each Kernel
5.3. The Detection Results of Our Proposed Approach
6. Discussion
6.1. The Size of Image Blocks
6.2. The Running Time for Processing Images in Different Sizes
7. Conclusions
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
References
- Li, W.; He, C.; Fu, H.; Luk, W. An FPGA-based tree crown detection approach for remote sensing images. In Proceedings of the IEEE 2017 International Conference on Field Programmable Technology (ICFPT), Melbourne, Australia, 11–13 December 2017; pp. 231–234. [Google Scholar]
- Pouliot, D.; King, D.; Bell, F.; Pitt, D. Automated tree crown detection and delineation in high-resolution digital camera imagery of coniferous forest regeneration. Remote Sens. Environ. 2002, 82, 322–334. [Google Scholar] [CrossRef]
- Tan, K.P.; Kanniah, K.D.; Cracknell, A.P. Use of UK-DMC 2 and ALOS PALSAR for studying the age of oil palm trees in southern peninsular Malaysia. Int. J. Remote Sens. 2013, 34, 7424–7446. [Google Scholar] [CrossRef]
- Li, W.; Fu, H.; Yu, L.; Cracknell, A. Deep Learning Based Oil Palm Tree Detection and Counting for High-Resolution Remote Sensing Images. Remote Sens. 2016, 9, 22. [Google Scholar] [CrossRef]
- Chirici, G.; Giuliarelli, D.; Biscontini, D.; Tonti, D.; Mattioli, W.; Marchetti, M.; Corona, P. Large-scale monitoring of coppice forest clearcuts by multitemporal very high resolution satellite imagery. A case study from central Italy. Remote Sens. Environ. 2011, 115, 1025–1033. [Google Scholar] [CrossRef] [Green Version]
- Li, W.; He, C.; Fang, J.; Zheng, J.; Fu, H.; Yu, L. Semantic Segmentation-Based Building Footprint Extraction Using Very High-Resolution Satellite Images and Multi-Source GIS Data. Remote Sens. 2019, 11, 403. [Google Scholar] [CrossRef]
- Ke, Y.H.; Zhang, W.H.; Quackenbush, L.J. Active contour and hill climbing for tree crown detection and delineation. Photogramm. Eng. Remote Sens. 2010, 76, 1169–1181. [Google Scholar] [CrossRef]
- Santos, A.M.D.; Mitja, D.; Delaître, E.; Demagistri, L.; Miranda, I.D.S.; Libourel, T.; Petit, M. Estimating babassu palm density using automatic palm tree detection with very high spatial resolution satellite images. J. Environ. Manag. 2017, 193, 40–51. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Ke, Y.; Quackenbush, L.J. A review of methods for automatic individual tree-crown detection and delineation from passive remote sensing. Int. J. Remote Sens. 2011, 32, 4725–4747. [Google Scholar] [CrossRef]
- Quackenbush, L.J.; Hopkins, P.F.; Kinn, G.J. Using template correlation to identify individual trees in high resolution imagery. In Proceedings of the American Society for Photogrammetry & Remote Sensing (ASPRS) 2000 Annual Conference Proceedings, Washington, DC, USA, 22–26 May 2000. [Google Scholar]
- Leckie, D.G.; Gougeon, F.A.; Tinis, S.; Nelson, T.; Burnett, C.N.; Paradine, D. Automated tree recognition in old growth conifer stands with high resolution digital imagery. Remote Sens. Environ. 2005, 94, 311–326. [Google Scholar] [CrossRef]
- López-López, M.; Calderón, R.; González-Dugo, V.; Zarco-Tejada, P.J.; Fereres, E. Early Detection and Quantification of Almond Red Leaf Blotch Using High-Resolution Hyperspectral and Thermal Imagery. Remote Sens. 2016, 8, 276. [Google Scholar] [CrossRef]
- Malek, S.; Bazi, Y.; Alajlan, N.; AlHichri, H.; Melgani, F. Efficient framework for palm tree detection in UAV images. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2014, 7, 4692–4703. [Google Scholar] [CrossRef]
- Nevalainen, O.; Honkavaara, E.; Tuominen, S.; Viljanen, N.; Hakala, T.; Yu, X.; Hyyppä, J.; Saari, H.; Pölönen, I.; Imai, N.; et al. Individual tree detection and classification with UAV-based photogrammetric point clouds and hyperspectral imaging. Remote Sens. 2017, 9, 185. [Google Scholar] [CrossRef]
- Guirado, E.; Tabik, S.; Alcaraz-Segura, D.; Cabello, J.; Herrera, F. Deep-learning versus OBIA for scattered shrub detection with Google earth imagery: Ziziphus Lotus as case study. Remote Sens. 2017, 9, 1220. [Google Scholar]
- Li, W.; Dong, R.; Fu, H.; Yu, L. Large-Scale Oil Palm Tree Detection from High-Resolution Satellite Images Using Two-Stage Convolutional Neural Networks. Remote Sens. 2019, 11, 11. [Google Scholar]
- Freudenberg, M.; Nölke, N.; Agostini, A.; Urban, K.; Wörgötter, F.; Kleinn, C. Large Scale Palm Tree Detection in High Resolution Satellite Images Using U-Net. Remote Sens. 2019, 11, 312. [Google Scholar] [CrossRef]
- Li, W.; Fu, H.; Yu, L.; Gong, P.; Feng, D.; Li, C.; Clinton, N. Stacked Autoencoder-based deep learning for remote-sensing image classification: A case study of African land-cover mapping. Int. J. Remote Sens. 2016, 37, 5632–5646. [Google Scholar] [CrossRef]
- Santos, L.; Berrojo, L.; Moreno, J.; López, J.F.; Sarmiento, R. Multispectral and hyperspectral lossless compressor for space applications (HyLoC): A low-complexity FPGA implementation of the CCSDS 123 standard. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2016, 9, 757–770. [Google Scholar]
- Bing, Z.; Wei, Y.; Gao, L.; Chen, D. Real-time target detection in hyperspectral images based on spatial-spectral information extraction. Eur. J. Adv. Signal Process. 2012, 2012, 142. [Google Scholar] [Green Version]
- Liu, D.; Zhou, G.; Huang, J.; Zhang, R.; Shu, L.; Zhou, X.; Xin, C.S. On-Board Georeferencing Using FPGA-Based Optimized Second-Order Polynomial Equation. Remote Sens. 2019, 11, 124. [Google Scholar]
- Bernabe, S.; Sanchez, S.; Plaza, A.; López, S.; Benediktsson, J.A.; Sarmiento, R. Hyperspectral unmixing on GPUs and multi-core processors: A comparison. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2013, 6, 1386–1398. [Google Scholar]
- Li, W.; Fu, H.; You, Y.; Yu, L.; Fang, J. Parallel Multiclass Support Vector Machine for Remote Sensing Data Classification on Multicore and Many-Core Architectures. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2017, 10, 4387–4398. [Google Scholar] [CrossRef]
- Wu, Z.; Liu, J.; Plaza, A.; Li, J.; Wei, Z. GPU implementation of composite kernels for hyperspectral image classification. IEEE Geosci. Remote Sens. Lett. 2015, 12, 1973–1977. [Google Scholar]
- González, C.; Bernabé, S.; Mozos, D.; Plaza, A. FPGA implementation of an algorithm for automatically detecting targets in remotely sensed hyperspectral images. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2016, 9, 4334–4343. [Google Scholar] [CrossRef]
- Mohammadnia, M.R.; Shannon, L. A multi-beam Scan Mode Synthetic Aperture Radar processor suitable for satellite operation. In Proceedings of the 2016 IEEE 27th International Conference on IEEE Application-Specific Systems, Architectures and Processors (ASAP), London, UK, 6–8 July 2016; pp. 83–90. [Google Scholar]
- Lee, C.A.; Gasster, S.D.; Plaza, A.; Chang, C.I.; Huang, B. Recent developments in high performance computing for remote sensing: A review. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2011, 4, 508–527. [Google Scholar] [CrossRef]
- Wang, S.; Niu, X.; Ma, N.; Luk, W.; Leong, P.; Peng, Y. A Scalable Dataflow Accelerator for Real Time Onboard Hyperspectral Image Classification. In International Symposium on Applied Reconfigurable Computing; Springer: Berlin, Germany, 2016; pp. 105–116. [Google Scholar]
- Gonzalez, C.; Lopez, S.; Mozos, D.; Sarmiento, R. A novel FPGA-based architecture for the estimation of the virtual dimensionality in remotely sensed hyperspectral images. J. Real-Time Image Process. 2015, 43, 1–12. [Google Scholar] [CrossRef]
- Shan, N.; Wang, X.S.; Wang, Z.S. Efficient FPGA implementation of cloud detection for real-time remote sensing image processing. In Proceedings of the 2010 Conference on Postgraduate Research in IEEE Asia Pacific Microelectronics and Electronics (PrimeAsia), Shanghai, China, 22–24 September 2010; pp. 190–193. [Google Scholar]
- Tang, J.W.; Shaikh-Husin, N.; Sheikh, U.U.; Marsono, M.N. FPGA-based real-time moving target detection system for unmanned aerial vehicle application. Int. J. Reconfigur. Comput. 2016, 2016. [Google Scholar] [CrossRef]
- Chen, Y.L.; Wu, C.C.; Lin, H.C.; Lin, C. A parallel approach of multi-level morphological active contour algorithm for individual tree detection and crown delineation. In Proceedings of the 2013 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Melbourne, Australia, 21–26 July 2013; pp. 2601–2604. [Google Scholar]
- Duncanson, L.; Cook, B.; Hurtt, G.; Dubayah, R. An efficient, multi-layered crown delineation algorithm for mapping individual tree structure across multiple ecosystems. Remote Sens. Environ. 2014, 154, 378–386. [Google Scholar] [CrossRef]
- Jiang, H.; Chen, S.; Li, D.; Wang, C.; Yang, J. Papaya Tree Detection with UAV Images Using a GPU-Accelerated Scale-Space Filtering Method. Remote Sens. 2017, 9, 721. [Google Scholar] [CrossRef]
- Srestasathiern, P.; Rakwatin, P. Oil palm tree detection with high resolution multi-spectral satellite imagery. Remote Sens. 2014, 6, 9749–9774. [Google Scholar] [CrossRef]
- Cheng, Y.; Le, Y.; Xu, Y.; Hui, L.; Cracknell, A.P.; Kanniah, K.; Peng, G. Mapping oil palm extent in Malaysia using ALOS-2 PALSAR-2 data. Int. J. Remote Sens. 2018, 39, 432–452. [Google Scholar] [CrossRef]
- Cheng, Y.; Le, Y.; Xu, Y.; Liu, X.; Hui, L.; Cracknell, A.P.; Kanniah, K.; Peng, G. Towards global oil palm plantation mapping using remote-sensing data. Int. J. Remote Sens. 2018, 39, 5891–5906. [Google Scholar] [CrossRef]
- Malaysian Oil Palm Plantation Dataset. Available online: https://github.com/dongrunmin/oil_palm_data/ blob/master/segmentation_map (accessed on 6 March 2019).
- Lachowicz, S.; Pfleiderer, H.J. Fast evaluation of nonlinear functions using FPGAs. Adv. Radio Sci. 2008, 6, 233–237. [Google Scholar] [CrossRef] [Green Version]
- Summers, S.; Rose, A.; Sanders, P. Using MaxCompiler for the high level synthesis of trigger algorithms. J. Instrum. 2017, 12, C02015. [Google Scholar] [CrossRef]
Resource | Kernel 1 | Kernel 2 | Kernel 3 | Manager | Total |
---|---|---|---|---|---|
LUTs | 1636 | 31,229 | 2852 | 38,621 | 74,338 (28.33%) |
FFs | 2847 | 50,062 | 6250 | 86,842 | 146,001 (27.82%) |
DSPs | 0 | 370 | 130 | 0 | 500 (25.47%) |
BRAMs | 0 | 304 | 1 | 455 | 760 (29.61%) |
Method | Index | Region 1 | Region 2 | Region 3 | Region 4 | Region 5 | Region 6 |
---|---|---|---|---|---|---|---|
PF-TCD algorithm | TP | 1033 | 1323 | 739 | 684 | 986 | 1380 |
FP | 206 | 196 | 121 | 164 | 104 | 259 | |
FN | 72 | 104 | 43 | 97 | 64 | 125 | |
Precision | 83.37% | 87.10% | 85.93% | 80.66% | 90.46% | 84.20% | |
Recall | 93.48% | 92.71% | 94.50% | 87.58% | 93.90% | 91.69% | |
F1-score | 88.14% | 89.82% | 90.01% | 83.98% | 92.15% | 87.79% | |
Original algorithm | TP | 1025 | 1320 | 741 | 681 | 980 | 1382 |
FP | 212 | 200 | 119 | 165 | 110 | 257 | |
FN | 80 | 106 | 41 | 100 | 70 | 123 | |
Precision | 82.86% | 86.84% | 86.16% | 80.50% | 89.91% | 84.32% | |
Recall | 92.76% | 92.57% | 94.76% | 87.20% | 93.33% | 91.83% | |
F1-score | 87.53% | 89.61% | 90.26% | 83.71% | 91.59% | 87.91% |
© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Li, W.; He, C.; Fu, H.; Zheng, J.; Dong, R.; Xia, M.; Yu, L.; Luk, W. A Real-Time Tree Crown Detection Approach for Large-Scale Remote Sensing Images on FPGAs. Remote Sens. 2019, 11, 1025. https://doi.org/10.3390/rs11091025
Li W, He C, Fu H, Zheng J, Dong R, Xia M, Yu L, Luk W. A Real-Time Tree Crown Detection Approach for Large-Scale Remote Sensing Images on FPGAs. Remote Sensing. 2019; 11(9):1025. https://doi.org/10.3390/rs11091025
Chicago/Turabian StyleLi, Weijia, Conghui He, Haohuan Fu, Juepeng Zheng, Runmin Dong, Maocai Xia, Le Yu, and Wayne Luk. 2019. "A Real-Time Tree Crown Detection Approach for Large-Scale Remote Sensing Images on FPGAs" Remote Sensing 11, no. 9: 1025. https://doi.org/10.3390/rs11091025