Advances in Multi-Sensor Data Fusion: Algorithms and Applications
<p>An overview of categorization of the fusion algorithms [<a href="#b8-sensors-09-07771" class="html-bibr">8</a>].</p> ">
<p>Generic flowchart of wavelet-based image fusion.</p> ">
<p>General schematic diagram of the ANN-based image fusion method.</p> ">
<p>Image fusion for change detection: an example in Yanqing city, Beijing, China.</p> ">
Abstract
:1. Introduction
2. Advances in Algorithms
2.1. Categorization of the algorithms
- Signal level fusion. In signal-based fusion, signals from different sensors are combined to create a new signal with a better signal-to noise ratio than the original signals.
- Pixel level fusion. Pixel-based fusion is performed on a pixel-by-pixel basis. It generates a fused image in which information associated with each pixel is determined from a set of pixels in source images to improve the performance of image processing tasks such as segmentation
- Feature level fusion. Feature-based fusion at feature level requires an extraction of objects recognized in the various data sources. It requires the extraction of salient features which are depending on their environment such as pixel intensities, edges or textures. These similar features from input images are fused.
- Decision-level fusion consists of merging information at a higher level of abstraction, combines the results from multiple algorithms to yield a final fused decision. Input images are processed individually for information extraction. The obtained information is then combined applying decision rules to reinforce common interpretation.
2.2. Advances in fusion algorithms
2.2.1. Standard fusion algorithms
2.2.2. Wavelet-based methods
2.2.3. Artificial neural network
3. Advances in Applications
3.1. Object identification
3.2. Classification
3.3. Change detection
3.4. Maneuvering target tracking
4. Discussion and Conclusions
(1) Improvements of fusion algorithms
(2) Development of “algorithm fusion” methods
- Design of a general framework for combination of different fusion approaches;
- Development of new approaches which can combine aspects of pixel/feature/decision level image fusion;
- Establishment of automatic quality assessment method for evaluation of fusion results, which is discussed as follows.
(3) Establishment of an automatic quality assessment scheme
Acknowledgments
References and Notes
- Hall, L.; Llinas, J. An introduction to multisensor data fusion. Proc. IEEE. 1997, 85, 6–23. [Google Scholar]
- Pohl, C.; Van Genderen, J.L. Multisensor image fusion in remote sensing: concepts, methods and applications. Int. J. Remote Sens. 1998, 19, 823–854. [Google Scholar]
- Simone, G.; Farina, A.; Morabito, F.C.; Serpico, S.B.; Bruzzone, L. Image fusion techniques for remote sensing applications. Inf. Fusion 2002, 3, 3–15. [Google Scholar]
- Vijayaraj, V.; Younan, N.; O'Hara, C. Concepts of image fusion in remote sensing applications. Proceedings of IEEE International Conference on Geoscience and Remote Sensing Symposium, Denver, CO, USA, July 31–August 4, 2006; pp. 3798–3801.
- Dasarathy, B.V. A special issue on image fusion: advances in the state of the art. Inf. Fusion 2007, 8. [Google Scholar] [CrossRef]
- Smith, M.I.; Heather, J.P. Review of image fusion technology in 2005. Proceedings of Defense and Security Symposium, Orlando, FL, USA; 2005. [Google Scholar]
- Blum, R.S.; Liu, Z. Multi-Sensor Image Fusion and Its Applications; special series on Signal Processing and Communications; CRC Press: Boca Raton, FL, USA, 2006. [Google Scholar]
- Dai, X.; Khorram, S. Data fusion using artificial neural networks: a case study on multitemporal change analysis. Comput. Environ. Urban Syst. 1999, 23, 19–31. [Google Scholar]
- Yun, Z. Understanding image fusion. Photogram. Eng. Remote Sens. 2004, 6, 657–661. [Google Scholar]
- Pouran, B. Comparison between four methods for data fusion of ETM+ multispectral and pan images. Geo-spat. Inf. Sci. 2005, 8, 112–122. [Google Scholar]
- Adelson, C.H.; Bergen, J.R. Pyramid methods in image processing. RCA Eng. 1984, 29, 33–41. [Google Scholar]
- Miao, Q.G.; Wang, B.S. Multi-sensor image fusion based on improved laplacian pyramid transform. Acta Opti. Sin. 2007, 27, 1605–1610. [Google Scholar]
- Xiang, J.; Su, X. A pyramid transform of image denoising algorithm based on morphology. Acta Photon. Sin. 2009, 38, 89–103. [Google Scholar]
- Mallat, S.G. A theory for multiresolution signal decomposition: the wavelet representation. IEEE Trans. Pattern Anal. Mach. Intell. 1989, 11, 674–693. [Google Scholar]
- Ganzalo, P.; Jesus, M.A. Wavelet-based image fusion tutorial. Pattern Recognit. 2004, 37, 1855–1872. [Google Scholar]
- Ma, H.; Jia, C.Y.; Liu, S. Multisource image fusion based on wavelet transform. Int. J. Inf. Technol. 2005, 11, 81–91. [Google Scholar]
- Krista, A.; Yun, Z.; Peter, D. Wavelet based image fusion techniques – An introduction, review and comparison. ISPRS J. Photogram. Remote Sens. 2007, 62, 249–263. [Google Scholar]
- Candes, E.J.; Donoho, D.L. Curvelets-A Surprisingly Effective Nonadaptive Representation for Objects with Edges.Curves and Surfcaces; Vanderbilt University Press: Nashville, TN, USA, 2000; pp. 105–120. [Google Scholar]
- Choi, M.; Kim, R.Y.; Nam, M.R. Fusion of multi-spectral and panchromatic satellite images using the Curvelet transform. IEEE Geosci. Remote Sens. Lett. 2005, 2, 136–140. [Google Scholar]
- Donoho, M.N.; Vetterli, M. Contourlets; Academic Press: New York, NY, USA, 2002. [Google Scholar]
- Minh, N.; Martin, V. The contourlet transform: an efficient directional multiresolution image representation. 2005. Available online: http://lcavwww.epfl.ch/~vetterli/IP-4-2005.pdf (accessed June 29, 2009).
- Louis, E.K.; Yan, X.H. A neural network model for estimating sea surface chlorophyll and sediments from thematic mapper imagery. Remote Sens. Environ. 1998, 66, 153–165. [Google Scholar]
- Dong, J.; Yang, X.; Clinton, N.; Wang, N. An artificial neural network model for estimating crop yields using remotely sensed information. Int. J. Remote Sens. 2004, 25, 1723–1732. [Google Scholar]
- Shutao, L.; Kwok, J.T.; Yaonan, W. Multifocus image fusion using artificial neural networks. Pattern Recognit. Lett. 2002, 23, 985–997. [Google Scholar]
- Thomas, F.; Grzegorz, G. Optimal fusion of TV and infrared images using artificial neural networks. Proceedings of Applications and Science of Artificial Neural Networks, Orlando, FL, USA, April 21, 1995; 2492, pp. 919–925.
- Huang, W.; Jing, Z. Multi-focus image fusion using pulse coupled neural network. Pattern Recognit. Lett. 2007, 28, 1123–1132. [Google Scholar]
- Wu, Y.; Yang, W. Image fusion based on wavelet decomposition and evolutionary strategy. Acta Opt. Sin. 2003, 23, 671–676. [Google Scholar]
- Sun, Z.Z.; Fu, K.; Wu, Y.R. The high-resolution SAR image terrain classification algorithm based on mixed double hint layers RBFN model. Acta Electron. Sin. 2003, 31, 2040–2044. [Google Scholar]
- Zhang, H.; Sun, X.N.; Zhao, L.; Liu, L. Image fusion algorithm using RBF neural networks. Lect. Notes Comput. Sci. 2008, 9, 417–424. [Google Scholar]
- Gail, A.; Siegfried, M.; Ogas, J. Self-organizing information fusion and hierarchical knowledge discovery- a new framework using ARTMAP neural networks. Neural Netw. 2005, 18, 287–295. [Google Scholar]
- Gail, A.; Siegfried, M.; Ogas, J. Self-organizing hierarchical knowledge discovery by an ARTMAP image fusion system. Proceedings of the 7th International Conference on Information Fusion, Stockholm, Sweden; 2004; pp. 235–242. [Google Scholar]
- Wang, R.; Bu, F.L.; Jin, H.; Li, L.H. A feature-level image fusion algorithm based on neural networks. Bioinf. Biomed. Eng. 2007, 7, 821–824. [Google Scholar]
- Jin, X.Y.; Davis, C.H. An integrated system for automatic road mapping from high-resolution multi-spectral satellite imagery by information fusion. Inf. Fusion 2005, 6, 257–273. [Google Scholar]
- Garzelli, A.; Nencini, F. Panchromatic sharpening of remote sensing images using a multiscale Kalman filter. Pattern Recognit. 2007, 40, 3568–3577. [Google Scholar]
- Wu, Y.; Yang, W. Image fusion based on wavelet decomposition and evolutionary strategy. Acta Opt. Sin. 2003, 23, 671–676. [Google Scholar]
- Sarkar, A.; Banerjee, A.; Banerjee, N.; Brahma, S.; Kartikeyan, B.; Chakraborty, M.; Majumder, K.L. Landcover classification in MRF context using Dempster-Shafer fusion for multisensor imagery. IEEE Trans. Image Processing 2005, 14, 634–645. [Google Scholar]
- Liu, C.P.; Ma, X.H.; Cui, Z.M. Multi-source remote sensing image fusion classification based on DS evidence theory. Proceedings of Conference on Remote Sensing and GIS Data Processing and Applications; and Innovative Multispectral Technology and Applications, Wuhan, China, November 15–17, 2007; 6790. part 2.
- Rottensteiner, F.; Trinder, J.; Clode, S.; Kubik, K.; Lovell, B. Building detection by Dempster-Shafer fusion of LIDAR data and multispectral aerial imagery. Proceedings of the 17th International Conference on Pattern Recognition, Cambridge, UK, August 23–26, 2004; 2, pp. 339–342.
- Ruvimbo, G.; Philippe, D.; Morgan, D. Object-oriented change detection for the city of Harare, Zimbabwe. Exp. Syst. Appl. 2009, 36, 571–588. [Google Scholar]
- Madhavan, B.B.; Sasagawa, T.; Tachibana, K.; Mishra, K. A decision level fusion of ADS-40, TABI and AISA data. Nippon Shashin Sokuryo Gakkai Gakujutsu Koenkai Happyo Ronbunshu 2005, 2005, 163–166. [Google Scholar]
- Duncan, S.; Sameer, S. Approaches to multisensor data fusion in target tracking: a survey. IEEE Trans. Knowl. Data Eng. 2006, 18, 1696–1710. [Google Scholar]
- Chen, Y.; Han, C. Maneuvering vehicle tracking based on multi-sensor fusion. Acta Autom. Sin. 2005, 31, 625–630. [Google Scholar]
- Liu, C.; Feng, X. An algorithm of tracking a maneuvering target based on ir sensor and radar in dense environment. J. Air Force Eng. Univ. 2006, 7, 25–28. [Google Scholar]
- Zheng, M.; Zhao, Y.; Tian, H. Maneuvering target tracking based on fusion of multi-sensor. J. Detect. Control 2006, 28, 43–45. [Google Scholar]
- Vahdati-khajeh, E. Tracking the maneuvering targets using multiple scan joint probabilistic data association algorithm. Proceedings of the Target Tracking 2004: Algorithms and Applications, IEE, Brighton, UK, January 1, 2004; Sussex University.
- Zhang, Y.; Hong, G. An HIS and wavelet integrated approach to improve pan-sharpening visual quality of natural color IKONOS and QuickBird image. Inf. Fusion. 2005, 6, 225–234. [Google Scholar]
- Chen, Y.; Xue, Z.Y.; Blum, R.S. Theoretical analysis of an information-based quality measure for image fusion. Inf. Fusion 2008, 9, 161–175. [Google Scholar]
- Zhao, J.Y.; Laganiere, R.; Liu, Z. Image fusion algorithm assessment based on feature measurement. Proceedings of the 1st International Conference on Innovative Computing, Information and Control, Beijing, China, August 30–September 1, 2006; 2, pp. 701–704.
- Goshtasby, A.; Nikolov, S. Image fusion: advances in the state of the art. Inf. Fusion. 2007, 8, 114–118. [Google Scholar]
© 2009 by the authors; licensee Molecular Diversity Preservation International, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).
Share and Cite
Dong, J.; Zhuang, D.; Huang, Y.; Fu, J. Advances in Multi-Sensor Data Fusion: Algorithms and Applications. Sensors 2009, 9, 7771-7784. https://doi.org/10.3390/s91007771
Dong J, Zhuang D, Huang Y, Fu J. Advances in Multi-Sensor Data Fusion: Algorithms and Applications. Sensors. 2009; 9(10):7771-7784. https://doi.org/10.3390/s91007771
Chicago/Turabian StyleDong, Jiang, Dafang Zhuang, Yaohuan Huang, and Jingying Fu. 2009. "Advances in Multi-Sensor Data Fusion: Algorithms and Applications" Sensors 9, no. 10: 7771-7784. https://doi.org/10.3390/s91007771