Image Forgery Detection and Localization via a Reliability Fusion Map
<p>Generic framework for image tampering detection and localization, including image pre-processing, feature extraction, and post-processing.</p> "> Figure 2
<p>Flowchart of our proposed classifier.</p> "> Figure 3
<p>Architecture of the convolution neural network (CNN) module, with 13 conventional layers, three fully-connected layers and a softmax layer.</p> "> Figure 4
<p>Illustration of the reliability fusion map (RFM) algorithm pipeline (<b>a</b>), and step by step clustering approach of [<a href="#B26-sensors-20-06668" class="html-bibr">26</a>] (<b>b</b>). “px” is the abbreviation of “pixel”.</p> "> Figure 5
<p>Accuracy curves on training dataset (<math display="inline"><semantics> <msub> <mi>D</mi> <mi>T</mi> </msub> </semantics></math>) for Constrained-CNN [<a href="#B23-sensors-20-06668" class="html-bibr">23</a>], SCI-CNN [<a href="#B47-sensors-20-06668" class="html-bibr">47</a>] and RFM-CNN proposed in this work.</p> "> Figure 6
<p>Tampering localization with different pre-processing stages: (<b>a</b>) forged image; (<b>e</b>) ground truth; (<b>b</b>) SCI-CNN denoting grayscale input image without pre-processing operation; (<b>c</b>) Constrained-CNN; (<b>d</b>) RFM-CNN with pre-processing operation; (<b>f</b>–<b>h</b>) visualization results generated by different methods.</p> "> Figure 7
<p>ROC curves of tampering detection results using our RFM-based detector with various thresholds <math display="inline"><semantics> <msub> <mi>τ</mi> <mn>1</mn> </msub> </semantics></math> and <math display="inline"><semantics> <msub> <mi>τ</mi> <mn>2</mn> </msub> </semantics></math>.</p> "> Figure 8
<p>Accuracy (ACC), true positive rate (TPR) of proposed method with various thresholds <math display="inline"><semantics> <msub> <mi>τ</mi> <mn>1</mn> </msub> </semantics></math>, <math display="inline"><semantics> <msub> <mi>τ</mi> <mn>2</mn> </msub> </semantics></math> (on the left of the gray dashed line) and the other competing algorithms (on the right of the gray dashed line). Blue and orange dashed lines denote the best ACC and TPR results of our proposed RFM-based detector, respectively.</p> "> Figure 9
<p>Comparison of localization performance between our RFM-based detector and the algorithm of [<a href="#B26-sensors-20-06668" class="html-bibr">26</a>].</p> "> Figure 10
<p>Tampering localization using our proposed RFM-based detector; from left to right: (<b>a</b>) forged image, (<b>b</b>) ground truth, (<b>c</b>) pre-processing result, (<b>d</b>) detection result without RFM (only relying on feature extraction), and (<b>e</b>) detection result with RFM (by adopting post-processing procedure).</p> ">
Abstract
:1. Introduction
2. State of the Art
2.1. Pre-Processing Based Algorithms
2.2. Feature Extraction Based Algorithms
2.3. Post-Processing Based Algorithms
3. Proposed Method
3.1. Pre-Processing
3.2. Feature Extraction
3.2.1. CNN Module
3.2.2. Content-Texture Module
3.3. Reliability Fusing
- Patch texture . The parameter can provide information about content texture of inquiry patch, which tends to be low for flat patches and high for patches with high variance. Since CNN module cannot perform in low-texture regions as well as in high-texture regions, let us accordingly decrease CNN confidence in low-texture regions.
- CNN confidence . represents the output result of the CNN module extracted from , among which sum of all vectors equals to 1. Rather than truncating confidence by an empirical threshold, our proposed algorithm combines the CNN confidence for each patch, meaning that the algorithm accumulates the CNN confidence of adjacent patches around the inspected (or central) patch.
- Density distribution . represents a tampering ratio of K adjacent patches. is proposed to remove the mismatched results generated by the CNN confidence . The larger indicates the more forged adjacent patches around the inspected patch.
3.3.1. Fusing and
3.3.2. Fusing and
3.3.3. Designing Binary Classifier
4. Experimental Results
Algorithm 1: Procedure of generating forged images |
4.1. Pre-Processing Performance Evaluation
4.2. Tampering Detection
4.3. Tampering Localization
5. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- Qiao, T.; Shi, R.; Luo, X.; Xu, M.; Zheng, N.; Wu, Y. Statistical model-based detector via texture weight map: Application in re-sampling authentication. IEEE Trans. Multimedia 2018, 21, 1077–1092. [Google Scholar] [CrossRef]
- Qiao, T.; Zhu, A.; Retraint, F. Exposing image resampling forgery by using linear parametric model. Multimed. Tools Appl. 2018, 77, 1501–1523. [Google Scholar] [CrossRef]
- Amerini, I.; Ballan, L.; Caldelli, R.; Del Bimbo, A.; Del Tongo, L.; Serra, G. Copy-move forgery detection and localization by means of robust clustering with J-Linkage. Signal Process. Image Commun. 2013, 28, 659–669. [Google Scholar] [CrossRef]
- Pan, X.; Lyu, S. Region duplication detection using image feature matching. IEEE Trans. Inf. Forensics Secur. 2010, 5, 857–867. [Google Scholar] [CrossRef]
- Zhao, Y.; Zheng, N.; Qiao, T.; Xu, M. Source camera identification via low dimensional PRNU features. Multimed. Tools Appl. 2019, 78, 8247–8269. [Google Scholar] [CrossRef]
- Qiao, T.; Retraint, F.; Cogranne, R.; Thai, T.H. Individual camera device identification from JPEG images. Signal Process. Image Commun. 2017, 52, 74–86. [Google Scholar] [CrossRef] [Green Version]
- Qiao, T.; Retraint, F. Identifying Individual Camera Device From RAW Images. IEEE Access 2018, 6, 78038–78054. [Google Scholar] [CrossRef]
- Chen, M.; Fridrich, J.; Goljan, M.; Lukás, J. Determining image origin and integrity using sensor noise. IEEE Trans. Inf. Forensics Secur. 2008, 3, 74–90. [Google Scholar] [CrossRef] [Green Version]
- Hsu, Y.F.; Chang, S.F. Camera response functions for image forensics: An automatic algorithm for splicing detection. IEEE Trans. Inf. Forensics Secur. 2010, 5, 816–825. [Google Scholar] [CrossRef] [Green Version]
- Zhao, X.; Wang, S.; Li, S.; Li, J. Passive image-splicing detection by a 2-D noncausal Markov model. IEEE Trans. Circuits Syst. Video Technol. 2015, 25, 185–199. [Google Scholar] [CrossRef]
- Salloum, R.; Ren, Y.; Kuo, C.C.J. Image Splicing Localization Using A Multi-Task Fully Convolutional Network (MFCN). J. Vis. Commun. Image Represent 2018, 51, 201–209. [Google Scholar] [CrossRef] [Green Version]
- Bunk, J.; Bappy, J.H.; Mohammed, T.M.; Nataraj, L.; Flenner, A.; Manjunath, B.; Chandrasekaran, S.; Roy-Chowdhury, A.K.; Peterson, L. Detection and localization of image forgeries using resampling features and deep learning. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Honolulu, HI, USA, 21–26 July 2017; pp. 1881–1889. [Google Scholar]
- Chen, C.; McCloskey, S.; Yu, J. Image Splicing Detection via Camera Response Function Analysis. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Honolulu, HI, USA, 21–26 July 2017; pp. 5087–5096. [Google Scholar]
- Rao, Y.; Ni, J. A deep learning approach to detection of splicing and copy-move forgeries in images. In Proceedings of the 2016 IEEE International Workshop on Information Forensics and Security, Abu Dhabi, United Arab Emirates, 4–7 December 2016; pp. 1–6. [Google Scholar]
- Wu, Y.; Abd-Almageed, W.; Natarajan, P. Deep matching and validation network: An end-to-end solution to constrained image splicing localization and detection. In Proceedings of the 2017 ACM on Multimedia Conference, Silicon Valley, CA, USA, 23–27 December 2017; pp. 1480–1502. [Google Scholar]
- Cozzolino, D.; Poggi, G.; Verdoliva, L. Efficient dense-field copy–move forgery detection. IEEE Trans. Inf. Forensics Secur. 2015, 10, 2284–2297. [Google Scholar] [CrossRef]
- Soni, B.; Das, P.K.; Thounaojam, D.M. Copy-Move Tampering Detection based on Local Binary Pattern Histogram Fourier Feature. In Proceedings of the 7th International Conference on Computer and Communication Technology, Allahabad, India, 24–26 November 2017; pp. 78–83. [Google Scholar]
- Verdoliva, L.; Cozzolino, D.; Poggi, G. A feature-based approach for image tampering detection and localization. In Proceedings of the 2014 IEEE Information Forensics and Security (WIFS), Atlanta, GA, USA, 3–5 December 2014; pp. 149–154. [Google Scholar]
- Bappy, J.H.; Roy-Chowdhury, A.K.; Bunk, J.; Nataraj, L.; Manjunath, B. Exploiting spatial structure for localizing manipulated image regions. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017; pp. 4970–4979. [Google Scholar]
- Jin, X.; Su, Y.; Zou, L.; Zhang, C.; Jing, P.; Song, X. Video logo removal detection based on sparse representation. Multimed. Tools Appl. 2018, 77, 29303–29322. [Google Scholar] [CrossRef]
- Pibre, L.; Jérôme, P.; Ienco, D.; Chaumont, M. Deep Learning for Steganalysis Is Better Than a Rich Model with An Ensemble Classifier, and Is Natively Robust to the Cover Source-Mismatch. Available online: https://arxiv.org/abs/1511.04855 (accessed on 16 November 2015).
- Bayar, B.; Stamm, M.C. Augmented convolutional feature maps for robust cnn-based camera model identification. In Proceedings of the 2017 IEEE International Conference on Image Processing (ICIP), Beijing, China, 17–20 September 2017; pp. 4098–4102. [Google Scholar]
- Bayar, B.; Stamm, M.C. Constrained convolutional neural networks: A new approach towards general purpose image manipulation detection. IEEE Trans. Inf. Forensics Secur. 2018, 13, 2691–2706. [Google Scholar] [CrossRef]
- Li, H.; Luo, W.; Qiu, X.; Huang, J. Image forgery localization via integrating tampering possibility maps. IEEE Trans. Inf. Forensics Secur. 2017, 12, 1240–1252. [Google Scholar] [CrossRef]
- Güera, D.; Zhu, F.; Yarlagadda, S.K.; Tubaro, S.; Bestagini, P.; Delp, E.J. Reliability map estimation for CNN-based camera model attribution. In Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, NV, USA, 12–15 March 2018; pp. 964–973. [Google Scholar]
- Bondi, L.; Lameri, S.; Güera, D.; Bestagini, P.; Delp, E.J.; Tubaro, S. Tampering detection and localization through clustering of camera-based CNN features. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA, 21–26 July 2017; pp. 1855–1864. [Google Scholar]
- Kang, X.; Stamm, M.C.; Peng, A.; Liu, K.R. Robust median filtering forensics using an autoregressive model. IEEE Trans. Inf. Forensics Secur. 2013, 8, 1456–1468. [Google Scholar] [CrossRef] [Green Version]
- Chierchia, G.; Cozzolino, D.; Poggi, G.; Sansone, C.; Verdoliva, L. Guided filtering for PRNU-based localization of small-size image forgeries. In Proceedings of the 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Florence, Italy, 4–9 May 2014; pp. 6231–6235. [Google Scholar]
- Popescu, A.C.; Farid, H. Exposing digital forgeries by detecting traces of resampling. IEEE Trans. Signal Process. 2005, 53, 758–767. [Google Scholar] [CrossRef]
- Kirchner, M. Fast and reliable resampling detection by spectral analysis of fixed linear predictor residue. In Proceedings of the 10th ACM Workshop on Multimedia and Security, Oxford, UK, 22–23 September 2008; pp. 11–20. [Google Scholar]
- Qiu, X.; Li, H.; Luo, W.; Huang, J. A universal image forensic strategy based on steganalytic model. In Proceedings of the 2nd ACM Workshop on Information Hiding and Multimedia Security, Salzburg, Austria, 11 June 2014; pp. 165–170. [Google Scholar]
- Cozzolino, D.; Verdoliva, L. Noiseprint: A CNN-based camera model fingerprint. arXiv 2019, arXiv:1808.08396. [Google Scholar]
- Zhang, Z.; Zhang, Y.; Zhou, Z.; Luo, J. Boundary-based image forgery detection by fast shallow cnn. In Proceedings of the 2018 24th International Conference on Pattern Recognition (ICPR), Beijing, China, 20–24 August 2018; pp. 2658–2663. [Google Scholar]
- Mazaheri, G.; Mithun, N.C.; Bappy, J.H.; Roy-Chowdhury, A.K. A Skip Connection Architecture for Localization of Image Manipulations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, USA, 16–17 June 2019; pp. 119–129. [Google Scholar]
- Bi, X.; Wei, Y.; Xiao, B.; Li, W. RRU-Net: The Ringed Residual U-Net for Image Splicing Forgery Detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, USA, 16–17 June 2019; pp. 30–39. [Google Scholar]
- Chen, Y.; Kang, X.; Shi, Y.Q.; Wang, Z.J. A multi-purpose image forensic method using densely connected convolutional neural networks. J. REAL-TIME IMAGE PR. 2019, 16, 725–740. [Google Scholar] [CrossRef]
- Bondi, L.; Güera, D.; Baroffio, L.; Bestagini, P.; Delp, E.J.; Tubaro, S. A preliminary study on convolutional neural networks for camera model identification. J. Electron. Imaging 2017, 2017, 67–76. [Google Scholar] [CrossRef] [Green Version]
- Farid, H. Exposing digital forgeries from JPEG ghosts. IEEE Trans. Inf. Forensics Secur. 2009, 4, 154–160. [Google Scholar] [CrossRef]
- Bianchi, T.; Piva, A. Image forgery localization via block-grained analysis of JPEG artifacts. IEEE Trans. Inf. Forensics Secur. 2012, 7, 1003–1017. [Google Scholar] [CrossRef] [Green Version]
- Cozzolino, D.; Gragnaniello, D.; Verdoliva, L. Image forgery localization through the fusion of camera-based, feature-based and pixel-based techniques. In Proceedings of the 2014 IEEE International Conference on Image Processing (ICIP), Paris, France, 27–30 October 2014; pp. 5302–5306. [Google Scholar]
- Gaborini, L.; Bestagini, P.; Milani, S.; Tagliasacchi, M.; Tubaro, S. Multi-clue image tampering localization. In Proceedings of the 2014 IEEE International Workshop on Information Forensics and Security (WIFS), Atlanta, GA, USA, 3–5 December 2014; pp. 125–130. [Google Scholar]
- Korus, P.; Huang, J. Multi-scale fusion for improved localization of malicious tampering in digital images. IEEE Trans. Image Process. 2016, 25, 1312–1326. [Google Scholar] [CrossRef]
- Korus, P.; Huang, J. Improved tampering localization in digital image forensics based on maximal entropy random walk. IEEE Signal Process Lett. 2016, 23, 169–173. [Google Scholar] [CrossRef]
- Qian, Y.; Dong, J.; Wang, W.; Tan, T. Deep learning for steganalysis via convolutional neural networks. In Proceedings of the Media Watermarking, Security, and Forensics 2015, San Francisco, CA, USA, 9–11 February 2015; Volume 9409, p. 94090. [Google Scholar]
- Pibre, L.; Pasquet, J.; Ienco, D.; Chaumont, M. Deep learning is a good steganalysis tool when embedding key is reused for different images, even if there is a cover sourcemismatch. Electron. Imaging 2016, 2016, 1–11. [Google Scholar] [CrossRef] [Green Version]
- Tuama, A.; Comby, F.; Chaumont, M. Camera model identification with the use of deep convolutional neural networks. In Proceedings of the 2016 IEEE International Workshop on Information Forensics and Security (WIFS), Abu Dhabi, United Arab Emirates, 4–7 December 2016; pp. 1–6. [Google Scholar]
- Yao, H.; Qiao, T.; Xu, M.; Zheng, N. Robust Multi-Classifier for Camera Model Identification Based on Convolution Neural Network. IEEE Access 2018, 6, 24973–24982. [Google Scholar] [CrossRef]
- Gloe, T.; Böhme, R. The dresden image database for benchmarking digital image forensics. J. Digit. Forensic Pract. 2010, 3, 150–159. [Google Scholar] [CrossRef]
- Bottou, L. Large-scale machine learning with stochastic gradient descent. In Proceedings of the 19th International Conference on Computational Statistics (COMPSTAT), Paris, France, 22–27 August 2010; pp. 177–186. [Google Scholar]
- Fan, B.; Kong, Q.; Zhang, B.; Liu, H.; Pan, C.; Lu, J. Efficient nearest neighbor search in high dimensional hamming space. Pattern Recognit. 2020, 99, 107082. [Google Scholar] [CrossRef]
- Chen, B.; Tan, W.; Coatrieux, G.; Zheng, Y.; Shi, Y.Q. A serial image copy-move forgery localization scheme with source/target distinguishment. IEEE Trans. Multimedia 2020. [Google Scholar] [CrossRef]
- Fan, B.; Liu, H.; Zeng, H.; Zhang, J.; Liu, X.; Han, J. Deep Unsupervised Binary Descriptor Learning through Locality Consistency and Self Distinctiveness. IEEE Trans. Multimedia 2020. [Google Scholar] [CrossRef]
ID | Input Size | Configuration | Type |
---|---|---|---|
conv 1 | 64 × 64-3 | stride = 2, ksize = 8 × 8 | convReLU |
conv 2 | 32 × 32-16 | stride = 1, ksize = 8 × 8 | convReLU |
conv 3 | 32 × 32-32 | stride = 2, ksize = 6 × 6 | convReLU |
conv 4 | 16 × 16-48 | stride = 1, ksize = 6 × 6 | convReLUmaxpool |
conv 5 | 16 × 16-64 | stride = 1, ksize = 3 × 3 | convReLU |
conv 6 | 16 × 16-128 | stride = 2, ksize = 3 × 3 | convReLU |
conv 7 | 8 × 8-256 | stride = 1, ksize = 3 × 3 | convReLUmaxpool |
conv 8 | 8 × 8-512 | stride = 2, ksize = 3 × 3 | convReLU |
conv 9 | 8 × 8-1024 | stride = 2, ksize = 3 × 3 | convReLUmaxpool |
conv 10 | 4 × 4-512 | stride = 1, ksize = 1 × 1 | convReLU |
conv 11 | 4 × 4-256 | stride = 1, ksize = 1 × 1 | convReLU |
conv 12 | 4 × 4-128 | stride = 2, ksize = 1 × 1 | convReLU |
conv 13 | 1 × 1-64 | stride = 2, ksize = 1 × 1 | convReLUmaxpool |
Threshold | ACC | TPR | FPR |
---|---|---|---|
= 0.0 = 0.015 | 0.904 | 0.828 | 0.020 |
= 0.4 = 0.015 | 0.942 | 0.910 | 0.026 |
= 0.6 = 0.015 | 0.949 | 0.942 | 0.044 |
= 0.6 = 0.012 | 0.892 | 0.792 | 0.008 |
Average | 0.922 | 0.868 | 0.025 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Yao, H.; Xu, M.; Qiao, T.; Wu, Y.; Zheng, N. Image Forgery Detection and Localization via a Reliability Fusion Map. Sensors 2020, 20, 6668. https://doi.org/10.3390/s20226668
Yao H, Xu M, Qiao T, Wu Y, Zheng N. Image Forgery Detection and Localization via a Reliability Fusion Map. Sensors. 2020; 20(22):6668. https://doi.org/10.3390/s20226668
Chicago/Turabian StyleYao, Hongwei, Ming Xu, Tong Qiao, Yiming Wu, and Ning Zheng. 2020. "Image Forgery Detection and Localization via a Reliability Fusion Map" Sensors 20, no. 22: 6668. https://doi.org/10.3390/s20226668