CN116091429A - A detection method for mosaic tampered images - Google Patents

Abstract

The invention discloses a method for detecting spliced tampered images. The method of the present invention first collects and downloads public image tampering datasets, including CASIA datasets and COLUMBIA datasets, a part as a training set, and the others as a test set; then use the training set to carry out model training for a hybrid transformer neural network, including: a feature extraction module , self-attention U-shaped block, cross-attention module, and feature decoding module; finally, use the trained network model to test the test set image to obtain the final detection effect. The method of the present invention integrates self-attention and cross-attention into U ² -Net, which can capture more text information and spatial correlation from different scales, and improve the correctness of prediction results. The present invention can avoid large-scale pre-training by using the inductive deviation of convolution, and can locate splicing and tampering regions of various scales.

Description

A method for detecting spliced tampered images

Technical Field

The present invention belongs to the field of computer technology, in particular to the field of computer vision and digital image processing technology, and relates to a method for detecting spliced tampered images, specifically an image tampering detection method based on a hybrid transformer neural network, which can accurately locate tampered areas at different scales and has good positioning performance.

Background Art

With the rapid development of modern mobile devices, it has become very easy to generate and transmit digital images. At the same time, image editing software is easy to operate, making it easy for anyone to modify images. Some spliced and tampered images may be abused maliciously and have a negative impact on society and the country. Therefore, it is becoming increasingly important to detect spliced and tampered images.

At present, the detection technology of image tampering area is mainly divided into two types: detection method based on traditional feature extraction and detection method based on convolutional neural network. Most traditional detection methods extract specific image fingerprints for detection, such as color filter array interpolation, sensor noise, and illumination inconsistency (PRNU). Traditional detection methods can only detect a specific image fingerprint, but when the specific fingerprint in the image does not exist or is not obvious, the detection will fail. In addition, the specific image fingerprint will be affected by post-processing effects such as image blur, JPEG compression and downsampling, which will reduce the detection results of traditional detection methods. In recent years, many CNN-based tampering detection methods have been proposed and have shown better performance than traditional methods. These CNN-based tampering detection methods can extract multiple image fingerprints at the same time, making up for the defects of traditional methods that rely on single image attributes, lack of generalization ability and poor robustness. For example, Faster R-CNN uses RGB streams and noise streams to detect tampered areas of a given tampered image. They use the SRM filter layer to extract noise features from the tampered image, which allows the model to capture the noise inconsistency between the tampered area and the real area. However, the limitation of this method is that it can only achieve regional-level tampering localization results. The RRU-Net structure captures the distinguishing features between tampered and non-tampered areas through residual propagation and residual feedback modules, thereby enhancing the learning mode of CNN. However, RRU-Net is not good at extracting global features, resulting in poor positioning of large-scale tampered areas and high false positive rates. MWC-Net can learn more comprehensive and representative features. However, this method still does not consider global features, which may lead to poor performance when locating large-scale tampered areas.

Most previous works ignore that the size of tampered regions varies and have difficulty locating tampered regions of different sizes. Due to the inherent locality of convolution operations, CNN-based methods have difficulty learning explicit global semantic information relationships and jointly utilizing local and global features. Therefore, most CNN-based detection methods can only handle limited scale variations. In addition, these methods may suffer from incomplete positioning or high false detection rates when locating large-scale tampered regions.

Summary of the invention

The purpose of the present invention is to provide an image tampering detection method based on convolutional neural network in view of the deficiencies in the prior art.

The method of the present invention specifically comprises:

Step (1) Collect and download public image tampering datasets, including the CASIA dataset and the COLUMBIA dataset; take 80% to 90% of them as training sets and the rest as test sets;

Step (2) using the training set to perform model training on the hybrid transformer neural network, including: feature extraction module, self-attention U-shaped block, cross attention module, feature decoding module;

Further, the feature extraction module uses the encoder part of U ² -Net to extract the feature map X of the input image; first, the input image of size 288×288×3 is sent to the residual U-shaped block-7, and two convolutions with a convolution kernel size of 3×3, a step size of 1, and a padding of 1 are performed, the batch normalization data flow layer norm, and the activation function Relu are performed; then five convolutions with a convolution kernel size of 3×3, a step size of 1, and a padding of 1 plus maximum pooling downsampling are performed, the batch normalization data flow layer norm, and the activation function Relu are performed; then a convolution with a convolution kernel size of 3×3, a step size of 1, a padding of 2, and a void rate of 2 is performed, the batch normalization data flow layer norm, and the activation function Relu are performed; then a convolution with a convolution kernel size of 3×3, a step size of 1, a padding of 2, and a void rate of 2 is performed, the batch normalization data flow layer norm, and the activation function Relu are performed; then a convolution with a convolution kernel size of 3×3, a step size of 3×3, a step size of The convolution with a kernel size of 1 and padding of 1, the batch normalized data flow layer norm, and the activation function Relu are performed; then five convolutions with a kernel size of 3×3, a step size of 1, and padding of 1 are performed plus sampling, the batch normalized data flow layer norm, and the activation function Relu are performed; then the output feature map after the first convolution and the feature map after the last convolution are added together, and finally 64 feature maps are obtained. The feature map output by the residual U-shaped block-7 is subjected to a maximum pooling downsampling operation to reduce the feature map resolution to half of the original; then the feature map is sent to the residual U-shaped block-6, residual U-shaped block-5, residual U-shaped block-4, and residual U-shaped block-4F in turn, and the final output size is a 9×9×512 feature map.

Furthermore, the self-attention U-shaped block uses self-attention to model the long-range dependency of the input image and extract the global information of the input image; the feature map of size 9×9×512 is used as input for two convolutions with a kernel size of 3×3, a step size of 1, and a padding of 1, a batch normalized data flow layer norm, and an activation function Relu; then a convolution with a kernel size of 3×3, a step size of 1, a padding of 2, and a void rate of 2 is performed, a batch normalized data flow layer norm, and an activation function Relu; then a convolution with a kernel size of 3×3, a step size of 1, a padding of 4, and a void rate of 4 is performed, a batch normalized data flow layer norm, and an activation function Relu; then a convolution with a kernel size of 3×3, a step size of 1, a padding of 8, and a void rate of 8 is performed, a batch normalized data flow layer norm, and an activation function Relu;

A∈R^n×n为注意力矩阵，上标T表示转置；The feature map is then fed into the self-attention module, which adds a positional encoding to the input feature map X∈Rd ^×H×W , where H and W are the height and width of the feature map, d is the number of channels, and R represents the real domain; X is then flattened and transposed into a sequence of size n×d, with parameter n=H×W; three 1×1 convolutional projections X are used for query, key, and value embedding: Q,K,V∈Rd ^×n , where Q represents the query matrix, K represents the key matrix, and V represents the value matrix; the output of the self-attention is a scaled dot product:

A∈R ^n×n is the attention matrix, and the superscript T indicates transposition;

Perform another convolution with a kernel size of 3×3, a stride of 1, a padding of 4, and a void rate of 4, batch normalize the data flow layer norm, and activate the Relu function; perform another convolution with a kernel size of 3×3, a stride of 1, a padding of 2, and a void rate of 2, batch normalize the data flow layer norm, and activate the Relu function; perform another convolution with a kernel size of 3×3, a stride of 1, and a padding of 1, batch normalize the data flow layer norm, and activate the Relu function; then add the output feature map of the first convolution and the feature map of the last convolution, and the final output size is a 9×9×512 feature map.

Further, the cross attention module uses cross attention to filter non-semantic features between the feature extraction module and the feature decoding module; first, position encoding is added to the low-level feature map U∈R ^d×2H×2W , H and W are the height and width of the feature map, d is the number of channels, and R represents the real number domain; then, the low-level feature map U is downsampled once and used as a value matrix of the cross attention module; position encoding is added to the high-level feature map N∈R ^2d×H×W , and the number of channels is changed to d by 1×1 convolution as a query matrix and a key matrix;

Then, the attention matrix A∈Rn ^×n is calculated using matrix multiplication and softmax function, where n=H×W; the calculated weight values are rescaled by the Relu activation function to obtain the result S as the filter, where low-amplitude elements represent noise or irrelevant areas to be reduced;

Take the Hadamard product of U and S to get the version of U with non-semantic features filtered out;

Finally, the result of the filtering operation is concatenated with the high-level feature map N.

其中(r,c)是像素坐标，p_(r,c)和y_(r,c)分别表示输入图像预测的像素值和标定真值的像素值。Furthermore, the feature decoding module decodes the feature map with global information and local information, and predicts the tampered area at the pixel level; the feature map obtained by the self-attention U-shaped block is upsampled and cascaded with the output feature map of the cross attention module, and the obtained feature map is sent to the residual U-shaped block-4F; the output feature map of the residual U-shaped block-4F is upsampled and cascaded with the output feature map of the cross attention module, and the obtained feature map is sent to the residual U-shaped block-4; the output feature map of the residual U-shaped block-4 is upsampled and cascaded with the output feature map of the cross attention module, and the obtained feature map is sent to the residual U-shaped block-5; the output feature map of the residual U-shaped block-5 is upsampled and cascaded with the output feature map of the cross attention module The output feature map of the attention module is cascaded, and the obtained feature map is sent to the residual U-shaped block-6; the output feature map of the residual U-shaped block-6 is upsampled and cascaded with the output feature map of the cross attention module, and the obtained feature map is sent to the residual U-shaped block-7; the feature map of the residual U-shaped block-7 with a size of 288×288×64 is output through a 3×3 convolution layer to output a feature map of size 288×288×1; and then the single-channel tampering probability mask map with a size of 288×288×1 is obtained through the activation function Sigmiod; the entire neural network optimizes the predicted results by minimizing the cross entropy loss function through the stochastic gradient descent optimization algorithm. The cross entropy loss function is:

Where (r, c) is the pixel coordinate, p _{(r, c)} and y _{(r, c)} represent the pixel values of the predicted input image and the calibrated true value, respectively.

Step (3) uses the trained network model to test the test set images to obtain the final detection effect.

z_i是卷积操作后的结果。The activation function Relu is defined as f(z _i ) = max(0,z _i ), where z _i is the result of the convolution operation. If z _i ≤ 0, f(z _i ) = 0, and if z _i > 0, f(z _i ) = z _i . The activation function Sigmiod is defined as

_zi is the result after the convolution operation.

The present invention integrates self-attention and cross-attention into U ² -Net, which can capture more text information and spatial correlation from different scales. The cross-attention module can filter out non-semantic features, enhance the low-level feature maps connected by jumps under the guidance of high-level semantic information, and achieve fine spatial recovery in the decoder, thereby ultimately improving the correctness of the prediction results. Secondly, the last block of the encoder of the present invention applies self-attention to combine the advantages of convolution and self-attention mechanisms. Therefore, the present invention can utilize the inductive bias of convolution to avoid large-scale pre-training, as well as the ability of Transformer to learn and model explicit global and long-range semantic information dependencies. In short, the present invention combines convolution and Transformer together, and can locate spliced tampering areas of various scales, thereby achieving state-of-the-art performance on two public image tampering datasets, CASIA and COLUMBIA.

DETAILED DESCRIPTION

The technical solution of the present invention is further described below through specific embodiments.

A method for detecting spliced tampered images is as follows:

Step (1) collect and download public image tampering datasets, including the CASIA dataset and the COLUMBIA dataset; take 80% to 90% of them as training sets and the rest as test sets. In this embodiment, 85% is taken as the training set.

Step (2) using the training set to perform model training on the hybrid transformer neural network, including: feature extraction module, self-attention U-shaped block, cross attention module, feature decoding module;

The feature extraction module uses the encoder part of U ² -Net to extract the feature map X of the input image; specifically:

The image of size 288×288×3 is fed into residual U-shaped block-7, and two convolutions with kernel size of 3×3, stride of 1, padding of 1 are performed, batch normalization data flow layer norm, activation function Relu; then five convolutions with kernel size of 3×3, stride of 1, padding of 1 plus maximum pooling downsampling are performed, batch normalization data flow layer norm, activation function Relu; then one more convolution with kernel size of 3×3, stride of 1, padding of 2, dilation rate of 2, batch normalization data flow layer norm, activation function Relu; Perform another convolution with a kernel size of 3×3, a stride of 1, and a padding of 1, batch normalization data flow layer norm, and activation function Relu; perform five more convolutions with a kernel size of 3×3, a stride of 1, and a padding of 1 plus sampling, batch normalization data flow layer norm, and activation function Relu; add the output feature map after the first convolution and the feature map after the last convolution to obtain an output feature map of size 288×288×64, and after the maximum pooling data flow layer pool, the final output size is 144×144×64 feature map;

The feature map of size 144×144×64 is further fed into the residual U-shaped block-6, and two convolutions with a kernel size of 3×3, a stride of 1, and a padding of 1 are performed, the batch normalized data flow layer norm, and the activation function Relu are performed; then four convolutions with a kernel size of 3×3, a stride of 1, and a padding of 1 plus maximum pooling downsampling are performed, the batch normalized data flow layer norm, and the activation function Relu are performed; and another convolution with a kernel size of 3×3, a stride of 1, a padding of 2, and a void rate of 2 is performed, the batch normalized data flow layer norm, and the activation function Relu is performed. ; Perform another convolution with a kernel size of 3×3, a stride of 1, and a padding of 1, batch normalization data flow layer norm, and activation function Relu; perform four more convolutions with a kernel size of 3×3, a stride of 1, and a padding of 1 plus sampling, batch normalization data flow layer norm, and activation function Relu; add the output feature map after the first convolution and the feature map after the last convolution to obtain an output feature map of size 144×144×128, and after the maximum pooling data flow layer pool, the final output size is 72×72×128 feature map;

The feature map of size 72×72×128 is further fed into the residual U-shaped block-5, and two convolutions with a kernel size of 3×3, a stride of 1, and a padding of 1 are performed, the batch normalized data flow layer norm, and the activation function Relu are performed; then three convolutions with a kernel size of 3×3, a stride of 1, and a padding of 1 plus maximum pooling downsampling are performed, the batch normalized data flow layer norm, and the activation function Relu are performed; and another convolution with a kernel size of 3×3, a stride of 1, a padding of 2, and a void rate of 2 is performed, the batch normalized data flow layer norm, and the activation function Relu is performed. ; Perform another convolution with a kernel size of 3×3, a stride of 1, and a padding of 1, batch normalization data flow layer norm, and activation function Relu; perform three more convolutions with a kernel size of 3×3, a stride of 1, and a padding of 1 plus sampling, batch normalization data flow layer norm, and activation function Relu; add the output feature map after the first convolution and the feature map after the last convolution to obtain an output feature map of size 72×72×256, and then pass through the maximum pooling data flow layer pool, and the final output size is 36×36×256 feature map;

The feature map of size 36×36×256 is further fed into the residual U-shaped block-4, and two convolutions with a kernel size of 3×3, a stride of 1, and a padding of 1 are performed, the data flow layer norm is batch normalized, and the activation function is Relu; then two convolutions with a kernel size of 3×3, a stride of 1, and a padding of 1 are performed plus maximum pooling downsampling, the data flow layer norm is batch normalized, and the activation function is Relu; then a convolution with a kernel size of 3×3, a stride of 1, a padding of 2, and a void rate of 2 is performed, the data flow layer norm is batch normalized, and the activation function is Relu ; Perform another convolution with a kernel size of 3×3, a stride of 1, and a padding of 1, batch normalization data flow layer norm, and activation function Relu; perform two more convolutions with a kernel size of 3×3, a stride of 1, and a padding of 1 plus sampling, batch normalization data flow layer norm, and activation function Relu; add the output feature map after the first convolution and the feature map after the last convolution to obtain an output feature map with an output size of 36×36×512, and then pass through the maximum pooling data flow layer pool, and the final output size is 18×18×512 feature map;

The feature map of size 36×36×256 is further fed into the residual U-shaped block-4F, and two convolutions with a kernel size of 3×3, a stride of 1, and a padding of 1 are performed, the batch normalized data flow layer norm, and the activation function Relu are performed; then a convolution with a kernel size of 3×3, a stride of 1, a padding of 2, and a void rate of 2 is performed, the batch normalized data flow layer norm, and the activation function Relu are performed; another convolution with a kernel size of 3×3, a stride of 1, a padding of 4, and a void rate of 4, the batch normalized data flow layer norm, and the activation function Relu are performed; another convolution with a kernel size of 3×3, a stride of 1, a padding of 8, and a void rate of 8 is performed, the batch normalized data flow layer norm, and the activation function Relu ; Perform another convolution with a kernel size of 3×3, a step size of 1, a padding of 4, and a void rate of 4, batch normalization data flow layer norm, and activation function Relu; perform another convolution with a kernel size of 3×3, a step size of 1, a padding of 2, and a void rate of 2, batch normalization data flow layer norm, and activation function Relu; perform another convolution with a kernel size of 3×3, a step size of 1, and a padding of 1, batch normalization data flow layer norm, and activation function Relu; add the output feature map of the first convolution and the feature map of the last convolution to obtain an output feature map of size 18×18×512, and then pass through the maximum pooling data flow layer pool, and the final output size is 9×9×512 feature map.

其中，超参数{γ,β}实现数据批次的打乱，防止训练发生偏移。μ_B和

分别表示批次B中的均值和方差，ε是为了避免除数为0而使用的微小正数。In the batch normalization data stream layer norm, the computational update for transforming each _xi to _yi in a data batch B = { _x1 , ..., _xi , ..., _xn } is defined as:

The hyperparameters {γ,β} are used to shuffle the data batches and prevent training from drifting _.

denote the mean and variance in batch B respectively, and ε is a small positive number used to avoid division by 0.

The self-attention U-shaped block uses self-attention to model the long-range dependency of the input image and extract the global information of the input image; specifically:

Take the feature map of size 9×9×512 as input and perform two convolutions with kernel size of 3×3, stride of 1, padding of 1, batch normalization data flow layer norm, activation function Relu; then perform one convolution with kernel size of 3×3, stride of 1, padding of 2, dilation rate of 2, batch normalization data flow layer norm, activation function Relu; then perform one convolution with kernel size of 3×3, stride of 1, padding of 4, dilation rate of 4, batch normalization data flow layer norm, activation function Relu; then perform one convolution with kernel size of 3×3, stride of 1, padding of 8, dilation rate of 8, batch normalization data flow layer norm, activation function Relu;

A∈R^n×n为注意力矩阵，上标T表示转置；注意力矩阵A用作权重，以考虑查询和键之间的所有交互；The feature map is then fed into the self-attention module. In order to consider the absolute semantic information, position encoding is first added to the input feature map X∈R ^d×H×W , where H and W are the height and width of the feature map, d is the number of channels, and R represents the real number domain; position encoding is particularly suitable for capturing the absolute and relative positions between tampered areas in self-attention; X is then flattened and transposed into a sequence of size n×d, with parameter n=H×W; three 1×1 convolutional projections X are used for query, key, and value embedding: Q,K,V∈R ^d×n , where Q represents the query matrix, K represents the key matrix, and V represents the value matrix; the output of self-attention is a scaled dot product:

A∈R ^n×n is the attention matrix, and the superscript T represents the transpose. The attention matrix A is used as a weight to consider all interactions between queries and keys.

Perform another convolution with a kernel size of 3×3, a stride of 1, a padding of 4, and a void rate of 4, batch normalize the data flow layer norm, and activate the Relu function; perform another convolution with a kernel size of 3×3, a stride of 1, a padding of 2, and a void rate of 2, batch normalize the data flow layer norm, and activate the Relu function; perform another convolution with a kernel size of 3×3, a stride of 1, and a padding of 1, batch normalize the data flow layer norm, and activate the Relu function; then add the output feature map of the first convolution and the feature map of the last convolution, and the final output size is a 9×9×512 feature map.

The cross attention module uses cross attention to filter non-semantic features between the feature extraction module and the feature decoding module, achieves more refined spatial recovery in the feature decoding module, and improves the accuracy of the prediction results; specifically:

First, add position encoding to the low-level feature map U∈R ^d×2H×2W , where H and W are the height and width of the feature map, d is the number of channels, and R represents the real number domain; then, downsample the low-level feature map U once and use it as the value matrix of the cross-attention module; add position encoding to the high-level feature map N∈R ^2d×H×W and use 1×1 convolution to change the number of channels to d as the query matrix and key matrix;

Then, the attention matrix A∈Rn ^×n is calculated using matrix multiplication and softmax function, where n=H×W; the calculated weight values are rescaled by the Relu activation function to obtain the result S as the filter, where low-amplitude elements represent noise or irrelevant areas to be reduced;

Take the Hadamard product of U and S to get the version of U with non-semantic features filtered out;

Finally, the result of the filtering operation is concatenated with the high-level feature map N. Through the cross-attention module, more detailed information can be retained than ordinary skip connections, thereby improving the detection performance.

The feature decoding module decodes the feature map with global information and local information, and predicts the tampered area at the pixel level. The feature decoding module has a similar structure to its symmetrical feature extraction module; specifically:

The feature map of the self-attention U-shaped block with a size of 9×9×512 is upsampled and the output feature map of the cross-attention module is cascaded to obtain a feature map of 18×18×1024, and then passed through the residual U-shaped block-4F, and two convolutions with a kernel size of 3×3, a step size of 1, and a padding of 1 are performed, the batch normalized data flow layer norm, and the activation function Relu; then a convolution with a kernel size of 3×3, a step size of 1, a padding of 2, and a void rate of 2 is performed, the batch normalized data flow layer norm, and the activation function Relu; then a convolution with a kernel size of 3×3, a step size of 1, a padding of 4, and a void rate of 4 is performed, the batch normalized data flow layer norm, and the activation function Relu; then a convolution with a kernel size of 3×3, a step size of 1, a padding of 4, and a void rate of 4 is performed, the batch normalized data flow layer norm, and the activation function Relu; then a convolution with a kernel size of 3× 3. Convolution with a step size of 1, a padding of 8, and a hole rate of 8, batch normalization of the data flow layer norm, and activation function Relu; perform another convolution with a convolution kernel size of 3×3, a step size of 1, a padding of 4, and a hole rate of 4, batch normalization of the data flow layer norm, and activation function Relu; perform another convolution with a convolution kernel size of 3×3, a step size of 1, a padding of 2, and a hole rate of 2, batch normalization of the data flow layer norm, and activation function Relu; perform another convolution with a convolution kernel size of 3×3, a step size of 1, a padding of 1, batch normalization of the data flow layer norm, and activation function Relu; then add the output feature map of the first convolution and the feature map of the last convolution, and the final output size is 18×18×512 feature map;

The feature map of size 18×18×512 output by the residual U-shaped block-4F is upsampled and the output feature map of the cross attention module is cascaded to obtain a feature map of 36×36×1024. Then, it passes through the residual U-shaped block-4, performs two convolutions with a kernel size of 3×3, a step size of 1, and a padding of 1, batch normalization of the data flow layer norm, and activation function Relu; then, two convolutions with a kernel size of 3×3, a step size of 1, and a padding of 1 plus maximum pooling downsampling, batch normalization of the data flow layer norm, and activation function Relu are performed; and then a convolution kernel size of 3 is performed. ×3, stride 1, padding 2, dilation rate 2 convolution, batch normalization data flow layer norm, activation function Relu; then perform a convolution with a kernel size of 3×3, stride 1, padding 1, batch normalization data flow layer norm, activation function Relu; then perform two convolutions with a kernel size of 3×3, stride 1, padding 1 plus sampling, batch normalization data flow layer norm, activation function Relu; then add the output feature map of the first convolution and the feature map of the last convolution, and the final output size is a 36×36×256 feature map;

The feature map of size 36×36×256 output by residual U-shaped block-4 is upsampled and the output feature map of the cross attention module is cascaded to obtain a feature map of 72×72×512, and then passed through residual U-shaped block-5, and two convolutions with a kernel size of 3×3, a step size of 1, and a padding of 1 are performed, the batch normalization data flow layer is norm, and the activation function is Relu; then three convolutions with a kernel size of 3×3, a step size of 1, and a padding of 1 are performed plus maximum pooling downsampling, batch normalization data flow layer norm, activation function Relu; and one more convolution with a kernel size of 3× 3. Convolution with a step size of 1, a padding of 2, and a void rate of 2, batch normalization of the data flow layer norm, and activation function Relu; perform another convolution with a kernel size of 3×3, a step size of 1, and a padding of 1, batch normalization of the data flow layer norm, and activation function Relu; perform three more convolutions with a kernel size of 3×3, a step size of 1, and a padding of 1 plus sampling, batch normalization of the data flow layer norm, and activation function Relu; then add the output feature map of the first convolution and the feature map of the last convolution, and finally output a feature map of size 72×72×128;

The feature map of size 72×72×128 output by residual U-shaped block-5 is upsampled and the output feature map of the cross attention module is cascaded to obtain a feature map of 144×144×256, and then passed through residual U-shaped block-6, and two convolutions with a kernel size of 3×3, a step size of 1, and a padding of 1 are performed, the batch normalization data flow layer norm, and the activation function Relu are performed; then four convolutions with a kernel size of 3×3, a step size of 1, and a padding of 1 are performed plus maximum pooling downsampling, batch normalization data flow layer norm, and activation function Relu are performed; and then a convolution kernel size of 3 is performed. ×3, stride 1, padding 2, dilation rate 2 convolution, batch normalization data flow layer norm, activation function Relu; then perform a convolution with a kernel size of 3×3, stride 1, padding 1, batch normalization data flow layer norm, activation function Relu; then perform four convolutions with a kernel size of 3×3, stride 1, padding 1 plus sampling, batch normalization data flow layer norm, activation function Relu; then add the output feature map of the first convolution and the feature map of the last convolution, and the final output size is 144×144×64 feature map;

The feature map of size 144×144×64 output by residual U-shaped block-6 is upsampled and the output feature map of the cross attention module is cascaded to obtain a feature map of 288×288×128, and then passed through residual U-shaped block-7, and two convolutions with a kernel size of 3×3, a step size of 1, and a padding of 1 are performed, the batch normalization data flow layer norm, and the activation function Relu are performed; then five convolutions with a kernel size of 3×3, a step size of 1, and a padding of 1 are performed plus maximum pooling downsampling, batch normalization data flow layer norm, and activation function Relu are performed; and then a convolution kernel size of 3 is performed. ×3, stride 1, padding 2, dilation rate 2 convolution, batch normalization data flow layer norm, activation function Relu; then perform a convolution with a kernel size of 3×3, stride 1, padding 1, batch normalization data flow layer norm, activation function Relu; then perform five convolutions with a kernel size of 3×3, stride 1, padding 1 plus sampling, batch normalization data flow layer norm, activation function Relu; then add the output feature map of the first convolution and the feature map of the last convolution, and the final output size is 288×288×64 feature map;

其中(r,c)是像素坐标，p_(r,c)和y_(r,c)分别表示输入图像预测的像素值和标定真值的像素值。The residual U-shaped block-7 outputs a feature map of size 288×288×64, which is passed through a 3×3 convolution layer to output a feature map of size 288×288×1; and then passes through the Sigmiod activation function to obtain a single-channel tampering probability mask map of size 288×288×1. The entire neural network optimizes the predicted results by minimizing the cross entropy loss function through the stochastic gradient descent optimization algorithm. The cross entropy loss function is:

Where (r, c) is the pixel coordinate, p _{(r, c)} and y _{(r, c)} represent the pixel values of the predicted input image and the calibrated true value, respectively.

Step (3) uses the trained network model to test the test set images to obtain the final detection effect, with an accuracy of about 81%.

The test uses an I7 9700K CPU, 32GB of memory, a GTX 3060 GPU graphics card, Python version 3.8, and Pytorch version 1.8.1. During the test, the learning rate is 0.0001, the weight decay is set to 0, and after 150 iterations of training, the final trained model is obtained. We use 85% of the CASIA data set and the COLUMBIA data set as training sets to train the hybrid transformer-based neural network, and 15% of them as test sets to test the detection accuracy based on the hybrid transformer neural network. The method of the present invention is compared with the existing methods ADQ, CFA, ELA, NOI, RRU-Net, U-Net and ManTra-Net, U ² -Net for analysis. The evaluation indicators used for comparative analysis include: Precision is the precision rate, Recall is the recall rate, and F1 is the balance between precision and recall rate. All evaluation indicators are the higher the better, and the closer to 1, the better. The following table shows the comparison results of tampering area positioning of different methods on different data sets.

It can be seen that the highest accuracy and F1 value are achieved on both CASIA and COLUMBIA datasets. Further subjective comparison shows that the positioning area of the present invention is more complete and the positioning edge is more accurate, which means that the present invention is effective in dealing with splicing tampering types.

On the CASIA dataset, as the variance of Gaussian noise increases, the precision and F1 value of the present invention are much better than the other six detection methods, the recall rate of the present invention is slightly lower than that of RRU-Net, and the detection index of the present invention is minimally affected. Experiments have shown that the present invention is robust to noise attacks on the dataset. In addition, as the JPEG quality factor decreases from 100 to 50, the precision and F1 value of the present invention are still better than other methods, and it is robust to compression attacks on the dataset.

The above embodiments should be understood as being only used to illustrate the present invention and not to limit the protection scope of the present invention. Any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the technical solution of the present invention still fall within the scope of the technical solution of the present invention.

Claims (5)

1. A method for detecting spliced tampered images, characterized in that: Step (1) Collect and download public image tampering datasets, including the CASIA dataset and the COLUMBIA dataset; take 80% to 90% of them as training sets and the rest as test sets; Step (2) using the training set to perform model training on the hybrid transformer neural network, including: feature extraction module, self-attention U-shaped block, cross attention module, feature decoding module; The feature extraction module uses the encoder part of U ² -Net to extract the feature map X of the input image; The self-attention U-shaped block uses self-attention to model the long-range dependency of the input image and extract the global information of the input image; The cross attention module uses cross attention to filter non-semantic features between the feature extraction module and the feature decoding module; The feature decoding module decodes the feature map with global information and local information, and predicts the tampered area at the pixel level; Step (3) uses the trained network model to test the test set images to obtain the final detection effect. 2. A detection method for spliced tampered images as described in claim 1, characterized in that the feature extraction module first feeds the input image of size 288×288×3 into the residual U-shaped block-7, performs two convolutions with a convolution kernel size of 3×3, a step size of 1, and a padding of 1, batch normalization of the data flow layer norm, and activation function Relu; then performs five convolutions with a convolution kernel size of 3×3, a step size of 1, and a padding of 1 plus maximum pooling downsampling, batch normalization of the data flow layer norm, and activation function Relu; then performs one convolution with a convolution kernel size of 3×3, a step size of 1, a padding of 2, and a void rate of 2, Batch normalize the data flow layer norm, activation function Relu; perform another convolution with a kernel size of 3×3, a stride of 1, and a padding of 1, batch normalize the data flow layer norm, and activate the function Relu; perform five more convolutions with a kernel size of 3×3, a stride of 1, and a padding of 1 plus sampling, batch normalize the data flow layer norm, and activate the function Relu; then add the output feature map of the first convolution and the feature map of the last convolution, and finally get 64 feature maps. Perform a maximum pooling downsampling operation on the feature map output by the residual U-shaped block-7 to reduce the feature map resolution to half of the original; The feature map is then sent to residual U-shaped block-6, residual U-shaped block-5, residual U-shaped block-4, and residual U-shaped block-4F in sequence, and the final output size is a 9×9×512 feature map. 3. A detection method for splicing tampered images as described in claim 1, characterized in that the self-attention U-shaped block specifically comprises: taking a feature map of size 9×9×512 as input, performing two convolutions with a convolution kernel size of 3×3, a step size of 1, and a padding of 1, batch normalization of the data flow layer norm, and activation function Relu; performing another convolution with a convolution kernel size of 3×3, a step size of 1, a padding of 2, and a void rate of 2, batch normalization of the data flow layer norm, and activation function Relu; performing another convolution with a convolution kernel size of 3×3, a step size of 1, a padding of 4, and a void rate of 4, batch normalization of the data flow layer norm, and activation function Relu; performing another convolution with a convolution kernel size of 3×3, a step size of 1, a padding of 8, and a void rate of 8, batch normalization of the data flow layer norm, and activation function Relu; The feature map is then fed into the self-attention module, which adds positional encoding to the input feature map X∈Rd ^×H×W , where H and W are the height and width of the feature map, d is the number of channels, and R represents the real domain; X is then flattened and transposed into a sequence of size n×d, with parameter n=H×W; three 1×1 convolutions are used to project X for query, key, and value embedding: Q,K,V∈R ^d×n , Q represents the query matrix, K represents the key matrix, and V represents the value matrix; the output of self-attention is a scaled dot product:

A∈R ^n×n is the attention matrix, and the superscript T indicates transposition; Perform another convolution with a kernel size of 3×3, a stride of 1, a padding of 4, and a void rate of 4, batch normalize the data flow layer norm, and activate the Relu function; perform another convolution with a kernel size of 3×3, a stride of 1, a padding of 2, and a void rate of 2, batch normalize the data flow layer norm, and activate the Relu function; perform another convolution with a kernel size of 3×3, a stride of 1, and a padding of 1, batch normalize the data flow layer norm, and activate the Relu function; then add the output feature map of the first convolution and the feature map of the last convolution, and the final output size is a 9×9×512 feature map. 4. A method for detecting spliced tampered images as claimed in claim 1, characterized in that the cross attention module specifically comprises: firstly adding position coding to the low-level feature map U∈R ^d×2H×2W , where H and W are the height and width of the feature map, d is the number of channels, and R represents the real number domain; then, downsampling the low-level feature map U once and using it as the value matrix of the cross attention module; adding position coding to the high-level feature map N∈R ^2d×H×W , and using 1×1 convolution to change the number of channels to d as the query matrix and key matrix; Then, the attention matrix A∈Rn ^×n is calculated using matrix multiplication and softmax function, where n=H×W; the calculated weight values are rescaled by the Relu activation function to obtain the result S as the filter, where low-amplitude elements represent noise or irrelevant areas to be reduced; Take the Hadamard product of U and S to get the version of U with non-semantic features filtered out; Finally, the result of the filtering operation is concatenated with the high-level feature map N. 5. A method for detecting spliced tampered images as described in claim 1, characterized in that the feature decoding module upsamples the feature map obtained from the self-attention U-shaped block and cascades the output feature map of the cross attention module, and the obtained feature map is sent to the residual U-shaped block-4F; upsamples the output feature map of the residual U-shaped block-4F and cascades the output feature map of the cross attention module, and the obtained feature map is sent to the residual U-shaped block-4; upsamples the output feature map of the residual U-shaped block-4 and cascades the output feature map of the cross attention module to obtain the feature map and sends it to the residual U-shaped block-5; upsamples the output feature map of the residual U-shaped block-5 and cascades the output feature map of the cross attention module The output feature maps of the modules are cascaded, and the obtained feature maps are sent to the residual U-shaped block-6; the output feature map of the residual U-shaped block-6 is upsampled and the output feature map of the cross attention module is cascaded, and the obtained feature map is sent to the residual U-shaped block-7; the feature map of the residual U-shaped block-7 with a size of 288×288×64 is output through a 3×3 convolution layer to output a feature map of size 288×288×1; and then a single-channel tampering probability mask map with a size of 288×288×1 is obtained through the Sigmiod activation function; the entire neural network optimizes the predicted results by minimizing the cross entropy loss function through the stochastic gradient descent optimization algorithm. The cross entropy loss function is:

Where (r, c) is the pixel coordinate, p _{(r, c)} and y _{(r, c)} represent the pixel values of the predicted input image and the calibrated true value, respectively.