CN113435376B - Bidirectional feature fusion deep convolution neural network construction method based on discrete wavelet transform - Google Patents

Abstract

The invention discloses a two-way feature fusion deep convolutional neural network construction method based on discrete wavelet transform, which belongs to the technical fields of image classification and artificial intelligence. The method includes the following specific steps: S101, constructing a bidirectional feature fusion module, the bidirectional feature fusion module is composed of a spatial domain feature fusion module, a pooling operation, and a channel domain feature fusion module; S102, embedding the bidirectional feature fusion module into a mainstream network architecture Replace the original pooling method; S103, use the network of the bidirectional feature fusion module embedded in the mainstream network architecture to perform feature map training and testing on the classic image classification data set. The present invention uses discrete wavelet transform and inverse discrete wavelet transform to provide a novel spatial domain feature fusion method, which can effectively suppress the information loss problem caused by direct use of pooling operations and improve image classification accuracy.

Description

Two-way feature fusion deep convolutional neural network construction method based on discrete wavelet transform

technical field

The invention belongs to the technical field of image classification and artificial intelligence, and specifically relates to a method for constructing a deep convolutional neural network based on discrete wavelet transform-based two-way feature fusion.

Background technique

Deep convolutional neural networks have become one of the important tools for computer vision tasks and image processing tasks such as image classification, object detection, and image restoration. The pooling layer is an important part of the deep convolutional neural network. It can increase the receptive field of the network, reduce the complexity of the network, increase the nonlinearity of the network and improve the generalization ability of the model. In deep convolutional neural networks, commonly used pooling methods include maximum pooling, average pooling, hybrid pooling, and random pooling. Among them, the classic maximum pooling and average pooling are widely used in deep convolutional neural networks because of their simple and efficient design principles. However, one of the main limitations of these two types of pooling operations is that part of the feature information in the image is lost and weakened as the image resolution decreases. Hybrid pooling and random pooling establish the connection between maximum pooling and average pooling by using probability and statistics. Although hybrid pooling and random pooling inherit the advantages of average pooling and max pooling, there are still problems of information loss and weakening. The loss and weakening of feature information caused by the pooling operation in the process of extracting features of the deep convolutional neural network will directly affect the expressive ability of the network, and the accuracy of classification will also be affected.

Researchers have made continuous improvements to the problem of feature information loss in pooling operations. Convolution with a step size reduces the resolution of the feature map without discarding data, but the step size convolution greatly increases the amount of computation and parameters for training the network. The researchers also observed that the direct downsampling pooling operation ignores the difference in the location distribution of low-frequency features and high-frequency features between the spatial domain and the channel, resulting in the aliasing effect between the frequency domain features. Therefore, it is proposed to use low-pass filtering to remove high-frequency features before down-sampling, which can effectively avoid the aliasing effect. Since the high-frequency part of the feature map contains a large amount of detail information and edge information, directly discarding high-frequency features will still cause the loss of feature information and affect the expressive ability of the network.

To solve the above problems, we propose a discrete wavelet transform-based bidirectional feature fusion deep convolutional neural network construction method.

Contents of the invention

Aiming at the deficiencies of the prior art, the present invention provides a method for constructing a deep convolutional neural network based on discrete wavelet transform-based two-way feature fusion, in order to solve the problems raised in the above-mentioned background technology.

In order to achieve the above object, the present invention provides the following technical solutions: a method for constructing a two-way feature fusion deep convolutional neural network based on discrete wavelet transform, comprising the following specific steps:

S101. Construct a bidirectional feature fusion module based on discrete wavelet transform and inverse discrete wavelet transform, the bidirectional feature fusion module is composed of a spatial domain feature fusion module, a pooling operation, and a channel domain feature fusion module;

S102. Embedding the bidirectional feature fusion module into the mainstream network architecture to replace the original pooling method;

S103. Using the deep convolutional neural network of the bidirectional feature fusion module embedded in the mainstream network architecture to perform feature map training and testing on the classic image classification dataset.

To further optimize the technical solution, the airspace feature fusion module includes the following steps when realizing the fusion between the airspace features:

S201. Using discrete wavelet transform to decompose the input features to obtain low-frequency features, high-frequency features in the horizontal direction, high-frequency features in the vertical direction, and high-frequency features in the diagonal direction;

S202. Using inverse discrete wavelet transform for group reconstruction to obtain four types of features with the same spatial dimension as the original feature, so as to realize the fusion of spatial features.

To further optimize this technical solution, the grouping reconstruction refers to single-frequency reconstruction of low-frequency features, reconstruction of low-frequency and high-frequency in the horizontal direction, reconstruction of low-frequency and high-frequency in the vertical direction, and reconstruction of low-frequency and high-frequency in the diagonal direction. structure.

To further optimize this technical solution, the four sets of features obtained by the airspace feature fusion module are airspace features and have similar distribution features, which have a considerable impact on the loss function. At the same time, pooling is performed on the four sets of fusion features obtained from the airspace.

To further optimize this technical solution, the spatial feature fusion module does not add any parameters during the process of spatial feature fusion using discrete wavelet transform and inverse discrete wavelet transform.

To further optimize this technical solution, the channel domain feature fusion module splices the pooled multiple groups of features in the channel dimension, and then uses 1x1 convolution to reduce the dimension, so as to realize information interaction, so that information in different frequency bands can be integrated in the channel domain. Complement and transfer each other.

To further optimize this technical solution, the feature map passes through the two-way feature fusion module, that is, through the airspace feature fusion module, the pooling operation, and the channel domain feature fusion module in turn. Therefore, the dual-domain feature fusion module can be summarized as:

LL,LH,HL,HH=Pooling(f _s (x))

x'=f _c (concat[LL,LH,HL,HH])

Given a feature map x as input, it goes through the spatial feature fusion module f _s , the pooling operation Pooling and the channel domain feature fusion module f _c in turn. Here, x' represents the result obtained after the feature map passes through the bidirectional feature fusion module.

To further optimize this technical solution, the bidirectional feature fusion module can be packaged as an independent module for replacing the pooling operation in the deep convolutional neural network.

To further optimize this technical solution, the deep convolutional neural network using the bidirectional feature fusion module embedded in the mainstream network architecture is trained and tested on public image classification data to improve the classification accuracy of the model.

Compared with the prior art, the present invention provides a method for constructing a two-way feature fusion deep convolutional neural network based on discrete wavelet transform, which has the following beneficial effects:

(1) The method sets up the spatial feature fusion module, which establishes the information fusion between different features in the airspace, and realizes the expansion of the channel dimension; uses the method of group reconstruction to make the distribution of different features approximate, And the impact on the loss function is similar. Since the relatively important low-frequency wavelet features are used in the group reconstruction, the features in the spatial domain and the channel domain are redundant, which reduces the feature loss caused by pooling.

(2) This method performs pooling processing on the four types of features, and then transfers the pooling results to the channel domain feature fusion module. Compared with the classic network, the popular network based on embedded attention mechanism and the latest wavelet-based deep convolution Convolutional neural network, a deep convolutional neural network embedded with a bidirectional feature fusion module, improves the accuracy of network classification with a small parameter increment, and has certain advantages.

(3) This method uses discrete wavelet transform and inverse discrete wavelet transform to give a novel spatial feature fusion method, which solves the shortcomings of existing wavelet-based methods, and can be directly embedded in the current popular neural network architecture to replace traditional pooling Operation, the information loss of pooling is suppressed from the two directions of space domain and channel domain.

Description of drawings

Fig. 1 is a schematic flow chart of a bidirectional feature fusion depth convolutional neural network construction method based on discrete wavelet transform proposed by the present invention;

Fig. 2 is a schematic flow diagram of a bidirectional feature fusion module in a bidirectional feature fusion depth convolutional neural network construction method based on discrete wavelet transform proposed by the present invention;

FIG. 3 is a schematic structural diagram of a bidirectional feature fusion module in a method for constructing a two-way feature fusion deep convolutional neural network based on discrete wavelet transform proposed by the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

Embodiment one:

Please refer to Figure 1-3, a method for constructing a deep convolutional neural network based on discrete wavelet transform for bidirectional feature fusion, including the following specific steps:

S101. Construct a bidirectional feature fusion module based on discrete wavelet transform and inverse discrete wavelet transform, the bidirectional feature fusion module is composed of a spatial domain feature fusion module and a channel domain feature fusion module;

S102. Embedding the bidirectional feature fusion module into the mainstream network architecture to replace the original pooling method;

S103. Using the deep convolutional neural network of the bidirectional feature fusion module embedded in the mainstream network architecture to perform feature map training and testing on the classic image classification dataset.

Specifically, the airspace feature fusion module includes the following steps when realizing the fusion between the airspace features:

S201. Using discrete wavelet transform to decompose the input features to obtain low-frequency features, high-frequency features in the horizontal direction, high-frequency features in the vertical direction, and high-frequency features in the diagonal direction;

S202. Using inverse discrete wavelet transform for group reconstruction to obtain four types of features with the same spatial dimension as the original feature, so as to realize the fusion of spatial features.

Specifically, the group reconstruction refers to single-frequency reconstruction of low-frequency features, reconstruction of low frequencies and high frequencies in the horizontal direction, reconstruction of low frequencies and high frequencies in the vertical direction, and reconstruction of low frequencies and high frequencies in the diagonal direction.

Specifically, the four sets of features obtained by the airspace feature fusion module are airspace features and have similar distribution features, which have a considerable impact on the loss function. At the same time, pooling is performed on the four sets of fusion features obtained from the airspace.

Specifically, the spatial feature fusion module does not add any parameters during the spatial feature fusion process using discrete wavelet transform and inverse discrete wavelet transform.

Specifically, the channel-domain feature fusion module splices the pooled multiple groups of features in the channel dimension, and then uses 1x1 convolution for dimensionality reduction, so as to achieve the purpose of feature selection and information interaction, so that information in different frequency bands Carry out mutual complementation and transfer of channel domains.

Specifically, the feature map passes through the two-way feature fusion module, that is, sequentially passes through the airspace feature fusion module, the pooling operation, and the channel domain feature fusion module. Therefore, the dual-domain feature fusion module can be summarized as:

LL,LH,HL,HH=Pooling(f _s (x))

x'=f _c (concat[LL,LH,HL,HH])

Given a feature map x as input, it goes through the spatial feature fusion module f _s , the pooling operation Pooling and the channel domain feature fusion module fc in sequence. Here, x' represents the result obtained after the feature map passes through the bidirectional feature fusion module.

Specifically, the bidirectional feature fusion module can be packaged as an independent module for replacing the pooling operation in the deep convolutional neural network.

Specifically, the deep convolutional neural network using the bidirectional feature fusion module embedded in the mainstream network architecture is trained and tested on public image classification data to improve the classification accuracy of the model.

Embodiment two:

Using the discrete wavelet transform-based two-way feature fusion deep convolutional neural network construction method described in Example 1, use three classic image classification data sets CIFAR-10, CIFAR-100, and Mini-ImageNet for training and verification Based on discrete wavelets Transform image features extracted by a deep convolutional neural network to obtain a feature fusion image classification model. Use the SGD optimizer to iteratively update the parameters of the convolution kernel and neurons in the model. The specific parameters of the optimizer are: 100 iterations of training, each training batch size is 32, the learning rate is 0.001, the weight penalty item is 0.0001, and the momentum is 0.8. When the loss of the training set and the verification set in it tends to converge, it means that the model is stable and a trained classification model is obtained.

Based on the discrete wavelet transform-based two-way feature fusion deep convolutional neural network construction method, first, construct a two-way feature fusion module, and use discrete wavelet transform to decompose the input features into low-frequency features, high-frequency features in the horizontal direction, and high-frequency features in the vertical direction. High-frequency features and high-frequency features in the diagonal direction, and then use the inverse discrete wavelet transform to group and reconstruct different frequency domain features. Among them, the single-frequency reconstruction of the low-frequency feature, the reconstruction of the low-frequency and high-frequency in the horizontal direction, the reconstruction of the low-frequency and high-frequency in the vertical direction, and the reconstruction of the low-frequency and high-frequency in the diagonal direction, respectively, obtain the same spatial dimension as the original feature Four types of characteristics. The second module pooling operation Pooling, this module performs pooling processing on the four types of features, and then transfers the pooling results to the third channel domain feature fusion module. The channel domain feature fusion module is to realize the feature fusion between channels. First, the channel dimensions are spliced, and then the 1×1 convolution is used to reduce the dimension to reduce the information redundancy between channels. Embed the bidirectional feature fusion module into the mainstream deep neural network architectures VGG, ResNet, and DenseNet. Taking VGG16 as an example, the network includes 13 convolutional layers, 5 pooling layers, and 3 volume connection layers. The 5 pooling layers of 2 are replaced with a bidirectional feature fusion module. The deep convolutional neural network embedded with the bidirectional feature fusion module is trained and tested on the public image classification data dataset. These include CIFAR-10, CIFAR-100, Mini-ImageNet. The CIFAR-10 dataset contains 50,000 training images and 10,000 testing images, with a total of 10 different categories. The image size of each figure is 32×32. The CIFAR-100 data set contains 50,000 training pictures and 10,000 test pictures, with a total of 100 different categories, and the size of each picture is 32×32. The Mini-Imagenet dataset contains 100 categories, each category contains 500 pictures for training, and 100 pictures for testing.

In this example, a large number of experiments were carried out on CIFAR-10, CIFAR-100, and Mini-ImageNet datasets, which proved that the two-way feature fusion module was used to replace the pooling operation in the model, and the two-way feature based on discrete wavelet transform was constructed. Integrating deep convolutional neural networks can improve the accuracy of image classification. It can be seen from Table 1 that using different types of wavelets can improve the classification accuracy compared with the original method. At the same time, it has been verified that different wavelet types have different effects on the classification structure. Among them, using Haar wavelet and biorthogonal wavelet The bidirectional feature fusion module constructed by the bior2.2 wavelet can obtain high classification accuracy in most experiments.

Table 1 Comparison of classification accuracy of deep convolutional neural network embedded with bidirectional feature fusion module (%)

In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the described specific features, structures, materials or characteristics may be combined in any suitable manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples and features of different embodiments or examples described in this specification without conflicting with each other.

Although the embodiments of the present invention have been shown and described, those skilled in the art can understand that various changes, modifications and substitutions can be made to these embodiments without departing from the principle and spirit of the present invention. and modifications, the scope of the invention is defined by the appended claims and their equivalents.

Claims (8)

1. A bidirectional feature fusion deep convolution neural network construction method based on discrete wavelet transform is characterized by comprising the following specific steps:

s101, constructing a bidirectional feature fusion module based on discrete wavelet transform and inverse discrete wavelet transform, wherein the bidirectional feature fusion module consists of a spatial domain feature fusion module, a pooling operation module and a channel domain feature fusion module;

s102, embedding the bidirectional feature fusion module into a mainstream network architecture to replace an original pooling method;

s103, performing feature map training and testing on the classical image classification data set by using a deep convolutional neural network of a bidirectional feature fusion module embedded into a mainstream network architecture;

the characteristic diagram passes through the bidirectional characteristic fusion module, namely sequentially passes through the airspace characteristic fusion module, the pooling operation and the channel domain characteristic fusion module, so that the bidirectional characteristic fusion module is summarized as follows:

given a feature mapxAs input, sequentially pass through a spatial feature fusion modulef _s Operation in pondsPoolingAnd channel domain feature fusion modulef _c ，x' represents the result of the feature graph after passing through the bidirectional feature fusion module.

2. The method for constructing the bidirectional feature fusion deep convolutional neural network based on the discrete wavelet transform as claimed in claim 1, wherein the spatial domain feature fusion module comprises the following steps when realizing fusion between spatial domain features:

s201, decomposing input features by utilizing discrete wavelet transform to obtain low-frequency features, high-frequency features in the horizontal direction, high-frequency features in the vertical direction and high-frequency features in the diagonal direction;

s202, grouping and reconstructing by utilizing inverse discrete wavelet transform to obtain four types of characteristics with the same spatial dimension as the original characteristics so as to realize the fusion of the spatial characteristics.

3. The method for constructing the bidirectional feature fusion deep convolution neural network based on the discrete wavelet transform as claimed in claim 2, wherein the grouping reconstruction refers to reconstructing a single frequency of a low frequency feature, reconstructing a high frequency in a low frequency and a horizontal direction, reconstructing a high frequency in a low frequency and a vertical direction, and reconstructing a high frequency in a low frequency and a high frequency in a diagonal direction.

4. The method for constructing the bidirectional feature fusion deep convolutional neural network based on the discrete wavelet transform as claimed in claim 2, wherein four groups of features obtained by the spatial domain feature fusion module are spatial domain features and distribution features are similar, the influence on the loss function is equivalent, and the four groups of fusion features obtained from the spatial domain are respectively pooled.

5. The method for constructing the bidirectional feature fusion deep convolutional neural network based on the discrete wavelet transform of claim 2, wherein the spatial domain feature fusion module does not add any parameter in the spatial domain feature fusion process by using the discrete wavelet transform and the inverse discrete wavelet transform.

6. The method for constructing the bidirectional feature fusion deep convolutional neural network based on the discrete wavelet transform as claimed in claim 1, wherein the channel domain feature fusion module performs channel dimension splicing on a plurality of groups of pooled features, and then performs dimension reduction by using 1x1 convolution to realize information interaction, so that different frequency band information performs mutual complementation and transmission of channel domains.

7. The method for constructing the bidirectional feature fusion deep convolutional neural network based on the discrete wavelet transform as claimed in claim 1, wherein the bidirectional feature fusion module is packaged as an independent module for replacing the pooling operation in the deep convolutional neural network.

8. The method for constructing the bidirectional feature fusion deep convolutional neural network based on the discrete wavelet transform of claim 1, wherein the deep convolutional neural network using the bidirectional feature fusion module embedded in the mainstream network architecture is trained and tested on public image classification data for realizing the purpose of improving the classification accuracy of the model.

Images