CN118037580A - Image denoising method, system, storage medium and electronic device - Google Patents

Abstract

The invention provides an image denoising method, an image denoising system, a storage medium and electronic equipment, wherein the method comprises the steps of obtaining an image to be processed, and obtaining a base layer data set and a detail layer data set of the image to be processed by utilizing guide filtering, wherein the base layer data set and the detail layer data set comprise a plurality of original image pairs, and each original image pair comprises a pair of original images and noise images; constructing an image denoising model based on a quaternion convolutional neural network, wherein the quaternion convolutional neural network comprises a first sub-network and a second sub-network, the first sub-network is used for processing a base layer data set, and the second sub-network is used for processing a detail layer data set; tensor conversion and standardization processing are carried out on all images in the base layer data set and the detail layer data set so as to obtain preprocessed images; the invention can obviously reduce the denoising cost of the color image.

Description

Image denoising method, system, storage medium and electronic device

Technical Field

The present invention relates to the field of image processing technology, and in particular to an image denoising method, system, storage medium and electronic equipment.

Background technique

Noise is ubiquitous, especially in images that are closely related to our daily lives, such as medical images, underwater images, sensor images, synthetic aperture radar images, and infrared images. Image noise refers to unnecessary or redundant interference information in image data, which has a great negative impact on image quality and hinders the transmission of information. Therefore, eliminating noise is crucial for both computers and humans. The purpose of denoising is to separate noise from noisy images to obtain a clean image.

Restoring noisy images has long been a research focus. With AlexNet's victory in the ImageNet competition in 2012, people discovered the potential of deep learning, which revitalized the exhausted field of denoising. Zhang et al. proposed a denoising convolutional neural network (DnCNN), which uses a stack of convolutional layers, batch normalization layers (BN), rectified linear unit layers (ReLU), and residual learning, which is simple and has achieved certain results. FFDNet uses the noise horizon map as part of the network input to generalize Gaussian noise to more complex real noise. CBDNet focuses on the noise horizon map part in FFDNet, and adaptively obtains the noise horizon map through a 5-layer fully convolutional network (FCN), thereby achieving a certain degree of blind denoising. DANet proposed a dual attention network that can adaptively inherit local features and global dependencies. NBNet uses subspace projection to reconstruct high-dimensional feature maps, suppresses the generation of noise, and realizes the combination of traditional subspace methods and CNN. NHNet is a non-local hierarchical network with a dual-path structure, namely the initial resolution path and the high-resolution path, with additional cross-connections and channel attention layers between the two paths, and a non-local mechanism is adopted in the upsampling process. KBNet proposes a learnable kernel-based attention module (KBA) that models spatial information and also designs a multi-axis feature fusion (MFF) module for image restoration.

In recent years, deep learning has become a hot research field. Convolutional neural networks (CNNs) have shown excellent performance in various applications such as robotics, autonomous driving systems, image recognition, facial expression analysis, handwritten digit recognition, natural language processing, etc. In addition, CNN is not limited to general image denoising, it has also achieved good results in blind denoising and real noisy images.

Existing technologies also have various shortcomings. For example, some methods have too large models, complex networks, and too many parameters. Some methods have poor denoising effects and insufficient detail recovery. Some methods rely heavily on convolutional layers and require a large amount of training data, resulting in long training times and slow denoising processes. But in fact, denoising is a low-level image task that does not require such a complex network and huge parameters. Therefore, it is necessary to design a simple and efficient denoising network to achieve high returns at low cost.

Summary of the invention

In view of the shortcomings of the prior art, the purpose of the present invention is to provide an image denoising method, aiming to solve the technical problems of the prior art denoising method such as too large model, too many parameters, too long training time and slow denoising process.

In order to achieve the above object, the present invention is implemented by the following technical solutions:

An image denoising method comprises the following steps:

S10, acquiring an image to be processed, and obtaining a base layer data set and a detail layer data set of the image to be processed by using guided filtering, wherein the base layer data set and the detail layer data set each include a plurality of original image pairs, and each of the original image pairs includes a pair of an original image and a noise image;

S20, constructing an image denoising model based on a quaternion convolutional neural network, wherein the quaternion convolutional neural network includes a first subnetwork and a second subnetwork, the first subnetwork is used to process the base layer data set, and the second subnetwork is used to process the detail layer data set;

S30, performing tensor conversion and standardization processing on all images in the base layer data set and the detail layer data set to obtain preprocessed images;

S40, setting a loss function and an optimization algorithm, inputting the preprocessed image into the image denoising model to train the image denoising model, and then obtaining an image denoising optimization model;

The step S40 comprises:

Calculating the error between an output image processed by the image denoising model and an initial image corresponding to the output image;

The loss function and the parameters in the optimization algorithm are adjusted according to the size of the error to optimize the image denoising model, thereby obtaining an image denoising optimization model.

Furthermore, the specific steps of step S10 include:

The image to be processed is cropped to obtain a plurality of patches, wherein the cropping is divided into two rounds, and the two rounds of cropping have the same effect on the image to be processed, Gaussian noise is added to the patches in the first round of cropping to obtain a noise image, and the patches in the second round of cropping remain the same to obtain an original image;

Using guided filtering to obtain a base layer and a detail layer of the patch;

The base layers of all the patches are integrated into a base layer data set, and the detail layers of all the patches are integrated into a detail layer data set.

Furthermore, in step S20, the depth of the first subnetwork and the second subnetwork both have several layers, and the first layer and the last layer of the first subnetwork and the second subnetwork both use ordinary real-valued convolution, and the intermediate layers both use quaternion convolution.

Furthermore, in the step S40, the loss function uses mean square error and structural similarity as a combined loss function, the optimization algorithm uses the AdamW algorithm, and in the process of training the image denoising model, the cosine annealing algorithm is used to adjust the learning rate;

The mean square error loss function is:

Wherein, _yi represents the value of the original image, represents the value of the noise image, and n represents the number of the original image pairs;

The structural similarity loss function is:

Where _μx is ^the mean of _x , _μy is the mean of y, _σx2 is the variance of x, ^σy2 is the variance of y, _σxy is the covariance of x and y, and _c1 and _c2 are constants.

Furthermore, before the step of calculating the error between the output image processed by the image denoising model and the initial image corresponding to the output image, the method further includes:

The features of the preprocessed image are extracted through the quaternion convolutional neural network and forward propagated.

Furthermore, after the step of calculating the error between the output image processed by the image denoising model and the initial image corresponding to the output image, the step further includes:

The value of the loss function is determined according to the error, and whether the quaternion convolutional neural network has converged is judged according to the value of the loss function. If not, it is determined that it is necessary to continue training the image denoising model.

Furthermore, after the step of adjusting the loss function and the parameters in the optimization algorithm according to the size of the error to optimize the image denoising model, the method further includes:

Testing and evaluating the trained image denoising optimization model according to the denoised image processed by the image denoising model and the original image corresponding to the denoised image;

The test evaluation indicators include:

Peak signal-to-noise ratio: used to measure the difference between the denoised image and the original image, the formula is as follows:

Wherein, (2 ⁿ -1) ² represents the maximum value of the image pixels; MSE represents the mean square error between the original image and the denoised image; and n represents the number of the original image pairs;

Structural similarity: used to measure the similarity between the denoised image and the original image, the formula is as follows:

Where μ _x is the mean of x, μ _y is the mean of y, σ _x ² is the variance of x, σ _y ² is the variance of y, σ _xy is the covariance of x and y, and c ₁ and c ₂ are constants;

The learnable perceived image block similarity is used to calculate the feature difference between the denoised image and the original image, and the formula is as follows:

where d is the distance between _x0 and x, Represent the true value and the predicted value respectively, H, W are the height and width of the denoised image and the original image respectively, l represents the number of layers of the quaternion convolutional neural network, and w _l represents the weight of the lth layer of the quaternion convolutional neural network.

The present invention also provides an image denoising system, comprising:

An acquisition module is used to acquire an image to be processed, and obtain a base layer data set and a detail layer data set of the image to be processed by using guided filtering, wherein the base layer data set and the detail layer data set each include a plurality of original image pairs, and each of the original image pairs includes a pair of an original image and a noise image;

The acquisition module is specifically used for:

The image to be processed is cropped to obtain a plurality of patches, wherein the cropping is divided into two rounds, and the two rounds of cropping have the same effect on the image to be processed, Gaussian noise is added to the patches in the first round of cropping to obtain a noise image, and the patches in the second round of cropping remain the same to obtain an original image;

Using guided filtering to obtain a base layer and a detail layer of the patch;

The base layers of all the patches are integrated into a base layer dataset, and the detail layers of all the patches are integrated into a detail layer dataset.

A construction module: used to construct an image denoising model based on a quaternion convolutional neural network, wherein the quaternion convolutional neural network includes a first subnetwork and a second subnetwork, the first subnetwork is used to process the base layer data set, and the second subnetwork is used to process the detail layer data set;

In the construction module, the depth of the first sub-network and the second sub-network are both several layers, and the first layer and the last layer of the first sub-network and the second sub-network both use ordinary real-valued convolution, and the intermediate layers both use quaternion convolution;

Preprocessing module: used for performing tensor conversion and standardization processing on all images in the base layer data set and the detail layer data set to obtain preprocessed images;

Training module: used to set the loss function and optimization algorithm, input the preprocessed image into the image denoising model to train the image denoising model, and then obtain the image denoising optimization model;

The training module includes:

Propagation unit: used for extracting the features of the preprocessed image through the quaternion convolutional neural network and performing forward propagation;

A calculation unit: used for calculating the error between the output image processed by the image denoising model and the initial image corresponding to the output image;

A judgment unit: used to determine the value of the loss function according to the error, and judge whether the quaternion convolutional neural network has converged according to the value of the loss function, and if not, it is determined that it is necessary to continue training the image denoising model;

Optimization unit: used for adjusting the loss function and the parameters in the optimization algorithm according to the size of the error, so as to optimize the image denoising model, and then obtain the image denoising optimization model;

In the training module, the loss function uses mean square error and structural similarity as a combined loss function, the optimization algorithm uses the AdamW algorithm, and in the process of training the image denoising model, the cosine annealing algorithm is used to adjust the learning rate;

The mean square error loss function is:

Wherein, _yi represents the value of the original image, represents the value of the noise image, and n represents the number of the original image pairs;

The structural similarity loss function is:

Where μ _x is the mean of x, μ _y is the mean of y, σ _x ² is the variance of x, σ _y ² is the variance of y, σ _xy is the covariance of x and y, and c ₁ and c ₂ are constants;

Evaluation module: used for testing and evaluating the trained image denoising optimization model according to the denoised image processed by the image denoising model and the original image corresponding to the denoised image;

The test evaluation indicators include:

Peak signal-to-noise ratio: used to measure the difference between the denoised image and the original image, the formula is as follows:

Wherein, (2 ⁿ -1) ² represents the maximum value of the image pixels; MSE represents the mean square error between the original image and the denoised image; and n represents the number of the original image pairs;

Structural similarity: used to measure the similarity between the denoised image and the original image, the formula is as follows:

Where μ _x is the mean of x, μ _y is the mean of y, σ _x ² is the variance of x, σ _y ² is the variance of y, σ _xy is the covariance of x and y, and c ₁ and c ₂ are constants;

The learnable perceived image block similarity is used to calculate the feature difference between the denoised image and the original image, and the formula is as follows:

where d is the distance between _x0 and x, Represent the true value and the predicted value respectively, H, W are the height and width of the denoised image and the original image respectively, l represents the number of layers of the quaternion convolutional neural network, and w _l represents the weight of the lth layer of the quaternion convolutional neural network.

The present invention also provides a storage medium on which a computer program is stored, wherein the computer program implements the image denoising method as described above when executed by a processor.

The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the image denoising method as described above when executing the computer program.

Compared with the prior art, the present invention has the following beneficial effects:

1) The present invention provides a lightweight color image denoising network with a simple network structure, fewer parameters, short training time, fast denoising process, and less required training data;

2) The quaternary convolutional neural network is used as the backbone of the image denoising model, which can process four channels in parallel and then fuse the information of each channel. It also reduces the number of parameters in the network and saves computing time without sacrificing feature information. In addition, the guided filtering technology is introduced. The guided filtering can obtain a sparser representation of the image, which is conducive to the training and convergence of the network.

3) The present invention demonstrates advanced performance in removing Gaussian noise from color images. Compared with other existing image denoising models, the present invention is simple in design but can achieve excellent results in evaluation indicators and image restoration details at a very low cost.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a flow chart of an image denoising method according to a first embodiment of the present invention;

FIG2 is a general flow chart of neural network training in the present invention;

FIG3 is a design diagram of a neural network structure in the present invention;

FIG4 is a structural block diagram of an image denoising system in a second embodiment of the present invention;

5 is a block diagram of a computer device in a third embodiment of the present invention;

The following specific implementation manner will further illustrate the present invention in conjunction with the above-mentioned drawings.

Detailed ways

In order to facilitate understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Several embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and/or" used herein includes any and all combinations of one or more related listed items.

Please refer to FIG. 1 and FIG. 2 , which show an image denoising method in a first embodiment of the present invention, comprising the following steps:

S10, acquiring an image to be processed, and obtaining a base layer data set and a detail layer data set of the image to be processed by using guided filtering, wherein the base layer data set and the detail layer data set each include a plurality of original image pairs, and each of the original image pairs includes a pair of an original image and a noise image;

S20, constructing an image denoising model based on a quaternion convolutional neural network, wherein the quaternion convolutional neural network includes a first subnetwork and a second subnetwork, the first subnetwork is used to process the base layer data set, and the second subnetwork is used to process the detail layer data set;

S30, performing tensor conversion and standardization processing on all images in the base layer data set and the detail layer data set to obtain preprocessed images;

S40, setting a loss function and an optimization algorithm, inputting the preprocessed image into the image denoising model to train the image denoising model, and then obtaining an image denoising optimization model;

The step S40 comprises:

Calculating the error between an output image processed by the image denoising model and an initial image corresponding to the output image;

The loss function and the parameters in the optimization algorithm are adjusted according to the size of the error to optimize the image denoising model, thereby obtaining an image denoising optimization model.

It can be understood that the present invention provides a lightweight color image denoising network with a simple network structure, fewer parameters, short training time, fast denoising process, and less required training data;

The quaternary convolutional neural network is used as the backbone of the image denoising model, which can process four channels in parallel and then fuse the information of each channel. It also reduces the number of parameters in the network and saves computing time without sacrificing feature information. In addition, the guided filtering technology is introduced. The guided filtering can obtain a sparser representation of the image, which is conducive to the training and convergence of the network.

The present invention demonstrates advanced performance in removing Gaussian noise from color images. Compared with other existing image denoising models, the present invention is simple in design but can achieve excellent results in evaluation indicators and image restoration details at a very low cost.

Specifically, the specific steps of step S10 include:

The image to be processed is cropped to obtain a plurality of patches, wherein the cropping is divided into two rounds, and the two rounds of cropping have the same effect on the image to be processed, Gaussian noise is added to the patches in the first round of cropping to obtain a noise image, and the patches in the second round of cropping remain the same to obtain an original image;

Using guided filtering to obtain a base layer and a detail layer of the patch;

The base layers of all the patches are integrated into a base layer dataset, and the detail layers of all the patches are integrated into a detail layer dataset.

It can be understood that the BSD-100 dataset is used in the present invention. BSD-100 is the test set of the Berkeley Segmentation Dataset (BSD-300), which is commonly used for image denoising and super-resolution reconstruction tasks. It consists of 100 different color images with two sizes of 480×320 and 320×480, including animals, humans, buildings, food, etc. 80 images are selected as training images and the other 20 images are selected as test images. A 64×64 patch is randomly cropped twice from each image, and the positions of the two crops are the same. At the same time, the base layer and detail layer of the patch are obtained by guided filtering. Gaussian noise is added to the first crop, which is a noise image. The second crop is processed as a label, which is the original image. The mean value of the noise is 0, and the standard deviation is [10, 20, 30, 40], with a total of four noise levels;

Exemplarily, in this embodiment, each image is randomly cropped 10 times, and a total of 1000 pairs of patches are obtained, of which 800 pairs of patches are used as training sets, and the remaining 200 pairs are used as test sets, so finally a base layer data set and a detail layer data set are obtained, and the base layer data set and the detail layer data set correspond to four noise levels.

Furthermore, in step S20, the depth of the first subnetwork and the second subnetwork both have several layers, and the first layer and the last layer of the first subnetwork and the second subnetwork both use ordinary real-valued convolution, and the intermediate layers both use quaternion convolution.

Exemplarily, in this embodiment, the first subnetwork and the second subnetwork both use simple convolutional layers and activation functions, and do not use batch normalization and pooling layers. The depth of the first subnetwork and the second subnetwork are both four layers, of which the first and last layers use ordinary real-valued convolutions, and the middle two layers use quaternion convolutions. For the first subnetwork, the convolution kernel sizes of the four layers are all 3×3; for the second subnetwork, the convolution kernel sizes of the first and last layers are 5×5, and the convolution kernel sizes of the two middle layers are 7×7.

Furthermore, in the step S40, the loss function uses mean square error and structural similarity as a combined loss function, the optimization algorithm uses the AdamW algorithm, and in the process of training the image denoising model, the cosine annealing algorithm is used to adjust the learning rate;

The mean square error loss function is:

Wherein, _yi represents the value of the original image, represents the value of the noise image, and n represents the number of the original image pairs;

The structural similarity loss function is:

Where _μx is ^the mean of _x , _μy is the mean of y, _σx2 is the variance of x, ^σy2 is the variance of y, _σxy is the covariance of x and y, and _c1 and _c2 are constants.

For example, in the AdamW algorithm, the weight decay is 1e-4, the mini-batch is set to 10, and the total number of training rounds is set to 50. During the training process, cosine annealing (CosineAnnealingLR) is used to adjust the learning rate, which allows the learning rate to change nonlinearly between two boundaries at a constant frequency, with a lower bound of 0.0001 and an upper bound of 0.001.

Furthermore, before the step of calculating the error between the output image processed by the image denoising model and the initial image corresponding to the output image, the method further includes:

Extracting features of the preprocessed image through the quaternion convolutional neural network and performing forward propagation;

After the step of calculating the error between the output image processed by the image denoising model and the initial image corresponding to the output image, the method further includes:

The value of the loss function is determined according to the error, and whether the quaternion convolutional neural network has converged is judged according to the value of the loss function. If not, it is determined that it is necessary to continue training the image denoising model.

It can be understood that if convergence is not achieved, the neural network re-extracts the image features and performs forward propagation. At the same time, the neural network automatically back-propagates and updates the learning to automatically adjust the loss function and the parameters in the optimization algorithm to optimize the image denoising model.

Furthermore, after the step of adjusting the loss function and the parameters in the optimization algorithm according to the size of the error to optimize the image denoising model and then obtaining the image denoising optimization model, the method further includes:

Testing and evaluating the trained image denoising optimization model according to the denoised image processed by the image denoising model and the original image corresponding to the denoised image;

The test evaluation indicators include:

Peak Signal to Noise Ratio (PSNR): used to measure the difference between the denoised image and the original image. The formula is as follows:

Wherein, (2 ⁿ -1) ² represents the maximum value of the image pixel; MSE represents the mean square error between the original image and the denoised image; n represents the number of the original image pairs; the larger the PSNR value, the better the denoising effect;

Structural Similarity (SSIM): used to measure the similarity between the denoised image and the original image. The formula is as follows:

Where, μ _x is the mean of x, μ _y is the mean of y, σ _x ² is the variance of x, σ _y ² is the variance of y, σ _xy is the covariance of x and y, c ₁ and c ₂ are constants to avoid the denominator being 0. The larger the value of SSIM, the more similar the denoised image is to the original image. When the two images are exactly the same, SSIM = 1.

The Learned Perceptual Image Patch Similarity (LPIPS) is used to calculate the feature difference between the denoised image and the original image. The formula is as follows:

where d is the distance between _x0 and x, Represent the true value and the predicted value respectively, H, W are the height and width of the denoised image and the original image respectively, l represents the number of layers of the quaternion convolutional neural network, and w _l represents the weight of the lth layer of the quaternion convolutional neural network.

As you can understand, the feature stack is extracted from the l layer and specified in units on the channel dimension. A vector is used to reduce the number of activated channels and finally the _l2 distance is calculated. Finally, the entire space is averaged and summed across channels. This metric learns the reverse mapping between the generated image and the ground truth, forcing the generator to reconstruct the reverse mapping of the real image from the fake image and prioritize the perceptual similarity between the two, which is more in line with human perception. The lower the LPIPS value, the higher the similarity between the two images, while the higher the value, the greater the difference.

Please refer to FIG. 2 . For ease of understanding, this embodiment provides an overall flowchart of neural network training, which corresponds to steps S30 - S40 .

Please refer to Figure 3. This embodiment also provides a neural network structure design diagram

It should be noted that the noisy image in FIG3 actually refers to the original image pair, and the denoised image refers to the denoised image output by the neural network model.

In order to make the present invention more comparable and convincing with the currently available related technologies, and to exclude the interference of other external factors on the denoising effect of the present invention, the technical experiments involved in the present invention were completed in the Python 3.8.5 environment, on a PC with Intel(R) Core(TM) i7-10875HCPU@2.30GHz and an NVIDIA GeForce RTX 2060.

The denoising effect of the final image denoising optimization model of the present invention is measured by the three test evaluation indicators and compared with the denoising effects of other models in the traditional method. The results are shown in the following table. The test data set is BSD-100. σ represents the noise level, and the bold part represents the best result.

As can be seen from the table, the network effect of the present invention is significantly better than other networks. On the one hand, the SSIM and LPIPS of the present invention are far ahead, indicating that the present invention has great advantages in restoring image structure and perceptual similarity. On the other hand, the PSNR of the present invention is also very good, especially under high noise levels, such as noise levels 30 and 40, which is significantly improved compared with other networks.

The comparison of the number of parameters (Parameter) and the number of floating-point operations (FLOPs) of each method is shown in the following table. The unit of the parameter is million, the unit of the floating-point operation is billion, and the bold part indicates the best result.

The above table shows that the lightweight network parameters proposed by the present invention are less than 500,000, and the FLOPs are also very short. Its biggest feature is that it is simple and efficient, but still has advanced denoising effects. These two indicators can also reflect the training cost of the entire network. Compared with real-valued networks and other networks, the advantages of the present invention are very obvious.

Please refer to FIG. 4 , which shows an image denoising system 40 in a second embodiment of the present invention, comprising:

Acquisition module 11: used to acquire an image to be processed, and obtain a base layer data set and a detail layer data set of the image to be processed by using guided filtering, wherein the base layer data set and the detail layer data set each include a plurality of original image pairs, and each of the original image pairs includes a pair of an original image and a noise image;

The acquisition module 11 is specifically used for:

The image to be processed is cropped to obtain a plurality of patches, wherein the cropping is divided into two rounds, and the two rounds of cropping have the same effect on the image to be processed, Gaussian noise is added to the patches in the first round of cropping to obtain a noise image, and the patches in the second round of cropping remain the same to obtain an original image;

Using guided filtering to obtain a base layer and a detail layer of the patch;

The base layers of all the patches are integrated into a base layer dataset, and the detail layers of all the patches are integrated into a detail layer dataset.

Construction module 12: used to construct an image denoising model based on a quaternion convolutional neural network, wherein the quaternion convolutional neural network includes a first subnetwork and a second subnetwork, the first subnetwork is used to process the base layer data set, and the second subnetwork is used to process the detail layer data set;

In the construction module 12, the depth of the first sub-network and the second sub-network are both several layers, and the first layer and the last layer of the first sub-network and the second sub-network both use ordinary real-valued convolution, and the intermediate layers both use quaternion convolution;

Preprocessing module 13: used for performing tensor conversion and standardization processing on all images in the base layer data set and the detail layer data set to obtain preprocessed images;

Training module 14: used to set a loss function and an optimization algorithm, input the preprocessed image into the image denoising model to train the image denoising model, and then obtain an image denoising optimization model;

The training module 14 comprises:

Propagation unit 141: used for extracting features of the preprocessed image through the quaternion convolutional neural network and performing forward propagation;

Calculation unit 142: used for calculating the error between the output image processed by the image denoising model and the initial image corresponding to the output image;

A judging unit 143 is used to determine the value of the loss function according to the error, and to judge whether the quaternion convolutional neural network has converged according to the value of the loss function. If not, it is determined that it is necessary to continue training the image denoising model.

Optimizing unit 144: used for adjusting the loss function and the parameters in the optimization algorithm according to the size of the error to optimize the image denoising model, thereby obtaining an image denoising optimization model;

In the training module 14, the loss function uses mean square error and structural similarity as a combined loss function, the optimization algorithm uses the AdamW algorithm, and in the process of training the image denoising model, the cosine annealing algorithm is used to adjust the learning rate;

The mean square error loss function is:

Wherein, _yi represents the value of the original image, represents the value of the noise image, and n represents the number of the original image pairs;

The structural similarity loss function is:

Where μ _x is the mean of x, μ _y is the mean of y, σ _x ² is the variance of x, σ _y ² is the variance of y, σ _xy is the covariance of x and y, and c ₁ and c ₂ are constants;

Evaluation module 15: used to test and evaluate the trained image denoising optimization model according to the denoised image processed by the image denoising model and the original image corresponding to the denoised image;

The test evaluation indicators include:

Peak signal-to-noise ratio: used to measure the difference between the denoised image and the original image, the formula is as follows:

Wherein, (2 ⁿ -1) ² represents the maximum value of the image pixels; MSE represents the mean square error between the original image and the denoised image; and n represents the number of the original image pairs;

Structural similarity: used to measure the similarity between the denoised image and the original image, the formula is as follows:

Where μ _x is the mean of x, μ _y is the mean of y, σ _x ² is the variance of x, σ _y ² is the variance of y, σ _xy is the covariance of x and y, and c ₁ and c ₂ are constants;

The learnable perceived image block similarity is used to calculate the feature difference between the denoised image and the original image, and the formula is as follows:

where d is the distance between _x0 and x, Represent the true value and the predicted value respectively, H, W are the height and width of the denoised image and the original image respectively, l represents the number of layers of the quaternion convolutional neural network, and w _l represents the weight of the lth layer of the quaternion convolutional neural network.

The functions or operation steps implemented when the above modules and units are executed are generally the same as those in the above method embodiments, and will not be repeated here.

The image denoising system provided in the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

Please refer to FIG5 . A third embodiment of the present invention further provides a computer device, comprising a memory 10, a processor 20, and a computer program 30 stored in the memory 10 and executable on the processor 20. When the processor 20 executes the computer program 30, the above-mentioned image denoising method is implemented.

The memory 10 includes at least one type of storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 10 may be an internal storage unit of a computer device, such as a hard disk of the computer device. In other embodiments, the memory 10 may also be an external storage device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card, etc. Further, the memory 10 may also include both an internal storage unit of a computer device and an external storage device. The memory 10 may be used not only to store application software and various types of data installed in the computer device, but also to temporarily store data that has been output or is to be output.

Among them, in some embodiments, the processor 20 can be an electronic control unit (Electronic Control Unit, abbreviated as ECU, also known as a vehicle computer), a central processing unit (Central Processing Unit, CPU), a controller, a microcontroller, a microprocessor or other data processing chip, used to run the program code stored in the memory 10 or process data, such as executing access restriction programs, etc.

It should be noted that the structure shown in FIG. 5 does not constitute a limitation on the computer device. In other embodiments, the computer device may include fewer or more components than shown in the figure, or a combination of certain components, or a different arrangement of components.

The embodiment of the present invention further provides a readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the image denoising method as described above is implemented.

Those skilled in the art will appreciate that the logic and/or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For purposes of this specification, a "computer-readable medium" may be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or a combination thereof: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that for ordinary technicians in this field, without departing from the concept of the present invention, several variations and improvements can be made, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be based on the attached claims.

Claims (10)

1. An image denoising method, comprising the following steps: S10, acquiring an image to be processed, and obtaining a base layer data set and a detail layer data set of the image to be processed by using guided filtering, wherein the base layer data set and the detail layer data set each include a plurality of original image pairs, and each of the original image pairs includes a pair of an original image and a noise image; S20, constructing an image denoising model based on a quaternion convolutional neural network, wherein the quaternion convolutional neural network includes a first subnetwork and a second subnetwork, the first subnetwork is used to process the base layer data set, and the second subnetwork is used to process the detail layer data set; S30, performing tensor conversion and standardization processing on all images in the base layer data set and the detail layer data set to obtain preprocessed images; S40, setting a loss function and an optimization algorithm, inputting the preprocessed image into the image denoising model to train the image denoising model, and then obtaining an image denoising optimization model; The step S40 comprises: Calculating the error between an output image processed by the image denoising model and an initial image corresponding to the output image; The loss function and the parameters in the optimization algorithm are adjusted according to the size of the error to optimize the image denoising model, thereby obtaining an image denoising optimization model. 2. The image denoising method according to claim 1, wherein the specific steps of step S10 include: The image to be processed is cropped to obtain a plurality of patches, wherein the cropping is divided into two rounds, and the two rounds of cropping have the same effect on the image to be processed, Gaussian noise is added to the patches in the first round of cropping to obtain a noise image, and the patches in the second round of cropping remain the same to obtain an original image; A base layer and a detail layer of the patch are obtained by using guided filtering; the base layers of all the patches are integrated into a base layer data set, and the detail layers of all the patches are integrated into a detail layer data set. 3. The image denoising method according to claim 1 is characterized in that, in the step S20, the depth of the first subnetwork and the second subnetwork both have several layers, and the first layer and the last layer of the first subnetwork and the second subnetwork both use ordinary real-valued convolution, and the intermediate layers all use quaternion convolution. 4. The image denoising method according to claim 1, characterized in that, in the step S40, the loss function uses mean square error and structural similarity as a combined loss function, the optimization algorithm uses the AdamW algorithm, and in the process of training the image denoising model, the cosine annealing algorithm is used to adjust the learning rate; The mean square error loss function is: Wherein, _yi represents the value of the original image, represents the value of the noise image, and n represents the number of the original image pairs; The structural similarity loss function is: Where _μx is ^the mean of _x , _μy is the mean of y, _σx2 is the variance of x, ^σy2 is the variance of y, _σxy is the covariance of x and y, and _c1 and _c2 are constants. 5. The image denoising method according to claim 1, characterized in that before the step of calculating the error between the output image processed by the image denoising model and the initial image corresponding to the output image, it also includes: The features of the preprocessed image are extracted through the quaternion convolutional neural network and forward propagated. 6. The image denoising method according to claim 1, characterized in that after the step of calculating the error between the output image processed by the image denoising model and the initial image corresponding to the output image, the method further comprises: The value of the loss function is determined according to the error, and whether the quaternion convolutional neural network has converged is judged according to the value of the loss function. If not, it is determined that it is necessary to continue training the image denoising model. 7. The image denoising method according to claim 1, characterized in that after the step of adjusting the loss function and the parameters in the optimization algorithm according to the size of the error to optimize the image denoising model, it also includes: Testing and evaluating the trained image denoising optimization model according to the denoised image processed by the image denoising model and the original image corresponding to the denoised image; The test evaluation indicators include: Peak signal-to-noise ratio: used to measure the difference between the denoised image and the original image, the formula is as follows: Wherein, (2 ⁿ -1) ² represents the maximum value of the image pixels; MSE represents the mean square error between the original image and the denoised image; and n represents the number of the original image pairs; Structural similarity: used to measure the similarity between the denoised image and the original image, the formula is as follows: Where μ _x is the mean of x, μ _y is the mean of y, σ _x ² is the variance of x, σ _y ² is the variance of y, σ _xy is the covariance of x and y, and c ₁ and c ₂ are constants; The learnable perceived image block similarity is used to calculate the feature difference between the denoised image and the original image, and the formula is as follows: where d is the distance between _x0 and x, Represent the true value and the predicted value respectively, H, W are the height and width of the denoised image and the original image respectively, l represents the number of layers of the quaternion convolutional neural network, and w _l represents the weight of the lth layer of the quaternion convolutional neural network. 8. An image denoising system, comprising: An acquisition module is used to acquire an image to be processed, and obtain a base layer data set and a detail layer data set of the image to be processed by using guided filtering, wherein the base layer data set and the detail layer data set each include a plurality of original image pairs, and each of the original image pairs includes a pair of an original image and a noise image; A construction module: used to construct an image denoising model based on a quaternion convolutional neural network, wherein the quaternion convolutional neural network includes a first subnetwork and a second subnetwork, the first subnetwork is used to process the base layer data set, and the second subnetwork is used to process the detail layer data set; Preprocessing module: used for performing tensor conversion and standardization processing on all images in the base layer data set and the detail layer data set to obtain preprocessed images; Training module: used to set the loss function and optimization algorithm, input the preprocessed image into the image denoising model to train the image denoising model, and then obtain the image denoising optimization model; The training module includes: A calculation unit: used for calculating the error between the output image processed by the image denoising model and the initial image corresponding to the output image; Optimization unit: used to adjust the loss function and the parameters in the optimization algorithm according to the size of the error to optimize the image denoising model, thereby obtaining an image denoising optimization model. 9. A storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the image denoising method according to any one of claims 1 to 7 is implemented. 10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the image denoising method according to any one of claims 1 to 7 when executing the computer program.