CN109559359B - Artifact removal method for reconstructed image from sparse angle data based on deep learning - Google Patents

Abstract

The invention provides an artifact removal method for reconstructing an image by sparse angle data based on deep learning, which comprises the steps of collecting sparse angle projection data and complete angle projection data; respectively reconstructing by using sparse angle projection data and complete projection data; constructing a neural network for removing artifacts; using the reconstructed image as training data; saving the trained neural network model, and throwing the test image into the neural network model; and E, subtracting the noise of the network prediction obtained in the step E from the test image to obtain a clear image. The deep learning method which is prominent in computer vision at present is introduced into the artifact removal research of the sparse angle projection analysis reconstructed image, and the characteristics of an acceptance-resnet network are utilized to construct a neural network with fine and various expression capacity, so that the method is suitable for artifact removal of the sparse angle data analysis reconstructed image.

Description

Artifact removal method for reconstructed images from sparse angle data based on deep learning

technical field

The invention relates to a method for removing artifacts of images reconstructed from sparse angle data based on deep learning.

Background technique

After more than 30 years of development, computerized tomography (CT) has made great progress in imaging speed and imaging quality. With the increasing application of CT scanning, people are concerned about the radiation of CT scanning More and more attention has been paid to medical research showing that X-ray exposure may induce metabolic abnormalities, cancer, leukemia and other diseases. In routine CT examinations, the radiation dose received by patients has reached a level that cannot be ignored. However, the reduction of CT scanning dose is often accompanied by the decline of imaging quality. The current research hotspot is to reduce the CT scanning dose as much as possible while ensuring the quality of imaging.

Sparse angle projection acquisition mainly achieves the purpose of reducing the scanning dose by reducing the CT scanning time, but its analytical reconstruction will bring serious strip artifacts. According to the redundancy of CT projection data, under the condition of sparse sampling, the missing The data is repaired and estimated to obtain a complete projection data set, and then reconstructed. Under this idea, Li et al. used the dictionary learning method to estimate the missing projection data and reconstructed it, and achieved better results than the TV method. Zhang et al. used S2etram's directional difference method to make a difference on the projection data to obtain missing data in the projection data space, and then perform correlation reconstruction. Compared with the traditional linear difference, this method has achieved better experimental results. In the approach based on Compressed Sensing (CS), Rudin uses a total variation method to remove the Gaussian noise in the projection data, and uses an improved total variation method to remove the Poisson-distributed noise. Ma et al. used the local smoothness of the non-local mean as a constraint and obtained better results than TV. proposed a progressive 2-layer iterative model, which introduced the zero-norm prior constraint into 3D CT reconstruction, and obtained results far superior to the one-norm prior constraint under highly sparse conditions. In addition, the iterative reconstruction algorithm is also a solution to the sparse angle projection, but compared with the analytical reconstruction algorithm, its large amount of calculation limits its wide application.

Among the existing methods based on analytical reconstruction, there are still the following problems:

1. Existing methods based on analytical reconstruction cannot completely and effectively suppress structural large-scale stripe artifacts, and iterative reconstruction methods take too long.

Second, in the traditional non-resnet type of neural network, the problem of gradient explosion and gradient disappearance. Generally speaking, the deeper the network, the stronger the expressive ability of the network. However, in practical applications, when the number of neural network layers exceeds a certain number, the expressive ability of the network will no longer increase, or even be inferior to shallower networks. This is because the gradient update rule of the network is based on the chain rule. The gradient of the front layer comes from the product of the gradients of all the layers behind. In this way, when the number of layers increases, the gradient of the front layer will become very small or very large. See 0.9^100≈0.00002, 1.1^100≈13781. In this way, during training, the update range of the previous layer will be too small or too large to eventually converge.

3. The overall receptive field of the Inception-resnet-v2 network is still not large enough to fully capture the characteristics of strip artifacts.

4. The traditional neural network has a fully connected layer, which will lead to a huge number of network parameters, long training time, and easy overfitting.

Therefore, it is of great significance to introduce the deep learning method, which is currently outstanding in computer vision, into the research on the removal of image artifacts in sparse angle-analytic reconstruction.

Contents of the invention

The purpose of the present invention is to provide an artifact removal method based on deep learning to reconstruct images from sparse angle data to solve the problem that the current analytical reconstruction methods in the prior art cannot effectively suppress large-scale stripe artifacts.

In the present invention, Inception represents the basic module of Google's Inception network, and Inception-resnet represents a new basic network module designed by Google in combination with the residual network and the Inception network. The specific structure is shown in FIG. 2 .

In the present invention, the FDK algorithm is a very classic three-dimensional cone-beam CT reconstruction algorithm in the field of CT image reconstruction, and its name is formed by combining the initials of the names of the three founders.

Technical solution of the present invention is:

A method for removing artifacts from reconstructed images from sparse angle data based on deep learning. Based on deep learning, a neural network for removing artifacts from sparse angle projection reconstructed images that combines full convolutional networks and Inception-resnet network modules is constructed for sparse angle CT. Image de-artifacts and noise are predicted and removed, specifically including the following steps,

S1. Collect several pairs of projection data, each pair of projection data includes sparse angle projection data and complete angle projection data;

S2. Use the sparse angle projection data and complete projection data collected in step S1 to reconstruct respectively, and save the reconstructed data set patchi (i=1, 2, 3, 4, 5...), each pair of data sets includes sparse angles Projection reconstruction datasets and complete projection reconstruction datasets;

S3. Constructing a sparse angle projection reconstruction image artifact elimination neural network;

S4, using the image reconstructed in step S2 as the training data set, complete the projection reconstruction data set as comparison data, train the sparse angle projection reconstruction image artifact elimination neural network constructed in step S3, and save the model with better training effect, Recorded as model model;

S5. Obtain the trained sparse angle projection reconstruction image artifact elimination neural network model model saved in step S4, and put the test image test_img into it, and finally predict and save the test image test_img and the corresponding artifact map noise_img;

S6. Using the test image test_img saved in step S5 to subtract the artifact image noise_img obtained in step S5, a clear image clean_img can be obtained.

Further, in step S1, the collection angle intervals of the collected sparse angle projection data are not less than 4° and not more than 8°, and the collection angle intervals of the complete angle projection data are not more than 0.5°, that is, the number of groups of data collection is less than 90 groups are greater than the range of 45 groups.

Further, in step S2, the FDK reconstruction algorithm is used to reconstruct the sparse angle projection data and the complete projection data collected in step S1, specifically,

S21. Calculating and multiplying the weighting factors of the two-dimensional projection matrix, that is, the projection data collected in step S1, and correcting it so that it satisfies the central slice theorem, and obtaining the corrected projection data;

S22. Perform line-by-line filtering on the projection data corrected in step S21 using a slope filtering method;

S23. Reconstructing an image by back-projecting the filtered data in step S22.

Further, in step S3, the specific structure of the neural network for constructing sparse angle projection reconstructed image artifacts is as follows:

Convolution->BN layer->relu->convolution->BN layer->relu->Resblock->Res_inception_block1->Res_inception_block2->Res_inception_block3->BN layer->relu->convolution;

Among them, the BN layer is batch normalization. During the training of the neural network, Z-score normalization is performed on the output data of the sparse angle projection reconstruction image artifact elimination neural network middle layer, that is, subtracting the mean and dividing the variance; relu is The activation function used by sparse angle projection reconstruction image artifact elimination neural network, its expression is relu(x)=max(0,x); concat represents the matrix splicing layer, that is, splicing the input of multiple branches in a specified dimension Input to the next layer together as a whole; Resblock represents the basic module of a residual network; Res_inception_block1, Res_inception_block2, Res_inception_block 3 respectively represent a basic module of Inception_resnet, different numbers represent different internal structures;

Further, in step S3,

The normal connection path of the Resblock: input->convolution->BN layer->relu->convolution->sum layer, and then it has an extra cross-layer connection, directly connected from the input to the sum layer, and the normal connection path The output of the sum is used as the input of the next layer;

The first connection path of the Res_inception_block1: input->convolution layer->concat layer is the channel splicing layer; the second connection path: input->convolution layer->BN layer->relu->convolution-> BN layer->relu->convolution->concat layer; the third path: input->convolution layer->BN layer->relu->convolution->BN layer->relu->convolution->concat layer; then the output of the above three paths after concat layer splicing and the input of the cross-layer connection path are added through the sum layer, and the result is used as the input of the next layer;

The first connection path of the Res_inception_block2: input->convolution->concat layer; the second connection path: input->convolution->BN layer->relu->convolution->BN layer->relu- >Convolution->concat layer; then add the output of the above two paths through the concat layer to the input of the cross-layer connection path in the sum layer, and the result is used as the input of the next layer;

The first connection path of the Res_inception_block3: input->convolution->concat layer; the second connection path: input->convolution->BN layer->relu->convolution->BN layer->relu- > Convolution; then add the output of the above two paths after concat layer splicing and the input of the cross-layer connection path under the sum layer, and the result is used as the input of the next layer.

Further, step S4 is specifically,

S41. Carry out block processing on the reconstructed data set patchi (i=1, 2, 3, 4, 5...) in step S2, and then invest in the sparse angle projection reconstructed image artifact elimination neural network;

其中mean为patchi的均值，std为其标准差；将patchi*投入稀疏角度投影重建图像伪影消除神经网络进行训练；并将其投入步骤S3中所构建的稀疏角度投影重建图像伪影消除神经网络进行训练，训练包括前向传播与反向传播；最终保存训练得到的模型model，供给步骤S5测试使用。S42. Normalize the data before putting the data set patchi (i=1,2,3,4,5...) into the sparse angle projection reconstruction image artifact elimination neural network, that is, the training samples obtained in step S2 to normalize, that is

Wherein, mean is the mean value of patchi, and std is its standard deviation; put patchi* into the sparse angle projection reconstruction image artifact elimination neural network for training; and put it into the sparse angle projection reconstruction image artifact elimination neural network constructed in step S3 Carry out training, the training includes forward propagation and back propagation; finally save the model model obtained from the training, and provide it for testing in step S5.

Further, in step S42, the mean square error mse is used as the final loss function of the training to correct the network parameters during the training process of the sparse angle projection reconstruction image artifact removal neural network, and the expression is as follows:

Among them, n is the Batch-size of each batch of input data, w and h are the length and width of each sample respectively, and D is the image after subtracting sparse angle projection reconstruction image artifacts to eliminate neural network prediction artifacts and noise The effect image clean_img, X is the comparison data, that is, the difference image between the sparse angle projection reconstruction data set and the complete angle projection reconstruction data set reconstructed in step S2; during the training process, the sparse angle projection reconstruction image artifacts are eliminated by the neural network according to the loss The value of the function, using the backpropagation algorithm and the Adam optimization algorithm to calculate the update amount layer by layer to update the parameters of the network successively to optimize the model model.

Further, step S5 is specifically,

S51. Save the model model trained in step S4;

S52. Put the test image test_img into the model model to predict the noise image noise_img, and save the predicted noise image noise_img.

Further, step S6 is specifically,

S61. Obtain the predicted noise image noise_img and test image test_img in step S5.

S62. Subtract the artifact image noise_img from the test image test_img to obtain the required clear image clean_img.

The beneficial effects of the present invention are:

1. This method of removing artifacts from reconstructed images based on sparse angle data based on deep learning introduces the current deep learning method, which is outstanding in computer vision, into the research on artifact removal of sparse angle projection analytically reconstructed images, using Inception- The characteristics of the resnet network can build a neural network with fine and diverse expressive capabilities, which is suitable for artifact removal of sparse angle data analysis and reconstruction images;

2. In the overall structure of the Inception-resnet network, the method of the present invention uses resnet to replace the simple convolution layer in the original Inception-resnet to construct a deeper neural network, so that the overall receptive field of the network is larger and more in line with Characterization of the global distribution of streaking artifacts.

3. The present invention uses the FCN full convolutional network mode, that is, only the convolutional layer is used instead of the fully connected layer. Due to the characteristics of convolutional weight sharing and local connection, the parameters of the network are greatly reduced, and the network parameters are greatly reduced. The training time also improves the anti-overfitting ability, and it is easier to design an image-to-image network by relying on FCN, saving a lot of trouble.

Description of drawings

FIG. 1 is a schematic flowchart of a method for removing artifacts of an image reconstructed from sparse angle data based on deep learning according to an embodiment of the present invention.

Fig. 2 is a schematic structural diagram of the basic module of Inception-resnet in the embodiment.

Fig. 3 is a schematic diagram of the basic structure of the neural network used in the embodiment for sparse angle projection reconstructed image artifact removal.

Fig. 4 is a schematic structural diagram of the corresponding Resblock module in the embodiment.

FIG. 5 is a schematic diagram of the basic structure of Res_inception_block1 corresponding to FIG. 3 .

FIG. 6 is a schematic diagram of the basic structure of Res_inception_block2 corresponding to FIG. 3 .

FIG. 7 is a schematic diagram of the basic structure of the corresponding Res_inception_block3 in FIG. 3 .

Fig. 8 is a schematic diagram of the convolution method in the embodiment.

Among them, Input means input, and Output means output;

Inception represents the basic module of Google's Inception network;

Inception-resnet represents a new basic network module designed by Google in combination with residual network (resnet) and Inception network;

Conv 1*1 means it is a convolutional layer with a convolution kernel size of 1*1; Conv 1*3 means it is a convolutional layer with a convolution kernel size of 1*3; Conv 3*1 means it is a Convolution layer, convolution kernel size is 3*1; Conv 3*3 means it is a convolution layer, convolution kernel size is 3*3; Conv 1*7 means it is a convolution layer, convolution kernel size is 1*7; Conv7*1 means that it is a convolutional layer, and the convolution kernel size is 7*1;

Relu is a modified linear unit, which represents an activation function used by the sparse angle projection reconstruction image artifact removal neural network;

The BN layer represents the batch normalization layer (Batch normalization);

Filter concat represents the matrix splicing layer, that is, the input of multiple branches is spliced together in a specified dimension and input to the next layer as a whole;

Resblock represents the basic module of a residual network;

Res_inception_block1, Res_inception_block 2, and Res_inception_block 3 respectively represent a basic module of Inception_resnet, and different numbers represent different internal structures (generally speaking, the combination of convolutions is different).

Detailed ways

Preferred embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings.

Example

A method for removing artifacts from reconstructed images from sparse angle data based on deep learning, by collecting sparse angle projection data and complete angle projection data; using sparse angle projection data and complete projection data to reconstruct respectively; constructing sparse angle projection reconstruction image artifacts Shadow elimination neural network; use the reconstructed image as training data; save the trained model and put the test image into it; use the test image to subtract the sparse angle projection obtained in step E to reconstruct image artifacts and eliminate the noise predicted by the neural network , a clear image can be obtained. Introduce the deep learning method, which is currently outstanding in computer vision, into the research on the artifact removal of sparse angle projection analytically reconstructed images, and use the characteristics of the Inception-resnet network to construct a sparse angle projection reconstruction image artifact with fine expressiveness and variety Elimination of neural networks, suitable for artifact removal in images reconstructed from sparse angle data analysis.

A method for removing artifacts of sparse angle data reconstruction images based on deep learning in the embodiment. Based on deep learning, construct a sparse angle projection reconstruction image artifact removal neural network combining full convolution network and Inception-resnet, aiming at sparse The anti-artifact and noise of the angle CT image are predicted and removed; as shown in Figure 1, the following steps are specifically included:

S1. Five pairs of projection data Ai (i=1, 2, 3, 4, 5) are collected, each pair includes a set of sparse angle projection data Ai_low and a set of complete angle projection data Ai_high.

The collection angle interval of the sparse angle projection data collected in step S1 is not less than 4° and not more than 8°, and the collection angle interval of the complete angle projection data is not more than 0.5°, that is, the number of data collection groups is less than 90 and greater than 45 within the scope of the group.

S2. Use the five pairs of projection data Ai collected in step S1 to reconstruct, and save the reconstructed data set as five pairs S2i (i=1, 2, 3, 4, 5), and each pair of data sets is divided into sparse angle projections The reconstruction data set S2i_low and the complete angle projection reconstruction data set S2i_high, the single-layer image size in each pair of data sets is 512*512.

The reconstruction in step S2 utilizes the classic FDK algorithm in the analytical reconstruction algorithm, specifically,

S21. Calculating and multiplying weighting factors for the two-dimensional projection matrix, and modifying them to roughly satisfy the central slice theorem.

S22. Perform line-by-line filtering on the corrected projection data, wherein a slope filtering method is adopted.

S23. Perform back projection on the filtered data to reconstruct an image.

S3. Constructing a sparse angle projection reconstructed image artifact elimination neural network.

S31. The overall structure of the constructed sparse angle projection reconstructed image artifact elimination neural network is shown in Figure 3.

Convolution->BN layer (batch-normalization) layer->relu->convolution->BN layer->relu->Resblock->Res_inception_block1->Res_inception_block2->Res_inception_block3->BN layer->relu->convolution.

Among them, the BN layer is batch normalization. During the training of the neural network, Z-score normalization is performed on the output data of the sparse angle projection reconstruction image artifact elimination neural network middle layer, that is, subtracting the mean and dividing the variance; relu is The activation function used by sparse angle projection reconstruction image artifact elimination neural network, its expression is relu(x)=max(0,x); concat represents the matrix splicing layer, that is, splicing the input of multiple branches in a specified dimension Input to the next layer together as a whole; Resblock represents the basic module of a residual network; Res_inception_block1, Res_inception_block 2, and Res_inception_block 3 respectively represent a basic module of Inception_resnet, and different numbers represent different internal structures;

S311, the Resblock described in step S31 is shown in Figure 4, the normal connection path: input->convolution->BN layer->relu->convolution->sum layer, and then compared with the ordinary convolutional network An additional cross-layer connection is directly connected from the input to the sum layer, which is added to the output of the normal connection path as the input of the next layer.

S312, the specific structure of Res_inception_block1 described in step S31 is shown in Figure 5, the first connection path: input -> convolution layer -> Filter concat layer (matrix splicing layer). The second connection path: input->convolution layer->BN layer->relu->convolution->BN layer->relu->convolution->concat layer. The third path: input->convolution layer->BN layer->relu->convolution->BN layer->relu->convolution->concat layer. Then, the output of the above three paths spliced by the concat layer and the input of the cross-layer connection path are added through the sum layer, and the result is used as the input of the next layer.

S313. The specific structure of Res_inception_block2 described in step S31 is shown in FIG. 6, the first connection path: input->convolution->concat layer. The second connection path: input->convolution->BN layer->relu->convolution->BN layer->relu->convolution->concat layer. Then, the output of the above two paths spliced by the concat layer is added to the input of the cross-layer connection path in the sum layer, and the result is used as the input of the next layer.

S314. The specific structure of Res_inception_block3 described in step S31 is shown in FIG. 7, the first connection path: input->convolution->concat layer. The second connection path: input->convolution->S2N->relu->convolution->S2N->relu->convolution. Then, the output of the above two paths spliced by the concat layer and the input of the cross-layer connection path are added under the sum layer, and the result is used as the input of the next layer.

Sparse Angle Projection Reconstruction Image Artifact Removal The neural network convolution method adopts boundary expansion convolution, so that the input and output dimensions of each layer are the same.

S4, using the data set Bi_low (i=1,2,3,4,5) preserved in step S2 as the training data set of the sparse angle projection reconstruction image artifact elimination neural network, the data set Bi_high (i=1,2, 3, 4, 5) As the control data, train the neural network for image artifact elimination for sparse angle projection reconstruction constructed in step S3, save the model with better training effect, and record it as model.

S41. Carry out block processing to the data Bi (i=1,2,3,4,5) obtained in step S2, and the block processing is about to divide each image into several 128*128 small block data sets patchi (i= 1, 2, 3, 4, 5) and then put into the sparse angle projection reconstructed image artifact removal neural network for learning.

其中mean为patchi的均值，std为其标准差；将patchi*投入稀疏角度投影重建图像伪影消除神经网络进行训练，并保存训练效果较好的模型记为model。S42. Before inputting the data patchi (i=1,2,3,4,5), it is necessary to perform Z-score normalization processing on the data, namely

Where mean is the mean value of patchi, and std is its standard deviation; put patchi* into the sparse angle projection reconstruction image artifact elimination neural network for training, and save the model with better training effect as model.

In step S42, during the sparse angle projection reconstruction image artifact elimination neural network training process, the mean square error mse is used as the final loss function of the training to correct the network parameters, and the expression is as follows:

Among them, n is the Batch-size of each batch of input data, w and h are the length and width of each sample respectively, and D is the image after subtracting sparse angle projection reconstruction image artifacts to eliminate neural network prediction artifacts and noise The effect image clean_img, X is the comparison data, that is, the difference image between the sparse angle projection reconstruction data set and the complete angle projection reconstruction data set reconstructed in step S2; during the training process, the sparse angle projection reconstruction image artifacts are eliminated by the neural network according to the loss The value of the function, using the backpropagation algorithm and the Adam optimization algorithm to calculate the update amount layer by layer to update the parameters of the network successively to optimize the model model.

S5. Obtain the model trained in step S4, put the image that needs to be processed for artifact removal as the test image test_img, put it into the model, and finally predict and save the test_img and the corresponding artifact image noise_img.

The specific steps in step S5 are to first obtain the model model saved in step S42, put the image test_img that needs to be de-artifacted into the model to predict the artifact map, and save the predicted artifact map noise_img.

S6. Using the test image test_img saved in step E to subtract the artifact image noise_img, a clear image clean_img can be obtained.

S61. Obtain the noise_img predicted in step E and the test_img used by it.

S62. Subtract the corresponding artifact image noise_img from the test image test_img to obtain the desired clear effect image clean_img.

The idea of many advanced neural network models, such as resnet, Inception-resnet, FCN, etc., is used in the sparse angle projection reconstructed image artifact elimination neural network constructed in the embodiment. The specific structure is shown in Figure 2;

In the embodiment, the sparse angle projection reconstruction image artifact elimination neural network is inspired by the idea of resnet, and adopts the "short connection" technique, as shown in the right half of Figure 2, which alleviates the gradient dispersion and gradient explosion that occur in traditional deep networks. . Gradient dispersion means that after the conventional network structure is stacked and the number of network layers increases to a certain level, the performance of sparse angle projection reconstruction image artifact removal neural network will not only not get better, but may even get worse. Gradient dispersion is a relatively common phenomenon in deep neural networks. How to deepen the network and improve the expressive ability of the network while avoiding the phenomenon of gradient dispersion is a hot spot in deep learning. The cross-layer connection directly connects two non-adjacent hidden layers to each other, so that when the network is very deep, even if there is a problem in some layers, due to the existence of the cross-layer connection, the results of the previous hidden layer can be passed through a short Connecting cross-layer input to the back will not affect the overall effect too much. In the Inception-resnet basic module, this advantage of resnet is absorbed.

Inception is characterized by the parallelism of multiple convolution kernel combinations, that is, it is "wider". In the past, when ordinary networks were used to extract different features, they only used a number of convolution kernels in the same combination, while the Inception structure used several convolution kernels in different combinations to extract different features. This kind of structure has a good fit for the characteristics of the large difference between the structural artifact and other noise features in the sparse angle CT image.

The Inception-resnet basic module combines the above resnet and Inception, as shown in Figure 2, which can take advantage of both. In the overall sparse angle projection reconstruction image artifact elimination neural network structure, using multiple Inception-resnet module combinations can build a deeper network to improve the overall expression ability a lot, and due to the existence of Inception, the way of extracting different features It is more effective, and the problem studied in the embodiment is that sparse angle artifacts and noise are mixed, that is, the features to be predicted have both high, medium and low levels, which is very suitable for the network built with the basic module of Inception-resnet. As shown in Table 1, several different networks are used for training and testing, from top to bottom are pure residual network, simple stacked convolutional network, and sparse angle projection reconstruction image artifact removal neural network in the embodiment. Network, 9 layers refers to the number of layers of the basic block used, such as Resblock, Inception block. Use psnr and ssim as evaluation indicators.

The overall structure of the sparse angle projection reconstruction image artifact elimination neural network is inspired by the FCN full convolutional neural network. Its advantage is that it uses convolution instead of full connection. Its local connection and parameter sharing characteristics greatly reduce the training parameters. It reduces the hardware requirements for experiments and reduces the possibility of overfitting at some levels. Because the full convolutional network cancels the full connection, it has no hard requirements for the input size, and is more suitable for the field of image processing.

The convolution method of sparse angle projection reconstructed image artifact removal neural network adopts boundary expansion convolution, as shown in Figure 8. Make the input and output dimensions of each layer the same. Using sparse angle data reconstruction images and complete angle data reconstruction images as training data, the sparse angle projection reconstruction image artifact elimination neural network can more accurately predict artifacts and other noises in sparse angle images.

The overall idea of deep learning to remove artifacts can be divided into two types, one is to directly map from an image with artifacts and noise to the original image, and the other is to map from an image with noise and artifacts to it. Noise and artifacts, and then use the image with noise and artifacts to subtract the predicted noise and artifacts to obtain the original information of the image. According to related research, the second method of removing artifacts has a simpler network topology than the first one, that is, the noise of artifacts is easier to learn than the normal organizational structure, so this method adopts the latter;

In the embodiment, the sparse angle projection reconstruction image artifact elimination neural network adopts mse as the final loss function, and the expression is as follows:

Among them, n is the Batch-size of each batch of input data, w and h are the length and width of each sample respectively, and D is the image after subtracting sparse angle projection reconstruction image artifacts to eliminate neural network prediction artifacts and noise Effect picture, X is the comparison picture, that is, the corresponding complete angle image. During the training process, the sparse angle projection reconstruction image artifact elimination neural network calculates the update amount layer by layer according to the value of the loss function, according to the back propagation algorithm (Back Propagation, BP) and the Adam optimization algorithm to update the weights and biases;

Sparse angle projection reconstruction image artifact elimination neural network parameter settings: the number of iterations is 150, the batch size is 48, the initial learning rate is 0.001, every 30 iterations decays to 60% of itself, the loss function uses mse, The optimizer uses Adam to create a sparse angle projection reconstruction image artifact removal neural network for training and testing. The test psnr of the reconstructed image from the final 90-angle projection data reaches 36.5915, and the ssim reaches 0.8818, as shown in Table 1.

Table 1 embodiment and training and test results of different networks

The specific implementation of this method has been explained above, certainly this method also can have other multiple specific implementation modes, and those who are familiar with this technical field can make various changes and distortions under the premise of not violating the spirit of the present invention, but These changes and deformations should be included in the scope of protection defined by the patent requirements of this application.

Claims (7)

1. A method for removing artifacts from sparse angle data reconstruction images based on deep learning, characterized in that: based on deep learning, a sparse angle projection reconstruction image artifact removal neural network module combining full convolutional network and Inception-resnet network module is constructed The network predicts and removes the artifacts and noise of the sparse-angle CT image, and specifically includes the following steps, S1. Collect several pairs of projection data, each pair of projection data includes sparse angle projection data and complete angle projection data; S2. Use the sparse angle projection data and complete projection data collected in step S1 to reconstruct respectively, and save the reconstructed data set patchi (i=1, 2, 3, 4, 5...), each pair of data sets includes sparse angles Projection reconstruction datasets and complete projection reconstruction datasets; S3. Constructing a sparse angle projection reconstruction image artifact elimination neural network; In step S3, the specific structure of the sparse angle projection reconstructed image artifact elimination neural network is as follows: Convolution->BN layer->relu->convolution->BN layer->relu->Resblock->Res_inception_block1->Res_inception_block2->Res_inception_block3->BN layer->relu->convolution; Among them, the BN layer is batch normalization. During the training of the neural network, Z-score normalization is performed on the output data of the sparse angle projection reconstruction image artifact elimination neural network middle layer, that is, subtracting the mean and dividing the variance; relu is The activation function used by sparse angle projection reconstruction image artifact elimination neural network, its expression is relu(x)=max(0,x); concat represents the matrix splicing layer, that is, splicing the input of multiple branches in a specified dimension Input to the next layer together as a whole; Resblock represents the basic module of a residual network; Res_inception_block1, Res_inception_block2, and Res_inception_block3 respectively represent a basic module of Inception_resnet, and different numbers represent different internal structures; The normal connection path of the Resblock: input->convolution->BN layer->relu->convolution->sum layer, and then it has an extra cross-layer connection, directly connected from the input to the sum layer, and the normal connection path The output of the sum is used as the input of the next layer; The first connection path of the Res_inception_block1: input->convolution layer->concat layer is the channel splicing layer; the second connection path: input->convolution layer->BN layer->relu->convolution-> BN layer->relu->convolution->concat layer; the third path: input->convolution layer->BN layer->relu->convolution->BN layer->relu->convolution->concat layer; then the output of the above three paths after concat layer splicing and the input of the cross-layer connection path are added through the sum layer, and the result is used as the input of the next layer; The first connection path of the Res_inception_block2: input->convolution->concat layer; the second connection path: input->convolution->BN layer->relu->convolution->BN layer->relu- >Convolution->concat layer; then add the output of the above two paths through the concat layer to the input of the cross-layer connection path in the sum layer, and the result is used as the input of the next layer; The first connection path of the Res_inception_block3: input->convolution->concat layer; the second connection path: input->convolution->BN layer->relu->convolution->BN layer->relu- > Convolution; then add the output of the above two paths after concat layer splicing and the input of the cross-layer connection path under the sum layer, and the result is used as the input of the next layer; S4, using the image reconstructed in step S2 as the training data set, complete the projection reconstruction data set as comparison data, train the sparse angle projection reconstruction image artifact elimination neural network constructed in step S3, and save the model with better training effect, Recorded as model model; S5. Obtain the trained sparse angle projection reconstruction image artifact elimination neural network model model saved in step S4, and put the test image test_img into it, and finally predict and save the test image test_img and the corresponding artifact map noise_img; S6. Using the test image test_img saved in step S5 to subtract the artifact image noise_img obtained in step S5, a clear image clean_img can be obtained. 2. The artifact removal method based on the sparse angle data reconstruction image realized by deep learning as claimed in claim 1, characterized in that: in step S1, the acquisition angle intervals of the sparse angle projection data collected are not less than 4° and If it is greater than 8°, the angular interval of complete angle projection data collection is not more than 0.5°, that is, the number of data collection groups is within the range of less than 90 groups and greater than 45 groups. 3. The method for removing artifacts based on the sparse angle data reconstruction image realized by deep learning as claimed in claim 1, wherein the sparse angle projection data collected by step S1 and the complete projection data are respectively obtained by using the FDK reconstruction algorithm in step S2. to rebuild, specifically, S21. Calculating and multiplying the weighting factors of the two-dimensional projection matrix, that is, the projection data collected in step S1, and correcting it so that it satisfies the central slice theorem, and obtaining the corrected projection data; S22. Perform line-by-line filtering on the projection data corrected in step S21 using a slope filtering method; S23. Reconstructing an image by back-projecting the filtered data in step S22. 4. The method for removing artifacts based on the sparse angle data reconstruction image realized by deep learning according to any one of claims 1-3, characterized in that: step S4 is specifically, S41. Carry out block processing on the reconstructed data set patchi (i=1, 2, 3, 4, 5...) in step S2, and then invest in the sparse angle projection reconstructed image artifact elimination neural network; S42. Normalize the data before putting the data set patchi (i=1,2,3,4,5...) into the sparse angle projection reconstruction image artifact elimination neural network, that is, the training samples obtained in step S2 to normalize, that is

Wherein, mean is the mean value of patchi, and std is its standard deviation; put patchi* into the sparse angle projection reconstruction image artifact elimination neural network for training; and put it into the sparse angle projection reconstruction image artifact elimination neural network constructed in step S3 Carry out training, the training includes forward propagation and back propagation; finally save the model model obtained from the training, and provide it for testing in step S5. 5. The artifact removal method of the sparse angle data reconstruction image realized based on deep learning as claimed in claim 4, characterized in that: in step S42, in the sparse angle projection reconstruction image artifact elimination neural network training process, the average method is specifically adopted. The square error mse is used as the final loss function of the training to correct the network parameters, and the expression is as follows:

Among them, n is the Batch-size of each batch of input data, w and h are the length and width of each sample respectively, and D is the image after subtracting sparse angle projection reconstruction image artifacts to eliminate neural network prediction artifacts and noise The effect image clean_img, X is the comparison data, that is, the difference image between the sparse angle projection reconstruction data set and the complete angle projection reconstruction data set reconstructed in step S2; during the training process, the sparse angle projection reconstruction image artifacts are eliminated by the neural network according to the loss The value of the function, using the backpropagation algorithm and the Adam optimization algorithm to calculate the update amount layer by layer to update the parameters of the network successively to optimize the model model. 6. The method for removing artifacts based on deep learning to reconstruct images based on sparse angle data according to any one of claims 1-3, characterized in that: Step S5 is specifically, S51. Save the model model trained in step S4; S52. Put the test image test_img into the model model to predict the noise image noise_img, and save the predicted noise image noise_img. 7. The method for removing artifacts based on deep learning to reconstruct images based on sparse angle data according to any one of claims 1-3, characterized in that: Step S6 is specifically, S61. Obtain the predicted artifact map noise_img and test image test_img in step S5; S62. Subtract the artifact image noise_img from the test image test_img to obtain the required clear image clean_img.

Images