CN111489404B - Image reconstruction method, image processing device and device with storage function - Google Patents

Abstract

The application discloses an image reconstruction method, an image processing device and a device with a storage function, wherein the image reconstruction method comprises the steps of obtaining an original image; obtaining input image data according to an original image; performing a first mapping operation on the input image data according to a first mapping function in the generator network to obtain output image data; a reconstructed image is formed from the output image data. According to the application, the first mapping operation is carried out on the input image data of the original image according to the first mapping function in the generator network to obtain the output image data, so that the image reconstruction is directly realized, the clear presentation of the image of the low-dose radioactive tracer can be realized, and the clinical diagnosis of doctors is facilitated; because the data volume that the mapping mode gathered is less for image reconstruction speed is faster, is favorable to improving work efficiency, and its required scanning time is also shorter, can avoid appearing the artifact, thereby improve image quality.

Description

Image reconstruction method, image processing device and device with storage function

technical field

The invention relates to the technical field of image processing, in particular to an image reconstruction method, an image processing device and a device with a storage function.

Background technique

Clinical examination imaging technology in the field of nuclear medicine can be applied to various medical clinical examination fields, such as PET (Positron Emission Computed Tomography, positron emission computed tomography) technology, etc., through injecting radioactive tracers to patients to achieve imaging, help Doctors can more accurately diagnose patients' conditions.

The inventors of the present application have found in long-term research and development that since the radioactive radiation of the radioactive tracer is potentially harmful to the human body, people pay more and more attention to the dose of the injected radioactive tracer. Large doses of radiotracers not only have safety issues, but also require a large amount of data to be collected for imaging, resulting in slower image reconstruction and, due to the longer scan time required, unavoidable physiological problems in patients. movement, causing artifacts to appear. Although reducing the dose of the radiotracer can reduce the radiation to the patient, it will affect the imaging effect of the image, which is not conducive to the doctor's clinical diagnosis.

Contents of the invention

The invention provides an image reconstruction method, an image processing device and a device with a storage function to solve the technical problem in the prior art that low-dose tracers affect the imaging effect of images.

In order to solve the above technical problems, a technical solution adopted by the present invention is to provide an image reconstruction method, including:

get the original image;

Obtaining input image data according to the original image, the input image data is sinusoidal data;

performing a first mapping operation on the input image data according to a first mapping function in the generator network to obtain output image data;

A reconstructed image is formed from the output image data.

In a specific embodiment, the first mapping operation includes encoding, conversion, and decoding, and the encoding method includes:

sequentially performing the first convolution operation on the input image data to obtain the first characteristic image data;

The number of convolutional layers in the first convolution operation is 14, wherein the step size of the 10-layer convolutional layer is 1, and the step size of the 4-layer convolutional layer is 2.

In a specific embodiment, the conversion method includes:

performing a second convolution operation on the first characteristic image data to obtain second characteristic image data;

The number of convolutional layers in the second convolution operation is 11, wherein the feature number of the 6-layer convolutional layer is 512, and the feature number of the 5-layer convolutional layer is 1024.

In a specific embodiment, the decoding method includes:

performing an upsampling operation and a third convolution operation on the second feature image data in sequence to obtain output image data;

Wherein, the number of convolution kernels of the last convolution layer in the third convolution operation is 1.

In a specific embodiment, a batch normalization operation is performed after the operation of each convolutional layer in the first convolution operation, the second convolution operation, and the third convolution operation and activation operations.

In a specific embodiment, after obtaining the output image data, it further includes:

Processing the output image data through a discriminator network to obtain generated adversarial loss data;

Identifying whether the reconstructed image meets a standard based on the generated adversarial loss data.

In a specific embodiment, the method for processing the output image data through a discriminator network includes:

performing a fourth convolution operation on the output image data to obtain a third feature image;

performing fully-connected layer processing on the third feature image to obtain the generated adversarial loss data;

Wherein, the number of layers of the convolutional layer in the fourth convolution operation is 8, the step size of the even-numbered layer convolutional layer is 2, the step size of the odd-numbered layer convolutional layer is 1, and the number of layers of the fully connected layer for 2.

In a specific embodiment, in the fourth convolution operation and the fully connected layer processing, the activation operation is performed after the operation of each convolution layer and after the processing of the first fully connected layer.

In a specific embodiment, after obtaining the output image data, it further includes:

obtaining a first data set according to the output image data, and obtaining a second data set according to the target image data;

Obtaining the mean square error loss function and the perceptual loss function according to the first data set and the second data set;

Judging whether the reconstructed image meets a standard according to the mean square error loss function and the perceptual loss function.

In a specific embodiment, after obtaining the mean square error loss function and the perceptual loss function, it further includes:

performing optimization calculations on the mean square error loss function and the perceptual loss function;

According to the optimized calculated mean square error loss function and perceptual loss function, it is judged whether the reconstructed image meets the standard.

In a specific embodiment, after the obtaining the output image data includes:

obtaining a third data set according to the input image data;

Extract corresponding data from the third data set, the first data set and the second data set as input data, output data and network labels respectively;

The first mapping function is trained according to the input data, the output data, the network label, the generated adversarial loss data, the mean square error loss function and the perceptual loss function to obtain the second mapping function.

In order to solve the above technical problems, another technical solution adopted by the present invention is to provide an image processing device, including:

Receiver, used to obtain the original image;

A processor, connected to the receiver, for obtaining input image data according to the original image; performing a first mapping operation on the input image data according to a first mapping function in the generator network to obtain output image data; wherein , the input image data is sinusoidal data;

a display, connected to the processor, for receiving the output image data, and forming a reconstructed image according to the output image data.

In order to solve the above technical problem, another technical solution adopted by the present invention is to provide a device with storage function, which stores program data, and the program data can be executed to implement the above method.

In the present invention, the first mapping operation is performed on the input image data of the original image according to the first mapping function in the generator network to obtain the output image data, thereby directly realizing image reconstruction and realizing the clear presentation of images of low-dose radioactive tracers, It is beneficial to the doctor's clinical diagnosis; due to the small amount of data collected by the mapping method, the image reconstruction speed is faster, which is conducive to improving work efficiency, and the required scanning time is also shorter, which can avoid the appearance of artifacts, thereby improving image quality.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings that need to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative work, in which:

Fig. 1 is a schematic flow chart of an embodiment of the image reconstruction method of the present invention;

Fig. 2 is a schematic flow chart of another embodiment of the image reconstruction method of the present invention;

Fig. 3 is a schematic flow chart of another embodiment of the image reconstruction method of the present invention;

4 is a schematic flow chart of the first mapping operation in another embodiment of the image reconstruction method of the present invention;

Fig. 5 is a schematic flow chart of discriminator network processing in another embodiment of the image reconstruction method of the present invention;

6 is a schematic structural diagram of an embodiment of an image processing device of the present invention;

Fig. 7 is a schematic structural diagram of an embodiment of a device with a storage function according to the present invention.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

The terms "first" and "second" in this application are only used for descriptive purposes, and should not be understood as indicating or implying relative importance or implicitly specifying the quantity of indicated technical features. In the description of the present application, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined. Furthermore, the terms "include" and "have", as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes unlisted steps or units, or optionally further includes For other steps or units inherent in these processes, methods, products or apparatuses. The term "and/or" is just an association relationship describing associated objects, which means that there can be three relationships, for example, A and/or B, which can mean: A exists alone, A and B exist simultaneously, and B exists alone These three situations. In addition, the character "/" in this article generally indicates that the contextual objects are an "or" relationship.

Referring to Fig. 1, an embodiment of the image reconstruction method of the present invention includes:

S110. Acquire an original image.

In this embodiment, the original image is an image directly collected after imaging of the patient injected with a low dose of tracer.

S120. Obtain input image data according to the original image.

In this embodiment, the input image data is sinusoidal data to realize the mapping.

S130. Perform a first mapping operation on the input image data according to the first mapping function in the generator network to obtain output image data.

In this embodiment, the first mapping function is a preset mapping function.

S140. Form a reconstructed image according to the output image data.

In the embodiment of the present invention, the first mapping operation is performed on the input image data of the original image according to the first mapping function in the generator network to obtain the output image data, thereby directly realizing image reconstruction, and realizing the clarity of images of low-dose radioactive tracers Presentation is beneficial to doctors' clinical diagnosis; due to the small amount of data collected by the mapping method, the image reconstruction speed is faster, which is conducive to improving work efficiency, and the required scanning time is also shorter, which can avoid the appearance of artifacts. Thereby improving image quality.

The image reconstruction method in this embodiment can be applied to PET (Positron Emission Computed Tomography, positron emission computed tomography) image reconstruction, and can also be applied to CT (Computed Tomography, electronic computer tomography) or CT after being adjusted according to actual conditions. Image reconstruction in technical fields such as SPECT (Single-Photon Emission Computed Tomography, Single-Photon Emission Computed Tomography) is not limited here.

Referring to Fig. 2, another embodiment of the image reconstruction method of the present invention includes:

S210. Acquire an original image.

S220. Obtain input image data according to the original image.

In this embodiment, the input image data is 288×269 sinusoidal data.

S230. Perform a first mapping operation on the input image data according to the first mapping function in the generator network to obtain output image data.

Referring to Figure 3 and Figure 4 together, the first mapping operation includes encoding, conversion, and decoding, specifically including:

Encoding: S231. Perform the first convolution operation on the input image data in order to obtain the first feature image number.

In this embodiment, the number of convolutional layers in the first convolution operation is 14, wherein the step size of the 10-layer convolutional layer is 1, and the step size of the 4-layer convolutional layer is 2. Specifically, the first convolution operation includes one set of first convolutional layer groups 310 and four sets of second convolutional layer groups 320 arranged in sequence, the first convolutional layer group 310 includes two layers of first convolutional layer 311, The step size of the first convolutional layer 311 is 1, and the size of the convolution kernel is 5*5. The second convolutional layer group 320 includes one layer of the second convolutional layer 321 and two layers of the third convolutional layer 322. The second The step size of the convolution layer 321 is 2, the size of the convolution kernel is 3*3, the step size of the third convolution layer 322 is 1, and the size of the convolution kernel is 3*3.

In this embodiment, the calculation formula of the convolutional layer is:

F _out1 ＝(F _in1 *α _down )↓ _p (1)

Among them, F _in1 is the input data of the convolutional layer, F _out1 is the output data of the convolutional layer, α _down is the convolution operation, and p is the reduction multiple. Among them, when the step size of the convolutional layer is 1, p is 1, which means that the size of the input data and output data remains unchanged; when the step size of the convolutional layer is 1, p is 2, which means that the output data is the size of the input data half.

Transformation: S232. Perform a second convolution operation on the first feature image data to obtain second feature image data.

In this embodiment, the number of convolutional layers in the second convolution operation is 11, wherein the feature number of the 6-layer convolutional layer is 512, and the feature number of the 5-layer convolutional layer is 1024. Specifically, the second convolution operation includes 11 groups of third convolutional layer groups 410 arranged in sequence, each group of third convolutional layer groups 410 includes one layer of fourth convolutional layer 411, and the steps of the fourth convolutional layer 411 The length is 1, the size of the convolution kernel is 3*3, and the feature number of the fourth convolution layer 411 of the first 3 layers is 512, the feature number of the fourth convolution layer 411 of the middle 4 layers is 1024, and the fourth layer of the last 3 layers The number of features of the convolutional layer 411 is 512.

Decoding: S233. Perform an upsampling operation and a third convolution operation on the second feature image data in sequence to obtain output image data.

In this embodiment, the third convolution operation includes a plurality of fourth convolutional layer groups 510 and fifth convolutional layer groups 520 arranged in sequence. Specifically, the fourth convolutional layer group 510 includes 1 layer of upsampling layer 511 and 3 layers of sixth convolutional layer 512, the step size of the sixth convolutional layer 512 is 1, and the size of the convolution kernel is 3*3. The fifth convolution layer 520 , that is, the number of convolution kernels of the last convolution layer in the third convolution operation is 1.

In this embodiment, the calculation formula of the upsampling layer is:

F _out2 ＝(F _in2 *α _up )↑ _q (2)

Among them, F _in2 is the input data of the up-sampling layer, F _out2 is the output data of the up-sampling layer, α _up is the convolution operation, q is the magnification multiple, and the multiple is set to 2, indicating that the input feature map is enlarged to the original twice as much.

In this embodiment, a batch normalization layer 312 and an activation layer 313 are arranged sequentially after the operation of each convolution layer in the first convolution operation, the second convolution operation, and the third convolution operation , used for batch normalization and activation operations. Wherein, the activation operation may be realized by a ReLU (Rectified Linear Unit, linear rectification function) function.

S240. Process the output image data through the discriminator network to obtain generation adversarial loss data.

Referring to FIG. 5 together, in this embodiment, the method for processing the output image data through the discriminator network includes:

S241. Perform a fourth convolution operation on the output image data to obtain a third feature image.

In this embodiment, the fourth convolution operation includes 8 convolutional layers, of which 4 layers are the seventh convolutional layer 610, and the other 4 layers are the eighth convolutional layer 620, and the seventh convolutional layer 610 and the first convolutional layer The eight convolutional layers 620 are arranged alternately in sequence.

In this embodiment, the step size of the seventh convolutional layer 610 , that is, the odd-numbered convolutional layer is 1. The eighth convolutional layer 620 , that is, the step size of the even-numbered convolutional layer is 2 to reduce the size of the input feature image and double the number of convolution kernels.

In this embodiment, the number of convolution kernels of the 8 convolution layers in the fourth convolution operation is 32, 32, 64, 64, 128, 128, 256, and 256 in sequence.

S242. Perform fully-connected layer processing on the third feature image to obtain generation adversarial loss data.

In this embodiment, two fully connected layers 710 are provided after the fourth convolution operation.

In this embodiment, in the fourth convolution operation and fully connected layer processing, an activation layer 630 is provided after the operation of each convolutional layer and after the processing of the first fully connected layer 710 for performing activation operations. . Among them, the activation operation can be realized by the Leaky ReLU (LeakyRectified Linear Unit, with leaky rectified linear unit) function.

In this embodiment, it can be identified whether the reconstructed image meets the standard according to the generated adversarial loss data.

S251. Acquire a first data set according to the output image data, and obtain a second data set according to the target image data.

In this embodiment, the target image data is preset image data of a standard image corresponding to the original image, and is used for training the first mapping function.

S252. Obtain a mean square error loss function and a perceptual loss function according to the first data set and the second data set.

In this embodiment, the calculation method of the mean square error loss function is as follows:

Among them, L _mse is the mean square error loss function, _xi is the element of the first data set, y _i is the element of the second data set, and m is the total number of pixels of the reconstructed image.

In this embodiment, the calculation method of the perceptual loss function is as follows:

Among them, L _vgg is the perceptual loss function, VGG(x) _i is the feature map of the reconstructed image after passing through the VGG network (Visual GeometryGroup Network, Visual Geometry Group Network), VGG(G(Y)) _i is the target image after passing through the VGG network n is the total number of pixels in the feature map of the reconstructed image after passing through the VGG network, and d is the number of feature maps of the reconstructed image after passing through the VGG network.

In this embodiment, whether the reconstructed image meets the standard can be judged according to the mean square error loss function and the perceptual loss function.

S260. Perform optimization calculation on the mean square error loss function and the perceptual loss function.

In this embodiment, an Adam (Adaptive Moment Estimation, adaptive moment estimation) algorithm may be used to optimize and calculate the mean square error loss function and the perceptual loss function.

In this embodiment, it may be judged whether the reconstructed image meets the standard according to the optimized calculated mean square error loss function and perceptual loss function.

S271. Acquire a third data set according to the input image data.

S272. Extract corresponding data from the third data set, the first data set, and the second data set as input data, output data, and network labels, respectively;

S280. Train the first mapping function according to the input data, output data, network label, generated adversarial loss data, mean square error loss function, and perceptual loss function, so as to obtain a second mapping function.

In this embodiment, the first mapping function is trained by generating adversarial loss data, a mean square error loss function, and a perceptual loss function, which can effectively solve the problems of being too smooth and losing detailed information in the reconstructed image, and can preserve the image. Details, making the structure in the image clearer.

Referring to Fig. 6, the image processing device of the present invention includes a receiver 810, a processor 820 and a display 830, the receiver 810 is used to obtain the original image; the processor 820 is connected to the receiver 810, and is used to obtain input image data according to the original image; The generator network performs a first mapping operation on the input image data according to a first mapping function to obtain output image data; wherein, the input image data is sinusoidal data; the display 830 is connected to the processor 820 for receiving the output image data, and according to The output image data forms a reconstructed image.

For the method for the processor 820 to process the input image data, refer to the above-mentioned embodiment of the image reconstruction method, which will not be repeated here.

In the embodiment of the present invention, the first mapping operation is performed on the input image data of the original image according to the first mapping function in the generator network to obtain the output image data, thereby directly realizing image reconstruction, and realizing the clarity of images of low-dose radioactive tracers Presentation is beneficial to doctors' clinical diagnosis; due to the small amount of data collected by the mapping method, the image reconstruction speed is faster, which is conducive to improving work efficiency, and the required scanning time is also shorter, which can avoid the appearance of artifacts. Thereby improving image quality.

Referring to FIG. 7 , an embodiment of a device 90 with a storage function in the present invention stores program data 910 , and the program data 910 can be executed to implement an image reconstruction method.

For the image reconstruction method, refer to the above-mentioned embodiment of the image reconstruction method, which will not be repeated here.

In the embodiment of the present invention, the first mapping operation is performed on the input image data of the original image according to the first mapping function in the generator network to obtain the output image data, thereby directly realizing image reconstruction, and realizing the clarity of images of low-dose radioactive tracers Presentation is beneficial to doctors' clinical diagnosis; due to the small amount of data collected by the mapping method, the image reconstruction speed is faster, which is conducive to improving work efficiency, and the required scanning time is also shorter, which can avoid the appearance of artifacts. Thereby improving image quality.

The above is only the embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process conversion made by using the description of the present invention and the contents of the accompanying drawings, or directly or indirectly used in other related technologies fields, are all included in the scope of patent protection of the present invention in the same way.

Claims (10)

1. An image reconstruction method, characterized in that, comprising: Acquiring an original image; wherein, the original image is an image directly collected by imaging after injecting a low dose of tracer; Obtaining input image data according to the original image, the input image data is sinusoidal data; Performing a first mapping operation on the input image data according to a first mapping function in the generator network to obtain output image data; wherein, the first mapping operation includes encoding, conversion, and decoding; forming a reconstructed image from said output image data; Wherein, the output image data is processed by a discriminator network to obtain generated anti-loss data; the first data set is obtained according to the output image data, and the second data set is obtained according to the target image data; the target image data is Preset image data corresponding to a standard image of an original image; obtaining a mean square error loss function and a perceptual loss function according to the first data set and the second data set; The mean square error loss function is: in, is the mean square error loss function, /> is the element of the first data set, /> is the element of the second data set, /> is the number of total pixels of the reconstructed image; The perceptual loss function is: in, is the perceptual loss function, /> To reconstruct the feature map of the image after passing through the VGG network, /> is the feature map of the target image after passing through the VGG network, /> is the total number of pixels in the feature map of the reconstructed image after the VGG network, /> is the number of feature maps after the reconstructed image passes through the VGG network; Judging whether the reconstructed image meets the standard according to the mean square error loss function and the perceptual loss function; and obtaining a third data set according to the input image data; from the third data set, the first data set and the Extract corresponding data from the second data set as input data, output data, and network label respectively; according to the input data, the output data, the network label, generate the adversarial loss data, the mean square error loss function and the The perceptual loss function is used to train the first mapping function to obtain a second mapping function. 2. The image reconstruction method according to claim 1, wherein the first mapping operation comprises encoding, conversion and decoding, and the encoding method comprises: sequentially performing the first convolution operation on the input image data to obtain the first characteristic image data; The number of convolutional layers in the first convolution operation is 14, wherein the step size of the 10-layer convolutional layer is 1, and the step size of the 4-layer convolutional layer is 2. 3. The image reconstruction method according to claim 2, wherein the method for converting comprises: performing a second convolution operation on the first characteristic image data to obtain second characteristic image data; The number of convolutional layers in the second convolution operation is 11, wherein the feature number of the 6-layer convolutional layer is 512, and the feature number of the 5-layer convolutional layer is 1024. 4. The image reconstruction method according to claim 3, wherein the decoding method comprises: performing an upsampling operation and a third convolution operation on the second feature image data in sequence to obtain output image data; Wherein, the number of convolution kernels of the last convolution layer in the third convolution operation is 1. 5. The image reconstruction method according to claim 4, wherein, in the first convolution operation, the second convolution operation and the third convolution operation, each layer of convolution After the operation of the layer, batch normalization operation and activation operation are performed. 6. The image reconstruction method according to claim 1, wherein said obtaining said output image data further comprises: Identifying whether the reconstructed image meets a standard based on the generated adversarial loss data. 7. The image reconstruction method according to claim 1, wherein the method for processing the output image data through the discriminator network comprises: performing a fourth convolution operation on the output image data to obtain a third feature image; performing fully-connected layer processing on the third feature image to obtain the generated adversarial loss data; Wherein, the number of layers of the convolutional layer in the fourth convolution operation is 8, the step size of the even-numbered layer convolutional layer is 2, the step size of the odd-numbered layer convolutional layer is 1, and the number of layers of the fully connected layer for 2. 8. The image reconstruction method according to claim 7, characterized in that, in the fourth convolution operation and fully connected layer processing, after the operation of each layer of convolution layer and the first layer of fully connected layer processing Afterwards, the activation operation is performed. 9. The image reconstruction method according to claim 7, characterized in that, after obtaining the mean square error loss function and the perceptual loss function, further comprising: performing optimization calculations on the mean square error loss function and the perceptual loss function; According to the optimized calculated mean square error loss function and perceptual loss function, it is judged whether the reconstructed image meets the standard. 10. An image processing device for realizing the image reconstruction method according to any one of claims 1-9, characterized in that it comprises: Receiver, used to obtain the original image; A processor, connected to the receiver, for obtaining input image data according to the original image; performing a first mapping operation on the input image data according to a first mapping function in the generator network to obtain output image data; wherein , the input image data is sinusoidal data; a display, connected to the processor, for receiving the output image data, and forming a reconstructed image according to the output image data.