CN112767407B - CT image kidney tumor segmentation method based on cascade gating 3DUnet model - Google Patents

Abstract

The invention discloses a CT image kidney tumor segmentation method based on a cascade gating 3DUnet model, which comprises the following steps: collecting image sequences containing kidneys in abdominal CT scanning, labeling the kidneys and tumors in each image sequence, generating corresponding labeling masks, and constructing a Dataset Dataset; performing P1 pretreatment operation on the Dataset, and constructing a U-shaped deep network model M1 for kidney (including tumor) segmentation; cutting the image sequence in the Dataset and the corresponding labeling mask, taking out the voxel part with only kidney (or tumor), performing P2 preprocessing operation, and constructing a depth network segmentation model M2 for tumor segmentation based on the gating convolution layer; training the models M1 and M2 respectively; kidney tumor areas were segmented by a cascade of models M1 and M2. According to the invention, a two-stage segmentation model of network cascade is combined with a gating convolution layer to construct a depth network model for tumor segmentation, so that robustness can be maintained to the shape change of cancerous kidneys, and tumors with different sizes can be effectively segmented.

Description

A method for segmenting kidney tumors in CT images based on cascaded gated 3DUnet model

Technical field

The invention relates to the fields of computer, software, artificial intelligence technology and the like, in particular to a method for segmenting kidney tumors in CT images based on a cascaded gated 3DUnet model.

Background technique

Medical image imaging technology has made great progress in the rapid development of science and technology. This technology integrates many technologies such as modern medicine, physics, electronic information and computer technology, and has become an indispensable and important means in the medical field. Computed Tomography (CT for short) is a widely used medical imaging method, which can accurately and comprehensively describe the detailed characteristics of human organs, tissues and lesions, enabling professional doctors to perform non-invasive detection of lesion parts more directly and clearly. In order to formulate an effective treatment plan in time, so as to improve the diagnosis efficiency and cure rate of the disease. The kidney is a key component of the urinary system, responsible for regulating acid-base balance and metabolism and other important functions. Due to the accelerated pace of people's life and increased work pressure, various kidney diseases continue to increase. Kidney tumor is a common kidney disease, namely "kidney cancer", also known as renal cell carcinoma, which is a malignant tumor originating from the renal epithelium. Its pathological types are complex and its clinical manifestations are different. Compared with other medical imaging methods, CT images can better present and distinguish the lesion characteristics of kidney cancer, and become an important basis for doctors to make a preliminary diagnosis and follow-up of kidney diseases.

In clinical treatment, accurate segmentation of kidney and diseased areas is very important for disease diagnosis, functional evaluation and treatment decision-making. The early segmentation work was manually drawn by experienced doctors. This segmentation method is highly subjective, inefficient, and the segmentation results cannot be reproduced. It cannot well meet the clinical requirements and cannot meet the current quantitative diagnosis needs. With the development of modern science and technology, it is possible to use computer technology to achieve medical image segmentation, and researchers have begun to explore automatic segmentation methods. However, there are some difficulties in accurately and reliably segmenting kidneys in CT images, such as: low contrast in CT images, blurred boundaries between kidneys and adjacent organs and tissues, individual shape differences, and water and air inside the kidneys are likely to cause noise and voids etc. For patients with renal cancer, due to the various sizes of tumors and the similar pixel values to normal renal tissue, the borders are difficult to distinguish, making segmentation of renal tumors in CT images posed many challenges. Therefore, it is of practical research significance to develop a fully automatic segmentation algorithm for renal tumors in CT images.

At present, deep learning has achieved remarkable results in various fields. Its superiority benefits from the ability of convolutional neural networks to automatically extract features, making deep learning models universally applicable to different tasks. The continuous deepening of the performance is becoming more and more obvious, making it rapidly develop into the mainstream technology in the era of big data. In recent years, deep learning models for medical image segmentation have gradually emerged in current research. Although the convolutional neural network can automatically extract effective features, there are still problems such as insufficient robustness to renal morphological changes and inaccurate tumor segmentation for the segmentation of kidney tumors in CT images. Establishing an accurate segmentation model of imaged renal tumors can provide quantitative diagnostic evidence and assist doctors in decision-making, which has important clinical significance and good application prospects.

Purpose of the invention:

The technical problem to be solved by the present invention lies in the segmentation of kidneys and their tumors in CT images. A kidney tumor segmentation algorithm based on the cascaded gated three-dimensional full convolution network 3DUnet model is proposed to achieve accurate segmentation of kidney tumor regions in CT image sequences.

Technical solutions:

In order to solve the above-mentioned technical problems, the present invention provides a kidney tumor segmentation algorithm based on the cascaded gated 3DUnet model, and its technical scheme is as follows:

A method for segmenting kidney tumors in CT images based on a cascaded gated 3DUnet model, comprising the following specific steps:

S101, collect images containing kidneys in the abdominal CT scan, take out slice images containing kidneys or tumors to form an image sequence, use labeling software to label kidneys and tumors in each slice image, generate corresponding labeling masks, and construct a data set Dataset;

S102, perform P1 preprocessing on Dataset, divide the Dataset after P1 preprocessing into M1 training set and M1 test set, train and test the constructed 3DUnet, and obtain the deep network model M1 of kidney segmentation;

S103, perform P2 preprocessing on Dataset, divide Dataset after P2 preprocessing into M2 training set and M2 test set, train and test the constructed three-dimensional gated residual full convolutional network, and obtain a deep network segmentation model for tumor segmentation M2;

S104, after performing P1 preprocessing on the image sequence to be segmented, use M1 to segment the kidney region, and stitch the segmentation results; crop the stitched segmentation results, take out only the voxels of the kidney or tumor, and perform P2 preprocessing , use M2 to segment out the tumor area.

Further, in S101, the ITK-SNAP medical image annotation software is used to annotate the kidneys and tumors in the slice images to generate corresponding annotation masks, and a Dataset is formed from the slice images and their corresponding annotation masks.

Further, performing P1 preprocessing on the Dataset in S102 specifically includes: interpolating all slice images and their corresponding label masks in the Dataset into the same resolution voxel spacing; The code is randomly cropped to obtain small voxel blocks, and the voxel small blocks are normalized; the resolution after interpolation is lower than that before interpolation.

Further, 3DUnet is constructed according to the size of the voxel block, which is used to obtain the segmentation mask of the kidney region.

Further, in S103, the Dataset is subjected to P2 preprocessing, which specifically includes: interpolating all the slice images in the Dataset and their corresponding labeling masks into the same resolution voxel spacing, and using edge detection to obtain the tumor boundary on the interpolated labeling masks , each slice image after interpolation and its corresponding annotation mask and tumor boundary are randomly cropped to obtain small voxel blocks, and the voxel small blocks are normalized; the resolution after interpolation is higher than that before interpolation .

Further, a three-dimensional gated residual full convolutional network is constructed according to the size of the voxel block. The three-dimensional gated residual full convolutional network includes a backbone network and a tumor shape branch network; the backbone network is 3DUnet, and the encoder and decoder Jump connections are made between feature maps corresponding to the same resolution; the tumor shape branch network includes a three-level cascade of gated convolutional layers, 3x3x3 convolutional layers, trilinear interpolation, and 1x1x1 convolutional layers. The output of the convolutional layer passes through the 1x1x1 convolutional layer, 3x3x3 convolutional layer, and trilinear interpolation as the input of the first-level gated convolutional layer, and the output of the second-level deconvolutional layer of the decoder passes through the 1x1x1 convolutional layer. Finally, it is used as the input of the first-level gated convolutional layer. The output of the first-level gated convolutional layer is used as the input of the second-level gated convolutional layer after passing through the 3x3x3 convolutional layer and trilinear interpolation. The third-level gated convolutional layer of the decoder The output of the first-level deconvolution layer passes through the 1x1x1 convolutional layer and is used as the input of the second-level gated convolutional layer, and the output of the second-level gated convolutional layer is used as the third-level gate after passing through the 3x3x3 convolutional layer and trilinear interpolation The input of the controlled convolutional layer, the output of the fourth-level deconvolution layer of the decoder passes through the 1x1x1 convolutional layer and is used as the input of the third-level gated convolutional layer, and the output of the third-level gated convolutional layer passes through the 1x1x1 convolutional layer After the product layer, together with the output of the decoder, it is used as the input of a fully connected layer. The output of the fully connected layer passes through a 1x1x1 convolutional layer and a Softmax classifier in turn to output a probability map; the output of the third-level gated convolutional layer passes through a 1x1x1 After the convolutional layer, it is used as the input of the sigmoid function, and the sigmoid function outputs the final prediction mask.

Compared with the prior art, the present invention has the following advantages:

A two-stage segmentation model based on network cascading is proposed, and a random cropping strategy is adopted to alleviate the impact of category imbalance and small objects to a certain extent. Aiming at the problem that the tumor boundary is difficult to distinguish, 3DUnet is used as the backbone network, and a tumor shape branch network is constructed based on the gated convolutional layer, which can predict the tumor boundary, thereby improving the tumor segmentation performance.

Description of drawings

Fig. 1 is the control flowchart of the inventive method;

Fig. 2 is the predictive operation schematic diagram of the inventive method;

Fig. 3 is M1 model structural diagram among the present invention;

Fig. 4 is a structural diagram of the M2 model in the present invention.

Detailed ways

In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention clearer and clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

In this embodiment, a renal tumor segmentation algorithm based on the cascaded gated 3DUnet model provided by the present invention is used to segment the renal tumor in the CT image, as shown in Figures 1 and 2, including the following specific steps:

S101, collect images containing kidneys in the abdominal CT scan, take out slice images containing kidneys or tumors to form an image sequence, use labeling software to label kidneys and tumors in each slice image, generate corresponding labeling masks, and construct data Set Dataset.

The data in this example comes from the KiTS19 competition data set. The original data are plain scan CT sequence data. The image and the corresponding manual annotation mask are provided in anonymous NIFTI format, and its shape is (number of slices, height, width). where the slice number corresponds to the axial view and increases with slice index from top to bottom in all patients supine during image acquisition. When a patient has multiple scan images, the case with the smallest slice thickness is selected, and the slice thickness of the data set is 1mm to 5mm.

S102, perform P1 preprocessing operation on the Dataset dataset, divide it into a training set and a test set, and construct a deep network model M1 for kidney segmentation based on 3DUnet (as shown in FIG. 3 ).

Perform P1 preprocessing operation: use trilinear interpolation to resample the sequence images of all samples to a voxel spacing of 3.22×1.62×1.62mm (lowest resolution) as the standard spacing, and use the nearest Neighboring interpolation samples to the same voxel spacing, and then randomly crops (50% overlap rate, that is, the step size is (40,80,80)) to take out the size of each sample with a size of 80×160×160 (middle value), the CT value of each sample sequence image is limited to the range of [-79,304] (5% to 95% of the range of all pixels), in order to eliminate the abnormal intensity caused by some substances Value, due to the nature of weight initialization in the network, use z-score normalization: subtract the mean from each pixel value and divide by the variance, so that the pixels of the image are in a range that is easier to be processed by CNN. Design 3DUnet according to the size of voxel blocks, including encoder and decoder. The parameter settings of the encoder are shown in Table 1, and the decoder adopts a symmetrical structure.

Table 1 Encoder parameter settings

type size/step output size Convolution layer (x2) 3x3x3/1 80x160x160x30 max pooling layer 1x2x2/1(2,2) 80x80x80x30 Convolution layer (x2) 3x3x3/1 80x80x80x60 max pooling layer 2x2x2/2 40x40x40x60 Convolution layer (x2) 3x3x3/1 40x40x40x120 max pooling layer 2x2x2/2 20x20x20x120 Convolution layer (x2) 3x3x3/1 20x20x20x240 max pooling layer 2x2x2/2 10x10x10x240 Convolution layer (x2) 3x3x3/1 10x10x10x320 max pooling layer 2x2x2/2 5x5x5x320

S103, crop the image sequence in the Dataset data set and the corresponding label mask, take out the voxel part of only the kidney (or tumor), perform P2 preprocessing operation, divide the data into training set and test set, based on three-dimensional gating The residual fully convolutional network constructs a deep network segmentation model M2 for tumor segmentation.

Perform P2 preprocessing operation: resample all samples, manual labeling masks and tumor border masks in the same way to a voxel spacing of 3×0.78×0.78mm (median) as a standard, and take out according to the manually labeled information For the VOI area containing the kidney (or tumor), reset the value of pixels other than the kidney (or tumor) to 0, and then randomly crop (50% overlap rate, step size (24,64,64)) The method takes out a small voxel block with a size of 48×128×128 (median value) in each VOI area, and limits the CT value of the sequence image of each sample to [-79,304] (5% of all pixel ranges to 95%) to remove abnormal intensity values caused by certain substances, due to the nature of weight initialization in the network, use z-score normalization: subtract the mean from each pixel value and divide by the variance. A U-shaped fully convolutional neural network was designed with small voxel blocks as network input, and a tumor shape branch network was formed by cascading gated convolutional layers, 1x1x1 convolutional layers, trilinear interpolation, and 1x1x1 convolutional layers.

As shown in Figure 4, the 3D gated residual full convolutional network includes a backbone network and a tumor shape branch network; the backbone network is 3DUnet, in which the encoder and decoder correspond to feature maps of the same resolution for skip connections, and then After 1x1x1 convolution, the number of channels is reduced, and finally the probability map is output through a Softmax classifier. The tumor shape branch network includes a three-level cascade of gated convolutional layers, 3x3x3 convolutional layers, trilinear interpolation, and 1x1x1 convolutional layers. The output of the first-level deconvolutional layer of the decoder passes through 1x1x1 convolutional layers, 3x3x3 convolutional layers The product layer and trilinear interpolation are used as the input of the first-level gated convolutional layer, and the output of the second-level deconvolution layer of the decoder is used as the input of the first-level gated convolutional layer after being processed by the 1x1x1 convolutional layer. The output of the first-level gated convolutional layer passes through the 3x3x3 convolutional layer and trilinear interpolation as the input of the second-level gated convolutional layer, and the output of the third-level deconvolutional layer of the decoder passes through the 1x1x1 convolutional layer. As the input of the second-level gated convolutional layer, the output of the second-level gated convolutional layer passes through the 3x3x3 convolutional layer and trilinear interpolation as the input of the third-level gated convolutional layer, and the fourth-level of the decoder The output of the deconvolution layer passes through the 1x1x1 convolution layer as the input of the third-level gated convolution layer, and the output of the third-level gated convolution layer passes through the 1x1x1 convolution layer and is used as a full connection with the output of the decoder. The input of the layer, the output of the fully connected layer passes through the 1x1x1 convolutional layer and a Softmax classifier in turn to output the probability map; the output of the third-level gated convolutional layer passes through the 1x1x1 convolutional layer as the input of the sigmoid function, and the sigmoid function outputs Final prediction mask. The encoder structure of the backbone network still adopts the settings in Table 1. Among them, the role of the 3x3x3 convolutional layer is to extract boundary features, the role of trilinear interpolation is to adjust the size of the feature map, and the role of 1x1x1 convolution is to reduce the dimensionality of the number of channels.

The three-dimensional gated residual full convolutional network sequentially passes through the downsampling and upsampling parts of the encoder and decoder of the backbone network for the input CT image to extract feature information, and then extracts the shape of the tumor area through the tumor shape branch network. The output boundary map of the shape branch is expressed as s∈R ^H×W×C , and the output feature map of the backbone network is z∈R ^H×W×C . The feature map of the shape branch network and the feature map output by the backbone network are connected in series, Finally, the final prediction mask is output by 1x1x1 convolution and soft-max. This patent can generate more accurate segmentation results by fusing the semantic features of the backbone network and the boundary features of the shape branch network.

S104. Select an appropriate optimization learning method, set relevant hyperparameters, and train the models M1 and M2 respectively.

The data set constructed by using S102 for the model M1 is trained, and the data set constructed for the model M2 by S103 is trained. The Adam optimizer is used for loss optimization, and the best average index on the verification set is taken as the optimal result of the model at the end of each training. The following hyperparameter settings are adopted: batchsize is set to 2, and 300 batches are iterated as one epoch, and a total of 150 epochs are iterated; the initial learning rate is set to 10 ^-3 , and it is automatically reduced by 0.1 times when training the 80th and 120th epochs; The momentum is set to 0.95, and the weight decay coefficient is constant to 10 ^-4 .

S105 , selecting a CT image sequence from the test set, performing a preprocessing operation, and segmenting a kidney tumor region by using the models M1 and M2 .

Randomly select a plain CT scan image of a patient with kidney cancer in the test set, select a CT slice image containing the kidney or tumor, and use the trained M1 model to predict after the P1 preprocessing operation, and then use the randomly cropped inverse Sequential splicing is performed, and small voxel blocks containing only kidneys or tumors are cut out according to the spliced results. After the P2 preprocessing operation, the trained M2 model is used for prediction, and then the segmentation results are spliced in the reverse order of random cropping. form the final segmentation result.

Table 2 Evaluation index values of different deep network models (%)

Table 2 shows the evaluation indicators of different deep network models, in which the two-dimensional fully convolutional network is formed by superimposing the results of segmenting each slice image, and 3DUnet and V-net adopt the structure of the original model. It can be seen that the patented algorithm has higher segmentation accuracy for kidney and tumor than other comparison algorithms, especially for tumor targets. Due to the addition of tumor-gated shape branch networks, the quantitative indicators of tumor segmentation have been significantly improved.

The above is only a specific implementation mode in the present invention, but the scope of protection of the present invention is not limited thereto. Anyone familiar with the technology can understand the conceivable transformation or replacement within the technical scope disclosed in the present invention. All should be covered within the scope of the present invention, therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims (4)

1. a kind of CT image renal tumor segmentation method based on cascade gating 3DUnet model, is characterized in that, comprises following concrete steps: S101, collect images containing kidneys in the abdominal CT scan, take out slice images containing kidneys or tumors to form an image sequence, use labeling software to label kidneys and tumors in each slice image, generate corresponding labeling masks, and construct a data set Dataset; S102, perform P1 preprocessing on the Dataset, divide the Dataset after P1 preprocessing into M1 training set and M1 test set, train and test the constructed three-dimensional full convolution network 3DUnet, and obtain the deep network model M1 of kidney segmentation; S103, perform P2 preprocessing on Dataset, divide Dataset after P2 preprocessing into M2 training set and M2 test set, train and test the constructed three-dimensional gated residual full convolutional network, and obtain a deep network segmentation model for tumor segmentation M2; S104, after performing P1 preprocessing on the image sequence to be segmented, use M1 to segment the kidney region, and stitch the segmentation results; crop the stitched segmentation results, take out only the voxels of the kidney or tumor, and perform P2 preprocessing , use M2 to segment the tumor area; Perform P1 preprocessing on the Dataset in S102, which specifically includes: interpolating all slice images and their corresponding label masks in the Dataset into the same resolution voxel spacing; Random cropping, obtaining small voxel blocks, and normalizing the small voxel blocks; the resolution after interpolation is lower than that before interpolation; In S103, P2 preprocessing is performed on the Dataset, which specifically includes: interpolating all slice images in the Dataset and their corresponding labeling masks into the same resolution voxel spacing, using edge detection to obtain the tumor boundary for the interpolated labeling masks, and After interpolation, each slice image and its corresponding annotation mask and tumor boundary are randomly cropped to obtain small voxel blocks, and the voxel small blocks are normalized; the resolution after interpolation is higher than that before interpolation; The three-dimensional gated residual fully convolutional network includes a backbone network and a tumor shape branch network; the backbone network is 3DUnet, and the tumor shape branch network includes a three-level cascade of gated convolutional layers. 2. A kind of CT image kidney tumor segmentation method based on the cascaded gated 3DUnet model as claimed in claim 1, it is characterized in that, utilize ITK-SNAP medical image labeling software to label the kidney and tumor in the slice image in S101 , to generate the corresponding label mask, and the Dataset is composed of sliced images and their corresponding label masks. 3. A kind of CT image renal tumor segmentation method based on cascading gated 3DUnet model as claimed in claim 1, is characterized in that, constructs 3DUnet according to the size of voxel block, is used to obtain the segmentation mask of kidney region. 4. A kind of CT image renal tumor segmentation method based on the cascaded gated 3DUnet model as claimed in claim 1, it is characterized in that, according to the size of voxel small piece, construct three-dimensional gated residual full convolutional network, backbone network The encoder and decoder in the corresponding feature maps of the same resolution perform skip connections; the tumor shape branch network also includes 3x3x3 convolutional layers, trilinear interpolation and 1x1x1 convolutional layers, and the first-level deconvolution layer of the decoder The output of the 1x1x1 convolutional layer, 3x3x3 convolutional layer, and trilinear interpolation are used as the input of the first-level gated convolutional layer, and the output of the second-level deconvolution layer of the decoder is processed by the 1x1x1 convolutional layer. The input of the first-level gated convolutional layer, the output of the first-level gated convolutional layer passes through the 3x3x3 convolutional layer and trilinear interpolation as the input of the second-level gated convolutional layer, and the third-level deconvolution of the decoder The output of the product layer passes through the 1x1x1 convolutional layer and is used as the input of the second-level gated convolutional layer, and the output of the second-level gated convolutional layer is used as the third-level gated convolution after passing through the 3x3x3 convolutional layer and trilinear interpolation Layer input, the output of the fourth-level deconvolution layer of the decoder passes through the 1x1x1 convolutional layer as the input of the third-level gated convolutional layer, and the output of the third-level gated convolutional layer passes through the 1x1x1 convolutional layer Together with the output of the decoder, it is used as the input of a fully connected layer. The output of the fully connected layer passes through a 1x1x1 convolutional layer and a Softmax classifier in turn to output a probability map; the output of the third-level gated convolutional layer passes through a 1x1x1 convolutional layer. Finally, as the input of the sigmoid function, the sigmoid function outputs the final prediction mask.

Images