CN112633416A - Brain CT image classification method fusing multi-scale superpixels - Google Patents

Abstract

A brain CT image classification method fused with multi-scale superpixels belongs to the field of medical image research. The method has the following characteristics: 1) The fusion of multi-scale superpixels and brain CT images removes redundant image information and reduces the grayscale similarity between lesions and surrounding brain tissue pixels. 2) A region- and boundary-based multi-scale superpixel encoder is designed to effectively extract the low-level information of lesions contained in multi-scale superpixels. 3) A multi-scale superpixel feature fusion model is designed, which comprehensively utilizes the high-level features extracted by the residual neural network and the low-level features of multi-scale superpixels to realize the classification of brain CT. 4) Compared with the traditional deep learning algorithm, the method of the present invention can effectively utilize the lesion information contained in the multi-scale superpixels, so as to classify the diseases existing in the brain CT images more accurately, and the method is reasonable and reliable, and can be used for The classification of brain CT images provides powerful help.

Description

A brain CT image classification method fused with multi-scale superpixels

technical field

The invention belongs to the field of medical image research, in particular, the invention relates to a brain CT image classification method fused with multi-scale superpixels.

Background technique

The diagnosis of brain injury in the clinical emergency department is extremely urgent, and even a short delay can lead to a worsening of the patient's condition. Computed Tomography (CT) is one of the most commonly used diagnostic tools. It has the characteristics of fast imaging, low cost, wide application range, and high detection rate of lesions. Although brain CT can detect critical and time-sensitive abnormalities such as intracranial hemorrhage, elevated intracranial pressure, and skull fractures, traditional disease classification methods often require radiologists to visually measure the size of the hemorrhage and estimate midline offset. This process is relatively time-consuming. In recent years, with the progress and development of medical imaging technology, the number of brain CT images has increased geometrically, but the growth rate of the number of radiologists is relatively slow, and the cost of training a qualified radiologist is high and the cycle is long. , resulting in increasing work tasks of in-service radiologists, which indirectly leads to social problems such as difficulty in seeing a doctor. Therefore, the automatic classification method of brain CT can assist radiologists in their work, improve the diagnostic efficiency, and reduce the rate of misdiagnosis and missed diagnosis, which is of great practical significance.

In recent years, the great success of deep learning (DL) in the field of computer vision has also promoted the rapid development of medical image analysis technology. Convolutional Neural Networks (CNN) is a traditional deep learning model. It can capture local area information of images and extract high-level semantic features, and is widely used in image feature extraction and classification tasks. Nowadays, CNN has been widely used in the recognition and processing of medical images, and after continuous iterative optimization, many classifiers based on CNN have been formed.

However, the existing work does not consider the differences between brain CT images and natural optical images when using traditional CNN to extract brain CT image features: First, brain CT images have low spatial resolution and low contrast; lack of brightness, color, texture, etc. Easy-to-recognize natural visual features; the demarcation line between regions is not clear, and the texture difference is not large; the image will be significantly different due to individual differences in patients and different imaging positions; the image also has artifacts caused by displacement and errors caused by volume effects , Many unstable factors that cause noise from equipment. And considering that medical images are mostly private data of hospitals, privacy protection regulations will hinder the sharing of medical images, and the number of datasets will directly affect the effect of deep learning. Studies have shown that unsupervised generated superpixels are composed of a series of pixel points with similar features, which can retain the local details of the original image and highlight local features. The number of image pixels, using superpixels instead of pixels as the primitives of image processing can greatly reduce the computational complexity of subsequent image processing and improve the efficiency of image processing algorithms.

SUMMARY OF THE INVENTION

Aiming at the problem that the above-mentioned existing methods ignore the visual characteristics of brain CT images, the present invention proposes a brain CT image classification method fused with multi-scale superpixel fusion (MSF). The method of the invention can optimize the image through multi-scale superpixels, and extract the information of the lesion area unsupervised, thereby enhancing the expressivity of the features generated by the residual neural network and improving the accuracy of the classification task.

In order to achieve the above object, the technical solution adopted in the present invention is a brain CT image classification method fused with multi-scale superpixels. The flow of the present invention is shown in FIG. 1 and includes the following steps. 1) First, construct a dataset and perform preprocessing to obtain multi-scale superpixels; secondly, perform data enhancement through multi-scale superpixel image fusion to obtain an optimized fused image; then use multi-scale superpixel features based on region and boundary information coding algorithm to obtain multi-scale superpixel low-level features; finally, the multi-scale superpixel feature fusion classification model is used to classify brain CT images;

Step (1) Acquire data and preprocess:

与脑CT分类标签向量Y＝[Y₁，Y₂，…Y_T]，Y_i∈{0，1}，其中N表示图像像素尺寸，T表示采集的疾病类别个数。Step (1.1) Data: collect brain CT images to construct a data set, each patient data contains the RGB matrix generated by its brain CT images

and brain CT classification label vector _Y =[ _{Y 1} , Y ₂ , .

Step (1.2) divides all patient data into training set, validation set and test set. Among them, the training set is used to learn the parameters of the neural network; the validation set is used to determine the network structure and hyperparameters; the test set is used to verify the classification effect of the neural network.

的计算过程如下：Step (1.3) Data preprocessing: Based on the super-hierarchy segmentation algorithm (Super Hierarchy, SH), for the given brain CT image I and the set superpixel segmentation scale {scale ¹ , scale ² , ... scale ^s }, where S represents the number of set superpixels, and generates a superpixel map at the sth segmentation scale

The calculation process is as follows:

P ^s =SH(I, scale ^s )

表示包含S个不同尺度超像素图的多尺度超像素。Among them, s∈{1, 2…S}, scale ^s is the s-th segmentation scale, and each segmentation scale is calculated to get

Represents a multi-scale superpixel containing S superpixel maps of different scales.

融合图像I′计算过程如下：Step (2) Multi-scale superpixel image fusion model: for a given brain CT image I and its multi-scale superpixels

The calculation process of the fusion image I' is as follows:

表示训练的权重，P^s表不

中第s个元素，W实现了各个尺度比重的自适应分配。Among them, ⊙ represents the dot product, f( ) is the SoftMax function,

Indicates the weight of training, P ^s represents

The s-th element in W realizes the adaptive allocation of the proportions of each scale.

Step (3) Multi-scale superpixel feature encoding:

中分割尺度为scale^s的超像素图

生成其像素值集合

对

中每个像素生成映射矩阵集合

其中，第k个映射矩阵

中第i，j个元素

的计算方式如下：Step (3.1) Multi-scale superpixels for brain CT image I

A superpixel map with a median segmentation scale of scale ^s

generate a collection of its pixel values

right

Generates a set of mapping matrices for each pixel in

Among them, the kth mapping matrix

The i,jth element in

is calculated as follows:

表示P^s中第k个超像素的像素值，M^s，k表示像素值为

的超像素区域映射。where k∈{1, 2,…scale ^s }, where

represents the pixel value of the kth superpixel in P ^s , M ^{s, k} represents the pixel value

superpixel region map.

计算过程如下：Step (3.2) encodes each mapping matrix in the set M ^s based on the area and boundary information, and obtains the encoding result of the superpixel map P ^s

The calculation process is as follows:

Among them, N ² represents the number of pixels contained in the superpixel map, and _sk represents the number of pixels contained in the k-th superpixel region.

中每个超像素图重复步骤(3.1)和步骤(3.2)，依次生成编码结果b¹，b²，…b^S，将其通过矩阵拼接成

得到多尺度超像素

的多尺度超像素特征编码B。Step (3.3) to

Steps (3.1) and (3.2) are repeated for each superpixel image in , and the encoding results b ¹ , b ² , . . . b ^S are generated in turn, which are spliced into

get multi-scale superpixels

The multi-scale superpixel feature encoding B.

Step (4) Multi-scale superpixel feature fusion classification model:

Step (4.1) Build a residual neural network (Residual Network, ResNet) as the backbone network, use the brain CT fusion image I' extracted in step (2) to input, and select the last residual structure (Basic Block) of the four Layers in ResNet. ) feature activation outputs l ₁ , l ₂ , l ₃ , l ₄ as high-level features.

将f₀自下而上依次和l₁、l₂、l₃、l₄进行多层融合，生成融合特征f₁、f₂、f₃、f₄。其中f_i(i∈{1，2，3，4})的计算如下：先采用池化操作将f_(i-1)转化为和l_i相同尺寸的特征矩阵，之后通过256个1×1卷积核构成的卷积层将l_i的通道数转换为256，之后通过矩阵相加进行特征融合，得到第i层融合特征f_i。Step (4.2) performs dimensionality reduction processing on the low-level feature B extracted in (3.3), and generates features through a convolutional layer composed of 256 3×3 convolution kernels

Perform multi-layer fusion of f ₀ with l ₁ , l ₂ , l ₃ , and l ₄ from bottom to top to generate fusion features f ₁ , f ₂ , f ₃ , and f ₄ . The calculation of f _i (i∈{1, 2, 3, 4}) is as follows: first, the pooling operation is used to convert f _(i-1) into a feature matrix of the same size as l _i , and then 256 1×1 The convolution layer formed by the convolution kernel converts the number of channels of li to 256, and then performs feature fusion through matrix addition to obtain the _i -th layer fusion feature f _i .

Step (4.3) Input the obtained fusion feature f ₄ into the convolution layer, pooling layer and fully connected layer composed of 512 3×3 convolution kernels to obtain a classification vector x of the same length as the number of labels T, and pass it through the Sigmoid linear Regression generates a predicted value vector y=[y ₁ , y ₂ ,...y _T ], where y _i ∈ [0, 1], the probability that the i-th element in x generates the corresponding label as a positive example y _i is expressed as: y _i =Sigmoid(x _i ), the classification result is determined according to the set classification threshold t, when y _i is greater than the set threshold t, the model determines that the brain CT has a corresponding label disease, and if y _i is less than the set threshold t, it is normal . t=0.5.

Step (4.4) The input of the brain CT image classification method fused with multi-scale superpixels of the present invention is the patient's brain CT image I and the brain disease classification label Y, and then the probability y of the subject belonging to each category is obtained. If a dataset of M patients is given D={(I ¹ , Y ¹ ), (I ² , Y ² ), ..., (I ^M , Y ^M )}, for a given brain CT image I ⁱ corresponds to the label Yi and the label prediction ^yi generated by the model ^, we calculate the classification error of each label in the sample through binary cross entropy loss (BCEloss), and then obtain the sample error by averaging all label classification errors , the loss function is calculated as follows:

表示样本的第j个标签的值，

表示模型预测的第j个标签的值，T表示样本中标签的个数。in

represents the value of the jth label of the sample,

Represents the value of the jth label predicted by the model, and T represents the number of labels in the sample.

Step (4.5) For the training set in step (1.2), use the Adam adaptive optimization algorithm to minimize the loss function described in step (4.4), by comparing the models under different learning rate λ settings, observe the model after training on the training set. For the classification accuracy on the validation set, the initial value of λ is generally set to 10 ^-3 to 10 ^-6 , the next learning rate is set to 3 times the previous one, and the maximum value of λ is set to 0.1 to 0.5, and then choose the accuracy rate. The highest learning rate to train the model.

Step (5) After completing all the above steps, a new brain CT data set can be input into the model, and these brain CT images can be classified according to the prediction results output by the model.

Compared with the existing method, the present invention has the following obvious advantages and beneficial effects:

The present invention proposes a brain CT image classification method fused with multi-scale superpixels. Compared with the traditional image classification network, the method has the following characteristics: 1) The fusion of multi-scale superpixels and brain CT images is used to remove image redundancy. information, reducing the grayscale similarity of pixels in the lesion and surrounding brain tissue. 2) A region- and boundary-based multi-scale superpixel encoder is designed to effectively extract the low-level information of lesions contained in multi-scale superpixels. 3) A multi-scale superpixel feature fusion model is designed, which comprehensively utilizes the high-level features extracted by the residual neural network and the low-level features of multi-scale superpixels to realize the classification of brain CT. 4) Compared with the traditional deep learning algorithm, the method of the present invention can effectively utilize the lesion information contained in the multi-scale superpixels, thereby more accurately classifying the diseases existing in the brain CT images.

Description of drawings

Figure 1: Flow chart of a brain CT image classification method fused with multi-scale superpixels.

Figure 2: Multi-scale superpixel feature fusion classification model.

Figure 3: Feature fusion models of different sizes.

Figure 4: Multiscale superpixel brain CT image fusion visualization.

Detailed ways

In this example, patients with cerebral hemorrhage are taken as the research object, but the method is not limited to this, and brain CT images of patients with other brain diseases can also be taken as the research object. The following takes the real intracerebral hemorrhage CT data set as an example to specifically describe the implementation steps of the method:

Step (1) Acquire data and preprocess:

与T＝14种脑疾病诊断标签向量Y＝[Y₁，Y₂，…Y₁₄]，Y_i∈{0，1}，其中标签的元素Y_i＝1表示经该脑CT图像存在第i个标签相应的脑疾病，Y_i＝0则是正常。Step (1.1) Data: The present invention uses the CQ500 data set (http://headctstudy.qure.ai/dataset) to collect brain CT images to construct a data set, and actually obtains 451 cases of scan data, a total of 22773 brain CT images, each patient The label information contains 14 diagnostic categories of brain diseases: intracranial hemorrhage, parenchymal hemorrhage, intraventricular hemorrhage, subdural hemorrhage, epidural hemorrhage, subarachnoid hemorrhage, left intracerebral hemorrhage, right intracerebral hemorrhage, chronic hemorrhage, Fractures, skull fractures, other fractures, midline offset, mass effect. First, for dataset labels, since three radiology experts may have different labels for the same label, we choose the choice of the majority of experts as the true label for the inconsistent labeling. Then, according to the determined label information of intracranial hemorrhage, the dataset was divided into 204 cases of patients with confirmed intracerebral hemorrhage and 247 cases of undiagnosed cases. From the cases of patients with confirmed intracerebral hemorrhage, 742 images corresponding to the location of the lesions were selected according to the label of the diagnosis category, and 742 images of the corresponding lesions were selected from the cases of patients with confirmed intracerebral hemorrhage. In the undiagnosed cases, 1045 images with the same location as the lesions in the patients with cerebral hemorrhage were selected, and a total of M=1787 brain CT images were used as the data set D={(I ¹ , Y ¹ ), (I ² , Y ² ), ..., (I ¹⁷⁸⁷ , Y ¹⁷⁸⁷ )}. Each data contains an RGB matrix generated from its brain CT image

and T= ₁₄ kinds of brain disease diagnosis label vector _{Y =} [Y ₁ , Y ₂ , . Each label corresponds to a brain disease, and Y _i = 0 is normal.

Step (1.2) divides all patient data into training set, validation set and test set according to 8:1:1. Among them, the training set is used to learn the parameters of the neural network; the validation set is used to determine the network structure and hyperparameters; the test set is used to verify the classification effect of the neural network.

的计算过程如下：Step (1.3) Data preprocessing: Based on the super-hierarchy segmentation algorithm (Super Hierarchy, SH), for the given brain CT image I and the set S=3 superpixel segmentation scale {5, 10, 15}, generate Superpixel maps at different segmentation scales

The calculation process is as follows:

P ¹ =SH(I,5)

P ² =SH(I, 10)

P ³ =SH(I, 15)

表示包含3个不同尺度超像素图的多尺度超像素。in,

Represents a multi-scale superpixel that contains 3 superpixel maps of different scales.

融合图像

计算过程如下：Step (2) Multi-scale superpixel image fusion model: for a given brain CT image I and its multi-scale superpixels

fused images

The calculation process is as follows:

表示训练的权重，P^s表不

中第s个元素，W实现了各个尺度比重的自适应分配。。Among them, ⊙ represents the dot product, f( ) is the SoftMax function,

Indicates the weight of training, P ^s represents

The s-th element in W realizes the adaptive allocation of the proportions of each scale. .

Step (3) Multi-scale superpixel feature encoding:

中分割尺度为5的超像素图

其中scale¹＝5，生成其像素值集合

对

中每个像素生成映射矩阵集合M¹＝{M^1，1，M^1，2，…，M^1，5}，其中，第k个映射矩阵

中第i，j个元素

的计算方式如下：Step (3.1) Multi-scale superpixels for brain CT image I

A superpixel map with a median segmentation scale of 5

Where scale ¹ = 5, generate its pixel value set

right

Each pixel in generates a set of mapping matrices M ¹ ={M ^1,1 , M ^1,2 ,...,M ^1,5 }, where the kth mapping matrix

The i,jth element in

is calculated as follows:

表示P¹中第k个超像素的像素值，M^1，k表示像素值为

的超像素区域映射where, k∈{1,2,…5}, where

represents the pixel value of the kth ^superpixel in P1, M1 ^,k represents the pixel value

The superpixel region map of

计算过程如下：Step (3.2) encodes each mapping matrix in the set M ¹ based on the area and boundary information, and obtains the encoding result of the superpixel map P ¹

The calculation process is as follows:

where _sk represents the number of pixels contained in the k-th superpixel region.

中每个超像素图重复步骤(3.1)和步骤(3.2)，依次生成编码结果b¹，b²，b³，将其通过矩阵拼接成

得到多尺度超像素

的多尺度超像素特征编码B。Step (3.3) to

Steps (3.1) and (3.2) are repeated for each superpixel image, and the encoding results b ¹ , b ² , and b ³ are generated in turn, and they are spliced into

get multi-scale superpixels

The multi-scale superpixel feature encoding B.

Step (4) Multi-scale superpixel feature fusion classification model:

作为高层次特征。Step (4.1) Build a 34-layer residual neural network ResNet-34 as the backbone network, use the brain CT fusion image I' extracted in step (2) to input, and select the last residual structure of the four layers in ResNet-34 ( BasicBlock) feature activation output

as a high-level feature.

将f₀自下而上依次和l₁、l₂、l₃、l₄进行多层融合，生成融合特征f₁、f₂、f₃、f₄。其中f_i(i∈{1，2，3，4})的计算如下：先采用池化操作将f_(i-1)转化为和l_i相同尺寸的特征矩阵，之后通过256个1×1卷积核构成的卷积层将l_i的通道数转换为256，之后通过矩阵相加进行特征融合，得到第f层融合特征f_i。Step (4.2) performs dimensionality reduction processing on the low-level feature B extracted in (3.3), and generates features through a convolutional layer composed of 256 3×3 convolution kernels

Perform multi-layer fusion of f ₀ with l ₁ , l ₂ , l ₃ , and l ₄ from bottom to top to generate fusion features f ₁ , f ₂ , f ₃ , and f ₄ . The calculation of f _i (i∈{1, 2, 3, 4}) is as follows: first, the pooling operation is used to convert f _(i-1) into a feature matrix of the same size as l _i , and then 256 1×1 The convolution layer formed by the convolution kernel converts the number of channels of li to 256, and then performs feature fusion through matrix addition to obtain the f- _{th layer fusion feature f i .}

Step (4.3) Input the obtained fusion feature f ₄ into the convolution layer, pooling layer and fully connected layer composed of 512 3×3 convolution kernels to obtain a classification vector x of the same length as the number of labels T, and pass it through the Sigmoid linear Regression generates a predicted value vector y=[y ₁ , y ₂ ,...y ₁₄ ], where y _i ∈ [0, 1], the probability that the i-th element in x generates the corresponding label as a positive example y _i is expressed as: y _i =Sigmoid(x _i ), at this time, the classification threshold is set to t=0.5. When y _i is greater than the set threshold of 0.5, the model determines that the brain CT has a corresponding label disease. If y _i is less than the set threshold of 0.5, it is normal.

Step (4.4) The input of the brain CT image classification method fused with multi-scale superpixels of the present invention is the patient's brain CT image I and the brain disease classification label Y, and then the probability y of the subject belonging to each category is obtained. For the input brain CT image I corresponding to the label Y and the label prediction y generated by the model, we use the binary cross entropy loss (Binarycross entropy loss, BCE loss) to calculate the classification error of each label in the sample, and then pass all The label classification error is averaged to obtain the sample error, and the loss function is calculated as follows:

Step (4.5) For the training set in step (1.2), use the Adam adaptive optimization algorithm to minimize the loss function described in step (4.4), by comparing the models under different learning rate λ settings, observe the model after training on the training set. For the classification accuracy rate on the validation set, the initial value of λ is set to 10 ^-5 , the next learning rate is set to 3 times the last time, and the maximum value of λ is set to 0.1, and then select the learning rate with the highest accuracy to train the model .

Step (5) After completing all the above steps, a new brain CT data set can be input into the model, and these brain CT images can be classified according to the prediction results output by the model.

In order to illustrate the beneficial effects of the method of the present invention, in the specific implementation process, we compared with the traditional classification model ResNet-34 as the backbone network in this paper, and compared the three parts of the brain CT classification model fused with multi-scale superpixels: Superpixel image fusion, superpixel feature encoding algorithm and different-level feature fusion models, ablation experiments are carried out to verify the effectiveness of each part. The F1 evaluation value (F-score, F) is used as the evaluation index, and the experimental results are shown in Table 1.

Multi-scale superpixel selection uses a combination of superpixel maps at scales 5, 10, 15 and 10, 15, and 20. The baseline of the control group in the experiment is to use the brain CT image for classification, MSF represents the method of the present invention, MSF ₀₀₁ represents the result of using the original brain CT image as input, and only the feature fusion model is used for classification; MSF ₀₁₁ Method represents the method of removing MSF Medium multi-scale superpixel image fusion part; MSF ₁₀₁ represents the removal of the superpixel feature encoding part in the MSF method; MSF ₁₀₀ represents the removal of the multi-scale superpixel image fusion and superpixel feature encoding part in the MSF method.

Table 1 Comparative experiments of brain CT classification models fused with multi-scale superpixels

By comparing the classification effects of MSF ₀₁₁ and MSF, it can be seen that the classification effect is slightly improved compared to Baseline without using multi-scale superpixel brain CT image fusion, but it is still not as good as the MSF method in this paper. By comparing the classification results of MSF ₁₀₁ and MSF, it can be seen that the classification effect is significantly improved compared to Baseline without the use of the superpixel feature encoding algorithm, but because the multi-scale superpixel map is directly used, there is no reference area and boundary information. The expression is not good and cannot be directly used as a low-level feature, resulting in a poor fusion feature with high-level features, and a large gap with the MSF classification effect. Through the comparison experiments of MSF ₀₀₁ and Baseline, MSF ₁₀₀ and MSF ₁₀₁ , it can be seen that whether the input is the original image or the image after multi-scale superpixel fusion, compared with only using ResNet to extract features, the use of the feature fusion model is significant. to improve the classification effect.

In addition, we also carried out a weighted map of multi-scale superpixels and weights in the process of multi-scale superpixel fusion and visualized the multi-scale superpixel fusion image I′, as shown in Figure 4. It can be clearly seen that the weighted image removes the redundant information outside the brain very well, and accurately divides the lesion area without over-segmentation; the fusion image I′ can clearly distinguish the lesion area and reduce the pixels of the lesion and surrounding brain tissue. The grayscale similarity of , better represents the lesion area.

In summary, by comparing the Baseline method and ablation experiments, the effectiveness of the MSF method proposed in this paper is verified in the brain CT image classification task. This is because the fusion image based on the fusion of the multi-scale superpixel brain CT image model effectively reduces the noise of the image and accurately divides the lesion area; the multi-scale superpixel encoder extracts accurate low-level features, which contain regions The area and boundary information can better pay attention to small-area lesions; the feature fusion model generates more discriminative fusion features by fusing two different levels of features, and has a more effective expression for the lesion area with variable area. Therefore, the method of the present invention is reasonable and reliable, and can provide powerful help for the classification of brain CT images.

Claims (1)

1. A brain CT image classification method fused with multi-scale superpixels, characterized in that: first, a data set is constructed and preprocessed to obtain multi-scale superpixels; secondly, data enhancement is performed through multi-scale superpixel image fusion to obtain The optimized fusion image; then the feature coding algorithm based on region and boundary information is used to process multi-scale superpixels to obtain multi-scale superpixel low-level features; finally, the multi-scale superpixel feature fusion classification model is used to classify brain CT images; Step (1) Acquire data and preprocess: Step (1.1) Data: collect brain CT images to construct a dataset, each patient data contains its RGB matrix generated by brain CT images

with brain CT classification label vector Y = [Y ₁ , Y ₂ , ... Y _T ], Y _i ∈ {0, 1}, where

represents the set of real numbers, N represents the pixel size of the image, and T represents the number of disease categories collected; Step (1.2) divides all patient data into training set, verification set and test set; wherein, the training set is used to learn the parameters of the neural network; the verification set is used to determine hyperparameters; the test set is used to verify the neural network classification effect; Step (1.3) Data preprocessing: Based on the super-hierarchy segmentation algorithm (Super Hierarchy, SH), for the given brain CT image I and the set superpixel segmentation scale {scale ¹ , scale ² , ... scale ^S }, where S represents the number of set superpixels, and generates a superpixel map at the sth segmentation scale

The calculation process is as follows: P ^s =SH(I, scale ^s ) Among them, s∈{1, 2…S}, scale ^s is the s-th segmentation scale, and each segmentation scale is calculated to get

represents a multi-scale superpixel containing S superpixel maps of different scales; Step (2) Multi-scale superpixel image fusion model: for a given brain CT image I and its multi-scale superpixels

The calculation process of the fusion image I' is as follows:

Among them, ⊙ represents the dot product, f( ) is the SoftMax function,

Represents the weight of training, P ^s represents

In the sth element, W realizes the adaptive distribution of the proportions of each scale; Step (3) Multi-scale superpixel feature encoding: Step (3.1) Multi-scale superpixels for brain CT image I

A superpixel map with a median segmentation scale of scale ^s

generate a collection of its pixel values

right

Generates a set of mapping matrices for each pixel in

Among them, the kth mapping matrix

The i,jth element in

is calculated as follows:

where k∈{1, 2,…scale ^s }, where

represents the pixel value of the kth superpixel in P ^s , M ^{s, k} represents the pixel value

The superpixel region map of ; Step (3.2) encodes each mapping matrix in the set M ^s based on the area and boundary information, and obtains the encoding result of the superpixel map P ^s

The calculation process is as follows:

where N ² represents the number of pixels contained in the superpixel map, _sk represents the number of pixels contained in the k-th superpixel region, and ⊙ represents the dot product; Step (3.3) to

Steps (3.1) and (3.2) are repeated for each superpixel image in , and the encoding results b ¹ , b ² , . . . b ^S are generated in turn, which are spliced into

get multi-scale superpixels

The multi-scale superpixel feature encoding B; Step (4) Multi-scale superpixel feature fusion classification model: Step (4.1) Build a residual neural network ResNet as the backbone network, use the brain CT fusion image I' extracted in step (2) as input, and select the feature activation output of the last residual structure (Basic Block) of the four Layers in ResNet l ₁ , l ₂ , l ₃ , l ₄ are used as high-level features; Step (4.2) performs dimensionality reduction processing on the low-level feature B extracted in (3.3), and generates features through a convolutional layer composed of 256 3×3 convolution kernels

Perform multi-layer fusion of f ₀ with l ₁ , l ₂ , l ₃ , and l ₄ from bottom to top to generate fusion features f ₁ , f ₂ , f ₃ , f ₄ ; where f _i (i∈{1, 2 , 3, 4}) are calculated as follows: first, the pooling operation is used to convert f _(i-1) into a feature matrix of the same size as l _i , and then l is passed through the convolution layer composed of 256 1×1 convolution kernels. The number of channels of _i is converted to 256, and then feature fusion is performed by matrix addition to obtain the i-th layer fusion feature f _i ; Step (4.3) Input the obtained fusion feature f ₄ into the convolution layer, pooling layer and fully connected layer composed of 512 3×3 convolution kernels to obtain a classification vector x of the same length as the number of labels T, and pass it through the Sigmoid linear Regression generates a predicted value vector y=[y ₁ , y ₂ ,...y _T ], where y _i ∈ [0, 1], the probability that the i-th element in x generates the corresponding label as a positive example y _i is expressed as: y _i =Sigmoid(x _i ), the classification result is determined according to the set classification threshold t, when y _i is greater than the set threshold t, the model determines that the brain CT has a corresponding label disease, and if y _i is less than the set threshold t, it is normal ; t is 0.5; In step (4.4), the input is the patient's brain CT image I and the brain disease classification label Y, and then the probability y of the subject belonging to each category is obtained; if the data set D={(I ¹ , Y ¹ ) of M patients is given, ^{( I 2} , ^{Y 2 ) ,} . The classification error of each label in the sample, and then the sample error is obtained by averaging the classification errors of all labels. The loss function is calculated as follows:

in

represents the value of the jth label of the sample,

Represents the value of the jth label predicted by the model, and T represents the number of labels in the sample; Step (4.5) For the training set in step (1.2), use the Adam adaptive optimization algorithm to minimize the loss function described in step (4.4), by comparing the models under different learning rate λ settings, observe the model after training on the training set. For the classification accuracy rate on the validation set, the initial value of λ is set to 10 ^-5 , the next learning rate is set to 3 times the last time, and the maximum value of λ is set to 0.1, and then select the learning rate with the highest accuracy to train the model ; Step (5) After completing all the above steps, input a new brain CT data set into the model, and classify these brain CT images according to the prediction results output by the model.

Images