CN113554131B - Medical image processing and analyzing method, computer device, system and storage medium - Google Patents
Medical image processing and analyzing method, computer device, system and storage medium Download PDFInfo
- Publication number
- CN113554131B CN113554131B CN202111108896.1A CN202111108896A CN113554131B CN 113554131 B CN113554131 B CN 113554131B CN 202111108896 A CN202111108896 A CN 202111108896A CN 113554131 B CN113554131 B CN 113554131B
- Authority
- CN
- China
- Prior art keywords
- medical image
- deep learning
- learning model
- image processing
- candidate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 37
- 238000012545 processing Methods 0.000 title claims abstract description 10
- 238000013136 deep learning model Methods 0.000 claims abstract description 69
- 238000012549 training Methods 0.000 claims abstract description 41
- 238000003672 processing method Methods 0.000 claims abstract description 18
- 230000008569 process Effects 0.000 claims abstract description 16
- 238000002372 labelling Methods 0.000 claims abstract description 13
- 238000004458 analytical method Methods 0.000 claims abstract description 10
- 238000010191 image analysis Methods 0.000 claims abstract description 10
- 230000011218 segmentation Effects 0.000 claims description 15
- 238000013145 classification model Methods 0.000 claims description 14
- 238000010586 diagram Methods 0.000 claims description 12
- 238000001514 detection method Methods 0.000 claims description 11
- 238000004422 calculation algorithm Methods 0.000 claims description 7
- 238000003703 image analysis method Methods 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 4
- 230000003252 repetitive effect Effects 0.000 claims description 4
- 238000002604 ultrasonography Methods 0.000 claims description 4
- 210000000988 bone and bone Anatomy 0.000 claims description 3
- 238000012800 visualization Methods 0.000 claims description 3
- 230000007170 pathology Effects 0.000 claims description 2
- 238000013135 deep learning Methods 0.000 description 20
- 210000004556 brain Anatomy 0.000 description 8
- 230000000694 effects Effects 0.000 description 7
- 210000001320 hippocampus Anatomy 0.000 description 7
- 230000006870 function Effects 0.000 description 5
- 230000003902 lesion Effects 0.000 description 5
- 238000003745 diagnosis Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000002474 experimental method Methods 0.000 description 3
- 210000000056 organ Anatomy 0.000 description 3
- 230000002829 reductive effect Effects 0.000 description 3
- 210000001519 tissue Anatomy 0.000 description 3
- PCTMTFRHKVHKIS-BMFZQQSSSA-N (1s,3r,4e,6e,8e,10e,12e,14e,16e,18s,19r,20r,21s,25r,27r,30r,31r,33s,35r,37s,38r)-3-[(2r,3s,4s,5s,6r)-4-amino-3,5-dihydroxy-6-methyloxan-2-yl]oxy-19,25,27,30,31,33,35,37-octahydroxy-18,20,21-trimethyl-23-oxo-22,39-dioxabicyclo[33.3.1]nonatriaconta-4,6,8,10 Chemical compound C1C=C2C[C@@H](OS(O)(=O)=O)CC[C@]2(C)[C@@H]2[C@@H]1[C@@H]1CC[C@H]([C@H](C)CCCC(C)C)[C@@]1(C)CC2.O[C@H]1[C@@H](N)[C@H](O)[C@@H](C)O[C@H]1O[C@H]1/C=C/C=C/C=C/C=C/C=C/C=C/C=C/[C@H](C)[C@@H](O)[C@@H](C)[C@H](C)OC(=O)C[C@H](O)C[C@H](O)CC[C@@H](O)[C@H](O)C[C@H](O)C[C@](O)(C[C@H](O)[C@H]2C(O)=O)O[C@H]2C1 PCTMTFRHKVHKIS-BMFZQQSSSA-N 0.000 description 2
- 206010006187 Breast cancer Diseases 0.000 description 2
- 206010028980 Neoplasm Diseases 0.000 description 2
- 230000002411 adverse Effects 0.000 description 2
- 201000007295 breast benign neoplasm Diseases 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000007423 decrease Effects 0.000 description 2
- 230000036541 health Effects 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000003062 neural network model Methods 0.000 description 2
- 230000036961 partial effect Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000002238 attenuated effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000001574 biopsy Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 201000011510 cancer Diseases 0.000 description 1
- 238000004195 computer-aided diagnosis Methods 0.000 description 1
- 230000001186 cumulative effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 238000012804 iterative process Methods 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000003211 malignant effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000010355 oscillation Effects 0.000 description 1
- 230000001575 pathological effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/217—Validation; Performance evaluation; Active pattern learning techniques
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
- Measuring And Recording Apparatus For Diagnosis (AREA)
Abstract
本发明属于医学图像处理技术领域，具体涉及一种医学图像处理和分析方法、计算机设备、系统和存储介质。本发明的医学图像处理方法步骤如下：基于初始的已标注医学图像样本训练得到深度学习模型；完成前向传播过程，基于香农定理将预测结果转化为熵图；挑选出不重复信息量总和最高的一批候选医学图像样本；挑选出的候选医学图像样本经过标注后与所有已标注医学图像样本合并，训练得到新的深度学习模型；重复以上步骤，直至遍历所有的候选医学图像样本或模型提前达到某一目标性能。本发明还提供了一种利用上述深度学习模型进行图像分析的方法。本发明能够减少对医学图像的标注量降低深度学习模型的使用成本，具有很高的应用价值。
The invention belongs to the technical field of medical image processing, and in particular relates to a medical image processing and analysis method, computer equipment, system and storage medium. The steps of the medical image processing method of the present invention are as follows: a deep learning model is obtained by training based on the initial labeled medical image samples; the forward propagation process is completed, and the prediction result is converted into an entropy map based on Shannon's theorem; A batch of candidate medical image samples; the selected candidate medical image samples are marked and merged with all marked medical image samples to train a new deep learning model; repeat the above steps until all candidate medical image samples or models are traversed in advance. a target performance. The present invention also provides a method for image analysis using the above-mentioned deep learning model. The invention can reduce the labeling amount of medical images and the use cost of the deep learning model, and has high application value.
Description
Technical Field
The invention belongs to the technical field of medical image processing, and particularly relates to a medical image processing and analyzing method, computer equipment, a system and a storage medium.
Background
Deep learning is a new research direction in the field of machine learning, and is a general term for a class of pattern analysis methods. Deep learning is the intrinsic law and expression level of the learning sample data, and the information obtained in the learning process is very helpful for the interpretation of data such as characters, images and sounds. The final aim of the method is to enable the machine to have the analysis and learning capability like a human, and to recognize data such as characters, images and sounds.
Deep learning has been widely used in medical image analysis, for example, patents such as CN202011259621.3 and CN202011618907.6 implement computer-aided diagnosis by using deep learning technology. They train a neural network model using the labeled data and then analyze the medical image data using the trained neural network model to assist in diagnosis. Deep learning has enabled a variety of purposes in medical image processing, such as classification, detection, and segmentation. However, in the deep learning model training process, a large amount of labeled data is needed, and the labeling of medical data requires strong professional knowledge. Therefore, deep learning model training often requires a professional doctor to spend a lot of time performing the data annotation function, which results in high cost of deep learning for medical image processing.
Therefore, it is very important to reduce the labeling amount of the sample in the medical image analysis. Zhou, Z et al (Zhou, Z. et al, 2017, In Proceedings of the IEEE conference on computer vision and pattern recognition, 7340) tried to reduce the cost of deep learning technology In the medical field by adding an active learning method In deep learning to reduce the amount of sample labeling, and achieved certain effect. However, the method is only oriented to the deep learning technology of image classification, and cannot be applied to the field of image detection and segmentation. Meanwhile, their active learning method is based on the sample information amount only, and the sample selection and annotation result causes serious sampling deviation and information repetition. Yang, L et al (Yang, L. et al 2017, In International reference on a dimensional image computing and computer-assisted interaction, 399-. The method reduces the sample annotation amount to a certain extent, and reduces the application cost of the depth technology in the medical field. But the convolutional neural network trained based on different samples in the method can cause selection difference, so that the difference of results is not completely dependent on the candidate samples, and the selection result is not accurate.
In a word, due to the limitation of the model modeling process, a proper deep learning method is still lacked in the prior art, and the labeling quantity of samples can be reduced aiming at various medical image analysis tasks such as classification, detection and segmentation. This leads to high cost of deep learning in the field of medical image analysis, and affects large-scale popularization and application thereof.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a medical image processing and analyzing method, computer equipment, a system and a storage medium, aiming at reducing the marking amount of medical image samples in the deep learning model modeling process, thereby obviously reducing the workload of professional doctors and reducing the cost of deep learning applied to medical image analysis.
A medical image processing method, comprising the steps of:
step 1, training based on an initial labeled medical image sample to obtain an initial deep learning model;
step 2, inputting the candidate medical image sample into the deep learning model established in the previous step, completing the forward propagation process, and converting the prediction result into an entropy diagram based on Shannon's theorem;
step 3, based on the prediction result and the entropy diagram, evaluating the total information amount and the information repetition degree of the candidate medical image samples, and selecting a batch of candidate medical image samples with the highest total non-repeated information amount;
step 4, outputting the candidate medical image samples selected in the step 3 for labeling;
step 5, combining the candidate medical image samples selected in the step 3 with all the labeled medical image samples after labeling to form a new sample set of the labeled medical image samples, and training by using the new sample set of the labeled medical image samples to obtain an intermediate deep learning model;
and 6, repeating the steps 2-5 until all the candidate medical image samples or models are traversed to reach the target performance, and obtaining the final deep learning model.
Preferably, the medical image is a CT image, MRI, ultrasound image, X-ray image, pathology slice image or bone visualization image, and the MRI is T1WI, T2WI, DWI or DTI.
Preferably, in step 1, step 5 and step 6, the initial deep learning model, the intermediate deep learning model and the final deep learning model are classification models, detection models or segmentation models of medical images.
Preferably, in step 1, step 5 and step 6, the initial deep learning model, the intermediate deep learning model and the final deep learning model are detection models or segmentation models of medical images;
in step 3, after a batch of candidate medical image samples with the highest total amount of non-repetitive information are selected, the candidate medical image samples are further selected through the following steps:
step 3a, training based on all labeled medical image samples to obtain a strong classification model;
step 3b, inputting the selected candidate medical image sample into the strong classification model to obtain a classification prediction result;
and 3c, performing regional comparison on the classification prediction result obtained in the step 3b and the training result of the deep learning model established in the previous step in the step 2, and further selecting candidate medical image samples with inconsistent comparison results.
Preferably, in step 3, the algorithm for selecting the candidate medical image samples with the highest total quantity of non-repetitive information is a greedy algorithm.
Preferably, in step 5, in the process of training the intermediate deep learning model, the training round and the learning rate are adaptively adjusted.
The invention also provides a medical image analysis method for non-diagnostic purposes, comprising the following steps:
and inputting the medical image to be analyzed into a deep learning model, analyzing the deep learning model and outputting an analysis result, wherein the deep learning model is a final deep learning model obtained after training according to the medical image processing method.
The invention also provides a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, which when executed by the processor implements the above-described deep learning-based medical image processing method or medical image analysis method for non-diagnostic purposes.
The present invention also provides a system comprising:
means for acquisition and/or input and/or storage of medical images;
the computer device described above.
The present invention also provides a computer-readable storage medium having stored thereon a computer program for implementing the above-described deep learning-based medical image processing method or medical image analysis method for non-diagnostic purposes.
In the present invention, the "initial labeled medical image sample" refers to a partial sample selected from all samples that can be used for modeling but are not labeled, and the partial samples are labeled by a professional doctor and then used for training an initial deep learning model. The "candidate medical image sample" refers to a part which is not selected as the "initial labeled medical image sample" in all samples which can be used for modeling but are not labeled, and the processing method of the invention needs to select a sample with a value as a training basis from the "candidate medical image sample".
When the total amount of information and the information repetition degree of the candidate medical image samples are evaluated, the total objective is to evaluate the information repetition among a batch of high-entropy samples and then eliminate the excessive repetition. And repeated labeling and cost waste are avoided. The total amount of information was evaluated based on entropy theory and the variability (degree of information repetition) was evaluated based on KL divergence and cross-correlation.
The classification refers to the division of lesion degree, lesion type, organ and tissue type, the detection refers to the determination of the region of the organ, tissue or lesion in the image, and the segmentation refers to the pixel-level division of the region of the organ, tissue or lesion in the image.
The term "non-diagnostic purpose" means that the analysis result outputted by the analysis method of the present invention is not the diagnosis result or health condition of the patient, but an intermediate result which requires further diagnosis by a medical professional to obtain the diagnosis result or health condition of the patient.
The invention has the following beneficial effects:
1. the invention integrates active learning into deep learning, and utilizes the evaluation results of the entropy diagram and the prediction result diagram to select, thereby realizing the purpose of reducing the sample mark amount in the deep learning model modeling process of medical image analysis. The method can effectively reduce the workload of professional doctors, reduce the application cost of deep learning in the field of medical image analysis, and is favorable for popularization and application of the medical image analysis method based on deep learning.
2. The invention can be widely adapted to the deep learning technology in the medical field, can be applied to various medical images and various deep learning models, and has extremely strong generalization.
3. In a preferred scheme, the invention utilizes a classification model to assist and guide the modeling of a detection model or a segmentation model based on the relatively easy-to-achieve characteristics of the classification task. When the deep learning model is a detection model or a segmentation model, the invention further screens the selected candidate medical image samples through a high-precision model (a strong classification model is obtained based on training of all labeled medical image samples), so that the amount of the samples needing to be labeled can be further reduced.
4. In a preferred scheme, the method adaptively adjusts the training round and the learning rate in the deep learning model modeling. A slight overfitting is formed by adjusting the training round so that the loss function does not drop for consecutive K rounds (e.g., 5-15 rounds). Therefore, the information carried in the labeled medical image sample can be extracted as much as possible, and the comprehensive analysis process of selecting the sample in the active learning process in the next iteration is facilitated. Meanwhile, the learning rate is adjusted to oscillate and attenuate, so that the adverse effect of slight overfitting on deep learning model training is overcome. Through the self-adaptive adjustment process, the influence of the training sample size change on deep learning model training in the iterative process can be eliminated, so that active learning and deep learning are better combined, and the cost of deep learning application is better reduced while the problems of over-fitting and under-fitting are overcome.
Obviously, many modifications, substitutions, and variations are possible in light of the above teachings of the invention, without departing from the basic technical spirit of the invention, as defined by the following claims.
The present invention will be described in further detail with reference to the following examples. This should not be understood as limiting the scope of the above-described subject matter of the present invention to the following examples. All the technologies realized based on the above contents of the present invention belong to the scope of the present invention.
Drawings
Fig. 1 is a flow chart of a medical image processing method based on deep learning according to the present invention.
Detailed Description
It should be noted that, in the embodiment, the algorithm of the steps of data acquisition, transmission, storage, processing, etc. which are not specifically described, as well as the hardware structure, circuit connection, etc. which are not specifically described, can be implemented by the contents disclosed in the prior art.
Embodiment 1 deep learning-based medical image processing method
The method of this embodiment is shown in fig. 1, and includes the following steps:
step 1, training based on an initial labeled medical image sample to obtain an initial deep learning model;
step 2, inputting the candidate medical image sample into the deep learning model established in the previous step, completing the forward propagation process, and converting the prediction result into an entropy diagram based on Shannon's theorem;
and 3, evaluating the total information amount and the information repetition degree of the candidate medical image samples based on the entropy diagram and the prediction result, and selecting a batch of candidate medical image samples with the highest total non-repeated information amount. Based on the mode, the method can effectively avoid selecting the sample with large absolute information amount but excessive repeated information, and avoid inefficient sample annotation caused by serious selection deviation.
The entropy diagram is an information quantity matrix obtained by converting the classification probability predicted by the model according to the Shannon theorem. For example:
1. in the face of the medical image classification task, a certain tumor image is benign or malignant. The prediction result of the whole image is a one-dimensional probability vector, and the information quantity result obtained according to the Shannon theorem is only 1 numerical value, namely the information quantity of the whole image relative to the model.
2. In the face of the medical image segmentation task, a certain brain structure is segmented (in multiple regions). The prediction result of the whole image is a 4-dimensional tensor, namely [ H, W, D, C ], wherein the first 3 dimensions are voxel positions, the 4 th dimension represents the probability that the voxels at the positions belong to a certain structure (region), similarly, the information quantity calculation is carried out on the C dimension according to the Shannon theorem, the information quantity matrix is 3 dimensions [ H, W, D ], and the value at each voxel position expresses the information quantity of the voxel relative to the model.
The entropy diagram has the following functions as a basic tool: 1) the information amount of the whole image can be measured according to the entropy diagram, for example, the simplest cumulative sum is used. 2) The information quantity distribution can be shown according to the entropy diagram, the difference of the information quantities of different positions of the sample is shown, and the difference sample is convenient to select.
When the deep learning model is a detection model or a segmentation model, after the candidate medical image samples are selected by using the greedy algorithm, the candidate medical image samples can be further selected by the following steps:
and 3a, training based on all labeled medical image samples to obtain a high-precision model, wherein the high-precision model is a strong classification model. For example, when the task of the deep learning model is to segment the brain hippocampus, the high-precision model is the brain hippocampus classification model (i.e., the probability that the prediction input is brain hippocampus).
Step 3b, inputting the candidate medical image sample selected by the greedy algorithm into the strong classification model to obtain a classification prediction result;
and 3c, performing regional comparison on the classification prediction result obtained in the step 3b and the training result of the deep learning model established in the previous step in the step 2, and further selecting candidate medical image samples with inconsistent comparison results. For example, for the above-mentioned division task of the hippocampus of the brain, the candidate medical image sample is input to the depth learning model which is not trained, and the division result is obtained, and then the division result is input to the high-precision model. If the high-precision model also considers that the input part has a high probability of being the hippocampus of the brain, the fact that the deep learning model has learned the characteristics of the candidate medical image sample can be well segmented indicates that the labeling of the candidate medical image sample is not necessary. Conversely, if the probability of the high-precision classification model considering the input as the hippocampus of the brain is low, it indicates that the candidate medical image sample needs to be labeled for learning.
Step 4, outputting the candidate medical image samples selected in the step 3 for labeling;
and 5, combining the candidate medical image samples selected in the step 3 with all the labeled medical image samples after labeling to form a new sample set of the labeled medical image samples, and training by using the new sample set of the labeled medical image samples to obtain an intermediate deep learning model. In the process of training the intermediate deep learning model, in order to solve the problems of over-fitting and under-fitting, the training round and the learning rate are adaptively adjusted based on the dynamic change of the sample data volume.
The adaptive adjustment includes adjustment of training rounds and learning rates.
The adjustment to the training round aims at controlling the sample fitting degree. In this example, the training round was controlled so that the degree of fit of the samples was slightly overfit (the loss function did not decrease for consecutive 5-15 rounds). For example: when the hardware condition is fixed (tesla V10032G × 4), when the iteration (iteration) of the model proceeds to the 4 th round, a total of 200 samples that have been picked and labeled, the deep learning training parameter batch size is 16 (determined by hardware), and the total number of training rounds (epoch) is 125. When the model iteration proceeds to round 5, the number of samples that have been picked and labeled increases to 250, and the batch size remains the same, then the round of training needs to be adjusted to 157 times as the number of samples increases.
For the adjustment of the learning rate, the purpose is to eliminate the adverse effect of slight overfitting on the parameter adjustment in the next round of deep learning model training. For example: in this embodiment, a learning rate of oscillation attenuation is adopted, in the iteration of the round, the first epochs of the model will use a larger learning rate to adjust parameters, the loss function deviates from the minimum value region of the previous round, the learning rate is gradually attenuated along with the progress of the epochs, and the global minimum point position is newly found. For example: in one iteration, 0.0005 is used as the initial learning rate (epoch _ 0), and the rate gradually decreases according to a fixed formula. The learning rate of the last epoch is half of the initial learning rate (which has been slightly overfit). In the next iteration, the global optimum of the previous round becomes the local optimum at that time, since a new sample is added. The initial learning rate setting needs to be greater than 0.00025 (e.g., 0.0004) to help the model adjust the model parameters in larger steps from the local optimum, which is a region where the model parameters are difficult to fall out of if the learning rate is maintained at 0.00025. This one iteration eventually gradually decays to 0.0002 epoch. And so on in iteration that follows.
(6) And (5) repeating the steps (2) to (5) until all the candidate medical image samples are traversed to obtain the final deep learning model.
This embodiment has a very strong generalization, and the medical images can be various medical images in the prior art, such as: CT images, MRI (including T1WI, T2WI, DWI, DTI), ultrasound images, X-ray images, pathological section images, bone visualization images. The deep learning model may be a classification model, a detection model or a segmentation model of the medical image.
Embodiment 2 method and System for medical image analysis
The embodiment provides a method and a system for analyzing a medical image. The system of the embodiment is composed of two parts connected through a data line and a data interface: a server and a computer device. The server is used for storing data of the medical image. Furthermore, as an equivalent alternative, the server may also be replaced by a device for acquiring medical images (e.g. a CT machine, an ultrasound imager, etc.) or a device for inputting medical image data.
The computer device is used for analyzing the medical image according to the following method:
inputting a medical image to be analyzed into a deep learning model, analyzing the medical image by the deep learning model, and outputting an analysis result, wherein the deep learning model is a final deep learning model obtained after training according to the method of the embodiment 1.
To illustrate further the beneficial effects of the present invention, the following experiments were performed:
experimental example 1
Data set: the Breakhis dataset contains microscopic biopsy images of benign and malignant breast tumors, obtained using magnification factors of 40X, 100X, 200X, 400X in a 3-channel RGB (red-green-blue) true color (24 bit color depth, 8 bit per color channel) color space, corresponding to objective 4X, 10X, 20X, 40X. The database consisted of 7909 images (2480 for each scale of benign tumor image and 5429 for malignant tumor) according to 3: the scale of 1 is randomly divided into a training set and a test set.
Deep learning model: the deep learning model of this experimental example is a classification model, specifically a multiscale benign and malignant breast tumor lesion classification network based on Res-net 50.
The experimental results are as follows: according to the method in the prior art, a full data set is adopted to train a deep learning model, and the classification accuracy of the trained model is 95%. After the active learning was added, the baseline effect was achieved using an average of 44.25% of the sample size (95% of the classification accuracy was used as baseline effect, 4 replicates) according to the method of example 1.
Experimental example 2
Data set: the data was derived from the ABCD neurocognitive prediction challenge (ABCD NP challenge 2019) for a total of 3736 annotated MRI (T1 WI) brain images with 1 channel image and 240x240x240 image resolution. 1000 of them were randomly selected for the experiment and tested according to 1: the scale of 1 is randomly divided into a training set and a test set.
Deep learning model: the deep learning model of the experimental example is a segmentation model, in particular to an MRI hippocampus segmentation model based on 3D U-net.
The experimental results are as follows: according to the method in the prior art, a full data set is adopted to train a deep learning model, and the segmentation dice index is 0.85. After active learning is added, the baseline effect can be achieved by using 67% of the sample amount on average (the baseline effect is achieved by using the segmentation dice index of 0.85, and the experiment is repeated for 4 times) according to the method of example 1.
It can be seen from the above embodiments that the present invention realizes the effect of reducing the amount of labeling on medical images by embedding the deep learning model into the active learning process. The method can effectively reduce the workload of professional doctors, reduce the use cost of the deep learning model, and is beneficial to large-scale popularization and application. This makes the present invention have high application value.
Claims (10)
1. A medical image processing method, characterized by comprising the steps of:
step 1, training based on an initial labeled medical image sample to obtain an initial deep learning model;
step 2, inputting the candidate medical image sample into the deep learning model established in the previous step, completing the forward propagation process, and converting the prediction result into an entropy diagram based on Shannon's theorem;
step 3, based on the prediction result and the entropy diagram, evaluating the total information amount and the information repetition degree of the candidate medical image samples, and selecting a batch of candidate medical image samples with the highest total non-repeated information amount;
step 4, outputting the candidate medical image samples selected in the step 3 for labeling;
step 5, combining the candidate medical image samples selected in the step 3 with all the labeled medical image samples after labeling to form a new sample set of the labeled medical image samples, and training by using the new sample set of the labeled medical image samples to obtain an intermediate deep learning model;
and 6, repeating the steps 2-5 until all the candidate medical image samples or models are traversed to reach the target performance, and obtaining the final deep learning model.
2. A medical image processing method according to claim 1, characterized in that: the medical image is a CT image, MRI, ultrasound image, X-ray image, pathology slice image, or bone visualization image, and the MRI is T1WI, T2WI, DWI, or DTI.
3. A medical image processing method according to claim 1, characterized in that: in step 1, step 5 and step 6, the initial deep learning model, the intermediate deep learning model and the final deep learning model are classification models, detection models or segmentation models of medical images.
4. A medical image processing method according to claim 3, characterized in that: in step 1, step 5 and step 6, the initial deep learning model, the intermediate deep learning model and the final deep learning model are detection models or segmentation models of medical images;
in step 3, after a batch of candidate medical image samples with the highest total amount of non-repetitive information are selected, the candidate medical image samples are further selected through the following steps:
step 3a, training based on all labeled medical image samples to obtain a strong classification model;
step 3b, inputting the selected candidate medical image sample into the strong classification model to obtain a classification prediction result;
and 3c, performing regional comparison on the classification prediction result obtained in the step 3b and the training result of the deep learning model established in the previous step in the step 2, and further selecting candidate medical image samples with inconsistent comparison results.
5. A medical image processing method according to claim 1, characterized in that: in step 3, an algorithm for selecting a batch of candidate medical image samples with the highest total amount of non-repetitive information is a greedy algorithm.
6. A medical image processing method according to claim 1, characterized in that: and 5, in the process of training the intermediate deep learning model, carrying out self-adaptive adjustment on the training round and the learning rate.
7. A method of medical image analysis for non-diagnostic purposes, comprising the steps of:
inputting a medical image to be analyzed into a deep learning model, analyzing the deep learning model and outputting an analysis result, wherein the deep learning model is a final deep learning model obtained after training according to the medical image processing method of any one of claims 1 to 6.
8. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the medical image processing method of any one of claims 1-6 or the non-diagnostic purpose medical image analysis method of claim 7 when executing the program.
9. A medical image processing and analysis system, comprising:
means for acquisition and/or input and/or storage of medical images;
the computer device of claim 8.
10. A computer-readable storage medium characterized by: stored thereon a computer program for implementing the medical image processing method of any one of claims 1-6 or the medical image analysis method for non-diagnostic purposes of claim 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111108896.1A CN113554131B (en) | 2021-09-22 | 2021-09-22 | Medical image processing and analyzing method, computer device, system and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111108896.1A CN113554131B (en) | 2021-09-22 | 2021-09-22 | Medical image processing and analyzing method, computer device, system and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113554131A CN113554131A (en) | 2021-10-26 |
| CN113554131B true CN113554131B (en) | 2021-12-03 |
Family
ID=78106489
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111108896.1A Active CN113554131B (en) | 2021-09-22 | 2021-09-22 | Medical image processing and analyzing method, computer device, system and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113554131B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114820952B (en) * | 2022-06-28 | 2022-09-16 | 四川大学华西医院 | Lung ultrasonic visualization three-dimensional reconstruction method and system |
| CN114818683B (en) * | 2022-06-30 | 2022-12-27 | 北京宝兰德软件股份有限公司 | Operation and maintenance method and device based on mobile terminal |
| CN115631391B (en) * | 2022-11-30 | 2023-03-21 | 北京阿丘科技有限公司 | Image selection method and device based on deep active learning and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101853392A (en) * | 2010-04-21 | 2010-10-06 | 河海大学 | Band Selection Method for Remote Sensing Hyperspectral Imagery Based on Conditional Mutual Information |
| CN111815633A (en) * | 2020-09-08 | 2020-10-23 | 上海思路迪医学检验所有限公司 | Medical image diagnosis apparatus, image processing apparatus and method, determination unit, and storage medium |
Family Cites Families (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6636623B2 (en) * | 2001-08-10 | 2003-10-21 | Visiongate, Inc. | Optical projection imaging system and method for automatically detecting cells with molecular marker compartmentalization associated with malignancy and disease |
| CN102254172B (en) * | 2011-06-16 | 2012-08-29 | 电子科技大学 | Method for segmenting fingerprint image based on cellular neural network and morphology |
| CN103617431B (en) * | 2013-11-05 | 2017-01-18 | 北京工业大学 | Maximum average entropy-based scale-invariant feature transform (SIFT) descriptor binaryzation and similarity matching method |
| CN106294535B (en) * | 2016-07-19 | 2019-06-25 | 百度在线网络技术（北京）有限公司 | The recognition methods of website and device |
| CN106570846A (en) * | 2016-11-16 | 2017-04-19 | 浙江工业大学 | Shannon entropy distribution and coverage rate-based newly born target birth strength estimation method |
| CN106951827A (en) * | 2017-02-21 | 2017-07-14 | 南京邮电大学 | A Global Abnormal Behavior Detection Method Based on Object Motion Characteristics |
| CN107256393B (en) * | 2017-06-05 | 2020-04-24 | 四川大学 | Feature extraction and state recognition of one-dimensional physiological signals based on deep learning |
| US10861152B2 (en) * | 2018-03-16 | 2020-12-08 | Case Western Reserve University | Vascular network organization via Hough transform (VaNgOGH): a radiomic biomarker for diagnosis and treatment response |
| CN109993726B (en) * | 2019-02-21 | 2021-02-19 | 上海联影智能医疗科技有限公司 | Medical image detection method, device, equipment and storage medium |
| CN110222831B (en) * | 2019-06-13 | 2022-05-17 | 百度在线网络技术（北京）有限公司 | Robustness evaluation method and device of deep learning model and storage medium |
| CN110543860B (en) * | 2019-09-05 | 2022-06-07 | 燕山大学 | Mechanical fault diagnosis method and system based on TJM transfer learning |
| CN110689510B (en) * | 2019-09-12 | 2022-04-08 | 北京航天控制仪器研究所 | Sparse representation-based image fusion method introducing dictionary information |
| CN112529172B (en) * | 2019-09-18 | 2024-09-10 | 华为技术有限公司 | Data processing method and data processing apparatus |
| CN111144448B (en) * | 2019-12-09 | 2024-08-20 | 江南大学 | Video barrage emotion analysis method based on multi-scale attention convolution coding network |
| CN111210414B (en) * | 2020-01-06 | 2023-12-26 | 上海联影智能医疗科技有限公司 | Medical image analysis method, computer device, and readable storage medium |
| CN111310659B (en) * | 2020-02-14 | 2022-08-09 | 福州大学 | Human body action recognition method based on enhanced graph convolution neural network |
| CN111340118B (en) * | 2020-02-27 | 2021-07-23 | 河南大学 | Conflicting evidence fusion method based on reliability entropy and BJS divergence |
| CN111461232A (en) * | 2020-04-02 | 2020-07-28 | 大连海事大学 | An MRI Image Classification Method Based on Multi-Strategy Batch Active Learning |
| CN112348036B (en) * | 2020-11-26 | 2025-01-14 | 北京工业大学 | Adaptive object detection method based on lightweight residual learning and deconvolution cascade |
| CN112489091B (en) * | 2020-12-18 | 2022-08-12 | 湖南华南光电(集团)有限责任公司 | Full strapdown image seeker target tracking method based on direct-aiming template |
| CN112800913B (en) * | 2021-01-20 | 2022-06-28 | 同济大学 | A spatiotemporal analysis method of pavement damage data based on multi-source feature fusion |
| CN113240040B (en) * | 2021-05-27 | 2023-04-18 | 西安理工大学 | Polarized SAR image classification method based on channel attention depth network |
| CN113379252A (en) * | 2021-06-11 | 2021-09-10 | 浙江工业大学 | Bayesian network high-voltage switch cabinet health system evaluation method based on multi-scale arrangement entropy |
-
2021
- 2021-09-22 CN CN202111108896.1A patent/CN113554131B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101853392A (en) * | 2010-04-21 | 2010-10-06 | 河海大学 | Band Selection Method for Remote Sensing Hyperspectral Imagery Based on Conditional Mutual Information |
| CN111815633A (en) * | 2020-09-08 | 2020-10-23 | 上海思路迪医学检验所有限公司 | Medical image diagnosis apparatus, image processing apparatus and method, determination unit, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113554131A (en) | 2021-10-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113554131B (en) | Medical image processing and analyzing method, computer device, system and storage medium | |
| CN110232383B (en) | Focus image recognition method and focus image recognition system based on deep learning model | |
| WO2018189541A1 (en) | Recist assessment of tumour progression | |
| Tiago et al. | A data augmentation pipeline to generate synthetic labeled datasets of 3D echocardiography images using a GAN | |
| CN110956634A (en) | Deep learning-based automatic detection method and system for cerebral microhemorrhage | |
| CN109949288A (en) | Tumor type determines system, method and storage medium | |
| CN114332132A (en) | Image segmentation method and device and computer equipment | |
| CN116797612B (en) | Ultrasonic image segmentation method and device based on weak supervision depth activity contour model | |
| US8224048B2 (en) | Computer implemented method for correction of magnetic resonance images | |
| La Rosa | A deep learning approach to bone segmentation in CT scans | |
| Mishra et al. | Improved interactive medical image segmentation using enhanced intelligent scissors (eis) | |
| Shen | [Retracted] Implementation of CT Image Segmentation Based on an Image Segmentation Algorithm | |
| US12106478B2 (en) | Deep learning based medical system and method for image acquisition | |
| EP3767634B1 (en) | Method and system for data augmentation using simulated contrasts | |
| CN116309647B (en) | Method for constructing craniocerebral lesion image segmentation model, image segmentation method and device | |
| Zhou et al. | Automatic segmentation algorithm of femur and tibia based on Vnet-C network | |
| CN116468103A (en) | Training method, application method and system for lung nodule benign and malignant recognition model | |
| JP7591781B2 (en) | Medical image processing apparatus and medical image processing method | |
| CN113327224B (en) | System and method for automatic field of view (FOV) definition | |
| Ramos et al. | Fast and accurate 3-D spine MRI segmentation using FastCleverSeg | |
| KR20210068189A (en) | Method for determining lesion based on medical image | |
| CN119359930B (en) | Cerebral blood vessel three-dimensional model construction method and system based on cerebral blood vessel image segmentation | |
| CN113421270B (en) | Method, system, device, processor and storage medium for realizing medical image domain adaptive segmentation based on single-center calibration data | |
| US20230309848A1 (en) | Magnetic resonance imaging apparatus, image processor, and image processing method | |
| Li et al. | MRI Super-Resolution Analysis via MRISR: Deep Learning for Low-Field Imaging |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |