CN111951217B - Model training method, medical image processing method and electronic device - Google Patents
Model training method, medical image processing method and electronic device Download PDFInfo
- Publication number
- CN111951217B CN111951217B CN202010648720.4A CN202010648720A CN111951217B CN 111951217 B CN111951217 B CN 111951217B CN 202010648720 A CN202010648720 A CN 202010648720A CN 111951217 B CN111951217 B CN 111951217B
- Authority
- CN
- China
- Prior art keywords
- medical image
- channel
- domain
- pixel
- processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012549 training Methods 0.000 title claims abstract description 124
- 238000000034 method Methods 0.000 title claims abstract description 55
- 238000003672 processing method Methods 0.000 title claims abstract description 11
- 238000012545 processing Methods 0.000 claims abstract description 160
- 238000010801 machine learning Methods 0.000 claims abstract description 58
- 238000002407 reforming Methods 0.000 claims description 35
- 238000010606 normalization Methods 0.000 claims description 17
- 238000000605 extraction Methods 0.000 claims description 13
- 230000009467 reduction Effects 0.000 claims description 13
- 239000013598 vector Substances 0.000 claims description 10
- 238000007781 pre-processing Methods 0.000 claims description 7
- 230000008521 reorganization Effects 0.000 abstract 1
- 230000008569 process Effects 0.000 description 23
- 239000011159 matrix material Substances 0.000 description 15
- 230000006870 function Effects 0.000 description 7
- 238000013527 convolutional neural network Methods 0.000 description 6
- 238000001514 detection method Methods 0.000 description 4
- 238000012935 Averaging Methods 0.000 description 2
- 238000013145 classification model Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
本申请实施例提供了模型训练方法、医疗图像处理方法及电子设备，模型训练方法包括：对训练医疗图像进行域统一处理，得到经过域统一处理的训练医疗图像，对训练医疗图像进行域统一处理包括：获取训练医疗图像的每一个单通道医疗图像；对所述每一个单通道医疗图像分别进行预设处理操作；对每一个经过预设处理操作的单通道医疗图像分别进行数值域重整；对每一个经过数值域重整的单通道医疗图像分别进行色彩空间混淆；对每一个经过色彩空间混淆的单通道医疗图像分别进行归一化处理；利用经过域统一处理的训练医疗图像对预设机器学习模型进行训练。提升机器学习模型的泛化性。大幅度提升在进行机器学习模型在未见域的医疗图像上的性能。
The embodiments of the present application provide a model training method, a medical image processing method, and an electronic device. The model training method includes: performing domain unified processing on training medical images, obtaining training medical images subjected to domain unified processing, and performing domain unified processing on the training medical images. The method includes: acquiring each single-channel medical image of the training medical image; performing a preset processing operation on each single-channel medical image; and performing numerical domain reformation on each single-channel medical image that has undergone the preset processing operation; Perform color space confusion on each single-channel medical image that has undergone numerical domain reorganization; normalize each single-channel medical image that has undergone color space confusion; use the training medical image that has undergone domain unified processing to preset Machine learning models are trained. Improve the generalization of machine learning models. Significantly improves the performance of ongoing machine learning models on unseen medical images.
Description
Technical Field
The application relates to the field of machine learning, in particular to a model training method, a medical image processing method and electronic equipment.
Background
When training a machine learning model for the medical field, the training set often includes medical images of multiple domains. Different camera types can be represented to have different color characteristics, different shooting qualities, different crowds and the like. For example, a training set of a machine model for analyzing the health condition of an eye portion often includes fundus medical images of a plurality of domains.
Currently, machine learning models are trained directly using a medical image training set that includes multiple domains. Since medical images of different domains are significantly different from the overall color, features such as those related to the overall color may become features that need to be learned.
However, in the collection process of the training medical image, many fields of medical images are difficult to obtain due to various limiting factors, and these fields are called unseen fields. For example, medical images taken by certain specific cameras are not available, medical images of certain specific people are difficult to acquire, and the like.
Meanwhile, when medical images are processed by using a machine learning model after training, some medical images to be processed in unknown domains may occur frequently. Due to the lack of the training medical image of the unseen domain in the training set, the machine learning model is difficult to learn the overall color-related features of the medical image of the unseen domain, and the performance of the machine learning model on the medical image of the unseen domain is greatly reduced.
This problem is more pronounced on medical images of the same person. For example, the medical images of the same person captured by different cameras have very large differences in overall color, for example, the medical images of the same person captured by each camera have very large differences in color appearance, brightness, and the like, resulting in very large differences in overall color. Multiple medical images of the same person may appear, resulting in different processing results being output by the machine learning model, simply due to belonging to different domains.
Disclosure of Invention
In order to overcome the problems in the related art, the application provides a model training method, a medical image processing method and an electronic device.
According to a first aspect of embodiments of the present application, there is provided a model training method, including:
carrying out domain unified processing on the training medical images to obtain the training medical images subjected to the domain unified processing, wherein the domain unified processing on the training medical images comprises the following steps: acquiring each single-channel medical image of the training medical images; and respectively carrying out preset processing operation on each single-channel medical image, wherein the preset processing operation comprises the following steps: carrying out mean value reduction treatment; respectively carrying out numerical value domain reforming on each single-channel medical image subjected to the preset processing operation; color space confusion is respectively carried out on each single-channel medical image which is reformed through a numerical value field; respectively carrying out normalization processing on each single-channel medical image subjected to color space confusion;
and training the preset machine learning model by using the training medical image subjected to domain unified processing.
According to a second aspect of embodiments of the present application, there is provided a medical image processing method including:
acquiring a medical image to be processed;
the method comprises the steps of preprocessing a medical image to be processed to obtain a preprocessed medical image, processing the preprocessed medical image by using a preset machine learning model to obtain a processing result, wherein the preset machine learning model is trained according to any one of the methods in advance, and the preprocessing comprises the following steps: carrying out mean value reduction processing on each single-channel medical image of the medical images to be processed respectively; performing numerical domain reformation on each single-channel medical image subjected to mean value reduction processing in the medical images to be processed respectively; and respectively carrying out normalization processing on each single-channel medical image subjected to numerical value domain reforming in the medical image to be processed.
The model training method, the medical image processing method and the electronic device provided by the embodiment of the application realize the domain unification processing of a large number of training medical images in a training set, and train the preset machine learning model by using a large number of training medical images subjected to the domain unification processing, which can also be called as training the preset machine learning model by adopting an ELDT (enhanced Local Color Transform) mode. On the one hand, a large number of training medical images which are subjected to domain unified processing are subjected to integral color disturbance on corresponding single-channel medical images respectively, so that the training medical images are subjected to integral color disturbance. Therefore, when a large number of training medical images subjected to domain unified processing are used for training the preset machine learning model, the machine learning model is insensitive to the features related to the whole color, correspondingly, the machine learning model cannot learn the features related to the whole color as the features with the discrimination, and the generalization of the machine learning model is improved. After training, when the medical image to be processed is processed, the performance of the machine learning model on the medical image in the unseen domain is greatly improved.
On the other hand, each of the training medical images is subjected to domain unification processing, and numerical domain reforming in the domain unification processing can make the distribution of pixel values of pixels in each of the training medical images subjected to the domain unification processing conform to a unified distribution. When a large number of training medical images subjected to domain unified processing are used for training the preset machine learning model, the preset machine learning model only needs to be processed aiming at the same distributed training medical images subjected to domain unified processing, and the stability of the preset machine learning model is improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.
FIG. 1 is a flow chart illustrating a model training method provided by an embodiment of the present application;
fig. 2 shows a flowchart of a medical image processing method provided in an embodiment of the present application.
Detailed Description
The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
Fig. 1 shows a flowchart of a model training method provided in an embodiment of the present application, where the method includes:
The training medical image is a medical image used for training a preset machine learning model.
The preset machine learning model may be a classification model, a target detection model, or the like. For example, the preset machine learning model may be a convolutional neural network for classification, a convolutional neural network for target detection, or the like.
The training medical images in step 101 are not particularly limited to a certain training medical image, and each training medical image of the preset machine learning model can be subjected to domain unified processing through step 101.
The operation described in step 101 is an operation performed in a process of exemplarily describing domain-unified processing on one training medical image.
In the present application, performing domain-unified processing on a training medical image includes: acquiring each single-channel medical image of the training medical image; respectively carrying out preset processing operation on each single-channel medical image; respectively carrying out numerical value domain reforming on each single-channel medical image subjected to the preset processing operation; color space confusion is respectively carried out on each single-channel medical image which is reformed through a numerical value field; and respectively carrying out normalization processing on each single-channel medical image subjected to color space confusion.
And carrying out domain unified processing on each training medical image, wherein color space confusion in the domain unified processing can carry out overall color disturbance on the training medical images. Therefore, when the obtained large number of training medical images subjected to domain unified processing are used for training the preset machine learning model, the machine learning model is insensitive to the features related to the whole color, and correspondingly, the machine learning model cannot learn the features related to the whole color as the features with the discrimination. And the generalization of the machine learning model is improved.
And carrying out domain unified processing on each training medical image, wherein numerical value domain reforming in the domain unified processing can ensure that the distribution of pixel values of pixels in each training medical image subjected to the domain unified processing conforms to a unified distribution. When a large number of training medical images subjected to domain unified processing are used for training the preset machine learning model, the preset machine learning model only needs to be processed aiming at the same distributed training medical images subjected to domain unified processing, and the stability of the preset machine learning model is improved.
In the present application, the training medical image is an RGB medical image. For one training medical image, channel extraction can be performed on the training medical image, and all single-channel medical images of the training medical image are obtained.
For a training medical image, all single-channel medical images of the training medical image include: an R-channel medical image of the training medical image, a G-channel medical image of the training medical image, and a B-channel medical image of the training medical image.
In the application, in the process of performing domain unified processing on a training medical image, each single-channel medical image of the training medical image is subjected to preset processing operation respectively, so that each single-channel medical image subjected to the preset processing operation is obtained.
In the present application, the preset processing operation includes: and (6) carrying out an average value reduction treatment. The averaging process may also be referred to as a regional difference process.
The process of the averaging process is briefly described below:
the k single-channel medical image f can be blurred through the following formula to obtain f ', then the average value can represent f-f', pixel subtraction of the same position of the two images is carried out, and a difference value graph corresponding to the k single-channel medical image is obtained.
k∈{1，2，3}，f’k(x, y) may represent a blurred pixel value for the pixel located at (x, y) in the kth single-channel medical image.
The difference map corresponding to the kth single-channel medical image can be used as the kth single-channel medical image subjected to the preset processing operation.
For a training medical image, the R-channel medical image of the training medical image may be subjected to a preset processing operation to obtain the R-channel medical image subjected to the preset processing operation. And carrying out preset processing operation on the G channel medical image of the training medical image to obtain the G channel medical image subjected to the preset processing operation. And carrying out preset processing operation on the B channel medical image of the training medical image to obtain the B channel medical image subjected to the preset processing operation.
In some embodiments, the pre-set processing operation further comprises: and a fuzzy operation of a preset step size is carried out after the average value reduction processing.
For a training medical image, performing mean value reduction processing on the R channel medical image of the training medical image to obtain a difference image corresponding to the R channel medical image of the training medical image, and then performing fuzzy operation with a preset step length on the difference image corresponding to the R channel medical image of the training medical image to obtain the R channel medical image subjected to preset processing operation. And then, performing fuzzy operation with preset step length on the difference image corresponding to the G channel medical image of the training medical image to obtain the G channel medical image subjected to the preset processing operation. And then, fuzzy operation with preset step length can be carried out on the difference image corresponding to the B channel medical image of the training medical image to obtain the B channel medical image after the preset processing operation.
In the application, in the process of carrying out domain unified processing on a training medical image, after each single-channel medical image subjected to preset processing operation is obtained, numerical domain reforming can be respectively carried out on each single-channel medical image subjected to preset processing operation, so that each single-channel medical image subjected to numerical domain reforming is obtained.
In the application, the sigmoid function can be utilized to respectively carry out numerical domain reforming on each single-channel medical image subjected to the preset processing operation, so that each single-channel medical image subjected to numerical domain reforming is obtained.
Respectively carrying out numerical domain reforming on the R channel medical image subjected to the preset processing operation, the G channel medical image subjected to the preset processing operation and the B channel medical image subjected to the preset processing operation by using a sigmoid function to obtain an R channel medical image subjected to numerical domain reforming, a G channel medical image subjected to numerical domain reforming and a B channel medical image subjected to numerical domain reforming.
The numerical domain reformation of each single-channel medical image subjected to the preset processing operation by using the sigmoid function can be represented as follows:
the medical image f' may be equivalent to a medical image obtained by stitching an R-channel medical image subjected to a preset processing operation, a G-channel medical image subjected to a preset processing operation, and a B-channel medical image subjected to a preset processing operation. The medical image G corresponds to a medical image obtained by stitching an R-channel medical image reformed in a numerical field, a G-channel medical image reformed in a numerical field, and a B-channel medical image reformed in a numerical field.
f 'max represents the maximum value among the pixel values of all the pixels of the single-channel medical image, and f' min represents the minimum value among the pixel values of all the pixels of the single-channel medical image.
Ψ may represent a sigmoid function. And respectively carrying out numerical domain reformation on the R channel medical image subjected to the preset processing operation, the G channel medical image subjected to the preset processing operation and the B channel medical image subjected to the preset processing operation by using a sigmoid function.
When the sigmoid is used to perform numerical domain reformation on the R-channel medical image subjected to the preset processing operation, f ' may refer to a pixel value of one pixel in the R-channel medical image subjected to the preset processing operation, f ' max may refer to a maximum value among pixel values of all pixels in the R-channel medical image subjected to the preset processing operation, and f ' min may refer to a minimum value among pixel values of all pixels in the R-channel medical image subjected to the preset processing operation.
Respectively calculating a value of a pixel subjected to numerical value domain reforming of each pixel in the R-channel medical image subjected to the preset processing operation by using a formula 255 Ψ (f ') - (f' min)/Ψ (f 'max) - Ψ (f' min), and setting the value of each pixel as the value of the pixel subjected to numerical value domain reforming, thereby obtaining the R-channel medical image subjected to numerical value domain reforming. In the R-channel medical image subjected to the preset processing operation, the pixel value of each pixel is a pixel value subjected to numerical value domain reformation.
When the G-channel medical image subjected to the preset processing operation is subjected to numerical domain reformation by using the sigmoid function, f ' may refer to a pixel value of one pixel in the G-channel medical image subjected to the preset processing operation, f ' max may refer to a maximum value among pixel values of all pixels in the G-channel medical image subjected to the preset processing operation, and f ' min may refer to a minimum value among pixel values of all pixels in the G-channel medical image subjected to the preset processing operation.
Respectively calculating a value of a pixel subjected to numerical value domain reforming of each pixel in the G channel medical image subjected to the preset processing operation by using a formula 255 Ψ (f ') - (f' min)/Ψ (f 'max) - Ψ (f' min), and setting the value of each pixel as the value of the pixel subjected to numerical value domain reforming, thereby obtaining the G channel medical image subjected to numerical value domain reforming. In the G-channel medical image subjected to the preset processing operation, the pixel value of each pixel is a pixel value subjected to numerical value domain reformation.
The process of obtaining the B-channel medical image reformed by the numerical field is the same as the above-described process of obtaining the R-channel medical image reformed by the numerical field and the G-channel medical image reformed by the numerical field.
In the application, in the process of carrying out domain unified processing on a training medical image, after each single-channel medical image subjected to numerical value domain reforming is obtained, color space confusion can be respectively carried out on each single-channel medical image subjected to numerical value domain reforming, and each single-channel medical image subjected to color space confusion is obtained.
When color space aliasing is performed on each single-channel medical image reformed through the numerical value domain, 3 random numbers can be generated, and each single-channel medical image reformed through the numerical value domain corresponds to one random number. For each numerical-domain-reformed single-channel medical image, the random number corresponding to the numerical-domain-reformed single-channel medical image may be added to the pixel value of each pixel in the single-channel medical image to obtain a color-space-confused pixel value of the pixel.
For example, for each single-channel medical image that is numerical field reformed, 3 random numbers may be generated by mean distribution. For example, 3 random numbers are generated by a mean distribution of-50 to 50. The values of the generated 3 random numbers are all between-50 and 50. Each single-channel medical image which is reformed through the numerical value field corresponds to a random number.
For each single-channel medical image subjected to numerical field reformation, the pixel value of each pixel in the single-channel medical image can be added to the random number corresponding to the single-channel medical image subjected to numerical field reformation, so as to obtain the color-space-confused pixel value of the pixel.
For each numerical-field-reformed single-channel medical image, the pixel value of each pixel of the numerical-field-reformed single-channel medical image is set to the color-space-obfuscated pixel value of each pixel. Thus, each single-channel medical image subjected to color space aliasing is obtained.
For the R-channel medical image reformed in the numerical value domain, the pixel value of each pixel in the R-channel medical image may be added to the random number corresponding to the R-channel medical image reformed in the numerical value domain, so as to obtain the color-space-confused pixel value of the pixel. The pixel value of each pixel in the R-channel medical image subjected to numerical domain reformation is set to the color-space-obfuscated pixel value of each pixel. Thus, an R-channel medical image subjected to color space aliasing is obtained. In the color-space-aliased R-channel medical image, the pixel value of each pixel is the color-space-aliased pixel value.
The process of obtaining the G channel medical image subjected to color space confusion, the B channel medical image subjected to color space confusion and the R channel medical image subjected to color space confusion is the same as the process of obtaining the G channel medical image subjected to color space confusion.
In some embodiments, separately color-space obfuscating each of the numerical-domain-reformatted single-channel medical images includes: acquiring random numbers with the same number as the single-channel medical images; calculating a scalar quantity corresponding to each single-channel medical image subjected to numerical value domain reforming based on a preset feature vector, a preset feature value and an acquired random number, wherein the preset feature vector and the preset feature value are obtained by extracting feature vectors and feature values of the medical image used for feature extraction, and the medical image used for feature extraction is selected from all training medical images; for each numerical-domain-reformed single-channel medical image, the pixel value of each pixel in the numerical-domain-reformed single-channel medical image is respectively added to the scalar corresponding to the numerical-domain-reformed single-channel medical image to obtain a color-space-obfuscated pixel value of each pixel in the numerical-domain-reformed single-channel medical image.
In the present application, before performing domain unification processing on each training medical image, one training medical image may be selected from all the training medical images as a medical image for feature extraction. Then, feature vector extraction and feature value extraction are carried out on the medical image for feature extraction, and a preset feature vector and a preset feature value are obtained.
The size of the medical image used to extract the features is (n, k, 3). n, k refer to the height and width of the medical image. And 3 represents the number of single channels. The medical images with the size of (n, k, 3) for extracting the features are integrated according to each channel to obtain a two-dimensional matrix with the size of (n x k, 3). Each column in the two-dimensional matrix is a pixel value for all pixels, i.e., n x k pixels, in a single-channel medical image of the medical image used to extract the features.
And when the medical image for extracting the features is subjected to feature vector extraction and feature value extraction, calculating a covariance matrix. Each column in the two-dimensional matrix corresponds to X in the covariance formula, where Xi is (n.k, 1) and i is {1, 2, 3 }. The covariance cov (xi, xj) of any two columns is calculated. cov (xi, xj) is a cross-covariance matrix calculation for xi and xj. Thus, 9 covariances can be obtained. Finally, a 3 x 3 covariance matrix can be obtained. Then, the eigenvector and eigenvalue of the 3 x 3 covariance matrix are calculated respectively, the eigenvector is used as the preset eigenvector, and the eigenvalue is used as the preset eigenvalue.
In the present application, neither step changes the size of the medical image. Thus, each numerical domain reformed single channel medical image includes n x k pixels each.
When color space aliasing is performed on each single-channel medical image subjected to numerical value domain reforming, the same number of random numbers as that of the single-channel medical images, that is, 3 random numbers, may be acquired first. For example, 3 random numbers can be generated by a mean distribution of-50 to 50. The values of the generated 3 random numbers are all between-50 and 50.
Then, a scalar quantity corresponding to each single-channel medical image subjected to numerical field reforming can be calculated based on the preset feature vector, the preset feature value and the acquired random number.
Can calculate [ P1, P2, P3][α1λ1，α2λ2，α3λ3]T. Wherein, [ p1, p2, p3]An eigenvector, [ alpha 1, alpha 2, alpha 3 ], representing a 3 x 3 covariance matrix]The eigenvalues of the covariance matrix are represented, and λ 1, λ 2, λ 3 represent 3 random numbers.
[P1，P2，P3]Is a 3 x 3 matrix, and the size of Pi is 3 x 1, [ alpha 1 lambda 1, [ alpha 2 lambda 2, [ alpha 3 lambda 3 ]]Is a 1 x 3 matrix with the rank of [ alpha 1 lambda 1, alpha 2 lambda 2, alpha 3 lambda 3]TIs 3X 1.
[P1，P2，P3][α1λ1，α2λ2，α3λ3]TOf (2) outputThe matrix with the size of 3 x 1, namely 3 rows and 1 column outputs 3 scalars in total. And each single-channel medical image subjected to numerical value field reforming corresponds to a scalar. For example, the scalar of row 1 in a matrix of 3 rows and 1 column may be taken as the scalar corresponding to the R-channel medical image that has undergone numerical field reformatting. The scalar of row 2 in the matrix of row 3 and column 1 may be taken as the scalar corresponding to the G-channel medical image that has undergone numerical field reformatting. The scalar of row 3 in the matrix of row 3 and column 1 may be used as the scalar corresponding to the B-channel medical image that has undergone numerical field reformatting.
For each numerical-domain reformed single-channel medical image, the pixel value of each pixel in the numerical-domain reformed single-channel medical image is added to the scalar corresponding to the numerical-domain reformed single-channel medical image to obtain a color-space-aliased pixel value for each pixel in the numerical-domain reformed single-channel medical image.
In the color-space-obfuscated R-channel medical image, the pixel value of each pixel is a color-space-obfuscated pixel value obtained by adding the pixel value subjected to numerical field reforming to a scalar corresponding to the R-channel medical image.
In the color-space-obfuscated G-channel medical image, the pixel value of each pixel is a color-space-obfuscated pixel value obtained by adding the pixel value subjected to numerical field reforming to a scalar corresponding to the G-channel medical image.
In the color-space-obfuscated B-channel medical image, the pixel value of each pixel is a color-space-obfuscated pixel value obtained by adding the pixel value subjected to numerical field reforming to a scalar corresponding to the B-channel medical image.
For each numerical-field-reformed single-channel medical image, the pixel value of each pixel of the numerical-field-reformed single-channel medical image is set to the color-space-obfuscated pixel value of each pixel. Thus, each single-channel medical image subjected to color space aliasing is obtained.
In some embodiments, when color space aliasing is performed on each single-channel medical image subjected to numerical domain reformation separately, the 3 random numbers obtained are random numbers that follow a normal distribution with a mean value of 0 and a standard deviation of σ.
In the present application, in the process of performing domain unification processing on one training medical image each time, when color space aliasing is performed on each single-channel medical image subjected to numerical domain reformation, respectively, random numbers can be generated through normal distribution with a mean value of 0 and a standard deviation of σ.
In some embodiments, σ can range from 0.1 to 2. The random number may be generated by a normal distribution with a mean of 0 and a standard deviation of one sigma between 0.1 and 2.
In the application, in the process of performing domain unified processing on a training medical image, after each single-channel medical image subjected to color space confusion is obtained, normalization processing can be performed on each single-channel medical image subjected to color space confusion respectively. Therefore, a single-channel medical image processed by each domain in a unified way is finally obtained.
The training medical images subjected to domain unified processing are obtained by splicing each single-channel medical image subjected to domain unified processing.
Specifically, the R-channel medical image subjected to color space confusion is subjected to normalization processing to obtain an R-channel medical image subjected to domain unification processing. In the R-channel medical image subjected to the domain unification processing, the pixel value of each pixel is the pixel value subjected to the normalization processing.
And carrying out normalization processing on the G channel medical image subjected to color space confusion to obtain the G channel medical image subjected to domain unification processing. In the G-channel medical image subjected to the domain unification processing, the pixel value of each pixel is the pixel value subjected to the normalization processing.
And carrying out normalization processing on the B channel medical image subjected to color space confusion to obtain the B channel medical image subjected to domain unified processing. In the B-channel medical image subjected to the domain unification processing, the pixel value of each pixel is the pixel value subjected to the normalization processing.
In some embodiments, the normalizing each color-space-aliased single-channel medical image comprises: for each pixel in the single-channel medical image subjected to color space aliasing, dividing a first difference value corresponding to the pixel by a second difference value corresponding to the pixel to obtain a normalized pixel value of the pixel, wherein the first difference value corresponding to the pixel is a minimum value obtained by subtracting pixel values of all pixels of the single-channel medical image subjected to color space aliasing from a pixel value of the pixel, and the second difference value corresponding to the pixel is a maximum value obtained by subtracting the minimum value from pixel values of all pixels of the single-channel medical image subjected to color space aliasing.
In the application, when each single-channel medical image subjected to color space confusion is subjected to normalization processing respectively, the single-channel medical image subjected to color space confusion can be subjected to normalization processing respectively by adopting a range transform method, so that a pixel value of a pixel in each single-channel medical image subjected to color space confusion, which is subjected to normalization processing, is obtained.
The normalized pixel value of a pixel in a color-space-aliased single-channel medical image may be expressed as: (g ') - (g ' min)/(g ' max) - (g ' min), wherein (g ') may represent a pixel value of one pixel in the color-space-aliased single-channel medical image, (g ' min) represents a minimum value among pixel values of all pixels of the color-space-aliased single-channel medical image, and (g ' max) may represent a maximum value among pixel values of all pixels of the color-space-aliased single-channel medical image. (g ') - (g' min) may represent a first difference value corresponding to the pixel, and (g 'max) - (g' min) may represent a second difference value corresponding to the pixel.
Specifically, for each pixel in the color space obfuscated R-channel medical image, a first difference value corresponding to the pixel is divided by a second difference value corresponding to the pixel to obtain a normalized pixel value of the pixel, where the first difference value corresponding to the pixel is a minimum value of the pixel minus pixel values of all pixels of the color space obfuscated R-channel medical image, and the second difference value corresponding to the pixel is a maximum value of the pixel values of all pixels of the color space obfuscated R-channel medical image minus the minimum value. Thereby, a normalized pixel value of each pixel in the color space obfuscated R-channel medical image is obtained, and the pixel value of each pixel in the color space obfuscated R-channel medical image is set as the normalized pixel value, so that the domain-unified R-channel medical image is obtained.
The process of obtaining the normalized pixel value of each pixel in the color space-confused G-channel medical image and the normalized pixel value of each pixel in the color space-confused B-channel medical image by the range transform method is the same as the process of obtaining the normalized pixel value of each pixel in the color space-confused R-channel medical image.
And splicing the R channel medical image subjected to domain unification processing, the G channel medical image subjected to domain unification processing and the B channel medical image subjected to domain unification processing to obtain the training medical image subjected to domain unification processing.
And 102, training a preset machine learning model by using the training medical image subjected to domain unified processing.
In the application, after the training medical images subjected to the domain unification processing are obtained, the preset machine learning model can be trained by using the training medical images subjected to the domain unification processing.
In the present application, for each of a large number of training medical images in a training set, a domain unification process may be performed in step 101 to obtain a corresponding domain unification processed medical image.
Each time, a medical image processed by the domain unification can be used for training the preset machine learning model. During each training, a medical image subjected to domain unified processing is input into a preset machine learning model.
For example, the preset machine learning model may be a convolutional neural network, and when the input is performed in the training process, the R-channel medical image subjected to domain unification processing, the G-channel medical image subjected to domain unification processing, and the B-channel medical image subjected to domain unification processing of the medical images subjected to domain unification processing may be input into the convolutional neural network in parallel. And calculating the loss between the predicted processing result and the labeling result through a loss function, and updating the parameters of the machine learning model according to the loss.
Fig. 2 shows a flowchart of a medical image processing method provided in an embodiment of the present application, where the method includes:
In the present application, the medical image to be processed may be a fundus medical image.
In the present application, a preset machine learning model is trained in advance by training medical images subjected to domain unification processing.
The pretreatment comprises the following steps: carrying out mean value reduction processing on each single-channel medical image of the medical image to be processed respectively; respectively carrying out numerical value domain reforming on each single-channel medical image subjected to mean value reduction processing in the medical images to be processed; and respectively carrying out normalization processing on each single-channel medical image subjected to numerical value domain reforming in the medical image to be processed.
The manner of performing the mean-reducing processing on each single-channel medical image of the medical image to be processed may refer to the manner of performing the preset processing operation including the mean-reducing processing on each single-channel medical image of the training medical image to obtain each single-channel medical image of the training medical image that is subjected to the preset processing operation in the above embodiment.
The method for performing numerical domain reformation on each single-channel medical image subjected to the mean value reduction processing on the medical image to be processed to obtain each single-channel medical image subjected to the numerical domain reformation of the medical image to be processed may refer to the method for performing numerical domain reformation on each single-channel medical image subjected to the preset processing operation on each training medical image to obtain each single-channel medical image subjected to the numerical domain reformation in the above embodiment.
After normalization processing is respectively carried out on each single-channel medical image subjected to numerical value domain reforming of the medical image to be processed, each single-channel medical image subjected to preprocessing of the medical image to be processed can be obtained. Each of the preprocessed single-channel medical images of the medical images to be processed may be stitched into a preprocessed medical image.
In the present application, the preset machine learning model may be a classification model, a target detection model, or the like. For example, the preset machine learning model may be a convolutional neural network for classification, a convolutional neural network for target detection, or the like.
In the application, before the preset machine learning model is used for performing corresponding processing on the medical image to be processed, the preset machine learning model is trained in advance through the training medical image subjected to domain unified processing. The process described in the above embodiment can be referred to as a process of performing domain unification processing on each training medical image in advance, and using each training medical image subjected to domain unification processing to preset a machine learning model.
When the medical image to be processed is processed by the preset machine learning model, the medical image to be processed is input into the preset machine learning model, and a processing result output by the preset machine learning model is obtained.
In the present application, a domain unification process is performed on each training medical image, and color space aliasing in the domain unification process may perform disturbance of the entire color for the training medical image.
Therefore, when a large number of training medical images subjected to domain unified processing are used for training a preset machine learning model in advance, the machine learning model is insensitive to the features related to the whole color, and accordingly the machine learning model cannot learn the features related to the whole color as the features with the discrimination. And the generalization of the machine learning model is improved.
After training, when the medical image to be processed is processed, the performance of the machine learning model on the medical image in the unseen domain is greatly improved.
The embodiment of the application also provides the electronic equipment. The electronic device includes one or more processors and memory resources, represented by memory, for storing instructions, such as application programs, that are executable by the processing components. The application program stored in the memory may include one or more modules that each correspond to a set of instructions. Further, the processing component is configured to execute the instructions to perform the model training method described above.
The embodiment of the application also provides the electronic equipment. The electronic device includes one or more processors and memory resources, represented by memory, for storing instructions, such as application programs, that are executable by the processing components. The application program stored in the memory may include one or more modules that each correspond to a set of instructions. Furthermore, the processing component is configured to execute the instructions to perform the medical image processing method described above.
Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.
It will be understood that the present application is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.
Claims (9)
1. A method of model training, the method comprising:
carrying out domain unified processing on the training medical images to obtain the training medical images subjected to the domain unified processing, wherein the domain unified processing on the training medical images comprises the following steps: acquiring each single-channel medical image of the training medical images;
and respectively carrying out preset processing operation on each single-channel medical image, wherein the preset processing operation comprises the following steps: carrying out mean value reduction treatment;
respectively carrying out numerical value domain reforming on each single-channel medical image subjected to the preset processing operation;
color space aliasing is respectively carried out on each single-channel medical image subjected to numerical value domain reforming, and the method comprises the following steps: acquiring the same number of random numbers as the number of the single-channel medical images; calculating a scalar quantity corresponding to each single-channel medical image subjected to numerical value domain reforming based on a preset feature vector, a preset feature value and an acquired random number, wherein the preset feature vector and the preset feature value are obtained by performing feature vector extraction and feature value extraction on the medical image used for feature extraction, and the medical image used for feature extraction is selected from all training medical images; for each numerical-domain-reformed single-channel medical image, adding the pixel value of each pixel in the numerical-domain-reformed single-channel medical image to the scalar corresponding to the numerical-domain-reformed single-channel medical image to obtain a color-space-obfuscated pixel value of each pixel in the numerical-domain-reformed single-channel medical image;
respectively carrying out normalization processing on each single-channel medical image subjected to color space confusion;
and training the preset machine learning model by using the training medical image subjected to domain unified processing.
2. The method of claim 1, wherein the pre-processing operation further comprises: and a fuzzy operation of a preset step size is carried out after the average value reduction processing.
3. The method according to claim 1, wherein the obtained random numbers are random numbers that follow a normal distribution with a mean of 0 and a standard deviation of σ.
4. The method of claim 3, wherein σ ranges from 0.1 to 2.
5. The method of claim 1, wherein separately normalizing each color-space aliased single-channel medical image comprises:
for each pixel in the single-channel medical image subjected to color space aliasing, dividing a first difference value corresponding to the pixel by a second difference value corresponding to the pixel to obtain a normalized pixel value of the pixel, wherein the first difference value corresponding to the pixel is a minimum value obtained by subtracting pixel values of all pixels of the single-channel medical image subjected to color space aliasing from a pixel value of the pixel, and the second difference value corresponding to the pixel is a maximum value obtained by subtracting the minimum value from pixel values of all pixels of the single-channel medical image subjected to color space aliasing.
6. The method of any one of claims 1-5, wherein the training medical image is a fundus medical image.
7. A medical image processing method, characterized in that the method comprises:
acquiring a medical image to be processed;
pre-processing a medical image to be processed to obtain a pre-processed medical image, and processing the pre-processed medical image by using a preset machine learning model to obtain a processing result, wherein the preset machine learning model is trained according to the method of any one of claims 1 to 6 in advance, and the pre-processing comprises: carrying out mean value reduction processing on each single-channel medical image of the medical images to be processed respectively; performing numerical domain reformation on each single-channel medical image subjected to mean value reduction processing in the medical images to be processed respectively; and respectively carrying out normalization processing on each single-channel medical image subjected to numerical value domain reforming in the medical image to be processed.
8. An electronic device, characterized in that the electronic device comprises:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the method of any one of claims 1 to 6.
9. An electronic device, characterized in that the electronic device comprises:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the method of claim 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010648720.4A CN111951217B (en) | 2020-07-07 | 2020-07-07 | Model training method, medical image processing method and electronic device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010648720.4A CN111951217B (en) | 2020-07-07 | 2020-07-07 | Model training method, medical image processing method and electronic device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111951217A CN111951217A (en) | 2020-11-17 |
| CN111951217B true CN111951217B (en) | 2022-01-14 |
Family
ID=73340486
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010648720.4A Active CN111951217B (en) | 2020-07-07 | 2020-07-07 | Model training method, medical image processing method and electronic device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111951217B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112966620A (en) * | 2021-03-15 | 2021-06-15 | 北京鹰瞳科技发展股份有限公司 | Fundus image processing method, model training method and equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107657237A (en) * | 2017-09-28 | 2018-02-02 | 东南大学 | Car crass detection method and system based on deep learning |
| CN109377462A (en) * | 2018-10-23 | 2019-02-22 | 上海鹰瞳医疗科技有限公司 | Method for processing fundus images and equipment |
| CN111091132A (en) * | 2020-03-19 | 2020-05-01 | 腾讯科技（深圳）有限公司 | Image recognition method and device based on artificial intelligence, computer equipment and medium |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9836839B2 (en) * | 2015-05-28 | 2017-12-05 | Tokitae Llc | Image analysis systems and related methods |
| US11080850B2 (en) * | 2018-01-16 | 2021-08-03 | Electronics And Telecommunications Research Institute | Glaucoma diagnosis method using fundus image and apparatus for the same |
| US10373291B1 (en) * | 2018-01-31 | 2019-08-06 | Google Llc | Image transformation for machine learning |
| CN110633758A (en) * | 2019-09-20 | 2019-12-31 | 四川长虹电器股份有限公司 | Method for detecting and locating cancer region aiming at small sample or sample unbalance |
| CN111209423B (en) * | 2020-01-07 | 2023-04-07 | 腾讯科技（深圳）有限公司 | Image management method and device based on electronic album and storage medium |
| CN111259968A (en) * | 2020-01-17 | 2020-06-09 | 腾讯科技（深圳）有限公司 | Illegal image recognition method, device, equipment and computer readable storage medium |
-
2020
- 2020-07-07 CN CN202010648720.4A patent/CN111951217B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107657237A (en) * | 2017-09-28 | 2018-02-02 | 东南大学 | Car crass detection method and system based on deep learning |
| CN109377462A (en) * | 2018-10-23 | 2019-02-22 | 上海鹰瞳医疗科技有限公司 | Method for processing fundus images and equipment |
| CN111091132A (en) * | 2020-03-19 | 2020-05-01 | 腾讯科技（深圳）有限公司 | Image recognition method and device based on artificial intelligence, computer equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111951217A (en) | 2020-11-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Hu et al. | Fc4: Fully convolutional color constancy with confidence-weighted pooling | |
| CN110046673B (en) | No-reference tone mapping image quality evaluation method based on multi-feature fusion | |
| Gao et al. | Efficient color constancy with local surface reflectance statistics | |
| Bianco et al. | Color constancy using CNNs | |
| Jiang et al. | A switched view of Retinex: Deep self-regularized low-light image enhancement | |
| CN107851193B (en) | Hybrid machine learning system | |
| Gu et al. | No-reference quality metric of contrast-distorted images based on information maximization | |
| CN108629306B (en) | Human body posture recognition method and device, electronic equipment and storage medium | |
| Zhang et al. | Rellie: Deep reinforcement learning for customized low-light image enhancement | |
| Qian et al. | Revisiting gray pixel for statistical illumination estimation | |
| BR102017026341A2 (en) | image processing system, learning method for main neural network and preprocessing of a system, image processing method and biometric identification method | |
| Isa | Pixel distribution shifting color correction for digital color images | |
| WO2019102608A1 (en) | Image processing device, image processing method, and image processing program | |
| CN109498037B (en) | Brain cognitive measurement method based on deep learning feature extraction and multiple dimensionality reduction algorithms | |
| CN111951217B (en) | Model training method, medical image processing method and electronic device | |
| Chin et al. | Facial skin image classification system using Convolutional Neural Networks deep learning algorithm | |
| CN111144413B (en) | Iris positioning method and computer readable storage medium | |
| Meylan | Tone mapping for high dynamic range images | |
| Guo et al. | Multi-scale feature guided low-light image enhancement | |
| KR102669584B1 (en) | Method and device for detecting animol biometric information | |
| Park et al. | Multiple transformation function estimation for image enhancement | |
| Poostchi et al. | Diabetic retinopathy dark lesion detection: preprocessing phase | |
| Ma et al. | Deep attention network for melanoma detection improved by color constancy | |
| Batard et al. | Variational models for color image correction inspired by visual perception and neuroscience | |
| Marco‐Detchart et al. | Sliding window based adaptative fuzzy measure for edge detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20210721 Address after: 100048 room 21, 4th floor, building 2, national defense science and Technology Park, Haidian District, Beijing Applicant after: Beijing Yingtong Technology Development Co.,Ltd. Applicant after: SHANGHAI YINGTONG MEDICAL TECHNOLOGY Co.,Ltd. Address before: 200011 room 01, 8th floor, building 1, No. 180, Yizhou Road, Xuhui District, Shanghai Applicant before: SHANGHAI YINGTONG MEDICAL TECHNOLOGY Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |