CN108185984A - The method that eyeground color picture carries out eyeground lesion identification - Google Patents

Abstract

The invention discloses a method for identifying fundus lesions in fundus color photographs. The automatic positioning and measurement of various parts in the fundus color photos can achieve the effect of disease pre-screening, automatically screen out pictures with suspected lesions, and quickly and effectively locate and identify hemorrhage, exudation and microvascular tumors, so as to assist in the diagnosis of diabetes Retinopathy and other diseases, reducing the workload of doctors; and the results do not depend on the doctor's experience, more objective, can effectively assist doctors in the diagnosis of diseases, and achieve the purpose of remote consultation.

Description

Method for identifying fundus lesions in fundus color photographs

technical field

The invention relates to a method for identifying fundus lesions in fundus color photographs.

Background technique

In order to quickly identify and draw the boundaries of common fundus lesions in fundus color photos in large quantities, it can quickly and effectively locate and identify hemorrhage, exudation and microvascular tumors, so as to assist in the diagnosis of diabetic retinopathy and other diseases. In clinical practice, due to the limited manpower of ophthalmologists in remote mountainous areas, grass-roots hospitals, and fundus-related film readers, such as mechanically reviewing a large number of fundus color photos one by one, the work content is heavy, single and repetitive, and the efficiency is not high. , wasting a lot of valuable human resources. The existing fundus color photo automatic recognition system also involves the automatic recognition and partition method of the fundus image, but it does not perform precise positioning of the fundus structure position and lesion identification. In addition, most of the existing methods use standard pictures on the Internet for reference comparison and identification. However, the color fundus pictures in clinical practice are not standard pictures, and even include many pictures with problems with focus and brightness. At present, many systems directly use the pictures Recognition, regardless of the quality of the picture, because the standard library on the Internet is all pictures of better quality, and the number is very small.

Contents of the invention

The primary purpose of the present invention is to provide a method for identifying fundus lesions in color fundus photos. The automatic positioning and measurement of various parts in the fundus color photos can achieve the effect of disease pre-screening, automatically screen out pictures with suspected lesions, and can quickly and effectively locate and identify hemorrhage, exudation and microvascular tumors, so as to assist in the diagnosis of diabetes and other diseases, reducing the workload of doctors; and the results do not depend on the experience of doctors, are more objective, and can effectively assist doctors in the diagnosis of diseases and achieve the purpose of remote consultation.

In order to solve the problems of the technologies described above, the technical solution adopted in the present invention is:

The method for identifying fundus lesions provided by the color fundus photos provided by the present invention has the following characteristics:

1. By combining the morphological method with the machine learning method, the localized area is preliminarily processed by the morphological method, then predicted by the machine learning method, and finally accurately positioned by the morphological method;

2. Automatically identify a large number of fundus color photos collected by hospitals or communities to assist doctors in diagnosing a large number of diabetic retinopathy screening and routine fundus color photos in physical examination centers, so as to assist in the diagnosis of diabetes and other diseases;

3. Using a large number of real fundus color photos to enable the model to fully learn the characteristics of image data in various situations, making the judgment more accurate and having better fault tolerance. At the same time, the system first performs preprocessing of the fundus color photos and identification of image quality. In order to ensure that the system can also have a good recognition ability and universal applicability even when the picture quality is uneven.

Description of drawings

The accompanying drawings constituting a part of this application are used to provide further understanding of the present invention, and the schematic embodiments and descriptions of the present invention are used to explain the present invention, and do not constitute an improper limitation of the present invention. In the attached picture:

Fig. 1 is the image example that the embodiment of the present invention selects;

Fig. 2 is the gray scale distribution of the selected image of the embodiment of the present invention;

FIG. 3 is a schematic diagram of a rough outline of a blood vessel during blood vessel identification according to an embodiment of the present invention;

4 is a schematic diagram of a blood vessel outline after deburring treatment during blood vessel identification according to an embodiment of the present invention;

Fig. 5 is a schematic diagram of breakpoint fitting of a schematic diagram of a blood vessel contour during blood vessel identification according to an embodiment of the present invention;

Fig. 6 is a schematic diagram of determining a blood vessel boundary during blood vessel identification according to an embodiment of the present invention;

Fig. 7 is an effect diagram of blood vessel recognition according to the embodiment of the present invention;

Fig. 8 is an image of the detail part of the green track when identifying a microvascular tumor according to the embodiment of the present invention;

Fig. 9 is an effect diagram of bleeding results according to the embodiment of the present invention;

Fig. 10 and Fig. 12 are exudation effect diagrams of the embodiment of the present invention;

Fig. 11 is the original picture corresponding to the embodiment of the present invention;

Fig. 13 is an image of candidate points of a microvascular tumor according to an embodiment of the present invention.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

Example

A method for identifying fundus lesions in color fundus photos, comprising the following steps:

Image quality inspection:

1. Feature extraction:

Using the skeleton of the image, extract its texture features, RGB, three layers, each layer extracts 15 features.

First use the canny operator to detect the edge of the image, and then use the median filter to denoise, and then use the preprocessed image to calculate the number of total pixels on the edge, the total perimeter of the edge, the maximum height of the edge area, Maximum width, number of chain codes for odd chains (number of points with discontinuous edges), target area, rectangularity, elongation.

Then extract the seven invariant moment features of the image:

The sum of horizontal and vertical directed variance, more distributed towards horizontal and vertical axes, the values are enlarged.

The covariance value of vertical and horizontal axes when the variance intensity of vertical and horizontal axes were similar.

The result emphasizing the values inclined to left/right and upper/lower axes.

The result emphasizing the values counterbalancing to left/right and upper/lower axes.

The extraction of values invariant against size, rotation, and location.

According to the judgment of sharpness, RGB, each layer extracts 5 features for judging sharpness.

a) Gray entropy:

It reflects the average amount of information in the image. The one-dimensional entropy of the image represents the amount of information contained in the aggregation features of the grayscale distribution in the image, and p _i represents the proportion of pixels with a grayscale value i in the image, then the unary grayscale entropy of the grayscale image is defined as:

b) Brenner gradient function

The Brenner gradient function is the simplest gradient evaluation function. It simply calculates the square of the gray difference between two adjacent pixels. The function is defined as follows:

f(x, y) represents the gray value of the pixel point (x, y) corresponding to image f.

c) variance function

Because sharply focused images have larger grayscale differences than blurred images, the variance function can be used as an evaluation function:

in, is the average gray value of the entire image, the function is sensitive to noise, the purer the image, the smaller the function value.

d) Energy gradient function

e) Gradient function

3. Gray histogram with 256 features.

Now convert RGB into a grayscale image, Gray=0.29900*R+0.58700*G+0.11400*B, and then use the grayscale histogram to extract features.

The gray level histogram is a function of the gray level distribution, which is the statistics of the gray level distribution in the image. The gray histogram is to count all the pixels in the digital image according to the size of the gray value and count the frequency of occurrence. The frequency of 0-255 gray values, a total of 256 features are extracted.

4. Transform the RGB space and extract 256 color and texture features.

Using the paper "Color and texture descriptors" to convert the original RGB space to HSV space, calculate the color histogram, and get 256 features.

So far, all the required features have been extracted, and then use all the extracted features as independent variables, and the quality of the picture as the dependent variable (0 or 1) to predict the picture quality. Here we use a random forest model for prediction.

1. Machine learning for prediction:

Random Forest Algorithm:

1. Given a training data set d=(X, y), where X is the extracted feature, and y is a categorical variable of 0 and 1 (0 means poor picture quality, 1 means good picture quality). Fixed m≤p (m is the number of randomly extracted features, p is the total number of features) and the number B of trees (decision tree algorithm).

2. For each b=1, 2, ..., B, do the following steps:

a) Construct a bootstrap training set by randomly sampling n times from n samples for training data d

b) use Construct a tree of maximum depth from the data in Randomly select m from p variables for splitting;

c) Store tree and bootstrap sample information.

3. For any prediction point x ₀ , perform random forest fitting and prediction. for each tree A category will be predicted, so since there are B trees, B 01 categories can be predicted. The final prediction result is the category with the most occurrences (0 or 1) among the B categories.

Since there are a large number of poor-quality pictures in real pictures, these pictures with poor quality are first screened out through the picture quality inspection, and only the fundus images that pass the picture quality are subjected to subsequent processing.

Image preprocessing:

Preprocessing uses histogram equalization. Firstly, select one of the images with the best recognition effect, as shown in Figure 1, and extract the gray distribution of its RGB three tracks, as shown in Figure 2. Make it a standard graph.

Video disc recognition:

Optic disc recognition is mainly divided into three main steps: initial positioning (ROI extraction), precise positioning, and smooth fitting

Preliminary positioning: First, based on the high brightness of the optic disc, which is most obvious on the red track, we first select the red track for analysis. Specifically, both the red and green tracks can identify the position of the optic disc with the naked eye, and the red track is more obvious. The region of interest (ROI) extraction mainly utilizes the method of adaptive threshold segmentation. Firstly, the brighter area of the whole picture is extracted by threshold segmentation, and the rest of the darker area is filled with the mean value. The modified image is thresholded again. Through multiple iterations, the area of the brighter area is reduced step by step. When the ROI area exceeds a predetermined threshold, the iteration is stopped. Then filter the extracted highlighted regions. Then extract the center of the ROI and intercept the ROI for further analysis. In the method of determining the ROI position, the current popular method is similar to the simple threshold cutting method: Optic cup and disclocalization for Detection of glaucoma using Matlab, Hanamant M.Havagondi, 2Mahesh S.Kumbhar. Kaiser Window positioning method: Blood vessel inpainting based technique for efficient localization and segmentation of optic disc in digital fundus images, Biomedical Signal Processing and Control 25(2016) 108–117 et al. Compared with other methods, our advantage lies in: only using the red track information, the influence of blood vessels is relatively small. For photos with overexposed borders. The method used in the present invention can quickly remove this part of the influence without causing trouble to ROI extraction. However, for some images with too many highlighted areas (or high-level lesions and insufficient brightness of the optic disc), the quality of these images is not very high, and the image program will automatically prompt quality problems without subsequent analysis.

Precise positioning and smooth fitting:

In the extracted ROI, the morphological processing method is mainly used to remove the noise effect first, and then the image is thresholded to obtain a relatively rough boundary position. Then ellipse fitting (minimum circumscribed ellipse) is performed on the boundary position, and a boundary parameter equation is fitted

x=a*cos(t)*cos(θ)-b*sin(t)*sin(θ)+x ₀

y=a*cos(t)*cos(θ)-b*sin(t)*sin(θ)+y ₀

Where θ is the inclination angle of the ellipse, a and b are the semi-major and minor axes, t is the parameter, x ₀ and y ₀ are the coordinates of the center of the ellipse. Finally, the boundary equations are drawn on the original graph. At present, there are mainly algorithms such as fixed threshold segmentation and region growing for optic disc boundary positioning. The fixed threshold has the worst segmentation stability and inaccurate boundary recognition, while our adaptive threshold method automatically selects the optimal threshold according to the optic disc area, which will not cause obvious misjudgment of the optic disc boundary. The region growing algorithm has certain requirements for the selection of initial seed points, and may cause the problem that the recognition area is too small.

Vessel Identification:

One, read in the picture.

Second, the first processing done on the image is to remove the black border around the circular fundus image.

Three, image processing

The image is processed to obtain a rough outline of the blood vessel. First, preprocessing to remove noise, followed by median filtering and threshold denoising, the results are shown in Figure 3;

Then, perform multiple corrosion operations on the pre-processed blood vessels to obtain the approximate distribution range of the blood vessels. At this time, due to the shooting angle, light and other reasons, some of the pictures may locally appear similar to dark spots in the picture quality, so The picture obtained in this step may have broken blood vessels at a certain place, so then pixel linking and diagonal filling are performed on the obtained discontinuous broken blood vessels to obtain continuous blood vessels. Then, deburring is performed on the obtained rough blood vessel to obtain the rough outline of the blood vessel, and the result is shown in Fig. 4 .

4. Vascular Identification

Then blood vessel identification is to treat each segment of blood vessel as an individual and identify each segment of blood vessel.

The main approach is to first find the centerline of the vessel. Because the initial identification of the outline of the blood vessel is not accurate, but the relatively accurate centerline of the blood vessel can be found through the corrosion operation. This step has already separated the segments of the blood vessels, except for the connection points of the blood vessels, each segment is a continuous line. Then use the algorithm to fit the breakpoints, as shown in Figure 5.

After the centerline is selected, use the outward gray gradient changes on both sides of the centerline to determine the boundaries of blood vessels, as shown in Figure 6.

Finally, draw the identified blood vessels on the original picture, as shown in Figure 7.

Hemorrhage recognition:

Hemorrhage and exudation are also trained using fundus images and their corresponding images that only contain marked areas of hemorrhage and exudation.

The classifier used here is Adaboost. The feature selection during training is similar to the feature selection during arteriovenous training, and the training process and the process of obtaining a classifier are also the same.

One, the first step in the recognition process is to remove the black frame on the outermost side of the picture.

Second, read the features of the picture, where the features are the same as those used to train the classifier.

Third, first use the morphological method to select a rough candidate area.

Here, the approximate area is first determined simply by grayscale, and then the selected area is morphologically processed to remove noise.

Fourth, use the trained classifier to classify. The results were obtained, wherein the bleeding results are shown in Figure 9 and Figure 12, and the exudation results are shown in Figure 10 and Figure 12. (Green in Figure 12 indicates hemorrhage, and blue indicates exudation. The corresponding original picture is Figure 11)

Microangioma identification:

First extract the green track of the picture to be processed, and then use median filtering on the green track, use the filtered image as the background, and then subtract the gray level of the original image from the gray level of the background to obtain an image that highlights the details of the green track , as shown in Figure 8.

Then, using the maximum mutual information method, the calculation formula is as follows:

p in the formula is the correlation matrix of 256 gray values of each image. The stronger the correlation, the greater the probability that two grayscales are adjacent in the image.

Adaptively select the threshold value in Figure 8 for binarization, then use the original image to extract the blood vessel area, remove the blood vessel area and the area serialized with the blood vessel, and obtain all candidate point images of microvascular tumors, as shown in Figure 13.

All candidate points are regarded as a suspected microvascular tumor area, and feature extraction is performed centering on the center of gravity of the candidate points. The features include the mean, standard deviation, minimum and maximum value of the red track of the small picture, the mean, standard deviation, minimum and maximum value of the green track, and the mean, standard deviation, minimum and maximum value of the small picture mapped to the green track from 0-256, connected The area of the domain, the length and width of the smallest circumscribed rectangle of the connected domain, the entropy of the image, the gradient of the image, the center of the image and the average gray level of the four regions in the corner. Export all these features, and mark each sample based on the mark.

Divide all the data into pictures, and use the SMOTE method to expand the positive samples of the training set. Finally, the training set is trained using the Xgboost model.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention should be included in the protection of the present invention. within range.

Claims (6)

1. a kind of fundus color photo carries out the method for fundus lesion identification, it is characterized in that comprising the following steps: Image quality detection, input the original image, extract image features, and conduct training, predict the image quality, detect the fundus image with acceptable image quality for subsequent processing; Image preprocessing, select one of the images with the best recognition effect, and extract the gray distribution of its RGB three tracks, and use it as a standard image; Bleeding and oozing recognition, select the area within a certain distance with the optic disc as the center of the circle as the target area, and divide the area into three sub-areas: the outer side of the blood vessel, the border of the blood vessel, and the inner side of the blood vessel according to the location of the blood vessel, with the blood vessel wall as the boundary; Two major features of gray gradient change and texture change are extracted from these three sub-regions respectively; Input the picture when identifying, and then identify the blood vessels in the target area, and then extract the features selected for training on the outside, blood vessel wall, and inside of the blood vessels; Use the trained machine learning classifier to identify, get the result, and display the result on the original picture. 2. the method for carrying out fundus focus identification according to the color fundus photo of claim 1, is characterized in that the extraction of described image features comprises the following steps: First use the canny operator to detect the edge of the image, and then use the median filter to denoise, and then use the preprocessed image to calculate the number of total pixels on the edge, the total perimeter of the edge, the maximum height of the edge area, The maximum width, the number of chain codes of odd chains, the target area, rectangularity, elongation, and then extract the seven invariant moment features of the image; Each layer extracts 5 features for judging clarity: gray entropy, Brenner gradient function, variance function, energy gradient function, gradient function; Use the gray histogram to extract features; Convert the original RGB space to HSV space and calculate the color histogram. 3. the method for carrying out fundus lesion identification according to the color fundus photo of claim 1, it is characterized in that said predicting picture quality comprises the following steps: 1) Given a training data set d=(X,y), where x is the extracted feature, y is 0, 1 is a classification variable, 0 means poor picture quality, 1 means good picture quality, fixed m≤p, m is The number of features extracted randomly, p is the total number of features, and the number B of trees in the decision tree algorithm; 2) For each b=1,2,...,B, do the following steps: a) Construct a bootstrap training set by randomly sampling n times from n samples for training data d b) use Construct a tree of maximum depth from the data in Randomly select m from p variables for splitting; c) store tree and bootstrap sample information; 3) For any prediction point X ₀ , perform random forest fitting and prediction, and for each tree A category will be predicted, so since there are B trees, B 01 categories can be predicted, and the final prediction result is the category with the most occurrences among the B categories. 4. The method for fundus focus identification according to claim 1, further comprising a blood vessel identification step: Image processing, denoising first, followed by median filtering and threshold denoising to obtain a rough outline of blood vessels; Obtain the vessel outline, preprocess the obtained vessel outline, perform multiple erosion operations to obtain the approximate distribution range of the vessel, and then perform pixel linking and diagonal filling on the obtained discontinuous broken vessels to obtain continuous vessels, and then The obtained rough blood vessels are deburred to the outline of the blood vessels; For blood vessel recognition, first find the centerline of the blood vessel, use the outward gray gradient changes on both sides of the centerline to determine the boundary of the blood vessel, and finally draw the recognized blood vessel on the original picture. 5. The method for fundus focus identification according to the color fundus photo of claim 1, is characterized in that it also includes the optic disc identification step: For initial positioning, first extract the brighter area of the entire image by threshold segmentation, and fill the rest of the darker area with the mean value, and then perform threshold segmentation on the modified image again. When the area of the region of interest has reached the predetermined threshold, the iteration is stopped, and then the extracted highlighted region is screened, and the center of the region of interest is extracted and intercepted for further analysis; Precise positioning and smooth fitting, use the morphological processing method to remove the noise first, then threshold the image to obtain a relatively rough boundary position, then perform ellipse fitting on the boundary position, and finally draw the boundary equation on the original image. 6. The method for fundus focus identification according to the color fundus photo of claim 1, is characterized in that it also includes the microvascular tumor identification step: First extract the green track of the picture to be processed, then use the median filter on the green track, use the filtered image as the background, and then subtract the gray level of the original image from the gray level of the background to obtain an image that highlights the details of the green track; Using the maximum mutual information method, the calculation formula is as follows: p in the formula is the correlation matrix of 256 gray values of each image; Adaptively select the threshold value of the image in the detail part of the green track for binarization, then use the original image to extract the blood vessel area, remove the blood vessel area and the area connected with the blood vessel serially, and obtain the candidate point images of all microvascular tumors; All candidate points are regarded as a suspected microvascular tumor area, and feature extraction is performed centering on the center of gravity of the candidate point. The features include the mean, standard deviation, minimum and maximum value of the red track of the small picture, and the mean and standard deviation of the green track. , minimum and maximum value, the mean, standard deviation, minimum and maximum value of the small picture mapped to the green track to 0-256, the area of the connected domain, the length and width of the smallest circumscribed rectangle of the connected domain, the entropy of the image, the gradient of the image, and the center of the image And information such as the average gray level of the four areas in the corner, export all these features, and mark each sample based on the mark; Divide all the data into pictures, use the SMOTE method to expand the positive samples of the training set, and finally use the Xgboost model to train the training set.