CN108230306A - Eyeground color picture blood vessel and arteriovenous recognition methods - Google Patents

Abstract

The invention discloses a kind of eyeground color picture blood vessel and arteriovenous recognition methods.Automatic positioning and measurement to each position in the color picture of eyeground, achieve the effect that disease prescreening, the picture Automatic sieve for having lesion suspicion is selected, can identification accurately and effectively be marked to vessel profile and vascular arteriovenous, then by calculating the diameter ratio of quiet artery, working doctor amount can be reduced with diseases such as auxiliary diagnosis retinal vessels；And its result is independent of doctors experience, more objective, can effectively assist a physician and carry out the diagnosis of disease, realizes the purpose of remote medical consultation with specialists.

Description

Identification method of blood vessels and arteries and veins in fundus color photographs

technical field

The invention relates to a method for identifying blood vessels and arteries and veins in fundus color photographs.

Background technique

In order to identify and draw the boundaries of blood vessels and arteries and veins in fundus color photos quickly and in large quantities, so as to accurately and effectively identify the outline of blood vessels and mark identification with blood vessels, arteries and veins, and then calculate the diameter ratio of veins and arteries to assist in the diagnosis of the retina diseases such as blood vessels. In clinical practice, due to the limited manpower of ophthalmologists in remote mountainous areas, grass-roots hospitals, and fundus-related film readers, such as mechanically reviewing a large number of fundus color photos one by one, the work content is heavy, single and repetitive, and the efficiency is not high. , wasting a lot of valuable human resources. The existing fundus color photo automatic recognition system also involves the automatic recognition and partition method of the fundus image, but it does not perform precise positioning of the fundus structure position and lesion identification. In addition, most of the existing methods use standard pictures on the Internet for reference comparison and identification. However, the color fundus pictures in clinical practice are not standard pictures, and even include many pictures with problems with focus and brightness. At present, many systems directly use the pictures Recognition, regardless of the quality of the picture, because the standard library on the Internet is all pictures of better quality, and the number is very small.

Contents of the invention

The primary purpose of the present invention is to provide a method for identifying blood vessels and arteries and veins in fundus color photographs. The automatic positioning and measurement of each part in the fundus color photo achieves the effect of disease pre-screening, and automatically screens out pictures with suspected lesions, so as to accurately and effectively mark and identify the outline of blood vessels and blood vessels. The diameter ratio of the artery can assist in the diagnosis of diseases such as retinal vessels and reduce the workload of doctors; and the results do not depend on the doctor's experience and are more objective, which can effectively assist doctors in the diagnosis of diseases and achieve the purpose of remote consultation.

In order to solve the problems of the technologies described above, the technical solution adopted in the present invention is:

The method for identifying blood vessels and arteries and veins in fundus color photographs provided by the present invention has the following characteristics:

1. By combining the morphological method with the machine learning method, the localized area is preliminarily processed by the morphological method, then predicted by the machine learning method, and finally accurately positioned by the morphological method;

2. Automatically identify the color fundus photos of a large number of patients collected in hospitals or communities, to assist doctors in the screening of a large number of diseases such as retinal vessels and the diagnosis of routine fundus color photos in physical examination centers;

3. Using a large number of real fundus color photos to enable the model to fully learn the characteristics of image data in various situations, making the judgment more accurate and having better fault tolerance. At the same time, the system first performs preprocessing of the fundus color photos and identification of image quality. In order to ensure that the system can also have a good recognition ability and universal applicability even when the picture quality is uneven.

Description of drawings

The accompanying drawings constituting a part of this application are used to provide further understanding of the present invention, and the schematic embodiments and descriptions of the present invention are used to explain the present invention, and do not constitute an improper limitation of the present invention. In the attached picture:

Fig. 1 is the image example that the embodiment of the present invention selects;

Fig. 2 is the gray scale distribution of the selected image of the embodiment of the present invention;

FIG. 3 is a schematic diagram of a rough outline of a blood vessel during blood vessel identification according to an embodiment of the present invention;

4 is a schematic diagram of a blood vessel outline after deburring treatment during blood vessel identification according to an embodiment of the present invention;

Fig. 5 is a schematic diagram of breakpoint fitting of a schematic diagram of a blood vessel contour during blood vessel identification according to an embodiment of the present invention;

Fig. 6 is a schematic diagram of determining a blood vessel boundary during blood vessel identification according to an embodiment of the present invention;

Fig. 7 is a schematic diagram of the blood vessel recognition effect of the embodiment of the present invention;

Fig. 8 is a schematic diagram of an arteriovenous recognition result of a fundus image according to an embodiment of the present invention.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

Example

A method for identifying blood vessels and arteries and veins in fundus color photos, comprising the following steps:

Image quality inspection:

1. Feature extraction:

Using the skeleton of the image, extract its texture features, RGB, three layers, each layer extracts 15 features.

First use the canny operator to detect the edge of the image, and then use the median filter to denoise, and then use the preprocessed image to calculate the number of total pixels on the edge, the total perimeter of the edge, the maximum height of the edge area, Maximum width, number of chain codes for odd chains (number of points with discontinuous edges), target area, rectangularity, elongation.

Then extract the seven invariant moment features of the image:

The sum of horizontal and vertical directed variance, more distributed towards horizontal and vertical axes, the values are enlarged.

The covariance value of vertical and horizontal axes when the variance intensity of vertical and horizontal axes were similar.

The result emphasizing the values inclined to left/right and upper/lower axes.

The result emphasizing the values counterbalancing to left/right and upper/lower axes.

The extraction of values invariant against size, rotation, and location.

According to the judgment of sharpness, RGB, each layer extracts 5 features for judging sharpness.

a) Gray entropy:

It reflects the average amount of information in the image. The one-dimensional entropy of the image represents the amount of information contained in the aggregation features of the grayscale distribution in the image, and let p _i represent the proportion of pixels with a grayscale value i in the image, then the unary grayscale entropy of the grayscale image is defined as:

b) Brenner gradient function

The Brenner gradient function is the simplest gradient evaluation function. It simply calculates the square of the gray difference between two adjacent pixels. The function is defined as follows:

f(x, y) represents the gray value of the pixel point (x, y) corresponding to image f.

c) variance function

Because sharply focused images have larger grayscale differences than blurred images, the variance function can be used as an evaluation function:

in, is the average gray value of the entire image, the function is sensitive to noise, the purer the image, the smaller the function value.

d) Energy gradient function

e) Gradient function

3. Gray histogram with 256 features.

Now convert RGB into a grayscale image, Gray=0.29900*R+0.58700*G+0.11400*B, and then use the grayscale histogram to extract features.

The gray level histogram is a function of the gray level distribution, which is the statistics of the gray level distribution in the image. The gray histogram is to count all the pixels in the digital image according to the size of the gray value and count the frequency of occurrence. The frequency of 0-255 gray values, a total of 256 features are extracted.

4. Transform the RGB space and extract 256 color and texture features.

Using the paper "Color and texture descriptors" to convert the original RGB space to HSV space, calculate the color histogram, and get 256 features.

So far, all the required features have been extracted, and then use all the extracted features as independent variables, and the quality of the picture as the dependent variable (0 or 1) to predict the picture quality. Here we use a random forest model for prediction.

1. Machine learning for prediction:

Random Forest Algorithm:

1. Given a training data set d=(X, y), where X is the extracted feature, and y is a categorical variable of 0 and 1 (0 means poor picture quality, 1 means good picture quality). Fixed m≤p (m is the number of randomly extracted features, p is the total number of features) and the number B of trees (decision tree algorithm).

2. For each b=1, 2, ..., B, do the following steps:

a) Construct a bootstrap training set by randomly sampling n times from n samples for training data d

b) use Construct a tree of maximum depth from the data in Randomly select m from p variables for splitting;

c) Store tree and bootstrap sample information.

3. For any prediction point x ₀ , perform random forest fitting and prediction. for each tree A category will be predicted, so since there are B trees, B 01 categories can be predicted. The final prediction result is the category with the most occurrences (0 or 1) among the B categories.

Since there are a large number of poor-quality pictures in real pictures, these pictures with poor quality are first screened out through the picture quality inspection, and only the fundus images that pass the picture quality are subjected to subsequent processing.

Image preprocessing:

Preprocessing uses histogram equalization. Firstly, select one of the images with the best recognition effect, as shown in Figure 1, and extract the gray distribution of its RGB three tracks, as shown in Figure 2. Make it a standard graph.

Video disc recognition:

Optic disc recognition is mainly divided into three main steps: initial positioning (ROI extraction), precise positioning, and smooth fitting

Preliminary positioning: First, based on the high brightness of the optic disc, which is most obvious on the red track, we first select the red track for analysis. Specifically, both the red and green tracks can identify the position of the optic disc with the naked eye, and the red track is more obvious. The region of interest (ROI) extraction mainly utilizes the method of adaptive threshold segmentation. Firstly, the brighter area of the whole picture is extracted by threshold segmentation, and the rest of the darker area is filled with the mean value. The modified image is thresholded again. Through multiple iterations, the area of the brighter area is reduced step by step. When the ROI area exceeds a predetermined threshold, the iteration is stopped. Then filter the extracted highlighted regions. Then extract the center of the ROI and intercept the ROI for further analysis. In the method of determining the ROI position, the current popular method is similar to the simple threshold cutting method: Optic cup and disclocalization for Detection of glaucoma using Matlab, Hanamant M.Havagondi, 2Mahesh S.Kumbhar. Kaiser Window positioning method: Blood vessel inpainting based technique for efficient localization and segmentation of optic disc in digital fundus images, Biomedical Signal Processing and Control 25(2016) 108–117 et al. Compared with other methods, our advantage lies in: only using the red track information, the influence of blood vessels is relatively small. For photos with overexposed borders. The method used in the present invention can quickly remove this part of the influence without causing trouble to ROI extraction. However, for some images with too many highlighted areas (or high-level lesions and insufficient brightness of the optic disc), the quality of these images is not very high, and the image program will automatically prompt quality problems without subsequent analysis.

Precise positioning and smooth fitting:

In the extracted ROI, the morphological processing method is mainly used to remove the noise effect first, and then the image is thresholded to obtain a relatively rough boundary position. Then ellipse fitting (minimum circumscribed ellipse) is performed on the boundary position, and a boundary parameter equation is fitted

x=a*cos(t)*cos(θ)-b*sin(t)*sin(θ)+x ₀

y=a*cos(t)*sin(θ)-b*sin(t)*cos(θ)+y ₀

Where θ is the inclination angle of the ellipse, a and b are the semi-major and minor axes, t is the parameter, x ₀ and y ₀ are the coordinates of the center of the ellipse. Finally, the boundary equations are drawn on the original graph. At present, there are mainly algorithms such as fixed threshold segmentation and region growing for optic disc boundary positioning. The fixed threshold has the worst segmentation stability and inaccurate boundary recognition, while our adaptive threshold method automatically selects the optimal threshold according to the optic disc area, which will not cause obvious misjudgment of the optic disc boundary. The region growing algorithm has certain requirements for the selection of initial seed points, and may cause the problem that the recognition area is too small.

Vessel Identification:

One, read in the picture.

Second, the first processing done on the image is to remove the black border around the circular fundus image.

Three, image processing

The image is processed to obtain a rough outline of the blood vessel. First, preprocessing to remove noise, followed by median filtering and threshold denoising, the results are shown in Figure 3;

Then, perform multiple corrosion operations on the pre-processed blood vessels to obtain the approximate distribution range of the blood vessels. At this time, due to the shooting angle, light and other reasons, some of the pictures may locally appear similar to dark spots in the picture quality, so The picture obtained in this step may have broken blood vessels at a certain place, so then pixel linking and diagonal filling are performed on the obtained discontinuous broken blood vessels to obtain continuous blood vessels. Then, deburring is performed on the obtained rough blood vessel to obtain the rough outline of the blood vessel, and the result is shown in Fig. 4 .

4. Vascular Identification

Then blood vessel identification is to treat each segment of blood vessel as an individual and identify each segment of blood vessel.

The main approach is to first find the centerline of the vessel. Because the initial identification of the outline of the blood vessel is not accurate, but the relatively accurate centerline of the blood vessel can be found through the corrosion operation. This step has already separated the segments of the blood vessels, except for the connection points of the blood vessels, each segment is a continuous line. Then use the algorithm to fit the breakpoints, as shown in Figure 5.

After the centerline is selected, use the outward gray gradient changes on both sides of the centerline to determine the boundaries of blood vessels, as shown in Figure 6.

Finally, draw the identified blood vessels on the original picture, as shown in Figure 7.

Blood vessel arteriovenous recognition:

Arteriovenous recognition is based on blood vessel recognition and solid disk recognition, using fundus pictures and corresponding blood vessel pictures with marked arteries and veins for training.

For arteriovenous recognition, we used machine learning methods.

Training is the use of machine learning methods. The first object to be recognized is blood vessels, and arteries and veins need to be recognized. Therefore, the image features recognized by the machine must be selected as the recognition standard. The image features required for training start with the location of the blood vessels. According to the location of the blood vessel, the region can be divided into three sub-regions: the outer side of the blood vessel, the border of the blood vessel, and the inner side of the blood vessel, taking the blood vessel wall as the boundary. Then, the features of gray gradient change and texture change are extracted from these three sub-regions respectively. This type of feature includes the average, median, mode, and pattern of texture features of the gray gradient change in the small area where each pixel is located (3*3 area centered on each pixel), and also includes A total of 141 training features were trained using linear and nonlinear combinations of these features. Here we choose SVM (Support Vector Machine) for training to get the classifier we need.

The area for arteriovenous identification is selected to identify areas with thicker blood vessels and fewer crossings. Therefore, an area within a certain distance with the optic disc as the center is selected as the target area.

Input the picture when identifying, and then identify the blood vessels in the target area, and then extract the 141 features selected by our training on the outside of the blood vessels, the blood vessel wall, and the internal test, and then use the trained machine learning classifier to identify and get the result , and display the result on the original picture, as shown in Figure 8.

The image processing in the identification process is basically the same as that in blood vessel identification, and will not be repeated here.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention should be included in the protection of the present invention. within range.

Claims (5)

1. a method for identifying fundus color photos of blood vessels and arteries and veins, characterized in that it may further comprise the steps: Image quality detection, input the original image, extract image features, and conduct training, predict the image quality, detect the fundus image with acceptable image quality for subsequent processing; Image preprocessing, select one of the images with the best recognition effect, and extract the gray distribution of its RGB three tracks, and use it as a standard image; Read in the picture and remove the border around the circular fundus picture; Image processing, denoising first, followed by median filtering and threshold denoising to obtain a rough outline of blood vessels; Obtain the vessel outline, preprocess the obtained vessel outline, perform multiple erosion operations to obtain the approximate distribution range of the vessel, and then perform pixel linking and diagonal filling on the obtained discontinuous broken vessels to obtain continuous vessels, and then The obtained rough blood vessels are deburred to the outline of the blood vessels; For blood vessel recognition, first find the centerline of the blood vessel, use the outward gray gradient changes on both sides of the centerline to determine the boundary of the blood vessel, and finally draw the recognized blood vessel on the original image. 2. the recognition method of fundus color photograph blood vessel and arteriovenous as claimed in claim 1, is characterized in that the extraction of described image feature comprises the following steps: First use the canny operator to detect the edge of the image, and then use the median filter to denoise, and then use the preprocessed image to calculate the number of total pixels on the edge, the total perimeter of the edge, the maximum height of the edge area, The maximum width, the number of chain codes of odd chains, the target area, rectangularity, elongation, and then extract the seven invariant moment features of the image; Each layer extracts 5 features for judging clarity: gray entropy, Brenner gradient function, variance function, energy gradient function, gradient function; Use the gray histogram to extract features; Convert the original RGB space to HSV space and calculate the color histogram. 3. the identification method of fundus color photograph blood vessel and arteriovenous as claimed in claim 1, is characterized in that described picture quality is predicted and comprises the following steps: 1) Given a training data set d=(X,y), where x is the extracted feature, y is 0, 1 is a classification variable, 0 means poor picture quality, 1 means good picture quality, fixed m≤p, m is The number of features extracted randomly, p is the total number of features, and the number B of trees in the decision tree algorithm; 2) For each b=1,2,...,B, do the following steps: a) Construct a bootstrap training set by randomly sampling n times from n samples for training data d b) use Construct a tree of maximum depth from the data in Randomly select m from p variables for splitting; c) store tree and bootstrap sample information; 3) For any prediction point X ₀ , perform random forest fitting and prediction, and for each tree A category will be predicted, so since there are B trees, B 01 categories can be predicted, and the final prediction result is the category with the most occurrences among the B categories. 4. the identification method of fundus color photograph blood vessel and arteriovenous as claimed in claim 1, is characterized in that also comprising blood vessel arteriovenous identification step: Select the area within a certain distance with the optic disc as the center of the circle as the target area. According to the location of the blood vessel, the area is divided into three sub-areas: the outer side of the blood vessel, the border of the blood vessel, and the inner side of the blood vessel; Two major features of gray gradient change and texture change are extracted from these three sub-regions respectively; Input the picture when identifying, and then identify the blood vessels in the target area, and then extract the features selected for training on the outside of the blood vessels, the blood vessel wall, and the inside; Use the trained machine learning classifier to identify, get the result, and display the result on the original picture. 5. the identification method of fundus color photograph blood vessel and arteriovenous as claimed in claim 4, is characterized in that also comprising optic disc identification step: For initial positioning, first extract the brighter area of the entire image by threshold segmentation, and fill the rest of the darker area with the mean value, and then perform threshold segmentation on the modified image again. When the area of the region of interest has reached the predetermined threshold, the iteration is stopped, and then the extracted highlighted region is screened, and the center of the region of interest is extracted and intercepted for further analysis; Precise positioning and smooth fitting, use the morphological processing method to remove the influence of noise first, then threshold the image to obtain a relatively rough boundary position, then perform ellipse fitting on the boundary position, and finally draw the boundary equation on the original image.