CN112669260B - Method and device for detecting optic disc and macula in fundus images based on deep neural network - Google Patents

Abstract

The invention discloses a fundus image optic disc macula lutea detection method and a device based on a deep neural network, which comprises the following steps: giving a data set containing a plurality of fundus color photographs, giving corresponding optic disc center position labels or whole optic disc region labels and macular center position labels, and dividing the data set into a training set and a verification set; preprocessing an original fundus color photograph to obtain an enhanced domain image, labeling a macular position of a video disc, and establishing a pseudo label image; establishing a model for positioning the macula lutea of the optic disc and segmenting the optic disc, and performing model training and verification on a training set and a verification set by utilizing an enhanced domain diagram and a pseudo label diagram; and positioning the optic disc and the macula lutea on the verification set by adopting the model constructed in the S13, extracting the region of interest, and segmenting by using the model to obtain the final macula lutea, optic disc positioning and optic disc segmentation results. The invention simultaneously covers two tasks of optic disc and yellow spot detection, is reliable and is easy to realize.

Description

Method and device for detecting optic disc and macula in fundus images based on deep neural network

technical field

The invention relates to the technical field of computer vision and image processing, in particular to a method and device for detecting optic disc and macula in fundus images based on a deep neural network.

Background technique

The analysis and processing of retinal fundus images is a hot issue in the field of computer-aided diagnosis at present, that is, input a color photo of the fundus, and the computer outputs information such as the organ structure and lesions of the fundus, so as to assist doctors in diagnosis and treatment, and at the same time save the labor costs of doctors. Realize large-scale screening, etc. The optic disc and macular structure in the fundus image can provide a wealth of medical information, so the computer automatic detection algorithm of the optic disc and macula is of great significance. At present, for the detection of optic disc and macula, according to the image processing method in the algorithm, it can be divided into methods based on deep learning and methods based on traditional image processing.

The methods based on traditional image processing are generally based on the color information of the optic disc and macula. The procedure is complex and the generalization is poor. It has been gradually replaced by the emerging method based on deep learning in recent years. In the method based on deep learning, it can be roughly divided into two ideas, one is to detect based on the color characteristics of the optic disc or macular area, and the other is to detect based on the structural information of the fundus. The feature of detection based on color features is that the idea is simple, and the results are more accurate under normal imaging conditions and mild lesions, but may fail under poor imaging conditions or severe lesions. The characteristic of detection based on structural information is more robust, but the results are usually worse than the former in non-extreme cases.

After retrieval, there are the following types in the prior art:

[1] B.Al-Bander, W.Al-Nuaimy, B.M.Williams, and Y.Zheng, "Multiscalesequential convolutional neural networks for simultaneous detection of fovea and optic disc," Biomedical Signal Processing and Control, vol.40, pp.91 –101, 2018.

[2] X.Meng, X.Xi, L.Yang, G.Zhang, Y.Yin, and X.Chen, "Fast and effective optical disk localization based on convolutional neural network," Neurocomputing, vol.312, pp.285 –295, 2018.

[3] K.-K.Maninis, J.Pont-Tuset, P.Arbelaez, and L.Van Gool, “Deep retinal image understanding,” in International conference on medical image computing and computer-assisted intervention. Springer, 2016, pp. 140–148

[4] J.Son, S.J.Park, and K.-H.Jung, "Towards accurate segmentation of retinal vessels and the optic disc in fundoscopic images with generative adversarial networks," Journal of digital imaging, vol.32, no.3, pp .499–512, 2019.

[5] Z.Gu, J.Cheng, H.Fu, K.Zhou, H.Hao, Y.Zhao, T.Zhang, S.Gao, and J.Liu, "Ce-net: Context encoder network for 2d medical image segmentation,” IEEE Transactions on Medical Imaging, pp.1–1, 2019.

In [1], the author regards the positioning of the optic disc and macula as a regression problem, and designs a convolutional neural network (CNN) to simultaneously predict the position of both. However, such methods cannot judge the reliability of the results through the output results of the model. In [2], the author introduced the idea of anomaly detection, but it requires a large amount of data set support and is not suitable for promotion. In [3][4][5], the various authors designed . Different networks and learning algorithms are used to segment the optic disc area, but the experiment is only for the image blocks of the optic disc area, and there is no positioning process of the image blocks of the optic disc area.

The above methods have their own limitations. Therefore, there is an urgent need to provide a reliable and easy-to-implement detection of the optic disc and macula in fundus images that simultaneously covers the two tasks of optic disc and macular detection.

Contents of the invention

Aiming at the problems existing in the above-mentioned prior art, the present invention proposes a method and device for detecting the optic disc and macula in fundus images based on a deep neural network, covering two tasks of detecting the optic disc and macula at the same time, and designing a new algorithm that takes into account both color information and structure information, so that the optic disc and macular detection indicators have been improved, and at the same time, the experiments of the present invention are verified on public data sets, which are easy to reproduce.

In order to solve the problems of the technologies described above, the present invention is achieved through the following technical solutions:

The invention provides a method for detecting the optic disc and macula of the fundus image based on a deep neural network, which includes:

S11: establish a data set;

A data set containing several color fundus photos is given, and the corresponding optic disc center position label or the entire optic disc area label, macular center position label is given, and the data set is divided into a training set and a verification set;

S12: preprocessing;

Preprocess the original color fundus photos to obtain the enhanced domain map, mark the position of the optic disc and macula, and establish a pseudo-label map;

S13: building a model;

Establishing a model for optic disc macula location and optic disc segmentation, using the enhanced domain map and pseudo-label map obtained in S12 to perform model training and verification on the training set and verification set divided in S11;

S14: test model;

On the verification set divided in S11, the model constructed in S13 is used to locate the optic disc and macula, and then the region of interest is extracted, and the model constructed in S13 is used for segmentation to obtain the final macula, optic disc positioning and optic disc segmentation result.

Preferably, the preprocessing in S12 includes: Gaussian filtering and background subtraction.

Preferably, the enhanced domain map I _eh obtained by preprocessing the original fundus color photo I _ori in S12 is further:

I _eh =4(I _ori −G(σ)*I _ori )+0.5,

Among them, G(σ) is a Gaussian filter, σ is its variance, * represents an image convolution operation, and σ is set to be 1/30 of the radius of the image field of view.

Preferably, the establishment of the pseudo-label map in S12 further includes: generating a circular area with a preset radius and a center located at the center of the optic disc and/or macula.

Preferably, the model training in S13 is to minimize the loss function, wherein the loss function used is a regression loss function constructed using the gap between the real coordinate label and the regression prediction result of the regression network; The segmentation loss function constructed by the gap; and the segmentation loss function constructed by the gap between the optic disc region annotation and the optic disc region segmentation structure.

Preferably, in said S13, the real coordinate labels are used for model training to obtain a regression network, and the pseudo-label map is used for model training to obtain a pseudo-label segmentation network; the model training is performed using real optic disc region labels to obtain an optic disc segmentation network; further,

Using the model constructed in S13 in S14 to test, including the pseudo-label segmentation result output by the pseudo-label segmentation network, the macula output by the regression network, the optic disc positioning result, and the optic disc segmentation result;

When the regional shape of the pseudo-label segmentation result is a regular circular shape, the pseudo-label segmentation result output by the pseudo-label segmentation network is used as the final macula and optic disc positioning result; when the regional shape of the pseudo-label segmentation structure is not If the shape is regular, the macula and optic disc positioning output by the regression network will be used as the final macula and optic disc positioning results.

Preferably, the region shape of the pseudo-label segmentation result is judged according to the shape factor SI, and the shape factor SI is:

Among them, C is the perimeter of the pseudo-label segmentation result area, and S is the area of the pseudo-label segmentation result area;

When the SI is within the preset range, it is judged to be a circle-like shape, and when the SI exceeds the preset range, it is judged to be an irregular shape.

Preferably, the regression loss function minimization problem is defined as:

Among them, θ represents the model parameters, and n is the number of images for each training, that is, the number of batches. The following assumes that P represents the coordinate vector, I represents the output image, pre in the subscript represents the predicted value of the model, and gt represents the real label value.

Preferably, the segmentation loss function minimization problem is defined as:

Among them, θ represents the model parameters, and n is the number of images for each training, that is, the number of batches. The remaining binomial loss functions are the cross-entropy loss L _CE and the Dice coefficient loss L _Dice respectively;

The cross entropy loss is:

The Dice coefficient loss is:

Among them, H and W represent the height and width of the image respectively, the unit is pixel, K is the number of categories, which is the number of target categories plus the number of backgrounds, and the coordinates (i, j, k) represent the k-th channel of the i-th row, j-th column of the image value.

The present invention also provides a deep neural network-based optic disc macular detection device, which is used to implement the above-mentioned deep neural network-based optic disc macular detection method, which includes: a data set establishment module, a preprocessing module, A model building module and a model testing module; wherein,

The data set building module is used to give a data set containing several fundus color photos, and give the corresponding optic disc center position label or the entire optic disc area label, macular center position label, and divide the data set into a training set and the validation set;

The preprocessing module is used to preprocess the original color fundus photo to obtain an enhanced domain map, mark the position of the macula on the optic disc, and establish a pseudo-label map;

The model building module is used to build a model for optic disc macula location and optic disc segmentation, using the enhanced domain map and pseudo-label map obtained by the preprocessing module to divide the training set and verification set in the data set building module model training and validation on

The model testing module is used to locate the optic disc and macula using the model built by the model building module on the verification set divided in the data set building module, and then extract the region of interest, and use the model building module to construct The model is segmented to obtain the final results of macula, optic disc positioning and optic disc segmentation.

Compared with the prior art, the present invention has the following advantages:

(1) The method and device for detecting optic disc and macula in fundus images based on deep neural network provided by the present invention, through labeling and preprocessing color fundus photos, not only color information but also structural information are taken into account, so that the positioning results are more accurate , robustness;

(2) The fundus image optic disc macula detection method and device based on the deep neural network provided by the present invention carry out model training through the enhanced domain map and the pseudo-label map to obtain two kinds of networks, which improves the segmentation accuracy;

(3) The deep neural network-based method and device for detecting optic disc and macula in fundus images provided by the present invention simultaneously cover two detection tasks of optic disc and macula, and realize macular positioning and optic disc segmentation at the same time.

Of course, any product implementing the present invention does not necessarily need to achieve all the above-mentioned advantages at the same time.

Description of drawings

Embodiments of the present invention will be further described below in conjunction with accompanying drawings:

Fig. 1 is the flow chart of the fundus image optic disc macula detection method based on deep neural network of an embodiment of the present invention;

Fig. 2 is a schematic diagram of a pseudo-label segmentation network (NetF) and a visual disc segmentation network (NetS) of a preferred embodiment of the present invention;

Fig. 3 is a schematic diagram of an input module of a preferred embodiment of the present invention;

Fig. 4 is a schematic diagram of ResNet basic module A of a preferred embodiment of the present invention;

Fig. 5 is a schematic diagram of ResNet basic module B of a preferred embodiment of the present invention;

Fig. 6 is a schematic diagram of a DAC module of a preferred embodiment of the present invention;

Fig. 7 is a schematic diagram of the RMP module of a preferred embodiment of the present invention;

Fig. 8 is a schematic diagram of a Dec module in a preferred embodiment of the present invention;

Fig. 9 is a schematic diagram of an output module of a preferred embodiment of the present invention;

Fig. 10 is a schematic diagram of a fully connected module in a preferred embodiment of the present invention.

Detailed ways

The embodiments of the present invention are described in detail below. This embodiment is implemented on the premise of the technical solution of the present invention, and detailed implementation methods and specific operating procedures are provided, but the protection scope of the present invention is not limited to the following implementation example.

FIG. 1 is a flow chart of a method for detecting optic disc and macula in a fundus image based on a deep neural network according to an embodiment of the present invention.

Please refer to Fig. 1, the fundus image optic disc macula detection method based on deep neural network of the present embodiment comprises:

S11: establish a data set;

Given a data set containing several fundus color photos, and given the corresponding optic disc center position label or the entire optic disc area label, macular center position label, and divide the data set into a training set and a verification set;

S12: preprocessing;

Preprocess the original color fundus photos to obtain the enhanced domain map, mark the position of the optic disc and macula, and establish a pseudo-label map;

S13: building a model;

Establish a model for optic disc macular location and optic disc segmentation, use the enhanced domain map and pseudo-label map obtained in S12 to perform model training and verification on the training set and verification set divided in S11;

S14: test model;

On the verification set divided in S11, the model constructed in S13 was used to locate the optic disc and macula, and then the region of interest was extracted, and the model constructed in S13 was used for segmentation to obtain the final results of macula, optic disc positioning and optic disc segmentation.

同时包含视盘位置标定

对于将进行视盘分割的数据，需包含完整的视盘区域图I_gt。一实施例中，模型训练使用4折交叉验证，即将数据集划分为四份，其中三份作为训练集，另一份作为验证集，重复该过程直到四份数据均作为验证集一次。In a preferred embodiment, the position calibration of the macula of the data set, that is, the coordinates

Also includes disc position calibration

For the data to be divided into optic discs, the complete optic disc area map I _gt should be included. In one embodiment, model training uses 4-fold cross-validation, that is, the data set is divided into four parts, three of which are used as training sets, and the other is used as a verification set, and this process is repeated until all four data sets are used as a verification set once.

In a preferred embodiment, the preprocessing in S12 includes: Gaussian filtering and background subtraction. S12 specifically includes: preprocessing the original RGB fundus image I _ori to obtain an enhanced image I _eh , the process can be expressed as:

I _eh ＝4(I _ori -G(σ)*I _ori )+0.5

Among them, G(σ) is a Gaussian filter, σ is its variance, * indicates image convolution operation, and σ is set to 1/30 of the radius of the image field of view.

和

生成一张视盘的伪标签图

为包含一个圆心位于

半径为

到

距离的五分之一的圆的二值图。同理可生成一张黄斑的伪标签图

In a preferred embodiment, the establishment of the pseudo-label map in S12 specifically includes: calibrating the positions of the optic disc and macula

and

Generate a pseudo-label map of the optic disc

To contain a circle centered at

Radius is

arrive

Binary map of circles of one-fifth of the distance. In the same way, a pseudo-label map of the macula can be generated

表示张量加法。In a preferred embodiment, the model training in S13 includes: using the real coordinate labels to carry out model training to obtain a regression network (NetP), using the pseudo-label map to perform model training to obtain a pseudo-label segmentation network (NetF), and using the real visual disc area label to perform model The optic disc segmentation network (NetS) is obtained through training. Model training includes: input module, Res module for feature extraction, DAC module, RMP module for encoding, Dec module, output module and fully connected module for decoding. Connect the input module, Res module, DAC module, RMP module, Dec module, and output module to form a pseudo-label segmentation network (NetF) and an optic disc segmentation network (NetS). Loss function, input the training set to train the model, iteratively update the network parameters. Connect the input module, Res module, DAC module, RMP module, and fully connected module to form a regression network (NetP), use the gap between the real position calibration and the regression result to construct a loss function, input the training set and verification set to train the model, and update the network parameters iteratively. The schematic diagram of the network model of pseudo-label segmentation network (NetF) and disc segmentation network (NetS) is shown in Figure 2, where Res represents the Res module, DAC represents the Dense Atrous Convolution module (DAC), and RMP represents the residual Residual Multi-kernel pooling (RMP), Dec represents the decoder module,

Represents tensor addition.

In a preferred embodiment, the input module in S13 is used to perform convolution, batch normalization, non-linear excitation, maximum pooling and summing on the two images. Specifically, the fundus image in the RGB domain and the enhanced domain image after the second preprocessing step are respectively input to the input module through the upper and lower channels. (BN), linear rectification function excitation (Relu) and maximum pooling (MaxPool). The convolution kernel size of the convolutional layer is 7×7, the number of input channels is 3, the number of output channels is 64, the step size is 2, and the filling pixel (pad) is 3. After the above processing, the two channels respectively obtain the feature maps of the fundus image in the RGB domain and the enhancement domain image, and output the two feature maps to the next module after summing. The process is shown in Figure 3.

In a preferred embodiment, the feature map output by the input module in S13 passes through 4 Res modules to extract deeper features. The Res module is a module proposed by ResNet, which is composed of a series of ResNet basic modules connected in series. The basic modules of ResNet can be divided into two types, A and B, as shown in Figure 4 and Figure 5 respectively. The Res1 module is composed of 3 ResNet basic modules B in series; the Res2 module is composed of 1 ResNet basic module A and 3 ResNet basic modules B in series; the Res3 module is composed of 1 ResNet basic module A and 5 ResNet basic modules B in series; The Res4 module consists of one ResNet basic module A and two ResNet basic module B connected in series. Conv3x3 and Conv1x1 in Figure 4 and Figure 5 represent convolutions with a convolution kernel size of 3×3 and 1×1, respectively, and s=2 means that the step size is 2, which includes a downsampling operation.

The number of input channels of all convolutions in the Res1 module is the number of feature map channels of the upper level, that is, 64, and the number of output channels is also 64; the number of input channels of the two convolutional layers with a step size of 2 in the Res2 module is 64, The number of output channels is 128, and the number of input and output channels of the other convolutional layers is 128; the number of input channels of the two convolutional layers with a step size of 2 in the Res3 module is 128, the number of output channels is 256, and the input and output channels of the other convolutional layers are The number of output channels is 256; the number of input channels of the two convolutional layers with a step size of 2 in the Res4 module is 256, the number of output channels is 512, and the number of input and output channels of the other convolutional layers is 512.

In a preferred embodiment, the DAC module in S13 includes a series of atrous convolution layers with different convolution kernel sizes and hole sizes. The RMP module includes pooling at scales 2, 3, 5, and 6, upsampling at scales 2, 3, 5, and 6, and tensor concatenation. Specifically: the feature map of 14×14, 512 channels extracted by the Res4 module, the information of different scales is extracted through the DAC module, and a feature map of 14×14, 512 channels containing information of different scales is obtained; multi-scale information is obtained by the RMP module Pooling and upsampling, and tensor splicing in the channel dimension to obtain a 14×14, 516-channel feature map. The DAC module is shown in Figure 6. The number before Conv in the figure indicates the size of the convolution kernel, rate indicates the size of the hole, and channel indicates the number of input and output channels. The RMP module is shown in Figure 7. Pooling in the figure indicates the maximum pooling operation, the number in front indicates the pooling scale, the number in front of Conv indicates the size of the convolution kernel, the convolution input channel is 512, the output channel is 1, and Upsample indicates up Sampling, Concatenate means tensor splicing.

[·]表示向下取整；Dec模块第一个反卷积层输入、输出的通道数均为

Dec模块第二个Conv1x1卷积层输入通道数为

输出的通道数为固定值，在本实施例中4个Dec模块该值依次为256、128、64、64。前3个Dec模块输出的特征图将分别与Res3、Res2和Res1模块输出的特征图做张量相加后输入下一个Dec模块。如图2所示。In a preferred embodiment, the feature map output by the RMP module in S13 will be decoded by four Dec modules. The schematic diagram of the Dec module is shown in Figure 8. ConvT3x3 in the figure represents a deconvolution operation with a convolution kernel size of 3×3 and a step size of 2. Specifically: for multiple Dec modules, let the number of feature map channels input by each Dec module be Ch, then the number of input channels of the first Conv1x1 convolutional layer of the Dec module is Ch, and the number of output channels is

[ ] indicates rounding down; the number of input and output channels of the first deconvolution layer of the Dec module is

The number of input channels of the second Conv1x1 convolutional layer of the Dec module is

The number of output channels is a fixed value. In this embodiment, the values of the four Dec modules are 256, 128, 64, and 64 in sequence. The feature maps output by the first three Dec modules will be added to the feature maps output by the Res3, Res2 and Res1 modules respectively, and input to the next Dec module. as shown in picture 2.

In a preferred embodiment, the feature map output by the Dec module at the end is analyzed by the output module to obtain the final output result. The schematic diagram of the output module is shown in FIG. 9 . Specifically: ConvT4x4 in the figure represents a deconvolution layer with a convolution kernel size of 4×4, with a step size of 2. In the present invention, the input channel of this layer is 64, and the output channel is 32; Conv3x3 represents a convolution kernel size of 3 The convolutional layer of × 3, the input channel of this layer is 32 in the present invention, and the output channel is 32; Conv1x1 represents the convolutional layer whose convolution kernel size is 1 × 1, and the input channel of this layer is 32 in the present invention, and the number of output channels Equal to the number of categories of the problem plus one, the number of output channels is 3 for the pseudo-label segmentation network (NetF), and 2 for the disc segmentation network (NetS).

In a preferred embodiment, the regression loss function minimization problem in S13 is defined as:

Among them, θ represents the model parameters, and n is the number of images for each training, that is, the number of batches. The following assumes that P represents the coordinate vector, I represents the output image, pre in the subscript represents the predicted value of the model, and gt represents the real label value.

In a preferred embodiment, the segmentation loss function minimization problem in S13 is defined as:

Among them, θ represents the model parameters, and n is the number of images for each training, that is, the number of batches. The remaining binomial loss functions are the cross-entropy loss L _CE and the Dice coefficient loss L _Dice respectively;

The cross entropy loss is:

The Dice coefficient loss is:

Among them, H and W represent the height and width of the image respectively, the unit is pixel, K is the number of categories, which is the number of target categories plus the number of backgrounds, and the coordinates (i, j, k) represent the k-th channel of the i-th row, j-th column of the image value. In one embodiment, both H and W are set to 448. For the pseudo-label segmentation network (NetF), K is 3, and for the video disc segmentation network (NetS), K is 2.

In a preferred embodiment, the regression network (NetP) input module, Res module, DAC module, RMP module have the same structure and connection mode as the corresponding modules in (NetF) and (NetS). The RMP module is connected to the fully connected module, and the schematic diagram of the fully connected module is shown in Figure 10.

In a preferred embodiment, the segmentation in S14 using the model constructed in S13 includes the pseudo-label segmentation results output by the pseudo-label segmentation network and the macula, optic disc location, and optic disc segmentation results output by the regression network. When the regional shape of the pseudo-label segmentation result is a regular circular shape, the pseudo-label segmentation result output by the pseudo-label segmentation network (NetF) is used as the final macula and optic disc positioning result; when the area of the pseudo-label segmentation structure If the shape is irregular, the macular and optic disc localization output by the regression network (NetP) will be used as the final macular and optic disc localization results.

Further, the region shape of the pseudo-label segmentation result is judged according to the shape factor SI, and the shape factor SI is:

Among them, C is the perimeter of the pseudo-label segmentation result area, and S is the area of the pseudo-label segmentation result area;

When the SI is within the preset range, it is judged to be a circle-like shape, and when the SI exceeds the preset range, it is judged to be an irregular shape. Theoretically, if the pseudo-label segmentation result area is circular, then the SI is 4π. If the SI is too high or too low, it can indicate that the segmentation result area has an irregular shape. In one embodiment, two thresholds T _min =11 and T _max =12.2 are set. When T _min ≤ SI ≤ T _max , the center of the pseudo-label segmentation area output by NetF is taken as the positioning result of the optic disc and macula. When SI<T _min or SI>T _max , or the segmentation result output by NetF contains multiple regions, the output of NetP is taken as the positioning result of optic disc and macula.

In a preferred embodiment, after the optic disc and macula are positioned in S14, an image block whose center is located at the center of the optic disc and whose side length is 1.6 times the distance from the optic disc to the macula is cut out, and the image block is input into NetS to obtain the segmentation result of the optic disc region.

The effect of the method for detecting the optic disc and macula of the fundus image based on the deep neural network in the above embodiment will be verified below with reference to specific examples. After performing four-fold cross-validation on different methods, the experimental results are shown in Table 1 and Table 2:

Table 1. ED comparison of different methods on different data validation sets for optic disc and macula positioning (the best results are indicated in bold)

Table 2. Comparison of different methods for optic disc segmentation Dice on different data validation sets (the best results are indicated in bold)

As can be seen from the results in Table 1 and Table 2, compared with other methods based on deep neural networks, the optic disc and macula detection method proposed by the present invention has better results on the verification set, is closer to the real calibration, and has better functions. Other methods are more comprehensive.

In one embodiment, a deep neural network-based fundus image optic disc macula detection device is also provided, which is used to implement the deep neural network-based optic disc macula detection method of the above embodiment, which includes: a data set establishment module, a pre-set processing module, model building module and model testing module; wherein,

The data set building module is used to give a data set containing several fundus color photos, and give the corresponding optic disc center position label or the entire optic disc area label, macular center position label, and divide the data set into a training set and a verification set. set;

The preprocessing module is used to preprocess the original color fundus photo to obtain an enhanced domain map, mark the position of the optic disc and macula, and establish a pseudo-label map;

The model building module is used to build a model for optic disc macula localization and optic disc segmentation, and use the enhanced domain map and pseudo-label map obtained by the preprocessing module to perform model training and verification on the training set and verification set divided in the data set building module ;

The model testing module is used to locate the optic disc and macula by using the model built by the model building module on the verification set divided in the data set building module, and then extract the region of interest, and use the model built by the model building module to segment to obtain The final macula, optic disc localization and optic disc segmentation results.

What is disclosed here are only preferred embodiments of the present invention. The purpose of selecting and describing these embodiments in this description is to better explain the principle and practical application of the present invention, not to limit the present invention. Any modifications and changes made by those skilled in the art within the scope of the description shall fall within the protection scope of the present invention.

Claims (9)

1. A fundus image optic disc macula lutea detection method based on a deep neural network is characterized by comprising the following steps:

s11: establishing a data set;

giving a data set containing a plurality of fundus color photographs, giving corresponding optic disc center position labels or whole optic disc region labels and macular center position labels, and dividing the data set into a training set and a verification set;

s12: pre-treating;

preprocessing an original fundus color photograph to obtain an enhanced domain image, labeling a macular position of a video disc, and establishing a pseudo label image;

s13: establishing a model;

establishing a model for positioning macula lutea of the optic disc and segmenting the optic disc, and performing model training and verification on the training set and the verification set which are divided in the S11 by using the enhanced domain map and the pseudo label map obtained in the S12;

the model in the S13 is trained to minimize a loss function, wherein the loss function is a regression loss function constructed by using the difference between the real coordinate label and the regression network regression prediction result; a segmentation loss function is constructed by utilizing the gap between the pseudo label graph and the pseudo label segmentation result; constructing a segmentation loss function by using the difference between the real optic disc region label and the optic disc region segmentation structure;

in the step S13, model training is carried out by using the real coordinate labels to obtain a regression network, model training is carried out by using the pseudo label graph to obtain a pseudo label segmentation network, and model training is carried out by using the real optic disc region labels to obtain an optic disc segmentation network; respectively inputting a training set and a verification set training model based on the obtained loss function, and iteratively updating each network parameter;

s14: testing the model;

and positioning the optic disc and the macula lutea on the verification set divided in the step S11 by adopting the model constructed in the step S13, extracting the region of interest, and segmenting by using the model constructed in the step S13 to obtain the final macula lutea, optic disc positioning and optic disc segmentation results.

2. The deep neural network-based fundus image optic disc macula detecting method according to claim 1, characterized in that the preprocessing in S12 includes: gaussian filtering and background subtraction.

3. The deep neural network-based fundus image optic disc macula detecting method of claim 2, wherein in the S12 original fundus color photograph I _ori Preprocessing to obtain an enhanced domain map I _eh Further comprises the following steps:

I _eh ＝4(I _ori -G(σ)*I _ori )+0.5，

where G (σ) is a gaussian filter, σ is its variance, and σ represents the image convolution operation, and σ is set to 1/30 of the image field radius.

4. The method for detecting the macula of an eye fundus image optic disc based on a deep neural network of claim 1, wherein the establishing of the pseudo tag map in S12 further comprises: and generating a circular area with the radius of a preset radius and the center of the circle positioned in the center of the optic disc and/or the yellow spots.

5. The deep neural network-based fundus image optic disc macula detecting method according to claim 1, characterized in that the model constructed in S13 in S14 is subjected to a test including a pseudo label segmentation result output by a pseudo label segmentation network, macula lutea output by a regression network, a optic disc positioning result, and a optic disc segmentation result;

when the region shape of the pseudo label segmentation result is a regular quasi-circular shape, the pseudo label segmentation result output by the pseudo label segmentation network is used as a final yellow spot and optic disc positioning result; and when the region of the pseudo label segmentation structure is in an irregular shape, positioning the yellow spots and the optic discs output by the regression network to obtain a final yellow spot and optic disc positioning result.

6. The method of detecting a fundus image based on a deep neural network for the macula lutea of an optic disc according to claim 5, wherein the region shape of the pseudo tag segmentation result is determined in accordance with a shape factor SI that is:

wherein C is the perimeter of the pseudo label segmentation result region, and S is the area of the pseudo label segmentation result region;

and when the SI is within the preset range, judging that the SI is in a round-like shape, and when the SI exceeds the preset range, judging that the SI is in an irregular shape.

7. The deep neural network-based fundus image optic disc macula detecting method of claim 1, wherein the regression loss function minimization problem is defined as:

wherein, theta represents a model parameter, and n is the number of images trained each time, namely the number of batchs; let P denote the coordinate vector, I denote the output graph, pre in the subscript denotes the model prediction value, gt denotes the true annotation value.

8. The method for fundus image-based macular disc detection by a deep neural network according to claim 1, wherein the segmentation loss function minimization problem is defined as:

wherein, theta represents a model parameter, and n is the number of images trained each time, namely the number of batchs; the other binomial loss functions are respectively cross entropy loss L _CE And Dice coefficient loss L _Dice ；

The cross entropy loss is:

the Dice coefficient loss is:

wherein H and W respectively represent the height and width of the image, the unit is pixel, K is the number of categories and is the number of target categories plus the number of backgrounds, and the coordinates (i, j, K) represent the value of the ith row and jth column of the image.

9. A fundus image optic disc macula lutea detecting apparatus based on a deep neural network, for realizing the fundus image optic disc macula lutea detecting method based on a deep neural network according to any one of claims 1 to 8, comprising: the system comprises a data set establishing module, a preprocessing module, a model establishing module and a model testing module; wherein,

the data set establishing module is used for giving a data set containing a plurality of fundus color photographs, giving corresponding optic disc center position labels or whole optic disc region labels and macular center position labels, and dividing the data set into a training set and a verification set;

the preprocessing module is used for preprocessing an original fundus color photograph to obtain an enhanced domain image, labeling the position of the macula lutea of a video disc and establishing a pseudo label image;

the model establishing module is used for establishing a model for the macular location of the optic disc and the segmentation of the optic disc, and performing model training and verification on a training set and a verification set which are divided in the data set establishing module by utilizing the enhanced domain map and the pseudo label map which are obtained by the preprocessing module; in the model building module, the model is trained to minimize a loss function, wherein the loss function is a regression loss function built by using the difference between a real coordinate label and a regression network regression prediction result; a segmentation loss function is constructed by utilizing the gap between the pseudo label graph and the pseudo label segmentation result; constructing a segmentation loss function by using the difference between the real optic disc region label and the optic disc region segmentation structure; carrying out model training by using a real coordinate label to obtain a regression network, carrying out model training by using a pseudo label graph to obtain a pseudo label segmentation network, and carrying out model training by using a real optic disc region label to obtain an optic disc segmentation network; respectively inputting a training set and a verification set training model based on the obtained loss function, and iteratively updating each network parameter;

the model testing module is used for positioning the optic disc and the yellow spot on the verification set divided in the data set establishing module by adopting the model established by the model establishing module, extracting the region of interest, and segmenting by using the model established by the model establishing module to obtain the final yellow spot, optic disc positioning and optic disc segmenting results.

Images