CN111222393A

CN111222393A - Self-learning neural network-based method for detecting signet ring cells in pathological section

Info

Publication number: CN111222393A
Application number: CN201910967864.3A
Authority: CN
Inventors: 应豪超; 宋庆宇; 吴健
Original assignee: Zhejiang University ZJU
Current assignee: Zhejiang University ZJU
Priority date: 2019-10-12
Filing date: 2019-10-12
Publication date: 2020-06-02

Abstract

The invention discloses a method for detecting signet ring cells in a pathological slice based on a self-learning neural network, comprising: (1) establishing a training set; (2) building a signet ring cell detection model: including a classification module and a detection module; (3) Model training: When training the classification module, use the detection network with the negative sample learning module to train under the original data set; when training the detection module, use self-learning training to compare the predicted signet ring cells with the original marked signet ring cells. Fusion to obtain new labeled data; (4) Signet ring cell detection: First, go through the classification module to determine whether the pathological picture is negative, if it is negative, no need to test; Cell detection, output the signet ring cell detection frame in the figure. With the present invention, a model with higher detection accuracy can be obtained under the condition that the training data is not completely marked.

Description

Self-learning neural network-based method for detecting signet ring cells in pathological section

Technical Field

The invention belongs to the field of medical artificial intelligence, and particularly relates to a method for detecting signet ring cells in pathological sections based on a self-learning neural network.

Background

The analytical test for pathological pictures is the gold standard for diagnostic screening of digestive system cancers. Digital pathological sections are widely applied to various hospitals in recent years due to the advantages that the digital pathological sections can store pathological pictures with high resolution and can be remotely viewed at any time. Especially, the influence is more serious for rural areas with scarce medical resources, and in the areas, due to the lack of experienced professional pathologists, the electronic slices obtained by scanning can be transmitted to urban areas for diagnosis and analysis by more professional pathologists. However, manual examination of pathological sections is time-consuming and labor-consuming, and thus, application of large-scale remote data pathological section examination is also hindered. Therefore, the target detection model in deep learning is used for automatically positioning and classifying the cells in the pathological section, so that the burden of doctors is effectively reduced, the remote pathological section examination is popularized in a large scale, and the problem of medical resource shortage in rural areas is solved.

In recent years, the rapid development of deep learning enables a deep learning network to be applied to medical data in a large scale, particularly for rapid diagnosis and analysis of pathological sections, in the field of deep learning, the main purpose of target detection is to use an algorithm to automatically frame the position of a target to be identified and classify the framed target, and in a pathological image, the target detection network is used to rapidly frame cell positions and cell types, so that the diagnosis speed can be greatly increased, and the reading pressure of doctors can be reduced.

Chinese patent publication No. CN109740626A discloses a method for detecting cancer regions in breast cancer pathological sections based on deep learning, which combines the feature that breast cancer pathological sections have no fixed direction, and systematically uses reasonable data enhancement techniques, including data enhancement techniques of geometric transformations such as random cutting, rotation, left-right turning, and the like; meanwhile, the data enhancement technology of color transformation such as random brightness, sharpening and the like is also used. Data enhancement is carried out in real time during training, so that the diversity of a data set can be increased, a training sample set is expanded, and the generalization capability of the classifier is effectively improved; and finally, solving the problem of unbalanced data of each category of the data set by using a real-time oversampling method.

Signet Ring Cell Carcinoma (SRCC) is a histological type, which is originally derived from the microscopic characteristics of tumors rather than the biological behaviors, shows that tumor cells are abundant in cytoplasm and full of mucus under a microscope, and nuclei are extruded on the cytoplasm side to form a signet ring shape, so that the signet ring cell carcinoma is named as a special type of mucus secretion type adenocarcinoma and commonly occurs in parts such as gastrointestinal tracts, mammary glands, bladder and prostate.

Since cell-by-cell labeling is time-consuming and labor-consuming, and a large number of signet ring cells exist in some areas, it is difficult to completely label each signet ring cell, so that training data used by us is not completely labeled in many cases. How to train with the incompletely labeled training set to obtain an algorithm close to that of the completely labeled training set is still a difficult problem.

Disclosure of Invention

The invention provides a self-learning neural network-based method for detecting signet ring cells in pathological sections, which has high recall rate and negative elimination rate for detecting the signet ring cells.

The technical scheme of the invention is as follows:

a method for detecting signet ring cells in pathological sections based on a self-learning neural network is characterized by comprising the following steps:

(1) establishing a training set: carrying out normalization pretreatment on pictures of pathological sections, then labeling, carrying out random section treatment on the pathological sections to obtain small-size section pictures with fixed sizes, and increasing the number of data sets by using a data enhancement method to serve as training data sets;

(2) constructing a signet ring cell detection model: the system comprises a classification module and a detection module, wherein the classification module is a detection network with a negative sample learning module, and the detection networks in the classification module and the detection module are retinets taking resnet18 as a bone network;

(3) model training: when a classification module is trained, training is carried out under a training set, when negative samples are encountered, the categories of all anchors are judged to be negative, so that the network can learn the characteristics from a negative image, and the training is stopped after the training is carried out to the fitting;

when the detection module is trained, self-learning training is adopted, and in the first stage, the original annotation image of a training set is used for training the detection network until an overfitting phenomenon occurs; in the second stage, prediction operation is carried out on the training set, a data enhancement method is used for obtaining more unmarked ring cells, and the predicted unmarked ring cell detection frames are fused with the original detection frame to generate new training data for next-stage detection network training; performing iterative training by using the method until the recall rate is not increased, and stopping training;

(4) detecting signet ring cells: inputting an unmarked original pathological picture, firstly judging whether the pathological picture is negative through a classification module, and if the pathological picture is negative, detecting is not needed; if the judgment result is positive, the detection module is used for detecting the signet ring cells, and a signet ring cell detection frame in the graph is output.

In the step (1), the normalization preprocessing is to perform an operation of subtracting a mean value and removing a variance on the pixel value. Through the operations, the learning capability of the network can be improved, and the robustness can be enhanced.

In the step (1), the random slicing treatment comprises the following steps: for each pathological section of 2000 × 2000 image of the original size, 512 × 512 areas were randomly cropped. Because the original size is too large, the original size cannot be put into a GPU for training, and the detail information of the ring-printing cells can be destroyed by using down-sampling, the problems of insufficient video memory and information destruction can be effectively solved by using a random slicing method.

The data enhancement method comprises the following steps: turning left and right, turning up and down, rotating by 90 degrees, rotating by 180 degrees, rotating by 270 degrees, and randomly rotating by-20 degrees to 20 degrees; more training data is generated by a random combination of the above data enhancement methods. The data enhancement method can improve the training data volume, especially under the condition of less training data volume, the training data can be forcibly fitted due to the strong learning capacity of the model, so that the serious over-fitting problem can be brought, the robustness and the accuracy of the model can be greatly improved by improving the training data volume by using the data enhancement method, and the over-fitting problem can be effectively solved.

Meanwhile, due to the fact that the number of the negative samples and the number of the positive samples are not balanced, the number of the negative samples is obviously more than that of the positive samples, resampling operation is conducted in a data preprocessing stage, the sampling rate of the negative samples is improved, and the number of the two types of slices is balanced.

In the step (3), when the detection module is trained, in the first stage, the detection module is trained by using original training data, indexes on the test set are calculated after each iteration is finished, the effective recall rate is monitored, and when the effective recall rate begins to decrease, the training is stopped.

All detection networks in the invention are retinets taking resnet18 as a bone network, and resnet18 as the bone network can relieve overfitting phenomena caused by incomplete labeling, has certain learning capacity, and can learn different characteristics of signet ring cells and normal cells. Retianet is used as a strong single-stage network, can effectively detect the signet ring cells, accurately selects the position of the signet ring cells through regression, and accurately distinguishes normal cells and the signet ring cells through classification.

And the classification module and a detection network in the detection module use an opencv tool to frame the position of the ring-printed cell in the pathological picture to obtain a ring-printed cell detection frame.

In the step (4), when the unmarked original pathological picture passes through the classification module, the classification module predicts the pathological picture, counts the number of ring-printing cell detection frames predicted by the whole picture, and judges that the pathological picture is negative when the number is less than 5.

When detecting the signet ring cells, under the original size of 2000 multiplied by 2000, uniformly selecting 9 slices with the block size of 1024 multiplied by 1024 by adopting a sliding window method, recording a detection frame predicted by each slice and converting a relative coordinate into an absolute coordinate after inputting a model because the slices can comprise an overlapped prediction part, and then fusing all prediction results by using a non-maximum inhibition algorithm.

Compared with the prior art, the invention has the following beneficial effects:

1. according to the method, when the detection module is trained, a self-learning training mode is adopted, and new marking data can be generated for training under the condition that the marking of the ring-printing cells is incomplete, so that the extremely high recall rate can be realized.

2. According to the method, the constructed ring-printing cell detection model comprises a classification module and a detection module, before ring-printing cell detection is carried out by using the detection module, the classification module is used for judging whether the pathological picture is negative or not, and extremely high negative elimination rate can be realized.

Drawings

FIG. 1 is a self-learning flow chart of the signet ring cell detection model in training.

Detailed Description

The invention will be described in further detail below with reference to the drawings and examples, which are intended to facilitate the understanding of the invention without limiting it in any way.

1) Establishing a training set

Carrying out uniform mean value reduction and variance removal normalization operation on a given original training set picture, wherein training data in each training process is a 512X 512 random cutting area in the original size picture; meanwhile, in the training sampling process, because the negative and positive samples are unbalanced, the invention adopts a resampling mode to improve the sampling rate of the negative samples, so that the negative samples and the positive samples are kept balanced in each iteration process.

In order to improve the data volume and improve the robustness, the invention uses data enhancement operations, including left-right flipping, up-down flipping, rotation of 90 °, rotation of 180 °, rotation of 270 °, random rotation of-20 ° to 20 °.

2) Model training

The model training comprises two parts, including a classification module training and a detection module training.

For the classification module: and training the detection network with the negative sample learning module under the original data set, and stopping training after training to be fit.

For the detection module: firstly, for the first stage, original training data is used for training a retinet, indexes of the algorithm on a test set are calculated after each iteration is finished during training, the effective recall rate is monitored, namely the highest recall rate which can be achieved by the algorithm under the accuracy of 20%, when the effective recall rate is found to start to be reduced, the fitting point is reached, the training is stopped, and at the moment, the generalization performance of the algorithm is strongest. After the training at the stage is finished, an algorithm is used for predicting on a training set, the prediction comprises part of unlabeled signet ring cells, and the predicted signet ring cells and the originally labeled signet ring cells are fused by using a non-maximum inhibition method to obtain new labeled data. And starting to enter the next stage of training, wherein the training method is similar to that of the first stage, and the only difference is that the newly generated training label replaces the original training label.

3) Model testing

Firstly, predicting a test picture by using a classification module, counting the number of predicted cell frames of the whole picture, and judging that the pathological image is negative when the number is less than 5; otherwise, the ring-printed cell is sent to a detection module for detection operation.

The detection module outputs the coordinates of the upper left corner and the lower right corner of the ring printing cell detection frame and the confidence coefficient selected by the detection frame, and the ring printing cell position is framed and selected in the pathological image by using an opencv tool.

Because random cropping operation is used in the training process, the same size of image needs to be used for prediction in the testing stage. For each original image, a sliding window method is used, 9 visual fields with the size of 1,024X 1,024 are taken and sent to a classification or detection module, the relative coordinates of a prediction frame are obtained, and then the relative coordinates are converted into absolute coordinates in the original image. And according to the absolute coordinate values, fusing the prediction frames of all the sliding windows by using a non-maximum suppression strategy to obtain a final prediction result.

The embodiments described above are intended to illustrate the technical solutions and advantages of the present invention, and it should be understood that the above-mentioned embodiments are only specific embodiments of the present invention, and are not intended to limit the present invention, and any modifications, additions and equivalents made within the scope of the principles of the present invention should be included in the scope of the present invention.

Claims

1. a method for detecting signet ring cells in a pathological slice based on a self-learning neural network, is characterized in that, comprises the following steps:

(1) Establish a training set: After normalizing and preprocessing the images of the pathological slices, mark them, and perform random slice processing on the pathological slices to obtain a small-sized slice image of a fixed size, and use the data enhancement method to increase the number of data sets as training. data set;

(2) Building a signet ring cell detection model: including a classification module and a detection module, the classification module is a detection network with a negative sample learning module, and the detection network in the classification module and the detection module is based on resnet18. The retinanet of the network;

(3) Model training: When training the classification module, the training is performed under the training set. When a negative sample is encountered, the category of all anchors is determined as negative, so that the network can learn the features from the negative image, and stop after training until fitting. train;

When training the detection module, self-learning training is adopted. In the first stage, the detection network is trained with the original labeled images of the training set until overfitting occurs; in the second stage, the prediction operation is performed on the training set, and the data augmentation method is used. , get more unlabeled signet ring cells, and fuse these predicted unlabeled signet ring cell detection frames with the original detection frame to generate new training data for the next stage of detection network training; by using the above method to iterate Train until it is found that the recall rate is not increasing, and stop training;

(4) Signet ring cell detection: input the original unlabeled pathological picture, first go through the classification module to determine whether the pathological picture is negative, if it is negative, no need to test; Cell detection, output the signet ring cell detection frame in the figure.

2. The method for detecting signet ring cells in pathological slices based on self-learning neural network according to claim 1, is characterized in that, in step (1), described normalization preprocessing is to carry out mean subtraction to pixel value and divide Variance operation.

3. the method for detecting signet ring cells in the pathological slice based on self-learning neural network according to claim 1, is characterized in that, in step (1), described random slice processing is: for each original size pathological slice, A 512×512 area is randomly cropped.

4. The method for detecting signet ring cells in a pathological slice based on a self-learning neural network according to claim 1, wherein the data enhancement method comprises: turning left and right, turning up and down, rotating 90°, rotating 180°, Rotate 270°, randomly rotate -20° to 20°; generate more training data through random combinations of the above data augmentation methods.

5. the method for detecting signet ring cells in the pathological slice based on self-learning neural network according to claim 1, is characterized in that, in step (3), when training detection module, in the first stage, use original training data to train detection module, which calculates the metrics on the test set after each iteration, monitors the effective recall rate, and stops training when the effective recall rate starts to drop.

6. the method for detecting signet ring cells in the pathological slice based on self-learning neural network according to claim 1, is characterized in that, in step (3), the detection network in described classification module and detection module uses opencv tool in pathological Select the position of the signet ring cells in the picture to get the signet ring cell detection frame.

7. The method for detecting signet ring cells in a pathological slice based on a self-learning neural network according to claim 1, is characterized in that, in step (4), when the unmarked original pathological picture passes through the classification module, the classification module is to the pathological picture. Prediction is performed, and the number of signet ring cell detection frames predicted for the entire image is counted. When the number is less than 5, the pathological image is determined to be negative.

8. the method for detecting signet ring cells in the pathological slice based on self-learning neural network according to claim 1, is characterized in that, in step (4), when carrying out signet ring cell detection, first adopt the method of sliding window to evenly select 9 slices with a size of 1024×1024 are input into the model, the detection frame predicted by each slice is recorded and the relative coordinates are converted into absolute coordinates, and then the non-maximum suppression algorithm is used to fuse all the prediction results.