CN118658159A - Indirect immunofluorescence image processing method and system - Google Patents

Abstract

The present invention relates to a method and system for processing indirect immunofluorescence images, and belongs to the technical field of medical detection image processing. The method comprises: performing target detection on the indirect immunofluorescence image to obtain multiple regions of interest; performing binarization processing on each region of interest; performing connected domain detection on the binarized image of each region of interest to screen out connected domains that do not meet cell-specific conditions, and taking the connected domains that meet the cell-specific conditions as cell regions and assigning the same grayscale value to obtain a mask of the region of interest; performing contour extraction on each cell region, and extracting characteristic values of preset multiple target features from the cell region based on the contour of each cell region; and constructing a classification model input based on the target characteristic values of a preset number of cell regions in a cell region set, and obtaining the antibody category of the detected sample as positive or negative through the classification model. The embodiment of the present invention improves the efficiency of antibody detection and reduces the misjudgment rate.

Description

Indirect immunofluorescence image processing method and system

Technical Field

The present invention relates to the technical field of medical detection image processing, and in particular to a method and system for processing indirect immunofluorescence images.

Background Art

Indirect immunofluorescence is an important technique widely used in clinical antibody detection, and has important significance and value in the screening and diagnosis of various diseases. This method is usually used as a cell matrix when used to detect pathogen-related antibodies, such as Epstein-Barr Virus (EBV), Coxsackievirus (CV), and Enterovirus (EV).

EBV infection is associated with a variety of diseases, such as infectious mononucleosis, lymphoma, and nasopharyngeal carcinoma. The detection of EBV capsid antigen antibodies (such as Viral Capsid Antigen -Immunoglobulin A, referred to as VCA-IgA) and early antigen antibodies (such as Early Antigen -Immunoglobulin A, referred to as EA-IgA) by indirect immunofluorescence has become an important means of screening and diagnosis of nasopharyngeal carcinoma. Similarly, CV and EA, as common pathogens, can cause a variety of infectious diseases. In addition, the detection of Coxsackie B virus antibodies (such as Coxsackievirus B- Immunoglobulin M, referred to as CBV-IgM) and enterovirus antibodies (such as EV-IgM and EV-IgG) is of great significance for the early diagnosis and treatment of these diseases.

The basic principle of indirect immunofluorescence is as follows: cells containing specific antigen components (such as the aforementioned VCA, EA, etc.) are made into a solid phase detection matrix, the patient serum sample is reacted with it, and then fluorescently labeled anti-human immunoglobulin antibodies are added. If there are target antibodies in the patient's serum (such as the aforementioned VCA-IgA, EA-IgA, CBV-IgM, EV-IgM or EV-IgG), they will bind to the antigen components on the matrix, and then bind to the fluorescently labeled antibodies to form specific fluorescent characteristics. By observing these characteristics, the presence or absence of the target antibody can be determined.

Currently, there are two common methods for interpretation in clinical applications: one is to take indirect immunofluorescence images with a camera connected to a microscope, and then have the test personnel manually interpret them; the other is to have experienced technicians directly observe and interpret them under a microscope. Both methods rely on manual identification of fluorescence and morphological characteristics to determine the positive and negative nature and titer of the antibodies in the test samples.

However, this manual interpretation method has some limitations. Due to the large fluctuations in specific fluorescence and morphological characteristics in different test samples and under different experimental conditions, manual quantitative and qualitative analysis is easily affected by subjective factors. Especially when the test sample contains non-specific cells and impurities, the fluorescence characteristics may not be significant enough, and the accuracy of the test results is highly dependent on the professional level and interpretation experience of the technicians. This may not only lead to a higher misjudgment rate, but also reduce the detection efficiency.

Therefore, it is of great significance to develop more objective, accurate and efficient indirect immunofluorescence image analysis methods.

Summary of the invention

In view of the technical problems existing in the prior art, the present invention proposes a method and system for processing indirect immunofluorescence images to improve the efficiency of antibody detection and reduce the misjudgment rate.

In order to solve the above technical problems, according to one aspect of the present invention, the present invention provides a method for processing an indirect immunofluorescence image, wherein the indirect immunofluorescence image is a fluorescent image of an antibody detection sample of a specific virus, and the method comprises:

Performing target detection on the indirect immunofluorescence image to obtain multiple regions of interest, wherein each region of interest includes a suspected cell region and a background region;

Performing binarization processing on each region of interest to obtain a binarized image, wherein the grayscale values of the binarized image are respectively a first grayscale value and a second grayscale value, and the first grayscale value and the second grayscale value are not equal;

Performing a connected domain detection on the binary image of each region of interest to screen out connected domains that do not meet the cell-specific conditions, and taking the connected domains that meet the cell-specific conditions as cell regions and assigning them the same grayscale value to obtain a region of interest mask, wherein the grayscale values of the cell region pixel points and the background region pixel points in the region of interest mask are the first grayscale value and the second grayscale value, respectively;

Performing contour extraction on each cell region, and extracting feature values of a plurality of preset target features from the cell region based on the contour of each cell region; and

A model input is constructed based on the characteristic values of multiple target features of a preset number of cell regions in the cell region set, and the model input is input into a trained classification model. The classification model determines whether the antibody category of the indirect immunofluorescence image corresponding to the detection sample is positive or negative.

According to another aspect of the present invention, the present invention also provides an indirect immunofluorescence image processing system, which includes a target detection module, a binarization module, a connected domain detection module, a mask module, a feature extraction module and a classification module, wherein the target detection module is configured to perform target detection on the indirect immunofluorescence image to obtain multiple regions of interest, wherein each region of interest includes a suspected cell region and a background region; the binarization module is connected to the target detection module, and is configured to perform binarization processing on each region of interest to obtain a binary image, wherein the grayscale values of the binary image are respectively a first grayscale value and a second grayscale value, and the first grayscale value and the second grayscale value are not equal; the connected domain detection module is connected to the binarization module, and is configured to perform connected domain detection on the binary image of each region of interest to screen out connected domains that do not meet cell-specific conditions; The mask module is connected to the connected domain detection module, and is configured to take the connected domain that meets the cell-specific conditions as the cell region and assign the same grayscale value to obtain the region of interest mask, wherein the grayscale values of the cell region pixel points and the background region pixel points in the region of interest mask are the first grayscale value and the second grayscale value respectively; the feature extraction module is connected to the mask module, and is configured to perform contour extraction on each cell region; based on the contour of each cell region, the feature values of multiple preset target features are extracted from the cell region; the classification module is connected to the feature extraction module, and is configured to construct a model input based on the feature values of multiple target features of a preset number of cell regions in the cell region set, and the model input is input to the trained classification model, and the classification model obtains the antibody category of the indirect immunofluorescence image corresponding to the detection sample as positive or negative.

According to another aspect of the present invention, the present invention also provides an electronic device, comprising a processor and a memory, wherein the memory stores a computer program instruction set, and when the processor executes the computer program instruction set on the memory, the aforementioned indirect immunofluorescence image processing method is performed.

According to another aspect of the present invention, the present invention also provides a computer-readable storage medium, wherein a computer program instruction set is stored on the computer-readable storage medium, and when the computer program instruction set is executed by a processor, the aforementioned indirect immunofluorescence image processing method is implemented.

According to another aspect of the present invention, the present invention further provides a computer program product, which includes a computer program instruction set, and when the computer program instruction set is executed by a processor, the aforementioned indirect immunofluorescence image processing method is implemented.

The processing method provided by the present invention replaces the manual interpretation of the fluorescent images of the test samples, effectively solves the defects caused by manual interpretation, has fast processing speed and high detection accuracy, and effectively improves the diagnosis efficiency of patients with related diseases.

BRIEF DESCRIPTION OF THE DRAWINGS

The preferred embodiments of the present invention will be further described in detail below with reference to the accompanying drawings, wherein:

FIG1 is a partial schematic diagram of a fluorescent image of EBV antibody detection using VCA as an antigen;

FIG2 is a partial schematic diagram of a fluorescent image of antibody detection of Coxsackie A virus;

FIG3 is a schematic diagram of device connections of an indirect immunofluorescence image processing system according to an embodiment of the present invention;

FIG4 is a flow chart of a method for processing an indirect immunofluorescence image according to an embodiment of the present invention;

FIG5 is a flow chart of a method for performing target detection on an indirect immunofluorescence image to obtain multiple ROIs according to an embodiment of the present invention;

6 is a flow chart of a method for determining a new region confidence level for each preliminary region of interest according to an embodiment of the present invention;

FIG7 is a partial schematic diagram of a VCA-IgA fluorescence image outputted through a target detection step according to an embodiment of the present invention;

FIG8 is a partial schematic diagram of a fluorescent image of an adenovirus antibody output after a target detection step according to a second embodiment of the present invention;

9 is a partial schematic diagram of a fluorescent image of Coxsackie B virus CBV-IgM outputted after a target detection step according to a third embodiment of the present invention;

10 is a partial schematic diagram of a fluorescent image of enterovirus EV-IgM outputted after a target detection step according to a fourth embodiment of the present invention;

FIG11 is a flow chart of a method for performing binarization processing on each region of interest to obtain a binarized image of the corresponding region of interest according to an embodiment of the present invention;

FIG12 is a flow chart of a further processing method when performing binarization processing according to an embodiment of the present invention;

FIG13 is a flow chart of a method for determining a first variance threshold according to an embodiment of the present invention;

FIG14 is a schematic diagram of an image obtained after processing a fluorescent image of a Coxsackie virus antibody according to an embodiment of the present invention;

FIG15 is a schematic diagram of an image obtained after processing a VCA-IgA fluorescence image of EBV according to an embodiment of the present invention;

FIG16 is a flow chart of a method for determining whether a connected domain meets cell-specific conditions according to an embodiment of the present invention;

FIG17 is a schematic diagram of cell shape features based on cell region contours according to one embodiment of the present invention;

FIG18 is a schematic diagram of cell edge morphological features based on cell region contours according to one embodiment of the present invention;

FIG19 is a schematic diagram of fluorescence regularity characteristics based on cell region contours according to an embodiment of the present invention;

FIG20 is a flow chart of a titer grade classification method according to one embodiment of the present invention;

FIG21 is a flow chart of a method for screening out ROIs before titer grade classification according to one embodiment of the present invention;

FIG22 is a principle block diagram of an indirect immunofluorescence image processing system according to an embodiment of the present invention; and

FIG. 23 is a schematic diagram of the hardware structure principle of an electronic device according to an embodiment of the present invention.

DETAILED DESCRIPTION

In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

In the following detailed description, reference may be made to the various specification drawings that are part of the present application and are used to illustrate specific embodiments of the present application. In the accompanying drawings, similar reference numerals describe substantially similar components in different figures. The various specific embodiments of the present application are described below in sufficient detail so that a person of ordinary skill in the art with relevant knowledge and skills in the art can implement the technical solutions of the present application. It should be understood that other embodiments may also be used or structural, logical or electrical changes may be made to the embodiments of the present application.

Through clinical findings, in a class of indirect immunofluorescence images used for virus antibody detection, the fluorescence characteristics of cells, especially the fluorescence characteristics of positive cells, are significantly different from the image background. As shown in Figures 1 and 2, Figure 1 is a local schematic diagram of a fluorescence image of EBV antibody detection using VCA as an antigen, in which a dotted area with large fluorescence intensity corresponds to a positive cell area, and an area with weak fluorescence intensity corresponds to a background area. Figure 2 is a local schematic diagram of a fluorescence image of antibody detection of Coxsackie A virus, in which an area with a strong fluorescence edge corresponds to a positive cell area, and other areas with weak fluorescence intensity correspond to background areas. However, in virus antibody detection using cells as a matrix, in addition to positive cells showing strong specific fluorescence characteristics, some false positive cells and impurities may also produce strong fluorescence signals, thereby interfering with the identification of cells and the judgment of positive and negative properties. It is highly dependent on the professional level and interpretation experience of technicians when making manual judgments, and is prone to misjudgment. In view of this, the present invention accurately identifies cell areas from fluorescence images based on deep learning technology, and extracts corresponding features from cell areas using the characteristics of positive cells in fluorescence images, thereby accurately judging the positive and negative properties of antibodies and the titer level of the test sample.

See Figure 3, which is a schematic diagram of the device connection of the indirect immunofluorescence image processing system according to an embodiment of the present invention. In this embodiment, the detection system is implemented by one or more computing devices 101, and the computing device 101 is, for example, a desktop computer, a laptop computer or a tablet computer. The computing device 101 is either connected to a fluorescent microscope 102, or connected to other storage devices 103 or connected to other devices through a network 104, and the computing device 101 obtains a fluorescent image of the test sample prepared based on the indirect immunofluorescence method from these devices. For example, when the computing device 101 is connected to the fluorescent microscope 102, the fluorescent image of the test sample can be obtained in real time. The specific operation steps are, for example: first, the patient's serum is dripped onto a glass slide containing a specific antigen, and then anti-human immunoglobulin labeled with fluorescent markers is added thereto to obtain a test sample slide; then the test sample slide is placed under a fluorescent microscope 102 connected to a camera; then the optical path is adjusted; then the camera is controlled to randomly select 1 to 8 fields of view for fluorescent film shooting, and then 1 to 8 fluorescent images of the same test sample are obtained. The fluorescence microscope 102 sends the obtained fluorescence image to the computing device 101. In addition, the fluorescence image of the test sample can also be taken by a fluorescence microscope connected to other computing devices and stored in the storage device 103, and then the storage device 103 is connected to the computing device 101, and the computing device 101 reads the fluorescence image of the test sample from the storage device 103. In addition, the computing device 101 can also be connected to other devices through the network 104, and read or request the fluorescence image of the test sample to be processed from other devices. The present invention implements the processing method of the indirect immunofluorescence image provided by the present invention through the processing system implemented by the computing device 101, and determines whether the antibody of the test sample is negative or positive by processing the fluorescence image of the test sample prepared based on the indirect immunofluorescence method. The processing method provided by the present invention replaces the manual interpretation of the fluorescence image of the test sample, effectively solves the defects caused by manual interpretation, has fast processing speed and high detection accuracy, and effectively improves the diagnostic efficiency of patients with related diseases.

FIG4 is a flow chart of a method for processing an indirect immunofluorescence image according to an embodiment of the present invention. In combination with FIG3 , the method shown in FIG4 can be implemented by the computing device 101 in FIG1 . The indirect immunofluorescence image in this embodiment is a fluorescent image of a detection sample of a specific virus. The method includes:

Step S1, performing target detection on the indirect immunofluorescence image to obtain a plurality of regions of interest (ROIs), wherein each ROI includes a suspected cell region and a background region.

Step S2, performing binarization processing on each ROI to obtain a corresponding binarized image, wherein the grayscale values in the binarized image are respectively a first grayscale value and a second grayscale value, and the first grayscale value and the second grayscale value are not equal.

Step S3, performing connected domain detection on the binary image of each region of interest to filter out connected domains that do not meet cell-specific conditions.

In step S4, the connected domains meeting the cell-specific conditions are regarded as cell regions and assigned the same grayscale value to obtain a region of interest mask (abbreviated as ROI mask).

Step S5, extracting the contour of each cell region in the ROI mask.

Step S6: based on the contour of each cell region, extracting feature values of a plurality of preset target features from the cell region.

Step S7, constructing a model input based on the feature values of multiple target features of a preset number of cell regions, inputting the model input into the trained classification model, and obtaining the antibody category of the indirect immunofluorescence image corresponding to the detection sample as positive or negative through the classification model. The classification model is a binary classification model obtained by training with labeled samples, wherein the sample is composed of multiple target features and categories of multiple cell regions extracted from the indirect immunofluorescence image, and the output value of the classification model indicates whether the antibody category of the indirect immunofluorescence image corresponding to the detection sample is positive or negative.

After obtaining the fluorescence image through the device shown in FIG3 , in order to facilitate subsequent processing, the fluorescence image is preprocessed to improve the stability of the image quality, for example, the image size is cropped to obtain an image of a specified size (hereinafter referred to as a fluorescence image), and then the pixel values of the fluorescence image are normalized, and then stored as an image file that can be used for processing by the method shown in FIG4 , or directly processed by the method shown in FIG4 to determine the positive and negative of the antibody corresponding to the test sample. Among them, the normalization process can use any algorithm, such as the Min-Max normalization algorithm, the z-score normalization algorithm, the log function normalization algorithm, the inverse tangent function normalization algorithm, etc., and ordinary technicians in this field can choose any one based on application habits, which will not be repeated here.

In one embodiment, in step S1, a neural network model based on deep learning is used to detect targets in indirect immunofluorescence images. The neural network model is a target detection model, which can use a one-stage detection network or a two-stage detection network. The algorithms corresponding to the one-stage detection network are, for example, YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector), and the algorithms corresponding to the two-stage detection network are, for example, Faster R-CNN (Faster Region-based Convolutional Neural Network), Mask R-CNN, Cascade R-CNN, etc. After the image is processed by the aforementioned algorithm, a rectangular area containing a cell area, that is, a region of interest (ROI), can be detected from the image, and the corresponding description information is output. In one embodiment, the description information includes region coordinates, region categories, and region confidence, wherein the region coordinate value is composed of four values of the coordinates of the upper left corner of the region, the width of the region, and the height. The region category is an integer starting from the number 0, which represents the classification category of the target when the target detection model detects the image. The region confidence is, for example, a number between 0 and 1. The larger the value, the greater the probability that the ROI belongs to the category, and the higher the credibility. The network used in the target detection model of the present invention can meet the robust performance in dense target scenarios, and can effectively perform multi-scale cell feature extraction, meet the requirements of generalization ability, and by conducting targeted training on the target detection model, a model suitable for specific virus antibody detection can be obtained. For example, using the data training model annotated with EBV antibody immunofluorescence images, a target detection model suitable for EBV antibody detection (such as VCA-IgA, EA-IgA) can be obtained; using the manually annotated data of Coxsackie virus, a model suitable for Coxsackie virus antibody detection can be obtained. This training method enables the model to be optimized for the detection of specific virus antibodies. It is worth noting that based on the existing model (such as the EBV antibody detection model), by introducing a small amount of annotated data of other virus antibodies, the model migration and update can be completed quickly. This migration learning method greatly improves the adaptability and efficiency of the model, so that it can be relatively easily extended to the detection of other virus antibodies, such as Coxsackie A virus antibody detection, Coxsackie B virus antibody detection, enterovirus antibody detection, adenovirus antibody detection, etc.

In one embodiment, in order to train the aforementioned target detection model, a sufficient number of indirect immunofluorescence images taken based on specific virus antibody detection are first collected, and then the image size of the fluorescence images is uniformly cropped and normalized, and then the processed fluorescence images are assigned negative or positive labels by clinical laboratory experts. Among them, the fluorescence image marked with a positive label contains a sufficient number of specific fluorescent cells. Then the image marked with a positive label is manually labeled with cells, including manually using a rectangular frame to frame the cell area, and annotating the area category according to the fluorescence characteristics, for example, cells with a sufficiently large size are classified into one category, and cells with a small size are classified into another category; or cells with similar fluorescence forms are classified into one category, for example, cells with large fluorescence intensity are classified into one category, and cells with small fluorescence intensity are classified into another category, etc. The present invention is not only conducive to obtaining an ideal ROI during target detection, but also conducive to the directionality of subsequent feature construction through detailed labeling of samples. Each fluorescent image after labeling is taken as a sample, and the sample includes multiple ROIs and corresponding area categories, thereby obtaining a sample set.

Then divide the sample set into a training set and a validation set, for example, 80% of the samples in the sample set are divided into a training set, and 20% of the samples in the sample set are divided into a validation set. The neural network is trained based on the samples in the training set, and the model performance is evaluated on the validation set. When the model loss converges and meets the performance evaluation requirements, the object detection model for recommending ROI is obtained.

After training a target detection model for a specific virus detection sample using the aforementioned method, its model parameters can be transferred to the target detection model of another specific virus detection sample, and the training can be completed using a small number of samples.

In addition, when the samples during training are mixed with multiple specific virus detection samples, the trained target detection model can be applicable to multiple specific virus detection samples at the same time.

Referring to FIG. 5 , FIG. 5 is a flow chart of a method for performing target detection on an indirect immunofluorescence image to obtain multiple ROIs according to an embodiment of the present invention, the method comprising:

Step S11, inputting the indirect immunofluorescence image into a target detection model, and obtaining a plurality of preliminary ROIs and their description information after being processed by the target detection model, wherein the description information includes region coordinates, region category and region confidence, and the plurality of preliminary ROIs constitute a preliminary ROI set.

Step S12, setting a preliminary ROI as a processing target.

Step S13, determining a new region confidence of the target preliminary ROI using the overlapping area determined based on the region coordinates and the region confidence as calculation factors.

Step S14: Calculate the difference or ratio between the new region confidence of the target preliminary ROI and the confidence threshold to compare the new region confidence of the target preliminary ROI with the confidence threshold.

Step S15, determining whether the new region confidence of the target preliminary ROI is less than the confidence threshold, if the new region confidence of the target preliminary ROI is less than the confidence threshold, in step S16, removing the target preliminary ROI from the preliminary ROI set, and then executing step S17. If the new region confidence of the target preliminary ROI is greater than or equal to the confidence threshold, executing step S17.

Step S17, determining whether there are any preliminary ROIs that have not been processed, if so, returning to step S12, if not, ending the processing flow.

Among them, in step S11, there are usually two or more ROIs overlapping among the multiple preliminary ROIs obtained by the target detection model. In order to reduce the amount of calculation, it is necessary to remove redundant ROIs from the multiple overlapping ROIs. For the present invention, when there is an overlapping area at the position of two positive cells, the corresponding ROI obtained is in an overlapping position, but the ROI in this case cannot be removed as a redundant ROI. Therefore, when screening out redundant ROIs, the present invention not only refers to the area of the overlapping area, but also refers to the regional confidence of the ROI, and determines the redundant ROI based on these two factors, thereby achieving the purpose of streamlining ROIs and avoiding the erroneous removal of ROIs representing positive cells.

Referring to FIG. 6 , FIG. 6 is a flow chart of a method for determining a new region confidence of each preliminary region of interest according to an embodiment of the present invention, wherein the method comprises the following steps:

Step S131, obtaining the target region coordinates and target region confidence of the target preliminary ROI.

Step S132, obtaining the first region coordinates of the first preliminary ROI with the maximum confidence in the current region.

Step S133: calculating an intersection over union (IoU) between the first preliminary ROI and the target preliminary ROI based on the first region coordinates and the target region coordinates.

Step S134, using the intersection-and-union ratio as an independent variable of a preset function to calculate a function value.

Step S135 , using the function value as the weight of the target region confidence to calculate a new region confidence of the target preliminary ROI.

In one embodiment, the i-th preliminary ROI obtained by the target detection model is expressed as , that is, a 6×1 matrix, the 6 elements in the matrix are the coordinates of the upper left corner of the region, the width of the region, the height of the region, and the category and confidence of the region s _{i of} the ROI, so the preliminary ROI set C output by the target detection model can be expressed as , where n is the number of preliminary ROIs. In this embodiment, the preset function in step S134 is, for example, a one-dimensional Gaussian variance function, so the step of finally obtaining the new region confidence S _i of the i-th preliminary ROI can be expressed by the following formula 1-1:

1-1

in Indicates the currently determined target preliminary ROI, It represents the first preliminary ROI with the highest confidence among all the current preliminary ROIs. IoU represents the intersection-over-union score of two preliminary ROIs, which is used to quantify the overlapping area of two preliminary ROIs. is the Gaussian variance parameter.

After the above processing, some ROIs are screened out from the preliminary ROI set by the regional confidence, and the remaining ROIs are used as the output of the target detection step for the binarization processing of step S2. Referring to Figures 7 to 10, Figure 7 is a partial schematic diagram of the fluorescence image of VCA-IgA output through the target detection step according to one embodiment of the present invention, Figure 8 is a partial schematic diagram of the fluorescence image of adenovirus antibodies output through the target detection step according to the second embodiment of the present invention, Figure 9 is a partial schematic diagram of the fluorescence image of Coxsackie B virus CBV-IgM output through the target detection step according to the third embodiment of the present invention, and Figure 10 is a partial schematic diagram of the fluorescence image of enterovirus EV-IgM output through the target detection step according to the fourth embodiment of the present invention.

In step S2, since the fluorescence disorder degree of the cell area and the fluorescence interference of the background can affect the determination of the cell area and the background area in the ROI, in one embodiment of the present invention, in order to improve the ability to accurately distinguish the cell area and the background area during binarization, different image processing methods are used according to the different fluorescence interference degrees of the ROI. Among them, the grayscale variance of the ROI is used as a parameter for evaluating the fluorescence interference degree. When the grayscale variance of the ROI is greater than or equal to a threshold (in order to distinguish it from the variance threshold used in other scenes, hereinafter referred to as the first variance threshold), it means that the background fluorescence interference is more serious. At this time, the adaptive threshold segmentation method is selected to obtain the binarization segmentation threshold, and the region of interest is binarized based on the segmentation threshold. When the grayscale variance of the ROI is less than the first variance threshold, it means that the background fluorescence interference is not strong. At this time, more attention should be paid to the acquisition of the fluorescence of the cell itself. Therefore, the ROI is edge detected at this time, and the region of interest is binarized based on the edge detection result. Therefore, in one embodiment, the specific process of performing binarization processing on each region of interest in step S2 to obtain a corresponding region of interest mask is shown in FIG11 , which is a flow chart of a method for performing binarization processing on each region of interest to obtain a binarized image of the corresponding region of interest according to one embodiment of the present invention, the method comprising:

Step S21, calculating the grayscale variance of each ROI.

Step S22: comparing the grayscale variance of each ROI with a first variance threshold.

Step S23, judging whether the grayscale variance is greater than or equal to a first variance threshold, when the grayscale variance is greater than or equal to the first variance threshold, in step S24, using adaptive threshold segmentation to perform binarization processing on the ROI to obtain a binary image of the ROI. When the grayscale variance is less than the first variance threshold, in step S25, using edge detection method to perform binarization processing on the ROI to obtain a binary image of the ROI.

In another embodiment, after binarization processing is performed on all ROIs in FIG. 11 , the processing steps in FIG. 12 are further included:

Step S25 , respectively calculating the structural similarity between the binary image of each ROI and its corresponding ROI.

In step S26, the loss amount of the binarization process is calculated based on the structural similarity between the binarized images of all ROIs and their corresponding ROIs.

Step S27, determine whether the loss amount is greater than or equal to the loss threshold, if the loss amount is greater than or equal to the loss threshold, then adjust the first variance threshold in step S28; then return to step S22, based on the adjusted first variance threshold, re-binarize each ROI. If the loss amount is less than the loss threshold, then end the process.

In this embodiment, after all ROIs are binarized, the total loss is calculated. When the total loss exceeds the loss threshold, it means that the loss of the binarization process is too large, and the distinguished cell area and background area do not meet the accuracy requirement. At this time, the first variance threshold is adjusted, so that the cell area in the binarized image of the ROI obtained by the present invention is closer to the original cell area, thereby improving the accuracy of subsequent judgment of the positive and negative detection of the sample.

In the above two embodiments of FIG. 11 and FIG. 12 , the process of binarizing a ROI using the adaptive threshold segmentation method can be shown as the following flow:

First, grayscale the ROI to obtain the ROI grayscale image; then traverse the grayscale values of the pixels in the ROI grayscale image to establish a grayscale histogram; then count the number of pixels in each grayscale level to obtain the probability of the pixel being at different grayscale levels; then traverse the grayscale values of the pixels in the ROI grayscale image from the minimum to the maximum as the threshold, and record all pixels with grayscale values higher than the current threshold as category A, otherwise they are recorded as category B; calculate the variance between category A and category B .variance The calculation formula 1-2 is as follows:

1-2

in, is the probability that the pixel is classified as category A when the threshold is t; is the probability that the pixel is classified as category B when the threshold is t; is the grayscale mean of the pixels in class A when the threshold is t, is the grayscale mean of the pixels in class B when the threshold is t.

When the threshold t When the value is the largest, it is considered that the difference between categories A and B is the most obvious, and the threshold is then positioned as the optimal segmentation threshold.

After obtaining the optimal segmentation threshold, the ROI is binarized using the optimal segmentation threshold, that is, the pixels in the ROI whose grayscale values are higher than the optimal segmentation threshold are assigned a value of 0, and the pixels whose grayscale values are lower than the threshold are assigned a value of 255. At this time, a binary image is obtained.

In another embodiment, a Triangle binary segmentation method may also be used, and the point in the histogram that is farthest from the line connecting the peak position and the trough position is used as the segmentation threshold.

Both of the above methods do not require manual input of parameters, can adapt to the ROI of the fluorescent cluttered area, and can obtain better binary images.

In the present invention, when determining to use an edge detection method to perform a binarization process on the ROI, any edge detection method may be used. In one embodiment, the process of performing a binarization process on a ROI based on the edge detection result, for example, includes the following steps:

First, the Sobel operator is used to calculate the first-order derivatives in the horizontal and vertical directions of the current ROI to obtain a gradient map, where the first-order derivative is the image gradient. Then, the gradient amplitude and direction of the boundary are determined based on the obtained gradient map. After that, each pixel is checked, and the maximum gradient amplitude is determined from the pixels with the same gradient direction around each pixel. Then, the maximum and minimum gradient thresholds are determined based on the double threshold method, and the true edge is determined based on the gradient threshold. The grayscale value of the pixel points belonging to the edge is set to 255, and the grayscale value of other pixel points is set to 0, thereby obtaining a binary image.

In another embodiment, an edge detection method of an adaptive Canny operator may be used. The high threshold and low threshold in the Canny operator can effectively suppress non-specific fluorescence, thereby improving the accuracy of the cell area in the ROI membrane. The threshold in the Canny operator can be obtained by clustering, for example: performing unsupervised cluster analysis on all pixel points of the ROI using edge detection to obtain two clusters, one is a first cluster corresponding to the cell area, and the other is a first cluster corresponding to the background, and then calculating the centers of the two clusters respectively, and using the grayscale value of the pixel corresponding to the center of the first cluster as the high threshold in the Canny operator, and using the grayscale value of the pixel corresponding to the center of the second cluster as the low threshold in the Canny operator. The specific detection process will not be repeated here.

In the aforementioned edge detection method, the two thresholds of the Canny operator are determined by clustering, without the need for manual parameter input, thus reducing the process of manual parameter adjustment, improving the generalization performance of edge detection in different scenarios, and improving the efficiency of cell mask extraction.

In addition, the aforementioned first variance threshold can be obtained by limited calculation. In one embodiment, referring to FIG. 13 , FIG. 13 is a flow chart of a method for determining a first variance threshold according to an embodiment of the present invention, and the method comprises the following steps:

Step S201 , obtaining a sample set, where the samples in the sample set are ROIs obtained after performing target detection on the sample indirect immunofluorescence image.

Step S202, setting the adjustment times of the first threshold value i = 1, wherein i = 1, 2, 3, ..., n. n is the highest number of times set.

Step S203, calculating the grayscale variance of each ROI.

Step S204, compare the grayscale variance of each ROI with the preset first threshold. When the grayscale variance of a ROI is greater than or equal to the first threshold, in step S205, the ROI is binarized by using the adaptive threshold segmentation method to obtain a ROI binary image. When the grayscale variance of a ROI is less than the first threshold, in step S206, the ROI is binarized by using the edge detection method to obtain a ROI binary image.

Step S207, calculating the structural similarity between each ROI binary image and its corresponding ROI.

Step S208 : calculating the loss of the binarization processing based on the structural similarity between all ROI binarized images and their corresponding ROIs.

Step S209, determine whether the adjustment number i is equal to n, if the adjustment number i is not equal to n, then in step S2010, adjust the first threshold to obtain a new first threshold, and return to step S204. If the adjustment number i is equal to n, execute step S2011.

Step S2011, sorting multiple (n) loss amounts by size.

Step S2012: determine the first threshold corresponding to the minimum loss amount as the first variance threshold.

It can be seen from the above process that in this embodiment, a specific value is first preset for the first threshold, and then the loss amount of the binarization process is finally obtained based on the specific value of the first threshold, and then the specific value of the first threshold is adjusted, and then another loss amount of the binarization process is finally obtained based on the new first threshold. After a finite number of calculations, multiple loss amounts are obtained, and the first threshold with the smallest loss amount is determined as the first variance threshold, so that the shape of the cell area in the binarized image in step S2 is closer to the shape of the cell area in the original ROI. In another embodiment, the termination condition for determining the first variance threshold in FIG. 13 may not be the finite number of times the first threshold is adjusted, but each time the loss amount of the binarization process is obtained, the loss amount of the binarization process is compared to see whether it is less than the maximum loss amount allowed. If the loss amount of the binarization process is less than the maximum loss amount allowed, it means that the current loss amount of the binarization process can be accepted and meets the accuracy requirement, then the adjustment of the first threshold is stopped, and the first threshold at this time is determined as the first variance threshold.

The loss amount involved in the above description can be calculated using the following cost function expressions 1-3:

1-3

in, is the scaling factor, which can adjust the cost function to be within a fixed interval to ensure that the overall loss will not approach infinity when the variance is small. k is the path of binarization processing. For example, k=1 indicates that the adaptive threshold segmentation method is used for binarization, and k=2 indicates that the edge detection method is used for binarization. represents the number of ROIs in the kth path, z represents the total number of ROIs, and MSSIM represents the average structural similarity of an ROI. Its calculation formula is shown in Formula 1-4:

1-4

in, and Represent ROI binary image and ROI respectively, , is the structural similarity, Calculated for a single ROI The number of windows of values, and are respectively the first In one embodiment, the window size is 11 pixels.

represents the MSSIM mean of all ROIs on the kth path. represents the MSSIM variance of all ROIs on the kth path.

The process of calculating the loss amount of binarization processing based on the above formula includes the following steps:

Count the first ROI and its number after binary processing based on the segmentation threshold .

Based on the first structural similarity MSSIM of each first ROI binary image and its corresponding first ROI and the number of first ROIs Calculate the average of the first structural similarity and its variance .

Count the second ROI and the number of second ROIs that are binarized based on edge detection results .

Based on the structure similarity MSSIM of each second ROI binary image and its corresponding second ROI and the number of second ROIs Calculate the average of the second structural similarity and its variance .

Calculate the average value of the first structural similarity respectively and its variance The first ratio and the second structural similarity average and its variance The second ratio of .

First ROI quantity For the total number of ROIs z (the number of first ROIs and the second ROI number The proportion of the sum of the two ratios is used as the weight of the first ratio, and the number of the second ROI is used as the weight of the first ratio. The proportion of the total number z of ROIs is used as the weight of the second ratio, and a first weighted value of the first ratio and a second weighted value of the second ratio are calculated respectively.

The first weighted value and the second weighted value are respectively used as preset cost functions The independent variable is used to calculate the corresponding cost function value, and the cost function value is used as the loss amount of the binarization processing.

From the expression and calculation process of the aforementioned cost function, it can be seen that if the cell position in the binary image of the ROI is more accurate, the similarity between it and the original image is higher, and the corresponding loss amount will be smaller. When calculating the loss amount, the present invention not only calculates the structural similarity between the binary image of each ROI and its corresponding ROI, but also calculates the proportion of each calculation path. Therefore, the overall loss amount obtained not only reflects the accuracy of the cell position in the binary image of all ROIs, but also reflects the influence of different calculation paths on the overall loss amount, so that the first variance threshold can be obtained more accurately.

In step S3, when connected domain detection is performed on the binary image of each ROI, the pixel points in the ROI binary image are traversed, the grayscale values of the pixel points are compared, and the pixel points with equal grayscale values in the neighborhood of each pixel point are determined as the same connected domain, thereby obtaining one or more connected domains, and then based on the cell-specific conditions, the connected domains that do not meet the cell-specific conditions are deleted, and the connected domains that meet the cell-specific conditions are retained. Then, in step S4, the pixel points in the connected domain that meets the cell-specific conditions are assigned values to obtain the ROI mask.

See Figure 14, which is a schematic diagram of images obtained after processing the fluorescent image of Coxsackie virus antibodies, wherein the first group of images is a schematic diagram of ROI obtained after target detection processing based on step S1, the second group of images is a schematic diagram of binary images obtained after binarization processing of the ROI in the first group, and the third group of images is a schematic diagram of ROI masks obtained after connected domain detection is performed on the binary images in the second group.

See Figure 15, which is a schematic diagram of images obtained after processing the fluorescence image of EBV VCA-IgA, wherein the first group of images is a schematic diagram of ROI obtained after target detection processing based on step S1, the second group of images is a schematic diagram of binary images obtained after binarization processing of the ROI in the first group, and the third group of images is a schematic diagram of ROI masks obtained after connected domain detection is performed on the binary images in the second group.

Referring to FIG. 16 , FIG. 16 is a flow chart of a method for determining whether a connected domain meets a cell-specific condition according to an embodiment of the present invention, the method comprising the following steps:

Step S41, calculating the area and centroid coordinates of the connected domain.

Step S42, calculating the center coordinates of the ROI based on the region coordinates of the ROI.

Step S43, calculating the distance between the centroid coordinates of the connected domain and the center coordinates of the corresponding ROI.

Step S44, determine whether the distance between the centroid coordinates of the connected domain and the center coordinates of the corresponding ROI is less than the distance threshold. If the distance between the centroid coordinates of the connected domain and the center coordinates of the corresponding ROI is greater than or equal to the distance threshold, it means that the connected domain deviates from the ROI area, and then determine to filter out the connected domain in step S46. If the distance between the centroid coordinates of the connected domain and the center coordinates of the corresponding ROI is less than the distance threshold, it means that the cell-specific condition may be met, and then step S45 is executed.

Step S45, judging whether the area of the connected domain is less than the area threshold, if the area of the connected domain is less than the area threshold, it means that the connected domain does not meet the size requirement of the positive cell, that is, it does not meet the cell-specific condition, and if it does not meet the cell-specific condition, it is determined to screen out the connected domain in step S46. If the area of the connected domain is greater than or equal to the area threshold, it is determined that the connected domain meets the cell-specific condition, and it is determined to retain the connected domain in step S47.

Only when both of the above two conditions are met, the connected domain is retained, otherwise the connected domain is screened out. In a special case, if the number of connected domains remaining in an ROI is less than 1 after the process shown in Figure 16, it means that there is no suitable cell area in the current ROI. At this time, in the subsequent feature construction, the feature construction step is skipped, and the feature values of all target features are assigned to -1.

In step S41, the contour can be obtained based on the connected domain, and the closed contour area is drawn based on the pixel center point in the contour. The number of pixels contained in the contour is set to , the number of pixels contained inside the contour is , Since the pixel center point is used for contour extraction, only half of the pixels on the edge contour will be retained when calculating the area. The final connected domain area contour_area is shown in the following formula 1-5:

1-5

The method to calculate the centroid coordinates of the connected domain is as follows: First, The horizontal and vertical coordinates of all pixels are accumulated respectively, and then divided by the total number of pixels contained in the connected domain to obtain the connected domain. Final center of mass coordinates , as shown in the following formulas 1-6 and 1-7, where Connected domain The total number of pixels contained in is the connected domain index.

1-6

1-7

The center coordinates of the ROI in step S42 are obtained based on the four values in the region coordinates, which can be implemented by a specified function in the cv2 library and will not be described in detail here.

The area threshold in step S44 is a constant determined based on statistical data, for example, the area range of each type of positive cells is counted for different cell types, thereby determining the area threshold of each type of positive cells. Similarly, the distance threshold is determined by counting the radius range of each type of positive cells.

The present invention targets the characteristics of various positive cells in fluorescent images, and in one embodiment, presets a plurality of target features, wherein the target features are cell shape features, cell edge morphology features and fluorescence regularity features.

When extracting the characteristic value of the cell shape feature from the cell region, the inscribed area of the cell region is first calculated according to the outline of the cell region, and the inscribed area is, for example, the number of pixels contained in the extracted cell region outline. Then the perimeter of the cell region is calculated according to the outline of the cell region, for example, the Euclidean distance between all adjacent points in the outline is calculated in a fixed direction and summed. Then the roundness of the cell region is calculated based on the inscribed area and perimeter of the cell region. The roundness calculation formula is shown in Table 1-8 below:

1-8

in, is the circularity, s is the inscribed area, and l is the circumference.

Among them, the inscribed area of the cell region is the first cell shape feature; the roundness of the cell region is the second cell shape feature. As shown in Figure 17, Figure 17 is a schematic diagram of the cell shape feature based on the cell region contour according to an embodiment of the present invention. The calculation of the roundness is related to the perimeter (blue area) and area (red area) of the cell region. The green is the reference circumscribed circle. When the blue contour is infinitely close to the green contour, the roundness is 1.

When calculating the characteristic value of the cell edge morphological feature based on the extracted cell region contour, the cell region contour is first sampled multiple times at intervals according to the preset number of pixels, and each sampling obtains the preset number of pixels. For example, the sampling interval is The outline of the cell area contains N pixels, and sampling is carried out clockwise from the pixel with the smallest horizontal coordinate of the outline. The number of pixels sampled in each sampling period is more than 2, preferably 3. In the following example, the number of pixels sampled in each sampling period is 3.

Then, the curvature is calculated based on the coordinates of a preset number of pixel points obtained in each sampling to obtain a curvature set K. For example, the pixel points obtained in each sampling period are classified into a subset, and after multiple (such as ) The sampling period obtains multiple sampling subsets, and the multiple sampling subsets are combined into multiple acquisition sets. In one embodiment, the curvature is calculated by the following formula 1-9:

1-9

Among them, the is the horizontal coordinate set of pixel points in the sampling set obtained in all sampling periods, and t is the number of sampling subsets; is the vertical coordinate set of the pixel points in the sampling set obtained in all sampling periods, and The contours are The first-order gradient in the direction and The first-order gradient of the direction. and The contours are The second-order gradient in the direction and The curvature set of the contour of the cell area calculated by the above formula 1-9 .

Then, the curvature variance is calculated based on multiple curvatures in the curvature set K according to the following formula 1-10: :

1-10

in is the mean of all elements in the curvature set K.

The curvature variance As the characteristic value of the cell edge morphological feature, it can effectively express the smoothness of the cell edge and effectively suppress the incomplete fluorescence of the cell edge. Referring to Figure 18, Figure 18 is a visualization diagram of the cell edge morphological feature based on the cell region contour according to an embodiment of the present invention, wherein the x mark is the sampled pixel point, and the circle is the approximate curvature circle obtained based on the sampling point of each sampling period.

When calculating the characteristic value of the fluorescence regularity feature based on the extracted cell area contour, the gray level co-occurrence matrix is first converted based on the gray value of the pixel point in the inner area of the cell area contour to obtain the gray level co-occurrence matrix Wherein, M represents the gray level in the matrix. In one embodiment, in order to include all possible gray values, M takes a value of 256. is the distance for pixel association calculation when gray-level co-occurrence matrix conversion is performed, is the angle of pixel association calculation. The gray level co-occurrence matrix is an M×M matrix, and the elements in the matrix are expressed as ,in are the horizontal and vertical coordinates of the element in the gray-level co-occurrence matrix.

Then, based on the gray-level co-occurrence matrix elements, the energy E of the inner area of the contour is calculated according to the following formula 1-11, and the energy E is used as the characteristic value of the fluorescence regularity feature.

1-11

Energy can effectively measure the uniformity of fluorescence and texture coarseness of the cell area. When the image texture is uniform and regular, the elements of the grayscale co-occurrence matrix are mainly concentrated near the diagonal, and the energy value is large. See Figure 19, which is a schematic diagram of the fluorescence regularity characteristics based on the cell area contour according to an embodiment of the present invention. The more bright spots in the matrix, the greater the energy value of the image; conversely, when the grayscale value distribution of the image is more random, the distribution of elements of the grayscale co-occurrence matrix will be more dispersed. Therefore, quantifying the texture feature regularity of the cell fluorescence image through energy helps to highlight the regular differences between specific fluorescence and non-specific fluorescence.

In one embodiment, the multiple eigenvalues of each ROI extracted above constitute a feature matrix F, for example, multiple eigenvalues of each ROI are used as matrix rows, and the matrix columns are different eigenvalues. In one embodiment, the matrix columns have 5 columns, corresponding to the inscribed area s of the cell region, the roundness of the cell region, , curvature variance , energy E and region category, so when the number of ROIs is n, the obtained feature matrix F can be expressed as . It is assumed that the input of the classification model is , then the eigenvalues of the ROIs whose regional confidences in the current ROI mask are ranked in the top t constitute the feature matrix F.

The classification model in the present invention can adopt various binary classification algorithms, such as logistic regression (LR), support vector machine (SVM), neural network, etc. In one embodiment, an SVM algorithm with a kernel function of a Gaussian kernel function is used as a classification model. When the aforementioned feature matrix F is input to the SVM model as a model input, the SVM model outputs a value of 0 or a value of 1, 0 corresponding to a negative result, and 1 corresponding to a positive result.

In order to train and obtain the classification model, fluorescent images with artificial positive and negative annotated are collected as original data, and based on the method of steps S1 to S6 in Figure 4, a feature matrix F is obtained for each fluorescent image, and each feature matrix F is used as a sample to form a sample set. In addition, the sample set can also be obtained by using the samples in the training set of the aforementioned target detection model as original data. The sample set is divided into a training set and an evaluation set according to 80% and 20%, and after training, the classification model is obtained after the model converges and evaluates and meets the relevant requirements. Since the sample data includes multiple cell area categories, the classification model after training is completed, for different categories in the input, matches the features in the input with the weights corresponding to their categories, thereby accurately classifying positive and negative. Compared with manual interpretation, the classification model can ensure the stability of processing cells of different fluorescent types, and has high processing efficiency and accurate classification.

In addition, the processing method provided by the present invention further includes a titer level classification processing step. After obtaining the ROI mask, the ROI mask and its ROI constitute a first titer level classification set. See FIG. 20 , which is a flow chart of a titer level classification method according to an embodiment of the present invention. The method includes the following steps:

Step S81, the grayscale means of the cell area of the ROI mask in the first titer level classification set are arranged in reverse order to form a grayscale mean sequence.

Step S82, calculating the average value of the grayscale mean value sequence.

Step S83, obtaining the grayscale mean of the corresponding number of bits from the grayscale mean sequence according to the preset number of bits of the sequence.

Step S84, calculating the average value of the grayscale mean value sequence and the weighted sum of the grayscale mean values of the corresponding bits, and using the weighted sum as the brightness value of the indirect immunofluorescence image.

Step S85, comparing the brightness value of the indirect immunofluorescence image with the brightness threshold defining the titer level to obtain a corresponding titer level, and determining the titer level as the titer level of the test sample.

The preset sequence digits in step S83 are, for example, the median, the quarter digit, etc. When the median or the quarter digit is used, the brightness value I of the indirect immunofluorescence image is calculated in step S84 according to the following formula 1-12:

1-12

in, is the 1/4 digit of the grayscale mean sequence, is the median of the grayscale mean sequence, is the average value of the grayscale mean sequence. are the corresponding weights respectively, which are known empirical values.

In one embodiment, four titer levels are set, and the corresponding brightness threshold intervals are shown in Table 1 below:

Table 1

After the brightness value I of the indirect immunofluorescence image is calculated according to formula 1-12, in step S85, the brightness value I is compared with the five brightness thresholds in Table 1 to determine the brightness threshold interval, thereby determining the corresponding titer level.

In another embodiment, before the aforementioned step S81, a step of screening out ROI is further included. Referring to FIG. 21 , FIG. 21 is a flow chart of a method for screening out ROI before titer grade classification according to an embodiment of the present invention, and the method includes the following steps:

Step S71, respectively calculating the grayscale variance of each ROI in the first titer level classification set.

Step S72: Screen out ROIs and corresponding ROI masks whose grayscale variance is greater than or equal to a second variance threshold from the first titer level classification set.

Step S73, based on the remaining ROIs and the corresponding ROI masks, respectively calculate the cell region grayscale mean A1, the background region grayscale mean B1, and the difference C1 between the cell region grayscale mean A1 and the background region grayscale mean B1 in each ROI.

Step S74, ROIs whose cell region grayscale mean A1 is less than the second threshold or whose difference C1 between the cell region grayscale mean and the background region grayscale mean is less than the third threshold are screened out from the current first titer level classification set. That is to say, for an ROI in the current first titer level classification set, it is retained only if it satisfies both the cell region grayscale mean A1 greater than or equal to the second threshold and the difference C1 between the cell region grayscale mean and the background region grayscale mean is greater than or equal to the third threshold.

When the grayscale variance of the ROI is greater than or equal to the second variance threshold, it indicates that the current ROI imaging is abnormal, or the brightness imaging is overexposed, and such areas are no longer meaningful for titer assessment. The embodiment of the present invention uses grayscale variance to screen ROIs suitable for titer assessment, thereby improving the overall calculation efficiency.

In addition, if the grayscale mean A1 of the cell area is less than the second threshold, it means that the brightness of the cell area is not enough. When the brightness of the cell area is not much different from that of the background area, it is also meaningless for titer assessment. By screening out ROIs that are not meaningful for titer assessment, the overall calculation efficiency is further improved, and the accuracy of titer grade classification is improved.

The embodiment of the present invention utilizes a deep learning algorithm to extract corresponding fluorescence features based on the characteristics of the fluorescence image of the detection sample, and can simultaneously realize quantitative analysis and qualitative classification of the immunofluorescence image, effectively solving the problem of false positive and false detection caused by noise interference such as nonspecific fluorescence and impurities. In addition, the method provided by the present invention has strong generalization performance and can be applied to the detection of different categories of viral antibodies. Since the characteristics of the fluorescence image of the detection sample are utilized during feature extraction, the influence of nonspecific fluorescence on the classification results is reduced, and thus the detection results of antibody positive and negative and the classification accuracy of titer levels are high.

On the other hand, referring to FIG. 22, FIG. 22 is a principle block diagram of an indirect immunofluorescence image processing system according to an embodiment of the present invention. The processing system 10 in this embodiment includes an image acquisition module 11, a target detection module 12, a binarization module 13, a connected domain detection module 14, a mask module 15, a feature extraction module 16 and a classification module 17, wherein the image acquisition module 11 is used to obtain a fluorescent image of a test sample prepared based on the indirect immunofluorescence method, and the test sample is, for example, an EBV antibody test sample with an antigen of VCA or EA, a Coxsackie A/B virus antibody test sample, an enterovirus antibody test sample, an adenovirus antibody test sample, etc. In one embodiment, the image acquisition module 11 can read the fluorescent image of the specified test sample from a storage device by data reading, or read the fluorescent image of the specified test sample from another device through a network, and can also be connected to a fluorescent microscope in real time to receive the fluorescent image of the test sample taken in real time by the fluorescent microscope. The image acquisition module 11 also performs corresponding preprocessing on the fluorescent image obtained above, such as cropping, normalization, etc.

The target detection module 12 receives the indirect immunofluorescence image pre-processed by the image acquisition module 11, and performs target detection to obtain multiple regions of interest, wherein each region of interest includes a suspected cell region and a background region. Referring to the first group of images in Figures 14 and 15, the specific processing flow is described in the description of the aforementioned method section, which will not be repeated here.

The binarization module 13 is connected to the target detection module 12, and performs binarization processing on each region of interest processed by the target detection module 12 to obtain a binarized image, wherein the grayscale values of the binarized image are respectively the first grayscale value and the second grayscale value, and the first grayscale value and the second grayscale value are not equal. Referring to the second group of images in FIG. 14 and FIG. 15, the first grayscale value and the second grayscale value are respectively 0 and 255. For the specific processing flow, please refer to the description of the aforementioned method part, which will not be repeated here.

The connected domain detection module 14 is connected to the binarization module 13 to perform connected domain detection on the binarized image of each region of interest to filter out connected domains that do not meet the cell-specific conditions.

The mask module 15 is connected to the connected domain detection module 14, and the connected domain that meets the cell-specific conditions is used as the cell region and assigned the same grayscale value to obtain the region of interest mask, wherein the grayscale values of the cell region pixel points and the background region pixel points in the region of interest mask are the first grayscale value and the second grayscale value, respectively, see the third group of images in Figures 14 and 15. For the specific processing flow, please refer to the description of the aforementioned method part. For the specific processing flow, please refer to the description of the aforementioned method part, which will not be repeated here.

The feature extraction module 16 is connected to the mask module 15, and performs contour extraction on each cell region, and based on the contour of each cell region, extracts the feature values of multiple preset target features from the cell region. For the specific processing flow, please refer to the description of the aforementioned method part, which will not be repeated here.

The classification module 17 is connected to the feature extraction module 16, and constructs a model input based on the feature values of multiple target features of a preset number of cell regions in the cell region set, and inputs the model input to the classification model, and the classification model obtains whether the antibody category of the indirect immunofluorescence image corresponding to the detection sample is positive or negative. Please refer to the description of the aforementioned method section for the specific processing flow, which will not be repeated here.

In another embodiment, the processing system 10 further includes a titer level module 18, as shown in the dotted box in FIG22, which is connected to the mask module 15 and is configured to obtain the brightness value of the indirect immunofluorescence image by calculating the gray value of the cell area of the region of interest mask; by comparing the brightness value of the indirect immunofluorescence image with the brightness threshold defining the titer level to obtain the corresponding titer level, the titer level is determined as the titer level of the test sample. Furthermore, before calculating the gray value of the cell area of the ROI mask, ROIs that have no titer significance can be screened out, thereby improving the overall calculation efficiency and the accuracy of titer level classification.

In addition, in order to facilitate human-computer interaction, the processing system 10 includes an interactive interface and a database. The staff can issue image reading instructions or shooting instructions, start processing instructions, etc. through the interactive interface, and can view various intermediate data and final result data of the processing system 10. After the processing system 10 processes a fluorescent image of the test sample, it stores its intermediate processing data and result data in the database for preservation. In addition, the processing system 10 can also provide configuration options in the interactive interface, such as configuring whether to include the titer level and the conditions for including the titer level in the processing results. For example, titer level classification is performed while the antibody positive and negative detection is performed; titer level classification is performed when the antibody test is positive, etc., and the staff can select the corresponding options as needed.

Table 2 shows the performance evaluation results obtained by testing the system and method provided by the present invention using Epstein-Barr virus fluorescence images as a test set.

Table 2

Among them, Sensitivity indicates the sensitivity of judgment, Specificity indicates the specificity of judgment, and Accuracy indicates the accuracy. It can be seen from the data in Table 2 that in the detection of VCA-IgA fluorescence images and in the detection of EA-IgA fluorescence images, high specificity performances of 98.6% and 95.8% were achieved respectively, while high sensitivity and accuracy were also guaranteed, indicating that false positive cells were efficiently filtered out.

FIG23 is a schematic diagram of the hardware structure principle of an electronic device according to an embodiment of the present invention, the electronic device can be implemented as a server or other terminal devices, such as a desktop personal computer, a tablet computer, a laptop computer, etc., including a processor 601 and a memory 602, the memory 602 stores a program instruction set, and when the processor 601 executes the program instruction set on the memory 602, any of the aforementioned indirect immunofluorescence image processing methods is implemented. The electronic device in this embodiment is, for example, the computing device 101 in FIG1 .

Specifically, the processor 601 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiment of the present invention.

The memory 602 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 602 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory 602 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 602 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 602 is a non-volatile solid-state memory.

The memory may include a read-only memory (ROM), a random access memory (RAM), a disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical/tangible memory storage device. Therefore, generally, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the indirect immunofluorescence image processing method provided by the present invention.

In one example, the electronic device may further include a communication interface 603 and a bus 604. The processor 601, the memory 602, and the communication interface 603 are connected via the bus 604 and communicate with each other.

The communication interface 603 is mainly used to implement communication between various modules, systems, units and/or devices in the embodiment of the present invention.

Bus 604 includes hardware, software or both, and the components of online data flow billing equipment are coupled to each other. For example, but not limitation, the bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front-side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. Where appropriate, bus 604 may include one or more buses. Although the embodiment of the present invention describes and shows a specific bus, the present invention considers any suitable bus or interconnection.

The present invention also provides a computer-readable storage medium having computer program instructions stored thereon, and the computer program instructions can be executed by a processor to implement any of the indirect immunofluorescence image processing methods in the aforementioned embodiments. The computer-readable storage medium can be any medium that can tangibly contain or store computer-executable instructions for use by or in combination with an instruction execution system, device, and equipment. The storage medium can be a transient computer-readable storage medium or a non-transient computer-readable storage medium. Non-transient computer-readable storage media may include, but are not limited to, magnetic storage devices, optical storage devices, and/or semiconductor storage devices. Examples of such storage devices include, for example, magnetic disks, optical disks based on CD, DVD, or Blu-ray technology, and persistent solid-state memories such as flash memory, solid-state drives, and the like.

The present invention also provides a computer program product, which includes a set of computer program instructions, and when the set of computer program instructions is executed by a processor, any one of the indirect immunofluorescence image processing methods in the aforementioned embodiments is implemented. The computer program product includes but is not limited to application installation packages published on websites and application stores, application plug-ins, and small programs that can be run in certain applications.

It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

The above embodiments are only used to illustrate the present invention, but not to limit the present invention. Ordinary technicians in the relevant technical field can make various changes and modifications without departing from the scope of the present invention. Therefore, all equivalent technical solutions should also fall within the scope of the present invention.

Claims (19)

1. A method for processing an indirect immunofluorescence image, wherein the indirect immunofluorescence image is a fluorescent image of an antibody detection sample of a specific virus, characterized in that the method comprises: Performing target detection on the indirect immunofluorescence image to obtain multiple regions of interest, wherein each region of interest includes a suspected cell region and a background region; Performing binarization processing on each region of interest to obtain a binarized image, wherein the grayscale value of the binarized image is respectively a first grayscale value and a second grayscale value, and the first grayscale value and the second grayscale value are not equal; Performing a connected domain detection on the binary image of each region of interest to screen out connected domains that do not meet the cell-specific conditions, taking the connected domains that meet the cell-specific conditions as cell regions and assigning them the same grayscale value to obtain a region of interest mask, wherein the grayscale values of the cell region pixel points and the background region pixel points in the region of interest mask are the first grayscale value and the second grayscale value, respectively; Performing contour extraction on each cell region, and extracting feature values of a preset plurality of target features from the cell region based on the contour of each cell region; and A model input is constructed based on the characteristic values of multiple target features of a preset number of cell regions, and the model input is input into a trained classification model. The classification model determines whether the antibody category of the indirect immunofluorescence image corresponding to the detection sample is positive or negative. 2. The method for processing indirect immunofluorescence images according to claim 1, wherein the step of performing target detection on the indirect immunofluorescence image to obtain a plurality of regions of interest comprises: Inputting the indirect immunofluorescence image to a trained target detection model, and obtaining a plurality of preliminary regions of interest and description information thereof after being processed by the target detection model, wherein the description information includes region coordinates, region categories, and region confidences, and the plurality of preliminary regions of interest constitute a preliminary region of interest set; Determine a new region confidence for each preliminary region of interest based on the overlapped area determined by the region coordinates and the region confidence as calculation factors; Comparing the new region confidence of each preliminary region of interest with the confidence threshold; and In response to the new region confidence being less than the confidence threshold, the preliminary region of interest corresponding to the new region confidence is removed from the preliminary region of interest set, and the remaining preliminary regions of interest are determined as regions of interest. 3. The method for processing indirect immunofluorescence images according to claim 2, wherein the step of determining the new regional confidence of each preliminary region of interest based on the overlapping area determined by the regional coordinates and the regional confidence as calculation factors comprises: Obtaining the first region coordinates of the first preliminary region of interest with the maximum current region confidence and the target region coordinates and the target region confidence corresponding to the target preliminary region of interest; Calculate the intersection-over-union ratio of the first preliminary region of interest and the target preliminary region of interest based on the first region coordinates and the target region coordinates; Using the intersection-and-union ratio as an independent variable of a preset function, a function value is calculated; and The function value is used as the weight of the target region confidence to calculate the new region confidence of the target preliminary region of interest. 4 . The method for processing indirect immunofluorescence images according to claim 3 , wherein the preset function is a one-dimensional Gaussian variance function. 5. The method for processing indirect immunofluorescence images according to claim 1, wherein the step of performing binarization processing on each region of interest comprises: Calculate the grayscale variance of each region of interest; Comparing the grayscale variance with a first variance threshold; In response to a grayscale variance of the region of interest being greater than or equal to a first variance threshold, performing adaptive threshold segmentation on the region of interest to obtain a binarized segmentation threshold, and performing binarization processing on the region of interest based on the segmentation threshold; and In response to the grayscale variance of the region of interest being less than a first variance threshold, edge detection is performed on the region of interest, and binarization is performed on the region of interest based on the edge detection result. 6. The method for processing indirect immunofluorescence images according to claim 5, characterized in that after binarization processing is performed on each region of interest, the method further comprises: Calculate the structural similarity between each region of interest binary image and its corresponding region of interest respectively; After the binarization of all the regions of interest is completed, the following steps are further performed: Calculate the loss of binarization processing based on the structural similarity between all the binarized images of the regions of interest and their corresponding regions of interest; comparing the loss amount to a loss threshold; In response to the loss amount being greater than or equal to a loss threshold, adjusting the first variance threshold; and Based on the adjusted first variance threshold, each region of interest is re-binarized until the loss amount of the binarization process is less than the loss threshold. 7. The method for processing indirect immunofluorescence images according to claim 5, further comprising the step of obtaining the first variance threshold: Acquire a sample set, where the samples in the sample set are regions of interest obtained after performing target detection on the sample indirect immunofluorescence image; Calculating the grayscale variance of each region of interest and comparing the grayscale variance of each region of interest with a preset first threshold; In response to a grayscale variance of a region of interest being greater than or equal to a first threshold, performing adaptive threshold segmentation on the region of interest to obtain a binarized segmentation threshold, and performing binarization processing on the region of interest based on the segmentation threshold to obtain a binarized image of the region of interest; in response to a grayscale variance of a region of interest being less than the first threshold, performing edge detection on the region of interest, and performing binarization processing on the region of interest based on the edge detection result to obtain a binarized image of the region of interest; Calculate the structural similarity between the binary image of each region of interest and its corresponding region of interest respectively; After all the regions of interest are binarized, the loss of the binarization is calculated based on the structural similarity between the binarized images of all the regions of interest and their corresponding regions of interest; Adjusting the size of the first threshold; Based on the adjusted first threshold, re-binarize each region of interest and calculate the loss amount of the binarization process; and After a preset number of adjustments to the first threshold and calculation of the loss amount, the first threshold with the smallest loss amount is determined as the first variance threshold. 8. The indirect immunofluorescence image processing method according to claim 6 or 7, characterized in that the step of calculating the loss amount of binarization processing based on the structural similarity between the binarized images of all the regions of interest and their corresponding regions of interest comprises: Counting the first regions of interest and their number after binarizing the regions of interest based on the segmentation threshold; Calculate the average value and variance of the first structural similarity based on the first structural similarity between the binary image of each first region of interest and its corresponding first region of interest and the number of first regions of interest; Counting the second regions of interest and their number after binarizing the regions of interest based on the edge detection results; Calculate the average value and variance of the second structural similarity based on the second structural similarity between the binary image of each second region of interest and its corresponding second region of interest and the number of second regions of interest; respectively calculating a first ratio of an average value of the first structural similarity and its variance and a second ratio of an average value of the second structural similarity and its variance; Taking the ratio of the number of the first regions of interest to the total number of regions of interest as the weight of the first ratio, taking the ratio of the number of the second regions of interest to the total number of regions of interest as the weight of the second ratio, respectively calculating a first weighted value of the first ratio and a second weighted value of the second ratio; and The first weighted value and the second weighted value are used as independent variables of a preset cost function to calculate a corresponding cost function value, and the cost function value is used as a loss amount of binarization processing. 9. The method for processing indirect immunofluorescence images according to claim 2, wherein the step of performing connected domain detection on the binary image of each region of interest comprises: Calculate the area and centroid coordinates of each connected domain in each region of interest; Calculate the center coordinates of the region of interest based on the region coordinates; Calculating the distance between the centroid coordinates of each connected domain and the center coordinates of the region of interest; and When the area of the connected domain is less than the area threshold or the distance between the centroid coordinates of the connected domain and the center coordinates of the region of interest is greater than or equal to the distance threshold, it is determined that the connected domain does not meet the cell-specific condition and the connected domain is screened out from the region of interest. When the area of the connected domain is greater than or equal to the area threshold and the distance between the centroid coordinates of the connected domain and the center coordinates of the region of interest is less than the distance threshold, it is determined that the connected domain meets the cell-specific condition. 10. The method for processing indirect immunofluorescence images according to claim 1, wherein the preset target feature is a cell shape feature, and correspondingly, the step of extracting a feature value of the cell shape feature from the cell region comprises: Calculate the inscribed area of the cell region based on its contour; Calculating the cell region perimeter based on the cell region contour; and The circularity of the cell region is calculated based on the inscribed area and the perimeter of the cell region, wherein the inscribed area of the cell region is the first cell shape feature; and the circularity of the cell region is the second cell shape feature. 11. The indirect immunofluorescence image processing method according to claim 1, characterized in that the preset target feature is a cell edge morphological feature, and correspondingly, the step of extracting a feature value of the cell edge morphological feature from the cell region comprises: The contour of the cell area is sampled multiple times at intervals of a preset number of pixels, and a preset number of pixels are obtained each time the sampling is performed; Calculating the curvature based on a preset number of pixel point coordinates obtained by each sampling to obtain a curvature set; and The curvature variance is calculated based on multiple curvatures in the curvature set, wherein the curvature variance is used as a characteristic value of the cell edge morphological feature. 12. The method for processing indirect immunofluorescence images according to claim 1, wherein the preset target feature is a fluorescence regularity feature, and correspondingly, the step of extracting a feature value of the fluorescence regularity feature from the cell region comprises: Performing gray-level co-occurrence matrix conversion based on the gray-level values of the pixels in the inner area of the cell area contour to obtain a gray-level co-occurrence matrix; and The energy of the inner area of the contour is calculated based on the gray level co-occurrence matrix elements; wherein the energy is used as the characteristic value of the fluorescence regularity feature. 13. The method for processing indirect immunofluorescence images according to claim 1, characterized in that after obtaining the region of interest mask, the region of interest mask and its region of interest constitute a first titer level classification set, and the method further comprises: Arranging the grayscale means of the cell regions of the region of interest mask in the first titer level classification set in reverse order to form a grayscale mean sequence; Calculating the average value of the grayscale mean sequence; According to the preset sequence number of bits, the grayscale mean of the corresponding bit number is obtained from the grayscale mean sequence; Calculating the average value of the grayscale mean value sequence and the weighted sum of the grayscale mean values of the corresponding digits, and using the weighted sum as the brightness value of the indirect immunofluorescence image; and The brightness value of the indirect immunofluorescence image is compared with the brightness threshold for defining the titer level to obtain the corresponding titer level. 14. The method for processing indirect immunofluorescence images according to claim 13, characterized in that before the grayscale means of the cell regions of the ROI mask in the first titer level classification set are arranged in reverse order to form a grayscale mean sequence, it also comprises: Calculate the grayscale variance of each region of interest in the first titer level classification set respectively; Screening out regions of interest whose grayscale variance is greater than or equal to a second variance threshold and corresponding region of interest masks from the first titer level classification set; Based on the remaining regions of interest, respectively calculate the grayscale mean of the cell region, the grayscale mean of the background region, and the difference between the grayscale mean of the cell region and the grayscale mean of the background region in each region of interest; and Regions of interest whose cell region grayscale mean is less than a second threshold or whose difference between the cell region grayscale mean and the background region grayscale mean is less than a third threshold are screened out from the current first titer level classification set. 15. A processing system for an indirect immunofluorescence image, wherein the indirect immunofluorescence image is a fluorescent image of an antibody detection sample of a specific virus, characterized in that it comprises: A target detection module is configured to perform target detection on the indirect immunofluorescence image to obtain a plurality of regions of interest, wherein each region of interest includes a suspected cell region and a background region; A binarization module, which is connected to the target detection module and is configured to perform binarization processing on each region of interest to obtain a binarized image, wherein the grayscale values of the binarized image are respectively a first grayscale value and a second grayscale value, and the first grayscale value and the second grayscale value are not equal; A connected domain detection module, which is connected to the binarization module and is configured to perform connected domain detection on the binarized image of each region of interest to screen out connected domains that do not meet cell-specific conditions; A mask module, which is connected to the connected domain detection module and is configured to use the connected domain that meets the cell-specific condition as the cell region and assign the same grayscale value to obtain a mask of the region of interest, wherein the grayscale values of the cell region pixel point and the background region pixel point in the mask of the region of interest are the first grayscale value and the second grayscale value respectively; a feature extraction module connected to the mask module and configured to extract the contour of each cell region; based on the contour of each cell region, extract feature values of a plurality of preset target features from the cell region; and A classification module is connected to the feature extraction module and is configured to construct a model input based on the feature values of multiple target features of a preset number of cell regions in the cell region set, and input the model input to the trained classification model. The classification model determines whether the antibody category of the indirect immunofluorescence image corresponding to the detection sample is positive or negative. 16. The indirect immunofluorescence image processing system according to claim 15 is characterized in that it further includes a titer level module, which is connected to the mask module and is configured to obtain the brightness value of the indirect immunofluorescence image by calculating the grayscale value of the cell area of the region of interest; and to obtain the corresponding titer level by comparing the brightness value of the indirect immunofluorescence image with the brightness threshold that defines the titer level. 17. An electronic device, characterized in that it comprises a processor and a memory, wherein a computer program instruction set is stored in the memory, and when the processor executes the computer program instruction set in the memory, the indirect immunofluorescence image processing method according to any one of claims 1 to 14 is implemented. 18. A computer-readable storage medium, characterized in that a computer program instruction set is stored on the computer-readable storage medium, and when the computer program instruction set is executed by a processor, the method for processing indirect immunofluorescence images according to any one of claims 1 to 14 is implemented. 19. A computer program product, characterized in that it comprises a computer program instruction set, and when the computer program instruction set is executed by a processor, the method for processing indirect immunofluorescence images according to any one of claims 1 to 14 is implemented.