CN113850796B

CN113850796B - Lung disease recognition method, device, medium and electronic device based on CT data

Info

Publication number: CN113850796B
Application number: CN202111187917.3A
Authority: CN
Inventors: 王雄; 裴璇
Original assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Current assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date: 2021-10-12
Filing date: 2021-10-12
Publication date: 2024-11-12
Anticipated expiration: 2041-10-12
Also published as: CN113850796A

Abstract

The disclosure provides a lung disease identification method and device based on CT data, a computer readable medium and electronic equipment, and relates to the technical field of image processing. The method comprises the following steps: acquiring a 3D tensor corresponding to lung CT data to be identified, and blocking the 3D tensor to obtain k block tensors; carrying out pulmonary disease classification and identification on the k segmented tensors to obtain k segmented classification results; and outputting the identification result corresponding to the lung CT data to be identified according to the k block classification results, and determining the focus block tensor corresponding to the lung CT data to be identified in the k block tensors when the identification result is an abnormal result. By partitioning the 3D tensor, the method can avoid sampling the 3D tensor, and further avoid the loss of key focus information or the introduction of repeated invalid noise; meanwhile, the focus block tensor can be determined in the k block tensors directly through the k block classification results, and further coarse positioning of the focus is realized based on the focus block tensor.

Description

CT data-based lung disease identification method and device, medium and electronic equipment

Technical Field

The disclosure relates to the technical field of image processing, in particular to a lung disease identification method based on CT data, a lung disease identification device based on CT data, a computer readable medium and electronic equipment.

Background

CT (Computed Tomography), i.e. electronic computed tomography, whose imaging principle is: the specific part of human body is scanned by X-ray beam, gamma ray, ultrasonic wave, etc. to obtain medical image after computer processing. Compared with the conventional imaging examination means, CT has the advantages of being capable of acquiring a real sectional image, high in density resolution, capable of quantitatively analyzing, convenient for subsequent image processing and the like, and therefore has wider and wider application in medical image detection.

In recent years, rapidly evolving computer-aided diagnosis techniques have greatly facilitated the diagnostic analysis of medical CT images. For lung CT images, there are typically three auxiliary analysis methods: firstly, 3D data of lung CT are sampled to 3D tensors with fixed sizes in an interpolation or sampling mode, and then the 3D tensors with fixed sizes are input into a 3D convolution model for diagnosis; secondly, realizing diagnosis by a feature classifier based on a preamble segmentation task, and inputting the type, the type and the number of features which are required to be extracted by artificial designs such as color, texture, shape features and the like of a focus into the classifier to realize diagnosis; and thirdly, realizing diagnosis by a feature classifier based on a fuzzy mode, constructing a feature space by manually designing or automatically extracting features by a convolutional neural network, and carrying out feature dimension reduction and weighting on the feature space based on the fuzzy mode.

However, in the three auxiliary analysis methods, the first method involves sampling of a fixed-size 3D tensor, due to lack of prior information, deletion of key focus information or introduction of repeated ineffective noise is easily caused; the second approach involves the need to provide accurate lesion area labels when training the feature classifier, thus labeling is time-consuming and labor-consuming; the third mode needs to manually design a feature space or a feature space screening strategy, has higher dependence on the feature quality of an input end, and is easy to cause model overfitting.

Disclosure of Invention

The present disclosure aims to provide a method for identifying lung diseases based on CT data, a device for identifying lung diseases based on CT data, a computer readable medium and an electronic device, which can provide spatial localization information of a lesion by determining a lesion block tensor on the basis of identifying lung diseases based on complete original CT data.

According to a first aspect of the present disclosure, there is provided a method for identifying a pulmonary disease based on CT data, comprising: acquiring a 3D tensor corresponding to lung CT data to be identified, and blocking the 3D tensor to obtain k block tensors; k is a positive integer greater than or equal to 2; carrying out pulmonary disease classification and identification on the k segmented tensors to obtain k segmented classification results; and outputting the identification result corresponding to the lung CT data to be identified according to the k block classification results, and determining the focus block tensor corresponding to the lung CT data to be identified in the k block tensors when the identification result is an abnormal result.

According to a second aspect of the present disclosure, there is provided a pulmonary disease recognition device based on CT data, comprising: the block acquisition module is used for acquiring a 3D tensor corresponding to the lung CT data to be identified and carrying out block segmentation on the 3D tensor to obtain k block tensors; k is a positive integer greater than or equal to 2; the classification and identification module is used for carrying out pulmonary disease classification and identification on the k segmented tensors to obtain k segmented classification results; and the result output module is used for outputting the identification result corresponding to the lung CT data to be identified according to the k block classification results, and determining the focus block tensor corresponding to the lung CT data to be identified in the k block tensors when the identification result is an abnormal result.

According to a third aspect of the present disclosure, there is provided a computer readable medium having stored thereon a computer program which, when executed by a processor, implements the method described above.

According to a fourth aspect of the present disclosure, there is provided an electronic apparatus, comprising: a processor; and a memory for storing one or more programs that, when executed by the one or more processors, cause the one or more processors to implement the methods described above.

According to the lung disease identification method based on the CT data, k segmented tensors can be obtained by obtaining the 3D tensors corresponding to the lung CT data to be identified and blocking the 3D tensors, then the k segmented tensors are respectively subjected to lung disease classification and identification to obtain k segmented classification results, further the identification results corresponding to the lung CT data to be identified can be output according to the k segmented classification results, and the focus segmented tensors are determined in the k segmented tensors. By partitioning the 3D tensor, not only can the sampling of the 3D tensor be avoided, but also the loss of key focus information or the introduction of repeated invalid noise can be avoided; meanwhile, the focus block tensor can be determined in the k block tensors directly through the k block classification results, and further coarse positioning of the focus is realized based on the focus block tensor.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure. It will be apparent to those of ordinary skill in the art that the drawings in the following description are merely examples of the disclosure and that other drawings may be derived from them without undue effort. In the drawings:

FIG. 1 illustrates a schematic diagram of an exemplary system architecture to which embodiments of the present disclosure may be applied;

FIG. 2 shows a schematic diagram of an electronic device to which embodiments of the present disclosure may be applied;

FIG. 3 schematically illustrates a flowchart of a method for identifying pulmonary diseases based on CT data in an exemplary embodiment of the present disclosure;

Fig. 4 schematically illustrates a flowchart for determining a recognition result and a focus blocking tensor in an exemplary embodiment of the present disclosure;

FIG. 5 schematically illustrates a schematic diagram of a training process of a pulmonary disease recognition neural network in an exemplary embodiment of the disclosure;

Fig. 6 schematically illustrates another flow diagram for determining a recognition result and a lesion blocking tensor in an exemplary embodiment of the present disclosure;

Fig. 7 schematically illustrates a composition diagram of a lung disease identification device based on CT data in an exemplary embodiment of the present disclosure.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments may be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus a repetitive description thereof will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software or in one or more hardware modules or integrated circuits or in different networks and/or processor devices and/or microcontroller devices.

Fig. 1 is a schematic diagram of a system architecture of an exemplary application environment in which a method and apparatus for identifying pulmonary diseases based on CT data according to an embodiment of the present disclosure may be applied.

As shown in fig. 1, the system architecture 100 may include one or more of the terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium to provide communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others. The terminal devices 101, 102, 103 may be various electronic devices having image recognition functions, including, but not limited to, desktop computers, portable computers, smart phones, tablet computers, and the like. It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation. For example, the server 105 may be a server cluster formed by a plurality of servers.

The lung disease identification method based on CT data provided by the embodiments of the present disclosure may be executed by the terminal devices 101, 102, 103, and accordingly, the lung disease identification apparatus based on CT data is generally provided in the terminal devices 101, 102, 103; may also be performed by the server 105, and accordingly, a lung disease recognition device based on CT data may also be provided in the server 105, which is not particularly limited in the present exemplary embodiment. For example, in an exemplary embodiment, the terminal device 101 may be a device capable of directly acquiring a 3D tensor corresponding to the lung CT data, for example, a tomographic device, and the other terminal devices 102, 103 and the server 105 may acquire the 3D tensor corresponding to the lung CT data acquired by the terminal device 101 through a network, so as to perform lung disease identification.

Exemplary embodiments of the present disclosure provide an electronic device, which may be a terminal device 101, 102, 103 or a server 105 in fig. 1, for implementing a method of pulmonary disease identification based on CT data. The electronic device comprises at least a processor and a memory for storing executable instructions of the processor, the processor being configured to perform a lung disease identification method based on CT data via execution of the executable instructions.

The configuration of the electronic device will be exemplarily described below using the mobile terminal 200 of fig. 2 as an example. It will be appreciated by those skilled in the art that the configuration of fig. 2 can also be applied to stationary type devices in addition to components specifically for mobile purposes. In other embodiments, mobile terminal 200 may include more or less components than illustrated, or certain components may be combined, or certain components may be split, or different arrangements of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware. The interfacing relationship between the components is shown schematically only and does not constitute a structural limitation of the mobile terminal 200. In other embodiments, the mobile terminal 200 may also employ a different interface from that of fig. 2, or a combination of interfaces.

As shown in fig. 2, the mobile terminal 200 may specifically include: processor 210, internal memory 221, external memory interface 222, universal serial bus (Universal Serial Bus, USB) interface 230, charge management module 240, power management module 241, battery 242, antenna 1, antenna 2, mobile communication module 250, wireless communication module 260, audio module 270, speaker 271, receiver 272, microphone 273, headset interface 274, sensor module 280, display screen 290, camera module 291, indicator 292, motor 293, keys 294, and subscriber identity module (subscriber identification module, SIM) card interface 295, among others. Wherein the sensor module 280 may include a depth sensor 2801, a pressure sensor 2802, a gyro sensor 2803, and the like.

Processor 210 may include one or more processing units such as, for example: the Processor 210 may include an application Processor (Application Processor, AP), a modem Processor, a graphics Processor (Graphics Processing Unit, GPU), an image signal Processor (IMAGE SIGNAL Processor, ISP), a controller, a video codec, a digital signal Processor (DIGITAL SIGNAL Processor, DSP), a baseband Processor and/or a neural network Processor (Neural-Network Processing Unit, NPU), and the like. Wherein the different processing units may be separate devices or may be integrated in one or more processors.

The NPU is a neural Network (Neural-Network, NN) computing processor, and can rapidly process input information by referencing a biological neural Network structure, such as referencing a transmission mode among human brain neurons, and can continuously learn. Applications such as intelligent awareness of the mobile terminal 200 may be implemented by the NPU, for example: image recognition, face recognition, speech recognition, text understanding, etc.

In some embodiments, the step of performing lung disease classification and identification on the k segmented tensors to obtain k segmented classification results, the step of outputting an identification result corresponding to the lung CT data to be identified according to the k segmented classification results, and determining a focus segmented tensor corresponding to the lung CT data to be identified in the k segmented tensors, and the step of adjusting and determining parameters in the lung disease identification neural network may be implemented through the NPU.

The processor 210 has a memory disposed therein. The memory may store instructions for implementing six modular functions: detection instructions, connection instructions, information management instructions, analysis instructions, data transfer instructions, and notification instructions, and are controlled to be executed by the processor 210.

The mobile terminal 200 implements display functions through a GPU, a display screen 290, an application processor, and the like. The GPU is a microprocessor for image processing, and is connected to the display screen 290 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 210 may include one or more GPUs that execute program instructions to generate or change display information. In some embodiments, the steps of blocking the 3D tensor may be implemented by a GPU.

In the related art, three auxiliary analysis methods are generally included for lung CT images.

Firstly, 3D data of lung CT is sampled to a 3D tensor with fixed size in an interpolation or sampling mode, and then the 3D tensor with fixed size is input into a 3D convolution model for diagnosis. For example, in Wang X, jiang L, li L et al, entitled "Joint learning of 3D lesion segmentation and classification for explainable COVID-19 diagnostics," a 3D convolutional neural network with 3 branches is proposed to extract underlying fusion features at multiple scales by scaling and interpolating downsampling CT data and using 3D convolutional neural network 1; then, using the 3D convolutional neural network branch 2 to process part of the characteristics output by the branch 1 to apply to a focus segmentation task and outputting a focus segmentation result; and finally, using the 3D convolutional neural network branch 3 to receive part of the characteristics output by the branch 1 and the segmentation result output by the branch 2, and outputting the final recognition result of the new coronary lung disease and the community acquired lung disease.

However, this method requires sampling the lung CT data to a 3D tensor of a fixed size by interpolation or sampling at the input end, and inputting the 3D tensor into a 3D convolution model, and due to lack of prior information, it is very easy to cause the loss of critical lesion information or the introduction of repeated ineffective noise.

Secondly, the feature classifier based on the preamble segmentation task is used for realizing diagnosis, and the type, the type and the number of the features which are required to be extracted by artificial designs such as the color, the texture, the shape features and the like of the focus are input into the classifier for realizing diagnosis. For example, in the patent application with publication number CN111724356a, an image processing method and system for identifying a lung disease of a CT image are provided, the image processing method firstly extracts a mask of a lung disease region based on a lung disease segmentation model, and then performs weight adjustment on a convolution feature at a spatial level by introducing an attention mechanism, so that the feature of the lung disease region is more prominent; scaling the three-dimensional CT sequence according to the transverse position, classifying each positive layer (layer with lung disease) of the sequence layer by layer, weighting the mask area and the classification result of the corresponding lung disease, and voting the class with the highest probability to obtain the lung disease classification result of the three-dimensional CT sequence.

However, the feature classifier based on the pre-segmentation task employed in this approach needs to provide accurate lesion area labels when training the model. Because the labeling of the labels is carried out manually, the time and the labor are consumed, and the labeling is limited by different understanding of labeling personnel on professional knowledge, so that the labeling quality is good and bad, the corresponding training sample size is small, and the generalization capability of the model obtained through training is required to be verified.

And thirdly, realizing diagnosis by a feature classifier based on a fuzzy mode, constructing a feature space by manually designing or automatically extracting features by a convolutional neural network, and carrying out feature dimension reduction and weighting on the feature space based on the fuzzy mode. For example, patent application publication No. CN111414956a provides a multi-example learning identification method for a blur pattern in a lung CT image, which uses a pre-training convolutional neural network to perform feature extraction on an acquired CT image, regards a single CT image as a packet, uses features extracted on the single CT image as an example, and uses a multi-example learning model (including a k nearest neighbor model Citation-KNN, a multi-example support vector machine model MI-SVM, and a desired maximization-diversity density model EM-DD) to perform classification tasks.

However, the method mainly uses the characteristics automatically extracted by the artificial design or the convolutional neural network to construct a characteristic space or a characteristic space screening strategy, has higher characteristic quality dependence on an input end, and is easy to cause model overfitting.

In view of one or more of the problems described above, the present exemplary embodiment provides a method for identifying a pulmonary disease based on CT data. The method for identifying lung diseases based on CT data may be applied to the server 105 or one or more of the terminal devices 101, 102, 103, which is not particularly limited in the present exemplary embodiment. Referring to fig. 3, the method for identifying lung diseases based on CT data may include the following steps S310 to S320:

In step S310, a 3D tensor corresponding to the CT data of the lung to be identified is obtained, and the 3D tensor is segmented to obtain k segmented tensors.

Wherein k is a positive integer greater than or equal to 2; the lung CT data to be identified may include lung CT data in different formats, such as DICOM format, NII format, and single channel image sequence patterns; correspondingly, the 3D tensor may also include 3D tensors obtained by converting lung CT data in different formats.

In an exemplary embodiment, the 3D tensor may be preprocessed before the 3D tensor is processed later. Specifically, the preprocessing process may include normalization processing, scaling processing, denoising processing, and the like.

Wherein the content stored in the CT data is different due to different formats. For example, HU values are stored in DICOM, NII format, and pixel gray values are stored in a single channel image sequence. Therefore, the 3D tensor needs to be normalized to be within a specific range, e.g., [0,1]. It should be noted that, the normalization process is to process the values, so the normalization process can also be directly performed on the lung CT data to be identified, and then the lung CT data can be converted into a 3D tensor.

Before the normalization processing, the 3D tensor may be windowed, i.e., the HU value or the pixel gray value outside the window range is screened out, and the HU value or the pixel gray value within the window range is reserved. Through the processing mode, some extra-large or extra-small HU values or pixel gray values can be screened out, and the problem that the normalized data are too aggregated and cannot be identified due to the existence of extra-large numerical values or extra-small numerical values is avoided. For example, assuming that the pixel gray scale distribution is between 0 and 100, if it is normalized to the range of [0,1], each value is simply divided by 100; assuming that the pixel gray levels are mostly distributed between 0 and 100, but there is a maximum value 255, each value needs to be divided by 255, which results in that the pixel gray levels distributed between 0 and 100 are compressively distributed between 0 and 0.39, and the portion between 0.39 and 1 has only 1 pixel value, and there is a high probability that the difference between gray levels is small due to the fact that the gray levels are too concentrated, thereby causing an unrecognizable problem.

The 3D tensors obtained after the transformation of the lung CT data to be identified acquired by the CT apparatus may be of different scales, for example, different resolutions. The scaling process can thus be performed on the basis of 3D tensors, with 3D tensors of different resolutions being nearest-neighbor interpolated over the cross-sectional slices, scaled to a predetermined size, e.g. H (image height) ×w (image width). It should be noted that, when scaling is performed, there is no need to scale the image data contained in the 3D tensor to avoid losing data at certain positions of the lung.

Noise may be introduced during the acquisition process, and thus noise that may be present in the 3D tensor may be removed by a denoising process.

In order to convert the 3D tensor into a form more suitable for analysis processing by a computer device such as a GPU, contrast enhancement may be performed on the 3D tensor. For example, contrast enhancement based on histogram equalization.

In an exemplary embodiment, when the 3D tensor is partitioned, the partitioning may be performed according to different partitioning rules. When the size of the 3D tensor is the number of images x the height of the images x the width of the images, the 3D tensor may be sequentially segmented based on preset segmentation parameters to obtain k-1 segmented tensors with the size of the segmentation parameters x the height of the images x the width of the images, and then the remaining part of the non-segmented tensors are used as 1 segmented tensor to obtain k tensors. In the case of performing the blocking, the size of the remaining portion is the remaining number×the image height×the image width, and the remaining number is smaller than or equal to the blocking parameter.

For example, assume that the size of a certain 3D tensor is d×h×w, and the preset blocking parameter is N. At this time, if D/N is divisible, D/n=k, and k block tensors with dimensions n×h×w are obtained in total, that is, the remaining number is equal to the block parameters; if D/N is not divisible, then D/N is rounded up to k to yield k-1 block tensors of size NXH XW and 1 block tensor of size (D- (k-1) N) XH XW. Wherein (D- (k-1) N) is the remaining number, and the remaining number is less than the chunking parameter.

In step S320, the pulmonary disease classification and identification is performed on the k segmented tensors to obtain k segmented classification results.

In an exemplary embodiment, after obtaining k segmented tensors, pulmonary disease classification and identification may be performed for each segmented tensor, and k segmented classification results are correspondingly obtained. Specifically, in the case of classifying and identifying lung diseases, machine learning, deep learning, and the like may be adopted. For example, the k segmented tensors may be used for pulmonary disease classification and identification via a trained 3D convolutional neural network (e.g., 3D ResNet-18 Model, 3D DenseNet, etc.).

In step S330, an identification result corresponding to the CT data of the lung to be identified is output according to the k block classification results, and if the identification result is an abnormal result, a focus block tensor corresponding to the CT data of the lung to be identified is determined in the k block tensors.

The recognition results corresponding to the lung CT data to be recognized may be of various preset types. For example, for pneumonia identification, the identification result corresponding to a certain lung CT data to be identified may include three types: normal state, common pneumonia, new coronaries pneumonia. Correspondingly, the preset type can be set as normal, common pneumonia and new coronaries pneumonia. On the basis, each block classification result comprises the probability that the block tensor belongs to each preset category. For example, after a block is identified, the obtained identification result may include: the probability of belonging to the normal state is 80%, the probability of belonging to the common pneumonia is 11%, and the probability of belonging to the new coronaries is 9%.

The abnormal result refers to that the identification result corresponding to the CT data of the lung to be identified is the result containing the lung diseases. For example, for pneumonia, the identification results include normal, and new coronaries, where both normal and new coronaries are abnormal results.

In an exemplary embodiment, after k segmented classification results are obtained, a recognition result and a focus segmented tensor corresponding to the lung CT data to be recognized may be determined based on the k segmented classification results. Specifically, machine learning, deep learning, and the like can be adopted. For example, the k segmented classification results may be integrated by a bayesian Noisy-Or model to output a recognition result corresponding to the lung CT data to be recognized and a focus segmented tensor.

In an exemplary embodiment, when the recognition result includes a plurality of preset categories and the block classification result includes probabilities that the block tensor belongs to each preset category, when the recognition result corresponding to the lung CT data to be recognized is output according to the k block classification results, for each preset category, a total probability corresponding to the preset category may be calculated based on the k block classification results, and then the preset category with the maximum total probability is determined as the recognition result corresponding to the lung CT data to be recognized.

In an exemplary embodiment, in calculating the total probability corresponding to the preset category, the calculation may be performed based on equation 1. For example, assume that k=2, and the block classification results corresponding to the 2 blocks are respectively: the probability of the new coronary pneumonia is 80 percent, the probability of the new coronary pneumonia is 10 percent, and the probability of the new coronary pneumonia is 10 percent; the probability of belonging to the normal state is 60%, the probability of belonging to the common pneumonia is 30%, and the probability of belonging to the new coronaries is 10%. At this time, based on the formula 1, the total probability of belonging to the normal state is 92%, the total probability of belonging to the general pneumonia is 37%, and the total probability of belonging to the new coronaries is 19%. The formula (1) is as follows:

Wherein, P is the total probability corresponding to a certain preset category, and pt is the probability that the t-th block belongs to the preset category.

In an exemplary embodiment, when the identification result is an abnormal result, that is, when a lung disease exists, a focus block tensor corresponding to the CT data of the lung to be identified may be determined from k block tensors according to the k block classification results. Specifically, when the block classification result includes probabilities that the block tensor belongs to each preset category, a target probability that the block tensor belongs to the recognition result in the block classification result can be obtained for each block classification result, and then the block tensor corresponding to the maximum target probability in the obtained k target probabilities is determined as the focus block tensor corresponding to the lung CT data to be recognized. By determining a focus blocking tensor in the 3D tensor, the most likely location of the abnormal result can be located to assist the medical staff in diagnosis based on determining the spatial location.

In addition, in an exemplary embodiment, when the above-described step S320 and step S330 are performed based on artificial intelligence such as machine learning or deep learning, referring to fig. 4, k segmented tensors may be first used as input, k segmented classification results may be obtained by classifying the segmented tensors based on a lung disease classification model, after the k segmented classification results are obtained, the k segmented classification results may be used as input, and integrated based on an integration model, so as to output an identification result corresponding to CT data of the lung to be identified, and when the identification result is an abnormal result, a focus segmented tensor corresponding to CT data of the lung to be identified may be determined in the k segmented tensors.

The lung disease classification model and the integration model can be obtained by training a constructed lung disease identification neural network. Specifically, during training, a plurality of groups of sample data can be acquired, wherein each group of sample data comprises a block tensor corresponding to lung CT data and an identification result label corresponding to the lung CT data; and dividing a plurality of groups of sample data into a training set, a verification set and a test set by taking the groups as a unit, constructing a lung disease identification neural network, and adjusting and determining parameters in the lung disease identification neural network according to lung CT data included in the training set, the verification set and the test set to obtain a lung disease classification model and an integration model. The constructed lung disease identification neural network may include a lung disease classification network and an integration network serially connected after the lung disease classification network.

The following is based on 3D ResNet-18 Model, and the training process of the model is described in detail with reference to fig. 5 and 6 by using a model weight initialization model pre-trained on an ImageNet dataset, using a model after the output dimension of the last full-connection layer of the modified model is 3 as a lung disease classification network, using a bayesian Noisy-Or model as an integration network, using k=4, and using a preset classification including normal, normal pneumonia and new coronary pneumonia as an example:

referring to fig. 5, when model training is performed, sample data may be divided into a training set, a verification set and a test set, and then training is performed based on the training set, verification is performed based on the verification set, and testing is performed based on the test set, so as to obtain a trained lung disease classification model and an integration model.

Before training based on the training set, data in the training set may be enhanced, for example, random horizontal, vertical overturn, random rotation angle (range 0-359 °) on the whole layer of the 3D tensor, random clipping (at least retaining 81% of the original image), adding random gaussian noise (gaussian noise mean value is 0, variance range 0-0.1, noise coefficient range is 0-4), and the like.

After the enhanced training set is obtained, the lung disease identification network can be trained based on the enhanced training set, and parameters and weights in the network are updated. Wherein the pulmonary disease recognition neural network can be obtained by concatenating the integrated network with the pulmonary disease classification network.

Specifically, each group of sample data contains 4 block tensors, and the 4 normalized block prediction information vectors (block classification results) with the size of 3×1 are input into a lung disease classification network, and vector elements represent the probability that the blocks belong to normal pneumonia, common pneumonia and new coronaries. After the block classification result is obtained, the block classification result is input into an integration network, the integration network outputs a final recognition result, and the final recognition result is compared with a recognition result label corresponding to the sample data to calculate a loss function. The random gradient descent method is then used to minimize the loss function, and the parameters, weights, in the pulmonary disease recognition neural network are updated by back propagation.

Wherein, cross entropy with L2 regularization can be used as a loss function of a new coronaries and common pneumonia identification task, and the following formula (2) is adopted:

Where yi is a boolean value, where yi=1 when the sample data belongs to the i-th class; the remaining cases yi=0. pi represents the final predicted probability of the i-th class output by the integration layer; the L/w ² represents the L2 norm of the model parameters, and the larger the model parameters, the higher the model complexity, and the easier the overfitting. Lambda is an L2 regularization coefficient used to constrain model complexity.

In this embodiment, λ=1×10 ^-4, the initial learning rate lr=2×10 ^-3, and a better recognition result can be obtained by multiplying the learning rate by the attenuation coefficient of 0.8 every 20 rounds of iteration.

After training, the training set may be updated to update the parameters of the pulmonary disease classification network based on the verification set. Specifically, the lung disease classification network is debugged based on the verification set data, and whether parameters and weights of the lung disease classification network are saved or not is determined. Specifically, the model performance may be evaluated using Kappa coefficient κ as an evaluation index. For the three classification problems of normal, and new coronaries, the Kappa coefficient κ can be determined by the following formula (3):

Where p _o represents the accuracy of the overall classification, i.e., the ratio of the number of correctly classified sample data to the total number of sample data, and p _e represents the probability of assumption of opportunistic consistency, the calculation formula is as follows (formula 4):

where M represents the total number of sample data, xi represents the total number of class i recognition result tags, zi represents the total number of sample data predicted as class i recognition results. Wherein, the closer Kappa coefficient Kappa is to 1, the better the recognition effect of the lung disease classification network after updating parameters based on the training set on the new coronaries pneumonia and the common pneumonia.

When the degree that the Kappa coefficient calculated on the verification set is close to 1 meets the preset requirement, parameters and weights can be reserved, and then the reserved parameters are tested based on the test set. Specifically, the test set data is input so that the lung disease classification network with the reserved parameters and weights can output the identification result and the focus blocking tensor. The block tensor with the largest probability corresponding to the identification result is the focus block tensor in the block classification result corresponding to the block tensor aiming at the identification result, and finally the focus is the most possible position.

The lung disease classification network in the lung disease classification network after the test is a lung disease classification model obtained through training, and the integration network in the lung disease classification network after the test is an integration model obtained through training.

After the trained lung disease classification model and the trained integration model are obtained, the identification of the lung disease can be completed based on the lung disease classification model and the trained integration model. Specifically, referring to fig. 6, after the 3D tensor corresponding to the lung CT data to be identified is referred to, the 3D tensor may be segmented to obtain 4 segmented tensors, then the 4 segmented tensors are input into the lung disease classification model, then 4 segmented classification results corresponding to the 4 segmented tensors may be output, and then the 4 segmented classification results are input into the value integration model, and the identification result corresponding to the lung CT data to be identified and the focus segmented tensor may be output.

It should be noted that, in the embodiment that the model after the output dimension of the last full-connection layer of the modified model is 3 is the lung disease classification network and the bayesian Noisy-Or model is the integration network, the experiment shows that when the 3D tensor is scaled to h=51, w=512, and n=2 Or 4, a better recognition effect can be obtained by initializing the model based on 3D ResNet-18 Model and using the model weight pre-trained on the ImageNet dataset.

Furthermore, data enhancement may not be performed for data input in the validation set and in the test set.

In summary, the present exemplary embodiment provides a pulmonary disease recognition method based on multi-example learning, which can realize accurate recognition of pulmonary disease based on pulmonary CT data. On one hand, the method does not depend on a specific interpolation or sampling technology, retains detailed information of original lung CT data, does not limit the size and resolution of the lung CT data of an input end in any priori, and can be suitable for lung CT data with different resolutions and different cross section slice numbers; on the other hand, when model training is carried out, extra labeling is not needed for the partitioned blocks, and the rough space position of the focus can be positioned under the condition of no partitioned area labeling.

It is noted that the above-described figures are merely schematic illustrations of processes involved in a method according to exemplary embodiments of the present disclosure, and are not intended to be limiting. It will be readily appreciated that the processes shown in the above figures do not indicate or limit the temporal order of these processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, for example, among a plurality of modules.

Further, referring to fig. 7, in this exemplary embodiment, there is further provided a pulmonary disease recognition apparatus 700 based on CT data, including a block acquisition module 710, a classification recognition module 720, and a result output module 730. Wherein:

the block acquisition module 710 may be configured to acquire a 3D tensor corresponding to the lung CT data to be identified, and block the 3D tensor to obtain k block tensors; k is a positive integer greater than or equal to 2.

The classification recognition module 720 may be configured to perform pulmonary disease classification recognition on the k segmented tensors to obtain k segmented classification results.

The result output module 730 may be configured to output, according to the k block classification results, a recognition result corresponding to the lung CT data to be recognized, and determine, when the recognition result is an abnormal result, a focus block tensor corresponding to the lung CT data to be recognized in the k block tensors.

In an exemplary embodiment, when the size of the 3D tensor is the number of images x the image height x the image width, the block obtaining module 710 may be configured to sequentially block the 3D tensor based on the block parameters to obtain k-1 block tensors having the size of the block parameters x the image height x the image width; taking the rest part which is not segmented in the 3D tensor as 1 segmented tensor; the size of the 1 blocking tensor is the residual quantity x the image height x the image width, and the residual quantity is smaller than or equal to the blocking parameter.

In an exemplary embodiment, when the block classification result includes probabilities that the block tensor belongs to each preset category, the result output module 730 may be configured to obtain, for each block classification result, a target probability that the block tensor belongs to the recognition result in the block classification result; and determining the blocking tensor corresponding to the maximum target probability in the k target probabilities as the focus blocking tensor corresponding to the lung CT data to be identified.

In an exemplary embodiment, when the block classification result includes probabilities that the block tensor belongs to each preset category, the result output module 730 may be configured to calculate, for each preset category, a total probability corresponding to the preset category based on the k block classification results; and determining the preset type with the maximum total probability as a recognition result corresponding to the CT data of the lung to be recognized.

In an exemplary embodiment, the block acquisition module 710 may be configured to pre-process the 3D tensor to acquire the 3D tensor that satisfies the preset condition; the pretreatment includes at least one of the following treatments: normalization processing, scaling processing, denoising processing, windowing processing and contrast enhancement processing.

In an exemplary embodiment, the classification recognition module 720 may be configured to classify the segmented tensor based on the pulmonary disease classification model with k segmented tensors as input, to obtain k segmented classification results.

In an exemplary embodiment, the result output module 730 may be configured to integrate the k segmented classification results based on the integration model with the k segmented classification results as input, to output a recognition result corresponding to the CT data of the lung to be recognized, and determine a focus segmented tensor corresponding to the CT data of the lung to be recognized from the k segmented tensors when the recognition result is an abnormal result.

In an exemplary embodiment, the pulmonary disease recognition device based on CT data may further include a model training module for acquiring a plurality of sets of sample data and dividing the plurality of sets of sample data into a training set, a verification set, and a test set in units of sets; each group of sample data comprises a group of block tensors corresponding to the lung CT data and an identification result label corresponding to the lung CT data; constructing a lung disease identification neural network, and adjusting and determining parameters in the lung disease identification neural network based on lung CT data included in a training set, a verification set and a test set to obtain a lung disease classification model and an integration model; wherein the lung disease identification neural network comprises a lung disease classification network and an integration network serially connected after the lung disease classification network.

The specific details of each module in the above apparatus are already described in the method section, and the details that are not disclosed can be referred to the embodiment of the method section, so that they will not be described in detail.

Those skilled in the art will appreciate that the various aspects of the present disclosure may be implemented as a system, method, or program product. Accordingly, various aspects of the disclosure may be embodied in the following forms, namely: an entirely hardware embodiment, an entirely software embodiment (including firmware, micro-code, etc.) or an embodiment combining hardware and software aspects may be referred to herein as a "circuit," module "or" system.

Exemplary embodiments of the present disclosure also provide a computer-readable storage medium having stored thereon a program product capable of implementing the method described above in the present specification. In some possible implementations, various aspects of the disclosure may also be implemented in the form of a program product comprising program code for causing a terminal device to carry out the steps according to the various exemplary embodiments of the disclosure as described in the "exemplary methods" section of this specification, e.g. any one or more of the steps of fig. 3, when the program product is run on the terminal device.

It should be noted that the computer readable medium shown in the present disclosure may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

Furthermore, the program code for carrying out operations of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any adaptations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It is to be understood that the present disclosure is not limited to the precise arrangements and instrumentalities shown in the drawings, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for identifying pulmonary disease based on CT data, comprising:

Acquiring a 3D tensor corresponding to lung CT data to be identified, and partitioning the 3D tensor to obtain k partitioned tensors; wherein k is a positive integer greater than or equal to 2;

Carrying out pulmonary disease classification and identification on the k segmented tensors to obtain k segmented classification results;

Outputting an identification result corresponding to the lung CT data to be identified according to the k block classification results, and determining a focus block tensor corresponding to the lung CT data to be identified in the k block tensors when the identification result is an abnormal result;

wherein when the size of the 3D tensor is the number of images x the height of images x the width of images, the partitioning the 3D tensor to obtain k partitioned tensors includes:

Sequentially blocking the 3D tensors based on blocking parameters to obtain k-1 blocking tensors with the sizes of the blocking parameters, the image height and the image width;

Taking the rest part which is not segmented in the 3D tensor as 1 segmented tensor; the size of the 1 blocking tensor is the residual quantity x the image height x the image width, and the residual quantity is smaller than or equal to the blocking parameter.

2. The method according to claim 1, wherein the block classification result includes probabilities that the block tensors belong to respective preset categories;

And when the identification result is an abnormal result, determining a focus blocking tensor corresponding to the lung CT data to be identified in the k blocking tensors, wherein the focus blocking tensor comprises the following steps:

aiming at each block classification result, obtaining the target probability that the block tensor belongs to the identification result in the block classification result;

and determining the blocking tensor corresponding to the maximum target probability in the k target probabilities as the focus blocking tensor corresponding to the lung CT data to be identified.

3. The method according to claim 1, wherein the block classification result includes probabilities that the block tensors belong to respective preset categories;

the step of outputting the identification result corresponding to the lung CT data to be identified according to the k block classification results comprises the following steps:

For each preset category, calculating the total probability corresponding to the preset category based on the k block classification results;

and determining the preset type with the maximum total probability as a recognition result corresponding to the lung CT data to be recognized.

4. The method of claim 1, wherein prior to the chunking the 3D tensor to obtain k chunked tensors, the method further comprises:

Preprocessing the 3D tensor to obtain the 3D tensor meeting the preset condition;

The pretreatment includes at least one of the following treatments: normalization processing, scaling processing, denoising processing, windowing processing and contrast enhancement processing.

5. The method of claim 1, wherein said classifying and identifying the pulmonary disease of the k segmented tensors to obtain k segmented classification results comprises:

taking the k segmented tensors as input, and classifying the segmented tensors based on a lung disease classification model to obtain k segmented classification results;

the step of outputting the identification result corresponding to the lung CT data to be identified according to the k block classification results, and determining the focus block tensor corresponding to the lung CT data to be identified in the k block tensors when the identification result is an abnormal result, comprising the following steps:

and integrating the k block classification results based on an integration model by taking the k block classification results as input, so as to output an identification result corresponding to the lung CT data to be identified, and determining focus block tensors corresponding to the lung CT data to be identified in the k block tensors when the identification result is an abnormal result.

6. The method of claim 5, wherein the method further comprises:

Obtaining a plurality of groups of sample data, and dividing the plurality of groups of sample data into a training set, a verification set and a test set by taking a group as a unit; each group of sample data comprises a group of block tensors corresponding to lung CT data and an identification result label corresponding to the lung CT data;

Constructing a lung disease identification neural network, and adjusting and determining parameters in the lung disease identification neural network based on the lung CT data included in the training set, the verification set and the test set to obtain a lung disease classification model and an integration model;

wherein the pulmonary disease recognition neural network comprises a pulmonary disease classification network and an integration network serially connected after the pulmonary disease classification network.

7. A pulmonary disease recognition device based on CT data, comprising:

The block acquisition module is used for acquiring a 3D tensor corresponding to the lung CT data to be identified and carrying out block division on the 3D tensor to obtain k block tensors; k is a positive integer greater than or equal to 2;

The classification and identification module is used for carrying out pulmonary disease classification and identification on the k segmented tensors to obtain k segmented classification results;

the result output module is used for outputting the identification result corresponding to the lung CT data to be identified according to the k block classification results, and determining focus block tensors corresponding to the lung CT data to be identified in the k block tensors when the identification result is an abnormal result;

8. A computer readable medium, on which a computer program is stored, which computer program, when being executed by a processor, implements the method according to any one of claims 1 to 6.

9. An electronic device, comprising:

A processor; and

A memory for storing executable instructions of the processor;

wherein the processor is configured to perform the method of any one of claims 1 to 6 via execution of the executable instructions.