CN111493935B - Method and system for automatic prediction and recognition of echocardiography based on artificial intelligence - Google Patents

Abstract

The application discloses an ultrasonic cardiogram automatic prediction and identification method and a system based on artificial intelligence, wherein the method comprises the following steps: acquiring a color Doppler video of at least one section of an echocardiogram of a detected object; extracting each video frame in the color Doppler video, and inputting each video frame into a trained convolutional neural network to obtain an N-dimensional feature vector corresponding to each video frame; generating the weight corresponding to each video frame by the N-dimensional feature vector of each video frame through an attention module; calculating a weighted sum of the N-dimensional feature vectors of each video frame by using the weights to obtain an overall feature representation of the color Doppler video; and calculating to obtain a predicted value containing the pre-identified image features based on the overall feature representation. By the method, whether the image features to be identified exist in the echocardiogram can be accurately predicted.

Description

Method and system for automatic prediction and recognition of echocardiography based on artificial intelligence

technical field

The present application relates to the technical field of intelligent identification of medical video images, and in particular, to a method and system for automatic prediction and identification of echocardiography based on artificial intelligence.

Background technique

Echocardiography is currently one of the most important methods for assessing cardiac structure and function. For the identification of heart valve regurgitation features in echocardiography, doctors usually identify the presence or absence of abnormal color flow with the naked eye, which is easily affected by velocity range and color gain, thereby overestimating or underestimating the severity of the above features. degree and is not suitable for precise assessment of the characteristics of valvular regurgitation. In addition, there are systolic flow method, continuous Doppler and other methods to quantitatively analyze the degree of heart valve regurgitation, but due to the obvious individual differences in image acquisition, measurement, analysis, judgment, etc. The influence of experience and ability is relatively large, making it difficult to guarantee the accuracy and consistency of the test, which often brings great difficulties to clinical identification.

Artificial intelligence technology has great advantages over the manual operation of professional doctors in the automatic measurement, analysis and interpretation of cardiac ultrasound images. It can realize the standardization of data analysis and identification, thereby eliminating the interference of human subjective factors and reducing the inter-individual differences and differences in cardiac ultrasound judgment. Intra-individual differences, improve the accuracy and consistency of ultrasound image discrimination. Artificial intelligence technology also makes cardiac ultrasound more efficient and more in line with clinical needs, thereby greatly improving the efficiency of the medical industry, reducing medical costs and family and socioeconomic burdens.

In recent years, although the use of artificial intelligence technology to process medical images is a research hotspot, more traditional machine learning algorithms such as decision trees, clustering, Bayesian classification, support vector machines, and EM are used. These algorithms do not use echocardiography. The dynamic video of the figure, which is classified only by randomly selected ultrasound images, loses the motion information of the heart, and the color Doppler video contains the flow direction information of the blood flow in the heart, which is helpful to identify valvular regurgitation. If such information is ignored It is easy to cause the classification to be inaccurate, and the effect of practical application is general.

SUMMARY OF THE INVENTION

In view of the above-mentioned defects or deficiencies in the prior art, the present application provides a method and system for automatic prediction and identification of echocardiography based on artificial intelligence. The method adopts a deep learning model specially designed for cardiac ultrasound images to automatically predict echocardiography. The pre-identified features in the figure and the video frames most relevant to the pre-identified features are output, which greatly improves the accuracy of the artificial intelligence technology in processing cardiac ultrasound images, meets the clinical needs of the medical system, and reduces the workload of medical staff.

A first aspect of the present invention provides a method and system for automatic prediction and identification of echocardiography based on artificial intelligence, including:

acquiring a color Doppler video of at least one slice in the echocardiogram of the test subject;

Extracting each video frame in the color Doppler video, and inputting each video frame to a trained convolutional neural network to obtain an N-dimensional feature vector corresponding to each video frame;

The N-dimensional feature vector of each video frame is generated by the attention module to the weight corresponding to each video frame;

Using the weights to calculate a weighted sum of N-dimensional feature vectors for each video frame to obtain an overall feature representation of the color Doppler video;

Based on the global feature representation, a prediction is calculated that the color Doppler video contains pre-identified image features.

Further, it also includes the following steps: outputting the frame with the largest weight as a key frame.

Further, the at least one view includes an apical four-chamber view.

Further, the pre-identified image feature is valve regurgitation.

Further, it also includes the following steps:

measuring the relative area of the left atrial region in the keyframe;

measuring the relative area of the regurgitation jet within the left atrial region in the keyframe;

The ratio of the relative area of the regurgitant jet to the relative area of the left atrium is calculated.

Further, before each video frame is input to the pre-trained convolutional neural network, the method further includes: inputting a color Doppler video of at least one slice of echocardiography containing pre-identified image features as a training sample into the pre-trained convolutional neural network. A convolutional neural network for training the pre-trained convolutional neural network.

Further, it also includes:

的值不再降低，或损失函数

的值低于预定值时，停止所述卷积神经网络的训练；When the loss function

The value of is no longer decreasing, or the loss function

When the value of is lower than a predetermined value, stop the training of the convolutional neural network;

表示为：The loss function

Expressed as:

；

;

表示在视频级别上计算的分类损失，所述

表示用来调节每一视频帧的权重的稀疏损失，

为

的归一化常数；Among them, the

represents the classification loss computed at the video level, the

represents the sparse loss used to adjust the weights of each video frame,

for

The normalization constant of ;

根据下式计算：said

Calculate according to the following formula:

表示第n个彩色多普勒视频中是否含有预识别图像特征，n的取值范围为（1,2,……N），N为彩色多普勒视频的总数量，若

，表示第n个彩色多普勒视频中不含有预识别图像特征；若

，表示第n个彩色多普勒视频中含有预识别图像特征；

表示第n个彩色多普勒视频中含有预识别图像特征的预测值；

为第n个彩色多普勒视频的整体特征表示，

则是

和

的交叉熵；in,

Indicates whether the nth color Doppler video contains pre-identified image features, the value range of n is (1, 2,...N), N is the total number of color Doppler videos, if

, indicating that the nth color Doppler video does not contain pre-identified image features; if

, indicating that the nth color Doppler video contains pre-identified image features;

Represents the predicted value of the pre-identified image feature in the nth color Doppler video;

is the overall feature representation of the nth color Doppler video,

is

and

the cross entropy;

根据下式计算：said

Calculate according to the following formula:

,

表示第n个彩色多普勒视频中的第t个视频帧的权重，t取值范围为（1,2,……,T），T为视频的帧数；n的取值范围为（1,2,……N），N为彩色多普勒视频的总数量。in,

,

Indicates the weight of the t-th video frame in the n-th color Doppler video, the value range of t is (1,2,...,T), T is the number of video frames; the value range of n is (1 ,2,...N), where N is the total number of color Doppler videos.

A second aspect of the present invention also provides an artificial intelligence-based echocardiography automatic prediction and identification system, comprising:

a video acquisition module for acquiring a color Doppler video of at least one slice in the echocardiogram of the detected object;

An input extraction module is used to extract each video frame in the color Doppler video, and each video frame is input to the trained convolutional neural network to obtain an N-dimensional feature vector corresponding to each video frame;

A weight generation module, which is used to generate the weight corresponding to each video frame by the N-dimensional feature vector of each video frame through the attention module;

an overall feature calculation module, using the weight to calculate the weighted sum of the N-dimensional feature vectors of each video frame to obtain the overall feature representation of the color Doppler video;

A prediction output module, based on the overall feature representation, calculates a prediction value that the color Doppler video contains pre-identified image features.

Further, it also includes: a key frame output module for outputting the frame with the largest weight as a key frame.

Further, the at least one view includes an apical four-chamber view; the pre-identified image feature is valve regurgitation; the system further includes: a pre-identified image feature measurement module for measuring the left atrial region in the key frame ; measure the relative area of the regurgitation bundle in the left atrium region in the key frame; calculate the ratio of the relative area of the regurgitation bundle to the relative area of the left atrium.

To sum up, an artificial intelligence-based echocardiographic automatic prediction and identification method and system of the present invention identify the important image features from the video by identifying a group of video frames composed of important image features of cardiac ultrasound, and the method is based on a A novel deep neural network that learns how to measure the importance of each frame in a video and automatically selects a sparse subset of representative frames to predict video-level classification. By using the above method and system provided by the present invention, it can be used to accurately identify the image features of mitral regurgitation in echocardiography, and can also be used to accurately identify tricuspid regurgitation and aortic regurgitation in echocardiography Cardiac ultrasound imaging features such as flow, pulmonary regurgitation, atrial septal defect, ventricular septal defect, and patent ductus arteriosus.

Description of drawings

Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

1 is a flowchart of an artificial intelligence-based automatic prediction and identification method for echocardiography provided by an embodiment of the present invention;

2 is a flowchart of an automatic prediction and identification method for echocardiography with a key frame output function provided by another embodiment of the present invention;

3 is a flowchart of an automatic prediction and identification method for echocardiography with a saliency evaluation function provided by another embodiment of the present invention;

4 is a flowchart of an automatic prediction and identification method for echocardiography with a pre-training function provided by another embodiment of the present invention;

5 is a functional block diagram of an automatic prediction and identification system for echocardiography provided by another embodiment of the present invention;

6 is a structural block diagram of an electronic device provided by another embodiment of the present invention;

FIG. 7 is a component composition diagram of an electronic device according to another embodiment of the present invention.

Detailed ways

The present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the related invention, but not to limit the invention. In addition, it should be noted that, for the convenience of description, only the parts related to the invention are shown in the drawings.

It should be noted that the embodiments in the present application and the features of the embodiments may be combined with each other in the case of no conflict. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

Echocardiography, as described herein, is an ultrasound image that examines the anatomical structure and functional status of the heart and great vessels using the special physical properties of ultrasound.

Referring to FIG. 1 , it shows an artificial intelligence-based echocardiographic automatic prediction and identification method according to an embodiment of the present invention. This embodiment takes the prediction and identification of mitral valve regurgitation features in echocardiography as an example for description, but the method proposed in this embodiment is also applicable to tricuspid regurgitation, aortic regurgitation, pulmonary regurgitation, Predictive identification of ultrasound images of septal defect, ventricular septal defect, patent ductus arteriosus and other cardiac blood flow characteristics.

Step S101, acquiring a color Doppler video of at least one slice in the echocardiogram of the detection object.

The most commonly used Doppler ultrasound techniques include pulsed Doppler, continuous Doppler and color Doppler flow imaging. Color Doppler belongs to the imaging technology, it is an area display, in the same area, there are many sound beams emitted and received back. The flow direction of color Doppler flow imaging is defined as the flow towards the probe as red and the flow away from the probe as blue. The valve orifices and great vessels of the heart are single antegrade blood flow, and once the retrograde blood flow occurs, it should be considered whether it is abnormal.

Specifically, for different body positions of the detection object and different slices of the heart during ultrasonic inspection, original echocardiographic images containing multiple sequences can be obtained, and each sequence corresponds to one slice in the ultrasonic inspection.

In this embodiment, a color Doppler video involving at least one slice in the echocardiogram (for example, it may be a video of one slice, or a combination of videos of multiple slices) is first acquired, and the color Doppler video With multiple video frames, this color Doppler video is used as input to the deep learning algorithm module. It should be pointed out that video is used instead of a single image as input because video provides more information in the temporal dimension, including the change information from frame to frame, and a single image often loses the movement of the heart information, it is easy to cause the classification to be inaccurate. In addition, the color Doppler video can show the direction of blood flow in the valve, and the features are more significant, which further improves the classification accuracy of the algorithm module.

Step S102: Extract each video frame in the color Doppler video, and input each video frame into the trained convolutional neural network to obtain an N-dimensional feature vector corresponding to each video frame.

Specifically, color Doppler video is composed of a group of video frames, each video frame can be understood as a frame of ultrasound image, firstly, all T video frames in a single video are extracted, and then input frame by frame to the in a trained convolutional neural network. This embodiment does not limit the type of the above-mentioned convolutional neural network. For example, an I3D network composed of a dual-stream network and a 3D convolution can be used, and a ResNet residual network can be used. This embodiment uses the ResNet residual network as an example to carry out illustrate.

,其中

为视频帧的标号，

的取值为（1，2……T），T为视频的帧数；N维度代表着对应帧所包含的信息，N既不能太小，也不能太大，N设置的太小会使视频帧包含过少的特征信息，N设置的太大，会使视频帧中包含过多的无用信息，浪费运算资源，本实施例中N优选为1024，实际运用时N可以通过多次人工尝试来获得合适的设置值。After each video frame passes through the Resnet neural network, an N-dimensional feature vector is obtained

,in

is the label of the video frame,

The value of N is (1, 2...T), T is the number of frames of the video; the N dimension represents the information contained in the corresponding frame, and N can neither be too small nor too large. If N is set too small, it will make the video The frame contains too little feature information. If N is set too large, the video frame will contain too much useless information and waste computing resources. In this embodiment, N is preferably 1024. In actual use, N can be determined by multiple manual attempts. to obtain the appropriate setting value.

Step S103, generating a weight corresponding to each video frame from the N-dimensional feature vector of each video frame through the attention module.

来表示第t帧的权重，t取值范围为（1,2,……,T），T为视频的帧数。In clinical inspection, only a few key frames in color Doppler video often contain the feature information of whether the sample is refluxed or not. Whether it has feature information and the amount of feature information with it determines the weight of the video frame. Here we use

To represent the weight of the t-th frame, the value range of t is (1, 2, ..., T), and T is the number of frames of the video.

输入至注意力模块（attention module）获得的。该注意力模块其实是深度学习算法重用于模拟人脑的注意力模型，举个例子来说，当我们观赏一幅画时，虽然我们可以看到整幅画的全貌，但是在我们深入仔细地观察时，其实眼睛聚焦的就只有很小的一块，这个时候人的大脑主要关注在这一小块图案上，也就是说这个时候人脑对整幅图的关注并不是均衡的，是有一定的权重区分的。N维特征向量

经由注意力模块计算后能够获得每一视频帧所对应的权重

。由于注意力模块的具体实现方法已经广泛应用于深度学习模型，因此本实施例不再赘述。The weights are obtained by combining the N-dimensional feature vector output by the Resnet neural network

The input is obtained by the attention module. The attention module is actually an attention model reused by deep learning algorithms to simulate the human brain. For example, when we look at a painting, although we can see the whole picture, but when we look deeply and carefully When observing, in fact, the eyes focus on only a very small piece. At this time, the human brain mainly focuses on this small pattern, which means that the human brain does not pay attention to the whole picture at this time. There is a certain amount of attention. weights are differentiated. N-dimensional feature vector

The weight corresponding to each video frame can be obtained after calculation by the attention module

. Since the specific implementation method of the attention module has been widely used in the deep learning model, it is not repeated in this embodiment.

Step S104, using the weight to calculate the weighted sum of the N-dimensional feature vectors of each video frame to obtain the overall feature representation of the color Doppler video.

能够表明该帧的视频表示，而视频的整体特征表示可以通过每帧的权重

与每帧的特征向量

的加权和来计算获得，即：Specifically, the 1024-dimensional feature vector corresponding to each video frame

can indicate the video representation of the frame, and the overall feature representation of the video can be determined by the weight of each frame

with the feature vector of each frame

to calculate the weighted sum of , namely:

为视频的整体特征表示，

为第t帧的视频特征表示，

为第t帧的权重，T为视频的帧数。in,

is the overall feature representation of the video,

is the video feature representation of the t-th frame,

is the weight of the t-th frame, and T is the frame number of the video.

Step S105 , based on the overall feature representation, a prediction value containing pre-identified image features is obtained by calculating through the FC fully connected layer and the sigmoid activation function.

后，在经过FC 全连接层和sigmod激活函数的两层运算后，得到最终的预测值

，

的取值在0和1之间，代表着由样本视频中存在二尖瓣膜反流特征的几率，即：Obtain the overall feature representation of the video

After the two-layer operation of the FC fully connected layer and the sigmod activation function, the final predicted value is obtained

,

The value of is between 0 and 1, which represents the probability of the presence of mitral valve regurgitation features in the sample video, namely:

Referring to Fig. 2, preferably on the basis of the embodiment shown in Fig. 1, it further includes:

最大的帧作为关键帧输出。Step S106, the weight

The largest frame is output as a keyframe.

最大的视频帧意味着体现出二尖瓣反流特征最明显或贡献最大的视频帧，将该关键帧自动输出并生成在超声报告上，节省了医生选取截图的操作步骤，提高了超声报告的生成效率。Weights

The largest video frame means the video frame that reflects the most obvious features of mitral valve regurgitation or contributes the most. The key frame is automatically output and generated on the ultrasound report, which saves the operation steps for doctors to select screenshots and improves the quality of the ultrasound report. Generation efficiency.

Further, at least one slice in the above embodiments preferably includes an apical four-chamber (A4C) slice. The apical four-chamber (A4C) view is a standard view in clinical echocardiography, and it is also one of the most widely used important views. The apical four-chamber (A4C) view can more accurately identify images such as mitral regurgitation feature.

Referring to FIG. 3 , preferably, on the basis that whether there is a pre-identified image feature in the color Doppler video of the detection object has been identified according to the embodiment shown in FIG. 1 , the method further includes:

Step S107, identifying the salient degree of the pre-identified image feature.

Taking the identification of mitral regurgitation features as an example, the specific method is to measure the ratio of the regurgitation area and the left atrial area on the obtained key frame with the most obvious mitral regurgitation, and output the ratio to the ultrasound report.

First, measure the left atrial area. Usually, there are hundreds of sets of echocardiograms that have been carefully delineated by the annotators in the early stage (the so-called delineation is to artificially draw the borders of key parts of the heart, such as the left atrium). These data are input into the neural network for learning to complete the automatic identification of the left atrium by the model. The relative area of the left atrium can be obtained by calculating the relative area of the left atrium in the image, that is, the number of pixels in the delineated area.

Then, the area of the reflux beam is measured. Because color Doppler ultrasound is used, the regurgitation beam is usually marked in blue (unregurgitated blood is usually red), so it is only necessary to count the number of blue pixels in the left atrium to calculate the regurgitation relative area of the bundle.

Finally, the ratio of the relative area of the regurgitation beam to the relative area of the left atrium was calculated, and the salience of the features of the regurgitation image in the ultrasound video can be known by this ratio.

Referring to Fig. 4, preferably, before executing the method shown in Fig. 1, the method further includes

Step S100, the color Doppler video of at least one slice in the echocardiogram containing the pre-identified image features is input to the pre-trained convolutional neural network as a training sample, and the step of training the pre-trained convolutional neural network .

也设计为两部分组成，分别为分类损失

和稀疏损失

。The process of training the model is actually the process of continuously adjusting the parameters to make the prediction results better and better. In this process, the loss function is usually used as a reference, and the loss function is usually used to evaluate the difference between the predicted value of the model and the real value. The neural network model of the present invention has two main tasks, one is to accurately judge whether the sample echocardiogram contains pre-identified image features, and the other is to output the video key frames that are most helpful for judging the result, so the corresponding loss function

It is also designed to be composed of two parts, namely the classification loss

and sparse loss

.

表示在视频级别上计算的分类损失：

Represents the classification loss computed at the video level:

表示第n个彩色多普勒视频中是否含有预识别图像特征，n的取值范围为（1,2,……N），N为彩色多普勒视频的总数量，若

，表示第n个彩色多普勒视频中不含有预识别图像特征；若

，表示第n个彩色多普勒视频中含有预识别图像特征；

表示第n个彩色多普勒视频中含有预识别图像特征的预测值；

为第n个彩色多普勒视频的整体特征表示，

则是

和

的交叉熵。in,

Indicates whether the nth color Doppler video contains pre-identified image features, the value range of n is (1, 2,...N), N is the total number of color Doppler videos, if

, indicating that the nth color Doppler video does not contain pre-identified image features; if

, indicating that the nth color Doppler video contains pre-identified image features;

Represents the predicted value of the pre-identified image feature in the nth color Doppler video;

is the overall feature representation of the nth color Doppler video,

is

and

the cross entropy.

，即用来调节视频中特定视频帧的权重

的损失函数，因为在临床检验中，往往只有关键的几帧代表着样本反流与否，因此在所有帧中只有少数帧是重要的，即只有少数帧对应的接近1，其余帧对应的都接近0，换言之这些帧的重要性

是稀疏的，本实施例使用L1 范数来衡量整个数据的稀疏性，依据数学定义，L1 范数是指向量中各个元素绝对值之和，L1 越小，则整个元素越稀疏，具体形式如下：so-called sparsity loss

, which is used to adjust the weight of a specific video frame in the video

The loss function of , because in clinical tests, only a few key frames often represent whether the sample is refluxed or not, so only a few frames are important in all frames, that is, only a few frames correspond to close to 1, and the remaining frames correspond to all close to 0, in other words the importance of these frames

is sparse. In this example, the L1 norm is used to measure the sparsity of the entire data. According to the mathematical definition, the L1 norm is the sum of the absolute values of each element in the pointer. The smaller the L1, the sparser the entire element. The specific form is as follows :

根据下式计算：said

Calculate according to the following formula:

,

表示第n个彩色多普勒视频中的第t个视频帧的权重，t取值范围为（1,2,……,T），T为视频的帧数；n的取值范围为（1,2,……N），N为彩色多普勒视频的总数量。in,

,

Indicates the weight of the t-th video frame in the n-th color Doppler video, the value range of t is (1,2,...,T), T is the number of video frames; the value range of n is (1 ,2,...N), where N is the total number of color Doppler videos.

的数学表达为：The loss function in the training process of this example

The mathematical expression is:

；

;

的值低于预定值、或损失函数

的值不再降低时，表明可以停止卷积神经网络的训练。In the specific training process, the samples used to train the neural network can be used cyclically. When the set number of cycles and the loss function are reached,

is lower than a predetermined value, or the loss function

When the value of is no longer decreasing, it indicates that the training of the convolutional neural network can be stopped.

It should be noted that although the operations of the method of the present invention are described in a particular order in FIG. 1, this does not require or imply that the operations must be performed in that particular order, or that all illustrated operations must be performed to achieve the desired result. Rather, the steps depicted in the flowcharts may be executed in a modified order as necessary.

Referring to FIG. 5, it shows an artificial intelligence-based echocardiography automatic prediction and identification system 200 according to another embodiment of the present invention, including:

A video acquisition module 201, configured to acquire a color Doppler video of at least one slice in the echocardiogram of the detection object;

Input extraction module 202, for extracting each video frame in the color Doppler video, and inputting each video frame to the trained convolutional neural network to obtain an N-dimensional feature vector corresponding to each video frame ;

The weight generation module 203 is used for generating the weight corresponding to each video frame by the N-dimensional feature vector of each video frame through the attention module;

the overall feature calculation module 204, using the weight to calculate the weighted sum of the N-dimensional feature vectors of each video frame to obtain the overall feature representation of the color Doppler video;

The prediction output module 205, based on the overall feature representation, obtains a predicted value containing the pre-identified image feature by calculating the FC fully connected layer and the sigmoid activation function.

Further, the automatic prediction and identification system 200 for echocardiography includes: a pre-identified image feature measurement module 206 for measuring the area of the left atrial region in the key frame; measuring the reflux in the left atrial region in the key frame area of the bundle; calculate the ratio of the area of the regurgitant bundle to the area of the left atrium.

It should be understood that the modules described in the automatic prediction and identification system 200 for echocardiography in this embodiment correspond to the steps in the method described in FIG. 1 . Therefore, the operations and features described above with respect to the method are also applicable to each module of this embodiment, and details are not described herein again. The system of this embodiment may be implemented in the electronic device in advance, or may be loaded into the electronic device by downloading or the like. Corresponding modules in the system of the present embodiment may cooperate with units in the electronic device to implement the solutions of the embodiments of the present application. In addition, the modules involved in the description in this embodiment may be implemented in a software manner, and may also be implemented in a hardware manner. The names of these units or modules do not limit the unit or module itself in some cases, for example, the video acquisition module 201 can also be described as "used to acquire the color of at least one slice in the echocardiogram of the test object. Module 201 of Doppler Video.

Referring to FIG. 6, it shows an electronic device 300 according to another embodiment of the present invention, including:

at least one processor 301; and,

a memory 302 in communication with the at least one processor 301; wherein,

The memory 302 stores instructions executable by the at least one processor 301 to enable the at least one processor 301 to perform the steps in the above method embodiments.

Referring to FIG. 7 , the electronic device in the embodiment shown in FIG. 6 may be, for example, a B-mode ultrasound machine. The ultrasound machine may also include a computer system 700 including a central processing unit (CPU) 701 that can be loaded into random access memory (RAM) 703 according to a program stored in read only memory (ROM) 702 or from storage section 708 to execute various appropriate actions and processes. In the RAM 703, various programs and data necessary for the operation of the system 700 are also stored. The CPU 701 , the ROM 702 , and the RAM 703 are connected to each other through a bus 704 . An input/output (I/O) interface 705 is also connected to bus 704 . The following components are connected to the I/O interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc. ; and a communication section 709 including a network interface card such as a LAN card, a modem, and the like. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I/O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is mounted on the drive 710 as needed so that a computer program read therefrom is installed into the storage section 708 as needed.

As another aspect, the present application also provides a computer-readable storage medium, and the computer-readable storage medium may be a computer-readable storage medium included in the system or electronic device described in the foregoing embodiments; or it may be a separate computer-readable storage medium. There is a computer-readable storage medium that does not fit into a device. The computer-readable storage medium stores one or more programs used by one or more processors to perform the automatic predictive identification method of echocardiography described in this application.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more logic for implementing the specified logic Executable instructions for the function. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented in dedicated hardware-based systems that perform the specified functions or operations , or can be implemented in a combination of dedicated hardware and computer instructions.

The above description is only a preferred embodiment of the present application and an illustration of the applied technical principles. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to the technical solution formed by the specific combination of the above-mentioned technical features, and should also cover the above-mentioned technical features without departing from the inventive concept. Other technical solutions formed by any combination of its equivalent features. For example, a technical solution is formed by replacing the above-mentioned features with the technical features disclosed in this application (but not limited to) with similar functions.

Claims (10)

1. an echocardiographic automatic prediction and identification method based on artificial intelligence, is characterized in that, comprises: acquiring a color Doppler video of at least one slice in the echocardiogram of the test subject; Extracting each video frame in the color Doppler video, and inputting each video frame to a trained convolutional neural network to obtain an N-dimensional feature vector corresponding to each video frame; The N-dimensional feature vector of each video frame is generated by the attention module to the weight corresponding to each video frame; Using the weights to calculate a weighted sum of N-dimensional feature vectors for each video frame to obtain an overall feature representation of the color Doppler video; Based on the global feature representation, a prediction is calculated that the color Doppler video contains pre-identified image features. 2. the echocardiographic automatic prediction and identification method based on artificial intelligence according to claim 1, is characterized in that, also comprises the steps: The frame with the highest weight is output as a keyframe. 3 . The method for automatic prediction and identification of echocardiography based on artificial intelligence according to claim 2 , wherein the at least one slice includes an apical four-chamber slice. 4 . 4 . The artificial intelligence-based echocardiography automatic prediction and identification method according to claim 3 , wherein the pre-identified image feature is valve regurgitation. 5 . 5. the echocardiographic automatic prediction and identification method based on artificial intelligence according to claim 4, is characterized in that, also comprises the steps: measuring the relative area of the left atrial region in the keyframe; measuring the relative area of the regurgitation jet within the left atrial region in the keyframe; The ratio of the relative area of the regurgitant jet to the relative area of the left atrium is calculated. 6. The echocardiographic automatic prediction and identification method based on artificial intelligence according to claim 1, characterized in that, before each video frame is input to the pre-trained convolutional neural network, further comprising: A color Doppler video of at least one slice of an echocardiogram containing pre-identified image features is input as a training sample to a pre-trained convolutional neural network, and the pre-trained convolutional neural network is trained. 7. The echocardiographic automatic prediction and identification method based on artificial intelligence according to claim 6, is characterized in that, also comprises: When the loss function

The value of is no longer decreasing, or the loss function

When the value of is lower than a predetermined value, stop the training of the convolutional neural network; The loss function

Expressed as:

; Among them, the

represents the classification loss computed at the video level, the

represents the sparse loss used to adjust the weights of each video frame,

for

The normalization constant of ; said

Calculate according to the following formula:

in,

Indicates whether the nth color Doppler video contains pre-identified image features. The value range of n is (1, 2,...N), and N is the color Doppler video used to train the convolutional neural network model. total number, if

, indicating that the nth color Doppler video does not contain pre-identified image features; if

, indicating that the nth color Doppler video contains pre-identified image features;

Represents the predicted value of the pre-identified image feature in the nth color Doppler video;

is the overall feature representation of the nth color Doppler video,

is

and

the cross entropy; said

Calculate according to the following formula:

in,

,

represents the nth color Doppler video in the

video frame weights,

The value range is (1,2,...,T), where T is the frame number of the video; the value range of n is (1,2,...N), where N is the total number of color Doppler videos. 8. The echocardiography automatic prediction and identification system based on artificial intelligence, is characterized in that, comprises: a video acquisition module for acquiring a color Doppler video of at least one slice in the echocardiogram of the detected object; an input extraction module, for extracting each video frame in the color Doppler video, and inputting each video frame into a trained convolutional neural network to obtain an N-dimensional feature vector corresponding to each video frame; A weight generation module, which is used to generate the weight corresponding to each video frame by the N-dimensional feature vector of each video frame through the attention module; an overall feature calculation module, using the weight to calculate the weighted sum of the N-dimensional feature vectors of each video frame to obtain the overall feature representation of the color Doppler video; A prediction output module, based on the overall feature representation, calculates a prediction value that the color Doppler video contains pre-identified image features. 9. The echocardiographic automatic prediction and identification system based on artificial intelligence according to claim 8, is characterized in that, also comprises: The keyframe output module is used to output the frame with the largest weight as a keyframe. 10. The echocardiographic automatic prediction and identification system based on artificial intelligence according to claim 9, is characterized in that: the at least one view includes an apical four-chamber view; The pre-identified image feature is valve regurgitation; The system further includes: The pre-identified image feature measurement module is used to measure the relative area of the left atrium area in the key frame; measure the relative area of the regurgitation beam in the left atrial area in the key frame; calculate the relative area of the regurgitation beam and all The ratio of the relative area of the left atrium.

Images