[go: up one dir, main page]

CN114943682B - Method and device for detecting anatomical key points in three-dimensional angiography images - Google Patents

Method and device for detecting anatomical key points in three-dimensional angiography images Download PDF

Info

Publication number
CN114943682B
CN114943682B CN202210179800.9A CN202210179800A CN114943682B CN 114943682 B CN114943682 B CN 114943682B CN 202210179800 A CN202210179800 A CN 202210179800A CN 114943682 B CN114943682 B CN 114943682B
Authority
CN
China
Prior art keywords
image
blood vessel
vascular
anatomical
key point
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210179800.9A
Other languages
Chinese (zh)
Other versions
CN114943682A (en
Inventor
冯建江
谭子萌
杨光明
印胤
卢旺盛
秦岚
刘文哲
周杰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Union Strong Beijing Technology Co ltd
Tsinghua University
Original Assignee
Union Strong Beijing Technology Co ltd
Tsinghua University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Union Strong Beijing Technology Co ltd, Tsinghua University filed Critical Union Strong Beijing Technology Co ltd
Priority to CN202210179800.9A priority Critical patent/CN114943682B/en
Publication of CN114943682A publication Critical patent/CN114943682A/en
Application granted granted Critical
Publication of CN114943682B publication Critical patent/CN114943682B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10088—Magnetic resonance imaging [MRI]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30101—Blood vessel; Artery; Vein; Vascular
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2211/00—Image generation
    • G06T2211/40—Computed tomography
    • G06T2211/404—Angiography
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Medical Informatics (AREA)
  • Magnetic Resonance Imaging Apparatus (AREA)
  • Apparatus For Radiation Diagnosis (AREA)

Abstract

The application provides a detection method of anatomical key points in a three-dimensional angiography image, which relates to the technical field of medical image processing, wherein the method comprises the following steps: acquiring a three-dimensional angiography image as a test image; preprocessing a test image, inputting the preprocessed image into a pre-trained multi-task deep learning network, and outputting an anatomic key point prediction probability map, wherein the multi-task deep learning model is obtained by training a training data set by using a three-dimensional angiography training image with the same blood vessel type as the test image and a labeling result of the three-dimensional angiography training image; and generating a detection result of the anatomical keypoints according to the prediction probability of the voxel positions in the anatomical keypoint prediction probability map. The application adopting the scheme can fully utilize the synergistic effect among different tasks, explicitly model the vessel topology variation type and combine the space priori information to realize good detection performance.

Description

Method and device for detecting anatomical key points in three-dimensional angiography image
Technical Field
The application relates to the technical field of medical image processing, in particular to a method and a device for detecting anatomical key points in a three-dimensional angiography image.
Background
Three-dimensional angiography techniques include Magnetic Resonance Angiography (MRA), computed Tomography Angiography (CTA), digital Subtraction Angiography (DSA), and the like, and can clearly and three-dimensionally display blood vessels and blood flow signal characteristics in a body by utilizing imaging characteristics of blood flow. Three-dimensional angiography techniques cover a variety of vascular structures such as intracranial vessels, coronary arteries, carotid arteries, and the aorta, and inspection and analysis of these structures is an important aid in diagnosing and treating related diseases. Taking intracranial vascular magnetic resonance radiography image as an example, it can reflect whether there is malformation in intracranial artery and vein, noninvasively, safely and clearly display the tumor body and the tumor-carrying artery morphology of intracranial aneurysm, and has become the first choice method for intracranial aneurysm diagnosis. In recent years, medical image intelligent analysis technology based on computer theory has been developed, and tasks such as automatic extraction of blood vessels, lesion positioning, lesion measurement and the like in three-dimensional angiography images have been widely studied and clinically applied.
The task of detecting anatomical keypoints in three-dimensional angiographic images focuses on the bifurcation of vessels at each level, where they are located at the bifurcation site of vessel segments at each level, with unique and important anatomical significance. Taking intracranial vessel keypoint detection as an example, a complete Willis loop region can be divided into 20 vessel segments (excluding ICA-C4 and beyond here) with independent anatomical nomenclature, according to the brain vessel topology definition, and a total of 19 keypoints can be defined at each vessel segment junction. Therefore, the anatomical key points explicitly model the whole topological structure of the blood vessel, and can provide abundant semantic information for blood vessel semantic segmentation, lesion positioning and disease diagnosis. In addition, the detection of the anatomical key points is an important enabling means for the intelligent analysis of the follow-up task of the medical image, can provide initialization conditions for vessel tracking and central line extraction, and is used for assisting in realizing vessel tree registration of multiple images of the same patient or images of different patients. However, because the blood vessel morphology is long and thin and is bent, the structure distribution is complex, various changes exist among different individuals, and part of blood vessel segments need to depend on surrounding tissue positions, the local appearance and gray distribution of images can be affected by pathology, and the like, the task of detecting the anatomical keypoints faces great challenges.
On the other hand, unlike other anatomical tree structures such as trachea, aorta, etc., there may be topological changes in the partial vascular structures such as intracranial vessels, coronary arteries, etc. Taking the Willis loop region in intracranial vessels as an example, only about 52% of individuals possess the complete Willis loop according to the related studies, and physiological changes represented by single-or double-sided PCoA deletions (PCA-P1 segment deletions (postembryonic traffic)) are widespread. Studies have shown that these physiological variations may be associated with the risk potential of disease, and how to model these types of variations is one of the focus of intracranial vascular analysis. Notably, the absence of a portion of the vessel segment will result in the relevant keypoints losing the local bifurcation feature and being indistinguishable. For example, when one side PCoA is present, the critical point PCoA-A (the bifurcation of the vessel segment PCoA with ICA) is located at the bifurcation site; when the PCoA is missing, the critical point will be on a smooth ICA vessel segment, without local bifurcation features. In clinical labeling, the location of these keypoints often needs to be determined based on physician experience and spatial symmetry, which makes automatic detection of the keypoints particularly difficult.
Disclosure of Invention
The present application aims to solve at least one of the technical problems in the related art to some extent.
Therefore, a first object of the present application is to provide a method for detecting anatomical keypoints in a three-dimensional angiographic image, which can fully utilize the synergistic effect between different tasks, explicitly model the vessel topology variation type, and combine spatial prior information to achieve good detection performance.
A second object of the present application is to propose a device for detecting anatomical keypoints in three-dimensional angiographic images.
To achieve the above object, an embodiment of a first aspect of the present application provides a method for detecting anatomical keypoints in a three-dimensional angiographic image, including: acquiring a three-dimensional angiography image as a test image; preprocessing a test image, inputting the preprocessed image into a pre-trained multi-task deep learning network, and outputting an anatomic key point prediction probability map, wherein the multi-task deep learning model is obtained by training a three-dimensional angiography training image containing the same blood vessel type as the test image and a labeling result of the three-dimensional angiography training image as a training data set; and generating a detection result of the anatomical keypoints according to the prediction probability of the voxel positions in the anatomical keypoint prediction probability map.
According to the method for detecting the anatomical key points in the three-dimensional angiography image, provided by the embodiment of the application, the three-dimensional angiography image data containing a specific vascular structure is obtained in an off-line stage and preprocessed, and the image is manually marked to generate corresponding prediction targets of the tasks of detecting the anatomical key points, segmenting blood vessel segment semantics, classifying blood vessel segment deletion and classifying the local bifurcation characteristics of the key points, so that a training data set is formed together to train the multi-task deep learning network; and in the online stage, outputting a key point probability heat map prediction result by using the trained network model from one image of the same type, and obtaining a final anatomical key point detection position by using the key point probability heat map prediction result. According to the application, the structure priori knowledge is explicitly introduced, the space semantic information is modeled, and good detection performance can be realized.
Optionally, in an embodiment of the present application, the test image is preprocessed, including unifying resolution, clipping to a preset size, and voxel gray value normalization.
Optionally, in one embodiment of the present application, pre-training the multi-task deep learning network includes:
Acquiring a three-dimensional angiographic image containing the same vessel type as the test image as an original data set;
Preprocessing an original data set, and obtaining a labeling result corresponding to the preprocessed data set, wherein the labeling result comprises a blood vessel anatomy key point labeling result, a blood vessel binary segmentation labeling result and a blood vessel segment semantic segmentation labeling result;
Generating a training data set according to the preprocessed data set and the corresponding labeling result;
and constructing a multi-task deep learning network, and training the multi-task deep learning network by using a training data set to obtain the trained multi-task deep learning network.
Optionally, in an embodiment of the present application, obtaining a labeling result corresponding to the preprocessed data set includes:
Using medical image processing software, manually labeling each image in the preprocessed data set with predefined vascular anatomy key points and vascular binary segmentation, wherein the labeling result of the vascular anatomy key points is a three-dimensional coordinate corresponding to each key point, and the labeling result of the vascular binary segmentation is a voxel-by-voxel binary map with the same size as the image;
Based on the labeling results of the vascular anatomy key points and the vascular binary segmentation, an automatic method is used for generating a blood vessel segment semantic segmentation labeling result corresponding to each image in the data set.
Optionally, in one embodiment of the present application, based on the vascular anatomy keypoints and the vascular binary segmentation labeling result, generating the blood vessel segment semantic segmentation labeling result corresponding to each image in the dataset by using an automated method includes:
obtaining a corresponding lumen central line from a blood vessel binary segmentation marking result by using a refinement algorithm, and dividing the central line into different semantic segments according to anatomical key point marking;
determining semantic labels of all blood vessel voxels in the blood vessel binary segmentation labels according to the nearest central line voxels;
and manually correcting the semantic segmentation automatic labeling result obtained by each image in medical image processing software so as to obtain a final semantic segmentation labeling result, wherein the semantic segmentation automatic labeling result comprises a semantic segment and a semantic label.
Optionally, in an embodiment of the present application, generating the training data set according to the preprocessed data set and the corresponding labeling result includes:
According to the labeling result, each image in the preprocessed data set is processed to obtain an anatomic key point multichannel probability heat map, a blood vessel segment semantic segmentation multichannel probability map, a blood vessel segment missing classification vector and a key point local bifurcation characteristic classification vector which are used as prediction targets corresponding to the images;
and forming a training data pair by each image in the preprocessed data set and the corresponding prediction target, wherein all training data pairs jointly form a training data set.
Optionally, in an embodiment of the present application, processing each image in the preprocessed dataset according to the labeling result to obtain an anatomical keypoint multichannel probability heat map, a vessel segment semantic segmentation multichannel probability map, a vessel segment missing classification vector, and a keypoint local bifurcation feature classification vector as prediction targets corresponding to the image, where the method includes:
outputting an anatomic key point multichannel probability heat map with the same size as the input image to each pre-processed image in the data set according to the labeling result of the anatomic key points of the blood vessels, wherein the corresponding probability heat map of each target key point takes the key point as the center to be in three-dimensional Gaussian distribution;
Generating a blood vessel segment semantic segmentation multichannel probability map according to the blood vessel segment semantic segmentation labeling result, wherein the last channel of the blood vessel segment semantic segmentation multichannel probability map is a background channel, and the rest channels respectively reflect the position distribution of each blood vessel segment in the input image;
Obtaining a blood vessel segment missing classification vector and a key point local bifurcation feature classification vector according to a labeling result of the semantic segmentation of the blood vessel segment, wherein when a certain blood vessel segment is missing in the labeling result of the semantic segmentation of the blood vessel segment, the anatomical key points at two ends of the blood vessel segment lose the local bifurcation feature, otherwise, the anatomical key points at two ends of the blood vessel segment have the local bifurcation feature.
Optionally, in one embodiment of the application, the multi-tasking deep learning network comprises a backbone portion and four branching portions, wherein,
A trunk part for extracting the characteristics of the input image and outputting a characteristic diagram;
the first branch is used for processing the feature map and generating a prediction result of the anatomic key point multichannel probability heat map;
The second branch is used for processing the feature map and generating a prediction result of the vascular segment semantic segmentation multichannel probability map;
the third branch is used for processing the feature map and generating a prediction result of the blood vessel segment missing classification vector;
And the fourth branch is used for processing the feature map and generating a prediction result of the feature classification vector of the local bifurcation of the key points.
Optionally, in one embodiment of the present application, training the initialized network using the training dataset includes:
Step S1: randomly selecting a training data pair from the training data set, inputting the preprocessed three-dimensional angiography image in the training data pair into a constructed multi-task deep learning network, and obtaining the output result of each branch of the network as a predicted result;
step S2: inputting a prediction result and a prediction target in the training data pair into a loss function to obtain a loss function value;
step S3: minimizing a loss function by using a gradient descent method based on the calculated loss function value, and adjusting network parameters;
Step S4: and repeating the steps S1, S2, S3 and S4, continuously adjusting the network parameters, and when the training times exceed the set upper limit times, completing training, determining the multi-task deep learning network parameters and obtaining the trained multi-task deep learning network.
In order to achieve the above object, a second aspect of the present invention provides a device for detecting anatomical keypoints in a three-dimensional angiographic image, which comprises an acquisition module, a processing module, and a result generation module, wherein,
The acquisition module is used for acquiring a three-dimensional angiography image as a test image;
The processing module is used for preprocessing the test image, inputting the preprocessed image into a pre-trained multi-task deep learning network and outputting an anatomic key point prediction probability map, wherein the multi-task deep learning model is obtained by training a training data set by taking a three-dimensional angiography training image with the same blood vessel type as the test image and a labeling result of the three-dimensional angiography training image;
And the result generation module is used for generating a detection result of the anatomical key point according to the prediction probability of the voxel position in the anatomical key point prediction probability map.
Additional aspects and advantages of the application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the application.
Drawings
The foregoing and/or additional aspects and advantages of the application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:
fig. 1 is a flowchart of a method for detecting anatomical keypoints in a three-dimensional angiographic image according to an embodiment of the application;
FIG. 2 is a flow chart of a method according to an embodiment of the present application;
FIG. 3 is a schematic diagram of intracranial vessel labeling and data generation results according to an embodiment of the present application;
FIG. 4 is a diagram showing the correspondence between the segment deficiency of an intracranial vessel and the change of the local bifurcation characteristic of a key point according to an embodiment of the present application;
FIG. 5 is a schematic diagram of an offline stage multi-task deep learning network according to an embodiment of the present application;
FIG. 6 is a schematic diagram of an online stage multi-task deep learning network according to an embodiment of the present application;
FIG. 7 is a graph of detection results of anatomical keypoints in an intracranial vascular magnetic resonance angiography image according to an embodiment of the application;
Fig. 8 is a schematic structural diagram of a device for detecting anatomical keypoints in a three-dimensional angiographic image according to an embodiment of the application.
Detailed Description
Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative and intended to explain the present application and should not be construed as limiting the application.
If the attribute of whether each blood vessel segment is missing or not is introduced into the detection algorithm of the blood vessel anatomical key points, the method can help the algorithm to model the blood vessel variation type explicitly, and better detection performance is realized. In addition, anatomical keypoints are located at both end points of the respective vessel segments, with very sharp structural features. Considering that the position distribution of specific blood vessel segments often has strong regularity and consistency, the auxiliary task of dividing each blood vessel segment (namely, the semantic division of blood vessels is realized) is introduced into a key point detection algorithm, the structure prior information can be introduced, and the guarantee is provided for further improving the detection precision, so that the application provides an anatomical key point detection method which is explicitly combined with the blood vessel topological structure variation type.
The following describes a method and apparatus for detecting anatomical keypoints in three-dimensional angiographic images according to an embodiment of the application with reference to the accompanying drawings.
Fig. 1 is a flowchart of a method for detecting anatomical keypoints in a three-dimensional angiographic image according to an embodiment of the application.
As shown in fig. 1, the method for detecting anatomical keypoints in the three-dimensional angiographic image comprises the following steps:
step 101, acquiring a three-dimensional angiographic image as a test image;
102, preprocessing a test image, inputting the preprocessed image into a pre-trained multi-task deep learning network, and outputting an anatomic key point prediction probability map, wherein the multi-task deep learning model is trained by taking a three-dimensional angiography training image containing the same blood vessel type as a test image and a labeling result of the three-dimensional angiography training image as a training data set;
and step 103, generating a detection result of the anatomical keypoints according to the prediction probability of the voxel positions in the anatomical keypoint prediction probability map.
According to the method for detecting the anatomical key points in the three-dimensional angiography image, provided by the embodiment of the application, the three-dimensional angiography image data containing a specific vascular structure is obtained in an off-line stage and preprocessed, and the image is manually marked to generate corresponding prediction targets of the tasks of detecting the anatomical key points, segmenting blood vessel segment semantics, classifying blood vessel segment deletion and classifying the local bifurcation characteristics of the key points, so that a training data set is formed together to train the multi-task deep learning network; and in the online stage, outputting a key point probability heat map prediction result by using the trained network model from one image of the same type, and obtaining a final anatomical key point detection position by using the key point probability heat map prediction result. According to the application, the structure priori knowledge is explicitly introduced, the space semantic information is modeled, and good detection performance can be realized.
According to the method for detecting the anatomical key points in the three-dimensional angiography image, provided by the embodiment of the application, the three-dimensional angiography image data containing a specific vascular structure is obtained in an off-line stage and preprocessed, and the image is manually marked to generate corresponding prediction targets of the tasks of detecting the anatomical key points, segmenting blood vessel segment semantics, classifying blood vessel segment deletion and classifying the local bifurcation characteristics of the key points, so that a training data set is formed together to train the multi-task deep learning network; and in the online stage, outputting a key point probability heat map prediction result by using the trained network model from one image of the same type, and obtaining a final anatomical key point detection position by using the key point probability heat map prediction result. According to the application, the structure priori knowledge is explicitly introduced, the space semantic information is modeled, and good detection performance can be realized.
Optionally, in an embodiment of the present application, the test image is preprocessed, including unifying resolution, clipping to a preset size, and voxel gray value normalization.
Optionally, in one embodiment of the present application, pre-training the multi-task deep learning network includes:
Acquiring a three-dimensional angiographic image containing the same vessel type as the test image as an original data set;
Preprocessing an original data set, and obtaining a labeling result corresponding to the preprocessed data set, wherein the labeling result comprises a blood vessel anatomy key point labeling result, a blood vessel binary segmentation labeling result and a blood vessel segment semantic segmentation labeling result;
Generating a training data set according to the preprocessed data set and the corresponding labeling result;
and constructing a multi-task deep learning network, and training the multi-task deep learning network by using a training data set to obtain the trained multi-task deep learning network.
Optionally, in an embodiment of the present application, obtaining a labeling result corresponding to the preprocessed data set includes:
Using medical image processing software, manually labeling each image in the preprocessed data set with predefined vascular anatomy key points and vascular binary segmentation, wherein the labeling result of the vascular anatomy key points is a three-dimensional coordinate corresponding to each key point, and the labeling result of the vascular binary segmentation is a voxel-by-voxel binary map with the same size as the image;
Based on the labeling results of the vascular anatomy key points and the vascular binary segmentation, an automatic method is used for generating a blood vessel segment semantic segmentation labeling result corresponding to each image in the data set.
Optionally, in one embodiment of the present application, based on the vascular anatomy keypoints and the vascular binary segmentation labeling result, generating the blood vessel segment semantic segmentation labeling result corresponding to each image in the dataset by using an automated method includes:
obtaining a corresponding lumen central line from a blood vessel binary segmentation marking result by using a refinement algorithm, and dividing the central line into different semantic segments according to anatomical key point marking;
determining semantic labels of all blood vessel voxels in the blood vessel binary segmentation labels according to the nearest central line voxels;
and manually correcting the semantic segmentation automatic labeling result obtained by each image in medical image processing software so as to obtain a final semantic segmentation labeling result, wherein the semantic segmentation automatic labeling result comprises a semantic segment and a semantic label.
Optionally, in an embodiment of the present application, generating the training data set according to the preprocessed data set and the corresponding labeling result includes:
According to the labeling result, each image in the preprocessed data set is processed to obtain an anatomic key point multichannel probability heat map, a blood vessel segment semantic segmentation multichannel probability map, a blood vessel segment missing classification vector and a key point local bifurcation characteristic classification vector which are used as prediction targets corresponding to the images;
and forming a training data pair by each image in the preprocessed data set and the corresponding prediction target, wherein all training data pairs jointly form a training data set.
Optionally, in an embodiment of the present application, processing each image in the preprocessed dataset according to the labeling result to obtain an anatomical keypoint multichannel probability heat map, a vessel segment semantic segmentation multichannel probability map, a vessel segment missing classification vector, and a keypoint local bifurcation feature classification vector as prediction targets corresponding to the image, where the method includes:
outputting an anatomic key point multichannel probability heat map with the same size as the input image to each pre-processed image in the data set according to the labeling result of the anatomic key points of the blood vessels, wherein the corresponding probability heat map of each target key point takes the key point as the center to be in three-dimensional Gaussian distribution;
Generating a blood vessel segment semantic segmentation multichannel probability map according to the blood vessel segment semantic segmentation labeling result, wherein the last channel of the blood vessel segment semantic segmentation multichannel probability map is a background channel, and the rest channels respectively reflect the position distribution of each blood vessel segment in the input image;
Obtaining a blood vessel segment missing classification vector and a key point local bifurcation feature classification vector according to a labeling result of the semantic segmentation of the blood vessel segment, wherein when a certain blood vessel segment is missing in the labeling result of the semantic segmentation of the blood vessel segment, the anatomical key points at two ends of the blood vessel segment lose the local bifurcation feature, otherwise, the anatomical key points at two ends of the blood vessel segment have the local bifurcation feature.
Optionally, in one embodiment of the application, the multi-tasking deep learning network comprises a backbone portion and four branching portions, wherein,
A trunk part for extracting the characteristics of the input image and outputting a characteristic diagram;
the first branch is used for processing the feature map and generating a prediction result of the anatomic key point multichannel probability heat map;
The second branch is used for processing the feature map and generating a prediction result of the vascular segment semantic segmentation multichannel probability map;
the third branch is used for processing the feature map and generating a prediction result of the blood vessel segment missing classification vector;
And the fourth branch is used for processing the feature map and generating a prediction result of the feature classification vector of the local bifurcation of the key points.
Optionally, in one embodiment of the present application, training the initialized network using the training dataset includes:
Step S1: randomly selecting a training data pair from the training data set, inputting the preprocessed three-dimensional angiography image in the training data pair into a constructed multi-task deep learning network, and obtaining the output result of each branch of the network as a predicted result;
step S2: inputting a prediction result and a prediction target in the training data pair into a loss function to obtain a loss function value;
step S3: minimizing a loss function by using a gradient descent method based on the calculated loss function value, and adjusting network parameters;
Step S4: and repeating the steps S1, S2, S3 and S4, continuously adjusting the network parameters, and when the training times exceed the set upper limit times, completing training, determining the multi-task deep learning network parameters and obtaining the trained multi-task deep learning network.
The application aims at adapting to the complex topological structure variation condition of the blood vessel, respectively modeling whether the blood vessel segment is missing or not and the change of the local bifurcation characteristic of the key point (namely whether the key point has the local bifurcation characteristic) caused by the blood vessel segment missing as the additional attribute of each blood vessel segment and bifurcation point, and requiring an algorithm to classify and predict the attribute. The method is realized based on a deep learning network, takes a multitask model as a framework, and simultaneously completes four subtasks of anatomy key point detection, blood vessel segment semantic segmentation, blood vessel segment missing classification and key point local bifurcation characteristic classification. The subtasks are highly correlated and share the spatial semantic features extracted by the network trunk part, the cooperative effect among the tasks is fully utilized, and the blood vessel variation type and the structure priori information are explicitly modeled. The application can be widely applied to detection tasks of various vascular anatomy key points, such as intracranial blood vessels, coronary arteries and the like, and can realize better detection performance.
The method of the embodiment of the present invention will be described in detail with reference to a specific embodiment.
The anatomical key point detection method provided by the application is applied to partial key point detection of intracranial vascular magnetic resonance angiography images, and the whole flow is shown in figure 2 and comprises an off-line stage and an on-line stage.
(1) Offline stage
(1-1) Acquiring an original data set and preprocessing;
Using a large number of three-dimensional angiographic images containing the same vessel type as the original dataset (intracranial vessel magnetic resonance angiographic images are used in this example), the images can be derived from a public dataset or a collaborative hospital, the number should be no less than 50. And preprocessing each image in the original data set, wherein the preprocessing comprises three parts of uniform resolution, clipping to the same size and voxel gray value normalization. The invention has no special requirements on the specific numerical values of the resolution and the size after cutting (the resolution is set to be 0.5 multiplied by 0.8mm 3 and the size after cutting is set to be 192 multiplied by 160 multiplied by 60 in the embodiment); the clipped image should contain the whole blood vessel structure to be detected (as the Willis ring region is required to be contained in the embodiment), the clipping process can remove the interference of noise such as bones and other irrelevant tissues, and the size of the clipping region can be determined according to the average statistical distribution of the blood vessel structure.
The preprocessed magnetic resonance angiography image of this embodiment is shown in fig. 3 (a).
(1-2) Labeling the preprocessed data set;
Each image in the preprocessed dataset is manually labeled using medical image processing software (in this embodiment, 3D slice software), requiring labeling of predefined vascular anatomical keypoints and vascular binary segmentations. The labeling result of the anatomical key points is a three-dimensional coordinate corresponding to each key point, and the labeling result of the vessel binary segmentation is a voxel-by-voxel binary image with the same size as the image (wherein, the voxel value of the vessel region is 1, and the voxel values of the rest background regions are 0).
Based on the anatomical key points and the blood vessel binary segmentation labels, an automatic method can be used for generating blood vessel segment semantic segmentation labels corresponding to each image in the data set. Specifically, a refinement algorithm is used to obtain a corresponding lumen centerline from a vessel binary segmentation labeling, and the centerline is divided into different semantic segments according to an anatomical key point labeling (in this embodiment, the centerline portion between the key point PCoA-A and the PCoA-P is the PCoA semantic segment). For each vessel voxel (voxel with voxel value of 1) in the vessel binary segmentation label, the semantic label is determined according to the nearest central line voxel. In particular, at the end of the peripheral vessel segment (e.g., the outer end of the MCA-M1 segment in this embodiment), the semantic segmentation markers are cut such that the cut plane is perpendicular to the centerline direction thereof. And then, manually correcting semantic segmentation automatic labeling results obtained by each image in medical image processing software, thereby obtaining final semantic segmentation labeling.
In this embodiment, the vessel binary segmentation labeling in the magnetic resonance angiography image is shown in fig. 3 (B); the corresponding anatomical keypoint labels of the image are shown in (C) of FIG. 3, in which the numbers are the predefined 19 keypoints; the semantic segmentation labels corresponding to the images are shown in (D) of fig. 3, and regions with different gray scales in the figures represent different blood vessel segments (i.e., different semantic tags), and english is abbreviated as anatomical naming of the blood vessel segments.
(1-3) Preparing a training dataset;
And (3) finishing the preparation work of the training data set by using the original data set preprocessed in the step (1-1) and the artificial labeling result obtained in the step (1-2), namely, each image in the data set, and obtaining the prediction targets of four subtasks including anatomical key point detection, blood vessel segment semantic segmentation, blood vessel segment missing classification and key point local bifurcation feature classification in the multi-task network.
(1-3-1) Anatomical keypoint detection prediction target generation;
in the application, the anatomical keypoint detection target is modeled as a multichannel Gaussian heat map regression task. Specifically, the network is required to output a probability heat map equal to the size of the input image for each pre-processed image in the dataset for each pre-defined keypoint. For each target key point, the corresponding probability heat map takes the key point as a center to form three-dimensional Gaussian distribution, and the value of each voxel reflects the probability that the voxel belongs to the target key point. The probability value is determined by Euclidean distance from the voxel position to the target key point, and the probability value is decreased from 1 to 0 outwards from the voxel position, and the decreasing rate is determined by Gaussian distribution standard deviation delta. Specifically, if the spatial coordinate of the ith key point of any preprocessed image in the dataset is x i, the probability value G i (x) of the corresponding probability heat map at any voxel spatial position x can be defined as:
Where N is the predefined total number of anatomical keypoints in each image. In this embodiment, the heat map generated for each anatomical keypoint in the magnetic resonance angiographic image is shown in fig. 3 (E). For ease of viewing, a three-dimensional heat map of all key points is projected into the same plane.
(1-3-2) Vessel segment semantic segmentation prediction target generation;
In the application, a vessel segment semantic segmentation task is modeled as a multi-channel single vessel segment binary segmentation task, namely, a prediction target is a multi-channel probability map generated by vessel segment semantic segmentation labeling. For S predefined vessel segments (i.e., S semantic classes), the prediction target should include s+1 channels, where the first S channels respectively reflect the position distribution of each vessel segment in the input image (i.e., whether each voxel in the input image belongs to each vessel segment, when a voxel belongs to the ith vessel segment, the value of the ith channel in the voxel position is 1, and the values of the rest channels are 0), and the s+1 channels are background channels (i.e., reflect whether each voxel in the input image belongs to the background class, when a voxel does not belong to any vessel segment, the value of the background channel in the voxel position is 1, and the values of the rest channels are 0).
In this embodiment, the predicted target for semantic segmentation of vessel segments in the magnetic resonance angiography image is shown in fig. 3 (F), which shows the channel corresponding to the MCA-M1 vessel segment in the predicted target.
(1-3-3) Generation of a blood vessel segment missing classification and key point local bifurcation characteristic classification prediction target;
Whether the blood vessel segment is missing or not and whether the key points at the two ends of the blood vessel segment have local bifurcation features corresponds to each other one by one can be obtained by the artificial labeling result of semantic segmentation of the blood vessel segment. Specifically, when a certain blood vessel segment is missing in the semantic segmentation labeling of the blood vessel segment (namely, the number of voxels belonging to the blood vessel segment in the labeling is 0), the anatomical key points at the two ends of the blood vessel segment lose the local bifurcation characteristic; when a vessel segment exists (i.e., the number of voxels belonging to the vessel segment in the label is greater than 0), the anatomical keypoints at the ends of the vessel segment have a local bifurcation feature. The correspondence relationship described above can be intuitively illustrated by fig. 4.
According to the application, the blood vessel segment missing classification and the key point local bifurcation characteristic classification are processed into a plurality of mutually independent classification tasks. The predicted targets for the predefined N anatomical keypoints and S vessel segments, the keypoint local bifurcation feature classification and the vessel segment deficiency classification are vectors y N and y S of length N and S, respectively, the values of the elements in the vector reflect whether each keypoint has a local bifurcation feature, whether each vessel segment is present (for any input image, the value of the i-th element in vector y S is 0 when the i-th vessel segment is missing, and is 1, and similarly, the value of the i-th element in vector y N is 0 when the i-th anatomical keypoint does not have a local bifurcation feature, and is 1, and vice versa).
And (1-3-4) constructing a training data pair by each preprocessed image and an anatomic key point multichannel probability heat map, a blood vessel segment semantic segmentation multichannel probability map, a blood vessel segment missing classification vector and a key point local bifurcation characteristic vector which are generated by corresponding manual labeling. All pairs of training data together form a training data set.
(1-4) Constructing a multi-task deep learning network;
The input of the multi-task deep learning network is a single three-dimensional angiographic image after preprocessing, and the input image is required to have uniform size and resolution, but specific values are not limited (the input image size used in the embodiment is 192×160×60, and the resolution is 0.5×0.5×0.8mm 3). The network is composed of a trunk portion and four branch portions, and the structure is shown in fig. 4. The backbone part improves the self-medical image processing classical network U-Net model, comprising symmetrical encoder and decoder structures. The encoder comprises 5 residual error modules and 4 largest pooling layers, wherein the largest pooling layers are sequentially distributed between every two residual error modules. The residual error module does not change the size of the input feature map and comprises two convolution layers with the convolution kernel size of 3 multiplied by 3 and a short connection structure between the input and the output of one module so as to avoid the gradient vanishing problem possibly occurring in the deep learning network training process. Each max pooling layer reduces the dimension size of the feature map to 1/2 of the original dimension size. The decoder comprises 3 residual modules and 4 deconvolution layers, and the residual modules are distributed between every two deconvolution layers in sequence. The decoder has the same structure as the residual module in the encoder, and each deconvolution layer expands the dimension size of the feature map by 2 times. The output end of the 5 th residual error module in the encoder is connected with the 1 st deconvolution layer input end in the decoder, and the maximum pooling layer in the encoder is kept consistent with the deconvolution layer number in the decoder, so that the input and output of the trunk part are ensured to have the same size.
In addition, in order to integrate the local spatial features with low dimensionality and the global semantic information with high dimensionality, a jumper structure is added between symmetrical layers in the encoder and decoder structures. Specifically, the output characteristic diagram of the 4 th residual error module in the encoder and the output characteristic diagram of the 1 st deconvolution layer in the decoder are spliced together to be used as the input of the 1 st residual error module in the decoder; splicing the output characteristic diagram of the 3 rd residual error module in the encoder with the output characteristic diagram of the 2 nd deconvolution layer in the decoder to be used as the input of the 2 nd residual error module in the decoder; splicing the output characteristic diagram of the 2 nd residual error module in the encoder and the output characteristic diagram of the 3 rd deconvolution layer in the decoder to be used as the input of the 3 rd residual error module in the decoder; and splicing the output characteristic diagram of the 1 st residual error module in the encoder with the output characteristic diagram of the 4 th deconvolution layer in the decoder to be used as the output characteristic diagram of the trunk part together. Thereafter, the output profile of the backbone portion is simultaneously transmitted into four branches of the network.
The four branch parts of the network are respectively formed by a residual module and a convolution layer with the convolution kernel size of 1 multiplied by 1. The output of the four branch parts respectively corresponds to the prediction results of an anatomic key point multi-channel probability heat map, a blood vessel segment semantic segmentation multi-channel probability map, a blood vessel segment missing classification vector and a key point local bifurcation characteristic vector, wherein the size and resolution of the first two branch prediction results are consistent with those of an input image, and the length of the last two branch prediction result vectors are consistent with the predefined blood vessel segment and the number of key points respectively.
The multi-task deep learning network constructed in this embodiment is shown in fig. 5, taking an intracranial vascular magnetic resonance angiography image as an example, and note that the numerical values in the figure are only examples, and other numerical values can be actually adopted.
(1-5) Applying the training data set generated in step (1-3), offline training the multi-task deep learning network constructed in step (1-4), the offline training comprising the steps of:
(1-5-1) randomly initializing the multi-task deep learning network parameters constructed in the step (1-4).
(1-5-2) Randomly selecting a training data pair from the training data set generated in the step (1-3), inputting the preprocessed three-dimensional angiography image into the multi-task deep learning network constructed in the step (1-4), and obtaining the output result of each branch of the network as the prediction result of each subtask. And respectively inputting the prediction result of each subtask and the prediction target of each subtask in the training data pair into a corresponding loss function to obtain a corresponding loss function value. Specifically, the invention uses an L2 loss function in an anatomical keypoint detection task, a Dice loss function in a blood vessel segment semantic segmentation task, and a cross entropy loss function in a blood vessel segment missing classification and keypoint local bifurcation feature classification task respectively. In order to avoid the problem that the network is difficult to converge due to serious class imbalance in training, the loss functions of the anatomic key point detection and the blood vessel segment semantic segmentation tasks are weighted, and the weights are respectively the ratio of the number of voxels of an input image to the number of voxels of the region where Gaussian hot spots and each blood vessel segment are located.
In addition, considering whether the blood vessel segment is missing or not and whether the local bifurcation feature one-to-one correspondence exists in the key points or not, a consistency loss function L self is introduced to monitor the prediction results of the classification tasks of the blood vessel segment and the key points to ensure that the prediction results of the classification tasks conform to the observation rule. In particular, the prediction of the classification by the absence of a vessel segmentDeriving corresponding key point local bifurcation feature categoriesKey point local bifurcation characteristic classification prediction result requiring actual output of key point local bifurcation characteristic classification prediction result and networkAnd keep the same. The consistency loss function may be defined using a cross entropy loss function:
Where the superscript i indicates the ith element in the vector (i.e., corresponds to the ith anatomical keypoint), Θ is a set of all keypoint sequence numbers that may cause a local bifurcation feature change due to a vascular variation (e.g., for the intracranial vascular magnetic resonance angiography image used in this embodiment, the predefined Willis loop anatomical keypoints, common keypoints that may cause a local bifurcation feature change due to physiological variation include two side PCoA, ACoA, PCA-P1, two side endpoints of ACA-A1, etc.).
The total loss function of the network training is obtained by linear combination of the loss functions:
L＝L1+αL2+β(L3+L4)+γLself(0＜α，β＜1)
Wherein, L 1、L2、L3、L4 is a loss function of anatomic key point detection, blood vessel segment semantic segmentation, blood vessel segment deletion classification and key point local bifurcation characteristic classification tasks, and L self is a consistency loss function. The super parameters alpha, beta and gamma can be flexibly adjusted in the actual scene so that the loss functions are in the same magnitude.
(1-5-3) Cyclically performing the training steps, wherein in each training, the loss function is minimized using a gradient descent method based on the calculated total loss function value, and the network parameters are continuously adjusted. When the training times exceeds the set upper limit times (the upper limit times are generally not less than 5000 times), the training is completed, and the multi-task deep learning network parameters are obtained.
(2) An online stage;
(2-1) acquiring a three-dimensional angiographic image containing the same vessel type as the original dataset of step (1-1) as a test image.
(2-2) Performing preprocessing on the test image obtained in the step (2-1), wherein parameters such as image resolution, image size after cutting and the like in the preprocessing operation should be kept consistent with the preprocessing step in the step (1-1).
And (2-3) inputting the preprocessed three-dimensional angiography image obtained in the step (2-2) into a trained multi-task deep learning network in an offline stage to obtain an output anatomic key point prediction probability map. And selecting the voxel position with the maximum prediction probability in each prediction probability map, namely the final detection result of the corresponding key point of the heat map. The multi-tasking deep learning network used in this step is shown in fig. 6. Note that the numerical values in the figures are for example only, and other numerical values may be used in practice.
By applying the anatomical keypoint detection method provided by the invention, the detection result of part of the keypoints of the intracranial vascular magnetic resonance angiography image in the embodiment is shown in fig. 7.
In order to realize the embodiment, the application also provides a device for detecting the anatomical key points in the three-dimensional angiography image,
Fig. 8 is a schematic structural diagram of a device for detecting anatomical keypoints in a three-dimensional angiographic image according to an embodiment of the application.
As shown in fig. 8, the device for detecting anatomical keypoints in the three-dimensional angiographic image comprises an acquisition module, a processing module and a result generation module, wherein,
The acquisition module is used for acquiring a three-dimensional angiography image as a test image;
The processing module is used for preprocessing the test image, inputting the preprocessed image into a pre-trained multi-task deep learning network and outputting an anatomic key point prediction probability map, wherein the multi-task deep learning model is obtained by training a training data set by taking a three-dimensional angiography training image with the same blood vessel type as the test image and a labeling result of the three-dimensional angiography training image;
And the result generation module is used for generating a detection result of the anatomical key point according to the prediction probability of the voxel position in the anatomical key point prediction probability map.
It should be noted that the foregoing explanation of the embodiment of the method for detecting an anatomical keypoint in a three-dimensional angiographic image is also applicable to the apparatus for detecting an anatomical keypoint in a three-dimensional angiographic image of this embodiment, and will not be repeated here.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.
Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In the description of the present application, the meaning of "plurality" means at least two, for example, two, three, etc., unless specifically defined otherwise.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and additional implementations are included within the scope of the preferred embodiment of the present application in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order from that shown or discussed, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the embodiments of the present application.
Logic and/or steps represented in the flowcharts or otherwise described herein, e.g., a ordered listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program is printed, as the program may be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.
It is to be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. As with the other embodiments, if implemented in hardware, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
Those of ordinary skill in the art will appreciate that all or a portion of the steps carried out in the method of the above-described embodiments may be implemented by a program to instruct related hardware, where the program may be stored in a computer readable storage medium, where the program, when executed, includes one or a combination of the steps of the method embodiments.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing module, or each unit may exist alone physically, or two or more units may be integrated in one module. The integrated modules may be implemented in hardware or in software functional modules. The integrated modules may also be stored in a computer readable storage medium if implemented as software functional modules and sold or used as a stand-alone product.
The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, or the like. While embodiments of the present application have been shown and described above, it will be understood that the above embodiments are illustrative and not to be construed as limiting the application, and that variations, modifications, alternatives and variations may be made to the above embodiments by one of ordinary skill in the art within the scope of the application.

Claims (10)

1.一种三维血管造影图像中解剖学关键点的检测方法，其特征在于，包括以下步骤：1. A method for detecting anatomical key points in a three-dimensional angiography image, characterized in that it comprises the following steps: 获取三维血管造影图像作为测试图像；acquiring a three-dimensional angiography image as a test image; 对所述测试图像进行预处理，并将预处理后的图像输入到预先训练的多任务深度学习网络中，输出解剖学关键点预测概率图、血管段语义分割多通道概率图、血管段缺失分类向量、关键点局部分叉特征分类向量，其中，所述多任务深度学习网络是以包含与所述测试图像同一血管类型的三维血管造影训练图像以及所述三维血管造影训练图像的标注结果作为训练数据集训练得到的；Preprocessing the test image, and inputting the preprocessed image into a pre-trained multi-task deep learning network, outputting an anatomical key point prediction probability map, a vascular segment semantic segmentation multi-channel probability map, a vascular segment missing classification vector, and a key point local bifurcation feature classification vector, wherein the multi-task deep learning network is trained using a three-dimensional angiography training image containing the same vascular type as the test image and the annotation results of the three-dimensional angiography training image as a training data set; 根据所述解剖学关键点预测概率图中体素位置的预测概率，生成解剖学关键点的检测结果；Generating a detection result of an anatomical key point according to the predicted probability of the voxel position in the anatomical key point prediction probability map; 其中，血管段是否缺失与其两端关键点是否具有局部分叉特征一一对应，当血管段语义分割标注中某一血管段缺失时，该血管段两端的解剖学关键点失去局部分叉特征；当某一血管段存在时，该血管段两端的解剖学关键点具有局部分叉特征；Among them, whether a blood vessel segment is missing corresponds to whether the key points at both ends of the segment have local bifurcation features. When a blood vessel segment is missing in the semantic segmentation annotation of the blood vessel segment, the anatomical key points at both ends of the blood vessel segment lose the local bifurcation features; when a blood vessel segment exists, the anatomical key points at both ends of the blood vessel segment have local bifurcation features. 在训练时，血管段缺失分类与关键点局部分叉特征分类的预测目标的生成过程包括：During training, the generation process of prediction targets for vessel segment missing classification and key point local bifurcation feature classification includes: 将血管段缺失分类和关键点局部分叉特征分类处理为多个相互独立的二分类任务，对预定义的N个解剖学关键点和S个血管段，关键点局部分叉特征分类和血管段缺失分类的预测目标分别为长度为N和S的向量yN和yS，向量中各元素的值反映了各关键点是否具有局部分叉特征、各血管段是否存在；The classification of missing blood vessel segments and the classification of local bifurcation features of key points are processed into multiple independent binary classification tasks. For the predefined N anatomical key points and S blood vessel segments, the prediction targets of the classification of local bifurcation features of key points and the classification of missing blood vessel segments are vectors y N and y S of length N and S respectively. The value of each element in the vector reflects whether each key point has a local bifurcation feature and whether each blood vessel segment exists. 所述多任务深度学习网络的损失函数表示为：The loss function of the multi-task deep learning network is expressed as: L＝L1+αL2+β(L3+L4)+γLself(0<α,β<1)L＝L 1 +αL 2 +β(L 3 +L 4 )+γL self (0<α,β<1) 其中，L1、L2、L3、L4分别为解剖学关键点检测、血管段语义分割、血管段缺失分类和关键点局部分叉特征分类任务的损失函数，L3、L4为交叉熵损失函数，α、β、γ为超参数，Lself为一致性损失函数，Among them, L1 , L2 , L3 , and L4 are the loss functions of anatomical key point detection, vascular segmentation, vascular segment missing classification, and key point local fork feature classification tasks, respectively. L3 and L4 are cross entropy loss functions. α, β, and γ are hyperparameters. Lself is the consistency loss function. 其中，上标i表示向量中第i个元素，θ为所有可能因血管变异而造成局部分叉特征变化的关键点序号集合，表示由血管段缺失分类的预测结果推导出的相对应的关键点局部分叉特征类别，为关键点局部分叉特征分类预测结果。Where, the superscript i represents the i-th element in the vector, and θ is the set of key point numbers that may cause local bifurcation feature changes due to vascular variation. Represents the prediction results classified by missing blood vessel segments The corresponding key point local bifurcation feature category is derived, The prediction results of the local fork feature classification for the key points. 2.如权利要求1所述的方法，其特征在于，所述对所述测试图像进行预处理，包括统一分辨率、裁剪至预设尺寸、体素灰度值归一化。2. The method according to claim 1 is characterized in that the preprocessing of the test image includes unifying the resolution, cropping to a preset size, and normalizing the voxel grayscale values. 3.如权利要求1所述的方法，其特征在于，预先训练所述多任务深度学习网络，包括：3. The method according to claim 1, wherein pre-training the multi-task deep learning network comprises: 获取包含与所述测试图像同一血管类型的三维血管造影图像作为原始数据集；Acquire a three-dimensional angiography image containing the same blood vessel type as the test image as an original data set; 对所述原始数据集进行所述预处理，并获取预处理后的数据集对应的标注结果，其中，所述标注结果包括血管解剖学关键点标注结果、血管二值分割标注结果和血管段语义分割标注结果；Performing the preprocessing on the original data set, and obtaining the annotation results corresponding to the preprocessed data set, wherein the annotation results include vascular anatomical key point annotation results, vascular binary segmentation annotation results, and vascular segment semantic segmentation annotation results; 根据预处理后的数据集和对应的标注结果生成训练数据集；Generate a training data set based on the preprocessed data set and the corresponding annotation results; 构建所述多任务深度学习网络，并使用所述训练数据集对所述多任务深度学习网络进行训练，得到训练后的多任务深度学习网络。Construct the multi-task deep learning network, and use the training data set to train the multi-task deep learning network to obtain a trained multi-task deep learning network. 4.如权利要求3所述的方法，其特征在于，所述获取预处理后的数据集对应的标注结果，包括：4. The method according to claim 3, wherein obtaining the annotation result corresponding to the preprocessed data set comprises: 使用医学图像处理软件，通过人工对预处理后的数据集中的每张图像标注预定义的血管解剖学关键点和血管二值分割两部分，其中，血管解剖学关键点标注结果为各关键点对应的三维坐标，血管二值分割标注结果为与图像相同大小的逐体素二值图；Using medical image processing software, each image in the preprocessed data set is manually annotated with predefined vascular anatomical key points and vascular binary segmentation. The vascular anatomical key point annotation results are the three-dimensional coordinates corresponding to each key point, and the vascular binary segmentation annotation results are voxel-by-voxel binary images of the same size as the image. 基于所述血管解剖学关键点和血管二值分割的标注结果，使用自动化方法生成数据集中每张图像对应的血管段语义分割标注结果。Based on the annotation results of the vascular anatomical key points and the vascular binary segmentation, an automated method is used to generate the semantic segmentation annotation results of the vascular segments corresponding to each image in the dataset. 5.如权利要求4所述的方法，其特征在于，所述基于所述血管解剖学关键点和血管二值分割标注结果，使用自动化方法生成数据集中每张图像对应的血管段语义分割标注结果，包括：5. The method according to claim 4, wherein the step of generating the semantic segmentation annotation result of the vascular segment corresponding to each image in the data set using an automated method based on the vascular anatomical key points and the vascular binary segmentation annotation result comprises: 使用细化算法由所述血管二值分割标注结果得到对应的管腔中心线，并依据解剖学关键点标注将中心线划分为不同的语义段；Using a thinning algorithm to obtain the corresponding lumen centerline from the blood vessel binary segmentation and annotation results, and dividing the centerline into different semantic segments according to the anatomical key point annotations; 对血管二值分割标注中的各个血管体素，依据最近的中心线体素确定其语义标签；For each vascular voxel in the vascular binary segmentation annotation, its semantic label is determined based on the nearest centerline voxel; 在医学图像处理软件中对各张图像所得的语义分割自动化标注结果进行手工修正，从而得到最终的语义分割标注结果，其中，所述语义分割自动化标注结果包括所述语义段和所述语义标签。Manually correct the automatic semantic segmentation annotation results obtained for each image in the medical image processing software to obtain a final semantic segmentation annotation result, wherein the automatic semantic segmentation annotation result includes the semantic segment and the semantic label. 6.如权利要求3所述的方法，其特征在于，所述根据预处理后的数据集和对应的标注结果生成训练数据集，包括：6. The method according to claim 3, characterized in that generating a training data set according to the preprocessed data set and the corresponding annotation results comprises: 根据标注结果，对预处理后的数据集中的每张图像进行处理，得到解剖学关键点多通道概率热图、血管段语义分割多通道概率图、血管段缺失分类向量、关键点局部分叉特征分类向量作为图像对应的预测目标；According to the annotation results, each image in the preprocessed data set is processed to obtain the multi-channel probability heat map of anatomical key points, the multi-channel probability map of vascular segment semantic segmentation, the classification vector of vascular segment missing, and the classification vector of key point local bifurcation features as the prediction target corresponding to the image; 将预处理后的数据集中的每张图像和对应的预测目标构成一个训练数据对，所有的训练数据对共同构成训练数据集。Each image in the preprocessed data set and the corresponding prediction target constitute a training data pair, and all training data pairs together constitute the training data set. 7.如权利要求6所述的方法，其特征在于，所述根据标注结果，对预处理后的数据集中的每张图像进行处理，得到解剖学关键点多通道概率热图、血管段语义分割多通道概率图、血管段缺失分类向量、关键点局部分叉特征分类向量作为图像对应的预测目标，包括：7. The method of claim 6, wherein the step of processing each image in the preprocessed data set according to the labeling result to obtain a multi-channel probability heat map of anatomical key points, a multi-channel probability map of semantic segmentation of vascular segments, a classification vector of missing vascular segments, and a classification vector of local bifurcation features of key points as prediction targets corresponding to the image comprises: 对数据集中每张预处理后的图像，根据所述血管解剖学关键点的标注结果对各预定义的关键点分别输出一张和输入图像尺寸相等的解剖学关键点多通道概率热图，其中，对各目标关键点，其对应的概率热图以该关键点为中心呈三维高斯分布；For each preprocessed image in the data set, a multi-channel probability heat map of anatomical key points with the same size as the input image is output for each predefined key point according to the annotation results of the vascular anatomical key points, wherein for each target key point, the corresponding probability heat map presents a three-dimensional Gaussian distribution with the key point as the center; 根据所述血管段语义分割标注结果生成血管段语义分割多通道概率图，其中，所述血管段语义分割多通道概率图的最后一个通道为背景通道，其余通道分别反映了输入图像中各血管段的位置分布；generating a multi-channel probability map of the semantic segmentation of the blood vessel segments according to the semantic segmentation and annotation results of the blood vessel segments, wherein the last channel of the multi-channel probability map of the semantic segmentation of the blood vessel segments is a background channel, and the remaining channels respectively reflect the position distribution of each blood vessel segment in the input image; 根据血管段语义分割的标注结果得到血管段缺失分类向量、关键点局部分叉特征分类向量，其中，当血管段语义分割标注结果中某一血管段缺失时，该血管段两端的解剖学关键点失去局部分叉特征，否则该血管段两端的解剖学关键点具有局部分叉特征。According to the annotation results of vascular segment semantic segmentation, the vascular segment missing classification vector and the key point local bifurcation feature classification vector are obtained. When a vascular segment is missing in the vascular segment semantic segmentation annotation results, the anatomical key points at both ends of the vascular segment lose the local bifurcation feature, otherwise the anatomical key points at both ends of the vascular segment have the local bifurcation feature. 8.如权利要求3所述的方法，其特征在于，所述多任务深度学习网络包括主干部分和四个分支部分，其中，8. The method of claim 3, wherein the multi-task deep learning network comprises a trunk part and four branch parts, wherein: 所述主干部分，用于对输入的图像进行特征提取，输出特征图；The backbone part is used to extract features from the input image and output a feature map; 第一分支，用于对所述特征图进行处理，生成解剖学关键点多通道概率热图的预测结果；The first branch is used to process the feature map to generate a prediction result of a multi-channel probability heat map of anatomical key points; 第二分支，用于对所述特征图进行处理，生成血管段语义分割多通道概率图的预测结果；The second branch is used to process the feature map to generate a prediction result of a multi-channel probability map of the semantic segmentation of the blood vessel segment; 第三分支，用于对所述特征图进行处理，生成血管段缺失分类向量的预测结果；The third branch is used to process the feature map to generate a prediction result of a blood vessel segment missing classification vector; 第四分支，用于对所述特征图进行处理，生成关键点局部分叉特征分类向量的预测结果。The fourth branch is used to process the feature map to generate a prediction result of a key point local fork feature classification vector. 9.如权利要求6所述的方法，其特征在于，所述使用所述训练数据集训练初始化后的网络，包括：9. The method according to claim 6, wherein the step of training the initialized network using the training data set comprises: 步骤S1：从所述训练数据集中随机选取一个训练数据对，将训练数据对中的经过预处理后的三维血管造影图像输入构建的多任务深度学习网络中，获取网络各分支的输出结果为预测结果；Step S1: randomly selecting a training data pair from the training data set, inputting the preprocessed three-dimensional angiography image in the training data pair into the constructed multi-task deep learning network, and obtaining the output results of each branch of the network as the prediction result; 步骤S2：将所述预测结果和所述训练数据对中的预测目标输入到损失函数中，得到损失函数值；Step S2: inputting the prediction result and the prediction target in the training data pair into a loss function to obtain a loss function value; 步骤S3：基于计算得到的损失函数值使用梯度下降法最小化损失函数，调整网络参数；Step S3: Based on the calculated loss function value, the loss function is minimized using the gradient descent method and the network parameters are adjusted; 步骤S4：重复进行步骤S1、步骤S2、步骤S3、步骤S4，不断调整网络参数，当训练次数超过设定的上限次数时，完成训练，确定多任务深度学习网络参数，得到训练好的多任务深度学习网络。Step S4: Repeat steps S1, S2, S3, and S4, and continuously adjust the network parameters. When the number of training times exceeds the set upper limit, the training is completed, the multi-task deep learning network parameters are determined, and a trained multi-task deep learning network is obtained. 10.一种三维血管造影图像中解剖学关键点的检测装置，其特征在于，包括获取模块、处理模块、结果生成模块，其中，10. A device for detecting anatomical key points in a three-dimensional angiography image, comprising an acquisition module, a processing module, and a result generation module, wherein: 所述获取模块，用于获取三维血管造影图像作为测试图像；The acquisition module is used to acquire a three-dimensional angiography image as a test image; 所述处理模块，用于对所述测试图像进行预处理，并将预处理后的图像输入到预先训练的多任务深度学习网络中，输出解剖学关键点预测概率图、血管段语义分割多通道概率图、血管段缺失分类向量、关键点局部分叉特征分类向量，其中，所述多任务深度学习网络是以包含与所述测试图像同一血管类型的三维血管造影训练图像以及所述三维血管造影训练图像的标注结果作为训练数据集训练得到的；The processing module is used to preprocess the test image, and input the preprocessed image into a pre-trained multi-task deep learning network, and output an anatomical key point prediction probability map, a vascular segment semantic segmentation multi-channel probability map, a vascular segment missing classification vector, and a key point local bifurcation feature classification vector, wherein the multi-task deep learning network is trained using a three-dimensional angiography training image containing the same vascular type as the test image and the annotation results of the three-dimensional angiography training image as a training data set; 所述结果生成模块，用于根据所述解剖学关键点预测概率图中体素位置的预测概率，生成解剖学关键点的检测结果；The result generating module is used to generate the detection result of the anatomical key point according to the prediction probability of the voxel position in the anatomical key point prediction probability map; 其中，血管段是否缺失与其两端关键点是否具有局部分叉特征一一对应，当血管段语义分割标注中某一血管段缺失时，该血管段两端的解剖学关键点失去局部分叉特征；当某一血管段存在时，该血管段两端的解剖学关键点具有局部分叉特征；Among them, whether a blood vessel segment is missing corresponds to whether the key points at both ends of the segment have local bifurcation features. When a blood vessel segment is missing in the semantic segmentation annotation of the blood vessel segment, the anatomical key points at both ends of the blood vessel segment lose the local bifurcation features; when a blood vessel segment exists, the anatomical key points at both ends of the blood vessel segment have local bifurcation features. 在训练时，血管段缺失分类与关键点局部分叉特征分类的预测目标的生成过程包括：During training, the generation process of prediction targets for vessel segment missing classification and key point local bifurcation feature classification includes: 将血管段缺失分类和关键点局部分叉特征分类处理为多个相互独立的二分类任务，对预定义的N个解剖学关键点和S个血管段，关键点局部分叉特征分类和血管段缺失分类的预测目标分别为长度为N和S的向量yN和yS，向量中各元素的值反映了各关键点是否具有局部分叉特征、各血管段是否存在；The classification of missing blood vessel segments and the classification of local bifurcation features of key points are processed into multiple independent binary classification tasks. For the predefined N anatomical key points and S blood vessel segments, the prediction targets of the classification of local bifurcation features of key points and the classification of missing blood vessel segments are vectors y N and y S of length N and S respectively. The value of each element in the vector reflects whether each key point has a local bifurcation feature and whether each blood vessel segment exists. 所述多任务深度学习网络的损失函数表示为：The loss function of the multi-task deep learning network is expressed as: L＝L1+αL2+β(L3+L4)+γLself(0<α,β<1)L＝L 1 +αL 2 +β(L 3 +L 4 )+γL self (0<α,β<1) 其中，L1、L2、L3、L4分别为解剖学关键点检测、血管段语义分割、血管段缺失分类和关键点局部分叉特征分类任务的损失函数，L3、L4为交叉熵损失函数，α、β、γ为超参数，Lself为一致性损失函数，Among them, L1 , L2 , L3 , and L4 are the loss functions of anatomical key point detection, vascular segmentation, vascular segment missing classification, and key point local fork feature classification tasks, respectively. L3 and L4 are cross entropy loss functions. α, β, and γ are hyperparameters. Lself is the consistency loss function. 其中，上标i表示向量中第i个元素，θ为所有可能因血管变异而造成局部分叉特征变化的关键点序号集合，表示由血管段缺失分类的预测结果推导出的相对应的关键点局部分叉特征类别，为关键点局部分叉特征分类预测结果。Where, the superscript i represents the i-th element in the vector, and θ is the set of key point numbers that may cause local bifurcation feature changes due to vascular variation. Represents the prediction results classified by missing blood vessel segments The corresponding key point local bifurcation feature category is derived, The prediction results of the local fork feature classification for the key points.
CN202210179800.9A 2022-02-25 2022-02-25 Method and device for detecting anatomical key points in three-dimensional angiography images Active CN114943682B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210179800.9A CN114943682B (en) 2022-02-25 2022-02-25 Method and device for detecting anatomical key points in three-dimensional angiography images

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210179800.9A CN114943682B (en) 2022-02-25 2022-02-25 Method and device for detecting anatomical key points in three-dimensional angiography images

Publications (2)

Publication Number Publication Date
CN114943682A CN114943682A (en) 2022-08-26
CN114943682B true CN114943682B (en) 2024-11-26

Family

ID=82905875

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210179800.9A Active CN114943682B (en) 2022-02-25 2022-02-25 Method and device for detecting anatomical key points in three-dimensional angiography images

Country Status (1)

Country Link
CN (1) CN114943682B (en)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115471499A (en) * 2022-10-19 2022-12-13 中国科学院空间应用工程与技术中心 Image target detection and segmentation method, system, storage medium and electronic equipment
CN116309591B (en) * 2023-05-19 2023-08-25 杭州健培科技有限公司 Medical image 3D key point detection method, model training method and device
CN116704248B (en) * 2023-06-07 2024-10-25 南京大学 Serum sample image classification method based on multi-semantic unbalanced learning
CN116524548B (en) * 2023-07-03 2023-12-26 中国科学院自动化研究所 Blood vessel structure information extraction method, device and storage medium
CN119399465B (en) * 2024-10-24 2025-04-18 江苏省电力试验研究院有限公司 Roof photovoltaic identification and evaluation method based on key point detection enhanced semantic segmentation
CN119963592B (en) * 2025-02-12 2025-10-24 强联智创（北京）科技有限公司 Blood vessel segmentation model training method, blood vessel segmentation method and electronic device

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112541893A (en) * 2020-12-11 2021-03-23 清华大学 Method for detecting tree structure branching key points in three-dimensional tomography image
CN112785581A (en) * 2021-01-29 2021-05-11 复旦大学附属中山医院 Training method and device for extracting and training large blood vessel CTA (computed tomography angiography) imaging based on deep learning
CN113192069A (en) * 2021-06-03 2021-07-30 清华大学 Semantic segmentation method and device for tree structure in three-dimensional tomography image

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110942466B (en) * 2019-11-22 2022-11-15 北京灵医灵科技有限公司 Cerebral artery segmentation method and device based on deep learning technology
CA3174674A1 (en) * 2020-03-10 2021-09-16 Bijoy K. Menon Systems and methods of prediction of ischemic brain tissue fate from multi-phase ct-angiography in patients with acute ischemic stroke using machine learning

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112541893A (en) * 2020-12-11 2021-03-23 清华大学 Method for detecting tree structure branching key points in three-dimensional tomography image
CN112785581A (en) * 2021-01-29 2021-05-11 复旦大学附属中山医院 Training method and device for extracting and training large blood vessel CTA (computed tomography angiography) imaging based on deep learning
CN113192069A (en) * 2021-06-03 2021-07-30 清华大学 Semantic segmentation method and device for tree structure in three-dimensional tomography image

Also Published As

Publication number Publication date
CN114943682A (en) 2022-08-26

Similar Documents

Publication Publication Date Title
CN114943682B (en) Method and device for detecting anatomical key points in three-dimensional angiography images
US20230104173A1 (en) Method and system for determining blood vessel information in an image
US12136220B2 (en) Method and system for providing an at least 3-dimensional medical image segmentation of a structure of an internal organ
CN112541893B (en) Method for detecting tree-structure bifurcation key points in three-dimensional tomography image
JP6657132B2 (en) Image classification device, method and program
CN113192069B (en) Semantic segmentation method and device for tree-like structure in three-dimensional tomography images
US20220284583A1 (en) Computerised tomography image processing
Xian et al. Main coronary vessel segmentation using deep learning in smart medical
US9189866B2 (en) Vascular tree from anatomical landmarks and a clinical ontology
EP2554120A1 (en) Projection image generation device, projection image generation programme, and projection image generation method
CN113506310A (en) Medical image processing method and device, electronic equipment and storage medium
Gu et al. Segmentation of coronary arteries images using global feature embedded network with active contour loss
Hepp et al. Fully automated segmentation and shape analysis of the thoracic aorta in non–contrast-enhanced magnetic resonance images of the German national cohort study
Li et al. Lumen segmentation of aortic dissection with cascaded convolutional network
CN115830016A (en) Medical image registration model training method and equipment
WO2025123880A1 (en) Methods, systems, and storage media for evaluating hemodynamic characteristics
CN116309346A (en) Medical image detection method, device, equipment, storage medium and program product
Guo et al. ELTS-Net: An enhanced liver tumor segmentation network with augmented receptive field and global contextual information
Essamlali et al. BDU-Net: A New Application of U-Net to the Segmentation of Bile Ducts from Cholangio-MRI Images
Zhang et al. SDLU-Net: A similarity-based dynamic linking network for the automated segmentation of abdominal aorta aneurysms and branching vessels
Li et al. Segmentation and reconstruction of cerebral vessels from 3D rotational angiography for AVM embolization planning
Rhu et al. 3D auto-segmentation of pancreas cancer and surrounding anatomical structures for surgical planning
Seada et al. Automatically seeded region growing approach for automatic segmentation of ascending aorta
Abbas et al. CV-Attention UNet: Attention-based UNet for 3D Cerebrovascular Segmentation of Enhanced TOF-MRA Images
Cai et al. [Retracted] Detection of 3D Arterial Centerline Extraction in Spiral CT Coronary Angiography

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant