CN115147334A - Bifurcation point detection method, image registration method, image segmentation method and system - Google Patents

Abstract

The invention provides a bifurcation point detection method, an image registration method, an image segmentation method and a system. The tree-like structure bifurcation point detection method detects bifurcation points in a tree-like structure through a deep learning network. The deep learning network trains the model used to detect the bifurcation points of the tree structure; in the inference step, using the bifurcation point topology atlas, Strengthen the bifurcation point.

Description

Bifurcation point detection method, image registration method, image segmentation method and system

technical field

The present invention relates to a tree structure branch point detection method and system for detecting tree structure branch points from an image, an image registration method and system using the method, and an image segmentation method and system.

Background technique

Trachea in the lung, blood vessels or blood vessels in the liver are important tissues of the human body. These internal blood vessels or trachea are usually tree-like structures, and detection or segmentation of these tree-like structures is an important topic. In addition, these tree-like structures are also important for segmentation and multimodal image registration of organs with this tree-like structure.

Patent Document 1 (WO2020/044840) describes a device for region division using a pulmonary trachea structure, which is used to divide a lung region included in a medical image. Regarding detection and localization of landmarks in the trachea as a tree-like structure, Patent Document 1 describes extracting a bronchial structure including bifurcation points to divide regions according to positions of a plurality of bifurcation points in the bronchial region. In addition, Patent Document 1 describes extraction of bronchial structures using machine learning.

SUMMARY OF THE INVENTION

Problems in the prior art

As described above, the bifurcation point is an important key point for the tree structure. Patent Document 1 proposes to use multiple bifurcation point positions in the bronchial region for area division, but the technical solution of Patent Document 1 only involves Region division instead of image division, specifically, Patent Document 1 divides the lung region into multiple regions in the up-down direction by using the positions of the bifurcation points, or divides the lung region into specific points from among the positions of the plurality of bifurcation points. The area where the cross point position is within a certain distance and the area where it is outside the certain distance (see paragraph 0042 of the specification of Patent Document 1).

In addition, most of the segmentation methods in the prior art are to segment the tree structure and then extract the center line and the bifurcation points of the tree structure from the segmented images of the tree structure. Most of these traditional methods are based on the image intensity, geometric shape, etc. to directly segment the tree structure from the image, or directly perform the tree structure image segmentation based on deep learning. Bifurcation points as key points are not fully utilized in the segmentation of tree structures.

In addition, as a detection technique for a branch point, although some methods for detecting a tree structure or a branch point of a tree structure based on machine learning are proposed in the prior art such as Patent Document 1, the branch point in these prior art The detection method does not consider the correspondence between the bifurcation points, and the detection precision and accuracy are insufficient.

Means for solving technical problems

The present invention is proposed to solve the above-mentioned problems of the prior art. The present invention proposes a method and system for detecting bifurcation points in a tree structure that considers the corresponding relationship between bifurcation points, and further proposes a method and system based on such bifurcation points. Image registration method and image segmentation method and system of point detection method.

According to an aspect of the present invention, there is provided a method for detecting bifurcation points in a tree structure, which detects bifurcation points in a tree structure through a deep learning network, characterized in that it includes the following steps: a training step, using the tree The bifurcation point topology atlas of the structure is used to train the model for detecting the bifurcation points of the tree structure through the deep learning network; in the inference step, the bifurcation point topology atlas is used to pass the The bifurcation points detected by the model trained in the training step are strengthened.

Therefore, the present invention uses the key point detection model of the tree structure trained by the deep learning network to detect the bifurcation points in the tree structure, and the deep learning network can use the topology information of the multiple key points of the tree structure during the training process. , in the inference process, the topology atlas of the bifurcation point is combined with the detected bifurcation points of the tree structure to strengthen the detection results, so that the corresponding information between the bifurcation points of the tree structure is used for Bifurcation point detection improves the accuracy and precision of bifurcation point detection.

In addition, the present invention provides an image registration method and an image segmentation method based on the above-mentioned bifurcation point detection method.

Specifically, according to another aspect of the present invention, an image registration method is provided, which uses the tree structure bifurcation points of an organ with a tree structure to match images of multiple modalities of the organ. It is characterized in that it includes the following steps: an acquisition step, acquiring images of the multiple modalities of the organ; a tree structure bifurcation point detection step, by using the above-mentioned tree structure bifurcation point detection method, after obtaining The bifurcation point of the tree structure is detected in the image of the organ; and the registration step is to register the images of the multiple modalities by using the detected bifurcation point.

According to another aspect of the present invention, there is provided an image segmentation method for segmenting an organ with a tree-like structure in a medical image, characterized in that it includes the following steps: an acquiring step, acquiring the medical image; a tree-like structure In the step of detecting a bifurcation point, the bifurcation point of the tree structure is detected in the obtained medical image by the above-mentioned method for detecting a bifurcation point of a tree structure; in the step of generating a center line, the detected bifurcation is used. point to generate the center line of the tree structure; and a dividing step, dividing the tree structure according to the generated center line of the tree structure.

The present invention can also be implemented as a tree structure bifurcation point detection system, an image registration system, and an image segmentation system having functional modules capable of implementing the various steps of the above-mentioned methods, and can also be implemented as a computer that executes the methods included in the above-mentioned methods. The steps are realized by a computer program, and can also be realized as a recording medium on which the above-mentioned computer program is recorded.

According to the present invention, the correspondence information between the branch points of the tree structure is used for branch point detection, thereby improving the accuracy and precision of branch point detection.

In addition, according to the present invention, the bifurcation point detection result that improves the accuracy and precision of bifurcation point detection is used for image registration such as registration of multimodal images, thereby improving the efficiency and precision of registration.

In addition, according to the present invention, the bifurcation point detection result which improves the accuracy and precision of bifurcation point detection is used for the image segmentation of the tree structure, thereby improving the efficiency and precision of the segmentation.

Description of drawings

FIG. 1 is a flow chart showing the training and inference process of the tree-structure bifurcation point detection model according to the first embodiment of the present invention;

2 is a schematic diagram showing a specific flow of a training process of a tree structure bifurcation point detection model according to the first embodiment of the present invention;

3 is a schematic diagram showing a specific flow of the reasoning process of the tree-structure bifurcation point detection model according to the first embodiment of the present invention;

FIG. 4 is a flowchart showing an image registration method according to a second embodiment of the present invention;

5 is an explanatory diagram showing a process of correcting a detection result in an image registration method according to a second embodiment of the present invention;

6 is a flowchart showing an image segmentation method according to a third embodiment of the present invention;

FIG. 7 is a functional block diagram showing a tree structure branch point detection system according to the first embodiment of the present invention.

Detailed ways

The present invention relates to methods and systems for bifurcation point detection, image registration, and image segmentation. These methods can be implemented by executing software programs on a device with a CPU (central process unit) such as an independent computer. These systems It can be implemented as the above-mentioned independent computer, and can also be implemented in hardware as a circuit capable of executing each step of the above-mentioned method. In addition, the system of the present invention may be pre-installed in the above medical image acquisition apparatus as a part of a medical image acquisition apparatus such as a magnetic resonance imaging apparatus (MRI).

Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.

In addition, in different embodiments, the same reference numerals are used for the same components, and overlapping descriptions are appropriately omitted.

<First Embodiment>

The present invention according to the first embodiment is a method and system for detecting bifurcation points of a tree structure, using a key point detection model of a tree structure trained by a deep learning network to detect bifurcation points in the tree structure.

The complete detection model deep learning framework consists of two main parts, namely, the training process and the inference process. The training process inputs a labeled (or ground truth: GT, Ground Truth) training data set to the model, calculates the objective function (loss function) between the output detection result and the true value, and uses gradient descent, stochastic gradient descent and other methods. The network parameters are corrected so that the loss function is minimized. This process is repeated until the error between the detection result output by the network and the true value meets the specified accuracy, so that the model reaches a state of convergence and reduces the error of the model predicted value. The inference process is the process of inputting unlabeled live data to the trained model to obtain the actual detection value. The method for detecting a branch point of a tree structure of the present invention is composed of the above-mentioned training process and inference process.

The first embodiment of the present invention will be described in detail below with reference to the accompanying drawings.

FIG. 1 is a flowchart showing the training and inference process of the tree structure bifurcation point detection model according to the first embodiment of the present invention, wherein FIG. 1( a ) represents the training process and FIG. 1( b ) represents the inference process.

As shown in FIG. 1, the training process of the tree structure bifurcation point detection model of the present invention includes the following steps: Step S100, input the ground truth (GT) of an image set and key points; Step S200, generate a heat map of samples and key points ; Step S300, use the deep learning network to train the key point detector; Step S400, output the trained model. The reasoning process of the tree structure bifurcation point detection model of the present invention includes the following steps: step S100', inputting an image set to the trained model; step S200', using the trained model to detect key points; step S300', using the key The point topology atlas strengthens the detection results.

The training process and the inference process are described in detail below with reference to FIG. 2 and FIG. 3 .

FIG. 2 is a schematic diagram showing a specific flow of the training process of the tree-structure bifurcation point detection model according to the first embodiment of the present invention.

As shown in FIG. 2 , the input in step S100 includes the input of the tree-structured 3D image set ( S101 ) and the input of branch point (ie, key point) information of the tree-structure as GT tags ( S102 ). Here, the GT label input in step S102 may be annotated by a known method for the 3D image set input in step S101, and may be prepared in advance. That is to say, step S101 and step S102 may be performed sequentially as shown in FIG. 2 , but may also be performed in parallel, in which case the arrow between step S101 and step S102 may be omitted.

Step S200 performs image preprocessing and transformation on the input image set, specifically, samples can be generated for the input image set through isotropic transformation, intensity transformation, elastic transformation, etc. (S201). Step S200 also transforms the input key points as GT labels and generates a GT heatmap (heatmap).

The above description of step S200 is just an example, and the preprocessing and transformation of the image set and the generation of the GT heat map can be implemented by various methods of the prior art known in the art.

In step S300, the bifurcation point detection model is trained. One of the features of the present invention is that when the bifurcation point detection model is trained, the bifurcation point topology atlas is introduced into the training process, so that the deep learning network can use the topology information of the multi-key points of the tree structure in the training process, Thus, the accuracy and precision of the bifurcation point detection are improved.

Specifically, the present invention first selects a network having characteristics suitable for the above-mentioned features of the present invention when constructing a network architecture. For example, in network design, a network type suitable for capturing the above topological information at each scale is selected. Since a stacked network with intermediate supervision can provide a mechanism to re-evaluate intermediate heatmaps and re-evaluate higher-order spatial relationships, this paper The inventive deep learning network preferably uses a stacked network. For example, the deep learning network of the present invention may be a Stacked Hourglass Network or a Coupled U-Network.

Second, the present invention includes a term representing the difference between the tree-structure bifurcation point topology atlas and the output value in the loss function of model training. The loss function (loss function) is a function used to estimate the degree of inconsistency (error value) between the output value P of the model and the true value G. The usual loss function can be expressed as L(P, G), one of the characteristics of the present invention. One is that the loss function in the training process is expressed by the following formula:

Loss _total = λ ₁ L(P, G)+λ ₂ L(P, A)

Among them, Loss _total represents the total error value of the model; L(P, G) represents the difference between the output value and the true value; L(P, A) represents the difference between the output value and the tree-structure bifurcation point topology atlas difference; λ ₁ and λ ₂ are weight coefficients.

L can use common loss functions in the field as needed, such as LogLoss logarithmic loss function, squared loss function, exponential loss function, Hinge loss function, etc. The details of these loss functions belong to common knowledge and will not be repeated here. The evaluation process of L(P, G) can adopt various methods in the prior art, which will not be repeated here. Specifically, the value of L(P,A) is to compare the similarity between the position and topology structure of the model output prediction point set P and the bifurcation point topology atlas A, and obtain the two by the function L(P,A). differences between topologies. Various methods in the prior art can be used to compare the similarity between the above-mentioned positions and topological structures, and details are not described herein again. The topology atlas A contains connections and side length relationships between bifurcation points at different levels, and the present invention calculates the difference between the position/topology structure between specific points in the predicted point set P and the position/topology structure in the topology atlas A, as part of the loss function.

λ ₁ and λ ₂ as weight coefficients are numbers between 0 and 1, and the sum of the two is 1. The specific values of λ ₁ and λ ₂ can be selected in advance according to experience, or the distance between the output detection result and the true value can be calculated by adjusting the weight coefficients λ ₁ and λ ₂ as in optimizing other parameters in the training process. The weighting coefficients λ ₁ and λ ₂ are optimized in such a way that the distance is shortened, and the value of the weighting coefficient that can obtain the optimal result is selected.

The above is the description of the training process of the branch point detection model of the tree structure according to the first embodiment of the present invention. As described above, the present invention introduces a tree-structured bifurcation point topology atlas in the training of the tree-structured keypoint detection model, and the deep learning network can utilize the tree-structured multi-keypoint topology information during the training process.

FIG. 3 is a schematic diagram showing a specific flow of the reasoning process of the tree-structure bifurcation point detection model according to the first embodiment of the present invention.

As shown in Figure 3, in step S100', an image set is input to the trained tree structure bifurcation point detection model. In step S200', key points are detected using the trained model. Here, in step S200', a heatmap is output for each detected bifurcation point, and the bright spots in the heatmap correspond to the bifurcation points in the tree structure. Ideally, each heatmap obtained should have only one bright spot, but There may be more than one bright spot in the resulting heatmap due to false detections caused by the accuracy and precision of the detection model and noise. In order to solve this problem, one of the features of the present invention is that in the next step S300', the detection result is enhanced by using the bifurcation point topology atlas.

The topological atlas of bifurcation points can be used to enhance the detection results because, for tubular organs with tree-like structures such as blood vessels or trachea, although the location of the bifurcation points of the tree-like structure may be different, overall Has a relatively stable topology. Such as liver blood vessels or pulmonary airways/vessels basically have a standard or can be standardized topology consisting of a first-order bifurcation point, a second-order bifurcation point 1, a second-order bifurcation point 2 . . . , so by applying the topological atlas of the bifurcation point to the bifurcation point detection results output by the bifurcation point detection model, and filtering out the detection results that do not conform to the bifurcation point topology atlas, the detection results can be strengthened, especially the higher level The bifurcation point of is more stably and accurately detected through the above enhancement.

To this end, in step S300 ′, the detected bifurcation points are screened through the bifurcation point topology atlas. FIG. 3 takes the portal vein of the liver as an example to illustrate the process of strengthening the detection results in the inference process of the tree structure detection model. , the output result of the detection model includes N heatmaps corresponding to the bifurcation point 1 to the bifurcation point N, wherein each heatmap contains the extreme point (the bright spot in the heatmap) for the bifurcation point. candidates, the number of these extreme point candidates may be more than one. As shown in the figure, the present invention is based on the portal vein topology of the liver, such as the first-order bifurcation point of the portal vein, the second-order right-1 bifurcation point of the portal vein, the second-order right-2 bifurcation point of the portal vein, and the left-1 point of the second-order portal vein. Fork points, the second left bifurcation point of the portal vein... The level relationship and spatial position relationship between such bifurcation points, etc., the extreme points in the heat map containing multiple extreme point candidates are screened, and the For extreme points that do not match the topological structure of the bifurcation points, only one extreme point is reserved for each heatmap as the enhanced detection result. The specific method for the above-mentioned elimination can be a method commonly used in the art, which will not be repeated here.

The above is the description of the reasoning process of the tree structure branch point detection model according to the first embodiment of the present invention. As mentioned above, the present invention introduces a tree-structured bifurcation point topology atlas in the reasoning of the tree-structured key point detection model, and bifurcates the bifurcation point topology atlas and the detected tree structure during the inference process. Points are combined to strengthen the detection results.

Therefore, in the tree structure bifurcation point detection method of the present invention including the above training process and reasoning process, by using the corresponding information between the bifurcation points of the tree structure for bifurcation point detection, the bifurcation point is improved. Detection accuracy and precision.

The present invention improves both the training process and the inference process in the tree structure bifurcation point detection method, and achieves the object of the present invention through the detection method composed of the improved training process and the improved inference process. However, it is not difficult to understand that improvements made only for either the training process or the reasoning process also belong to the protection scope of the present invention, and can also achieve the purpose of the present invention.

In addition, the present invention may include various modifications. For example, the bifurcation point topology of the tree structure may be standard for different organs or it may vary due to individual factors such as gender. Therefore, the topological atlas of the bifurcation point in the present invention may use a pre-prepared standard topological atlas, or may be a non-standard topological atlas obtained by, for example, a training process for these individual factors.

The tree structure branch point detection system according to the first embodiment will be described below. FIG. 7 is a functional block diagram showing a tree structure branch point detection system according to the first embodiment of the present invention. As shown in FIG. 7 , the tree structure bifurcation point detection system 1 according to the first embodiment is a tree structure bifurcation point detection system for detecting bifurcation points in a tree structure through a deep learning network, and includes: training The device 10 uses the bifurcation point topology atlas of the tree structure to train a model for detecting the bifurcation points of the tree structure through a deep learning network; the inference device 20 uses the bifurcation point topology atlas to perform training The bifurcation points detected by the trained model are then reinforced.

The method and system for detecting a branch point of a tree structure according to the first embodiment have been described above. According to the present invention, the key point detection model of the tree structure trained by the deep learning network is used to detect the bifurcation points in the tree structure, and the deep learning network can use the topology information of the multiple key points of the tree structure in the training process, In the inference process, the topological atlas of bifurcation points is used to strengthen the detection results. Therefore, the present invention uses the corresponding information between the branch points of the tree structure for branch point detection, which improves the accuracy and precision of branch point detection.

<Second Embodiment>

The present invention according to the second embodiment applies the tree-structure bifurcation point detection method of the first embodiment to the registration of medical images, especially the registration of multimodal medical images.

The second embodiment of the present invention will be described in detail below with reference to the accompanying drawings.

FIG. 4 is a flowchart showing an image registration method according to a second embodiment of the present invention. As shown in FIG. 4 , the image registration method of the present invention includes the following steps.

In step S4100, images of multiple modalities of the organ are acquired. The present embodiment will be described by taking the registration between the 3D liver image as the preoperative image and the 2D liver image as the intraoperative image as an example. In this case, step S4100 may include the step of acquiring a 3D preoperative MR/CT image set (step S4101) and a step of acquiring a 2D intraoperative US (ultrasound) image set for the same organ (S4102).

In step S4200, the bifurcation points of the liver blood vessels as a tree structure are detected in the obtained images of the organs; in step S4300, the detection results of the bifurcation points of the liver blood vessels are enhanced by using the 3D topology atlas of the liver blood vessels.

Here, steps S4200 and S4300 can be specifically implemented through the respective steps in the tree structure bifurcation point detection method described in the first embodiment. In other words, the detection and enhancement of the branch point of the tree structure formed by the steps S4200 and S4300 surrounded by the dotted line in FIG. For example, step S4200 corresponds to step S200' in the first embodiment, and step S4300 corresponds to step S300' in the first embodiment.

In the case where the present invention is used to register a 3D image as a preoperative image and a 2D image as an intraoperative image, step S4200 may be divided into two steps S4201 and S4202 for the 3D image and the 2D image, respectively, wherein The bifurcation point detection step S4201 of the 3D image and the bifurcation point detection step S4202 of the 2D image can be respectively performed by using the tree structure bifurcation point detection method of the present invention described in the first embodiment. On the other hand, the 2D image is relatively simple to detect the bifurcation point, so in order to speed up the processing speed and reduce the processing load, the bifurcation point detection step S4202 of the 2D image may not use the tree structure bifurcation point detection method of the present invention, but only by, for example, Do it manually. In this case, step S4300 may only be executed for the part of the 3D image (the output of step S4201 ) to which the tree structure bifurcation point detection method of the present invention is applied.

In step S4400, the preoperative liver image and the intraoperative liver image are registered using, for example, the bifurcation points of liver blood vessels detected in steps S4200 and S4300. Using the blood vessel bifurcation points to perform image registration including blood vessels may use any known prior art, and thus will not be described again.

Here, in order to improve the accuracy of the registration and reduce the processing load of the registration, after the branch point detection of the tree structure is performed in the present invention, before the registration is performed in step S4400, the segmentation result of the organ can also be used for the tree structure. The detection result of the bifurcation point is corrected. FIG. 5 is an explanatory diagram showing a process of correcting the detection result before registration in step S4400 in the image registration method according to the second embodiment of the present invention. As shown in FIG. 5 , the segmentation result image of liver tissue is obtained by segmenting, for example, the liver, and the segmentation result image is used as a template to apply to the detection result of the bifurcation point of liver blood vessels. The bright spots in the figure are retained as the bifurcation point detection results, and the bright spots outside the liver tissue area of the segmentation result image are excluded. Therefore, the bifurcation point is corrected before the registration, thereby improving the accuracy of the registration and reducing unnecessary processing load. PV and HV in Figure 5 represent the portal and hepatic veins of the liver, respectively.

Returning to FIG. 4 to continue the description, after the multi-modal image registration is performed through the bifurcation point in step S4400, the multi-modal image can be further finely registered by other structures in step S4500. The process of fine registration can obviously be omitted, so the operation of step S4500 is not necessary.

In addition, according to the image registration method of the present invention, in addition to using the bifurcation points of the tree structure in the registration in step S4400, of course, the tree structure itself can also be used. In this case, for example, after the branch point of the tree structure is detected, the tree structure center line of the organ is generated using the detected branch point of the tree structure, and the tree structure of the organ is performed based on the generated tree structure center line. segmentation to generate a tree-structured segmented image. Furthermore, in the registration step S4400, images of multiple modalities are registered according to the generated tree structure image.

The above-mentioned registration using the bifurcation point, the registration using the tree structure, and the fine registration can be implemented using techniques known in the art, and thus will not be described again.

The image registration system according to the second embodiment is an image registration system for registering images of a plurality of modalities of an organ using bifurcation points of an organ having a tree-like structure, and includes an acquisition device that acquires multiple modalities of an organ. an image of each modality; a device for detecting branching points of a tree-like structure, by using the method for detecting branching-points of a tree-like structure described in the first embodiment, to detect the branching points of the tree-like structure in the obtained image of the organ; and registration The device uses the detected bifurcation points to register images of multiple modalities.

The image registration method and system of the second embodiment have been described above. In the above description, only the registration between the 3D image as the preoperative image and the 2D image as the intraoperative image is shown, but the image registration method of the present invention can obviously also be applied to images of other modalities. Registration between images for more modalities. In addition, the second embodiment is described above by taking the liver blood vessel as an example, but the image registration method of the present invention can obviously be applied to the registration of other organs having a tree-like structure.

In addition to the effect of the first embodiment, the second embodiment can also obtain the effect of improving the registration accuracy of multimodal images such as preoperative images and intraoperative images.

<Third Embodiment>

The present invention according to the third embodiment applies the tree structure branch point detection method of the first embodiment to image segmentation.

The third embodiment of the present invention will be described in detail below with reference to the accompanying drawings.

FIG. 6 is a flowchart showing an image segmentation method according to a third embodiment of the present invention.

As shown in FIG. 6 , the image segmentation method of the present invention includes the following steps: step S6100, obtaining a medical image; step S6200, detecting the bifurcation points of the tree structure in the obtained medical image, wherein step S6200 can use the first embodiment Each step of the tree structure bifurcation point detection method of the present invention described in S6300 is used to generate the center line of the tree structure by using the detected bifurcation points; step S6400, according to the generated tree structure center line Segmentation of tree structures.

Taking the tree-like structure as a blood vessel as an example, step S6300 can be implemented by a method known in the art for extracting centerlines by using key points, such as a blood vessel tracking method, etc. Step S6400 can be achieved by threshold-based segmentation, image intensity-based Or traditional blood vessel segmentation methods such as geometric features, segmentation methods based on deep learning, etc. are implemented, so the details are not repeated here.

The image segmentation system according to the third embodiment is an image segmentation system for segmenting organs with a tree-like structure in a medical image, and includes: an acquisition device for acquiring a medical image; a tree-like structure bifurcation point detection device for The tree-like structure bifurcation point detection method described in the embodiment, detects bifurcation points of the tree-like structure in the acquired medical image; the centerline generating device generates the centerline of the tree-like structure by using the detected bifurcation points; and The dividing means divides the tree structure according to the center line of the generated tree structure.

The image segmentation method and system of the third embodiment have been described above. In addition to the effect of the first embodiment, the third embodiment can also obtain the effect of improving the accuracy of tree structure segmentation.

<Other modification examples>

The present invention is not limited to the above-described embodiments, and various modifications are possible.

For example, in each of the above embodiments, a blood vessel has been described as an example, but the present invention can also be applied to organs having other tubular structures such as a trachea.

The system of the present invention may be installed in a medical device as a circuit capable of realizing the functions described in the respective embodiments, or may be stored in a magnetic disk (a floppy disk, a hard disk, etc.), an optical disk (CD-ROM, etc.) as a program that can be executed by a computer , DVD, BD, etc.), magneto-optical disk (MO), semiconductor memory and other storage media.

Furthermore, OS (Operating System), database management software, MW (middleware) such as network software, etc. that operate on the computer based on an instruction from a program installed in the computer from the storage medium may execute each process for realizing the above-described embodiments. a part of.

Several embodiments of the present invention have been described above, but these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and changes can be made without departing from the gist of the invention. These embodiments and modifications thereof are included in the scope and spirit of the invention, and are also included in the invention described in the scope of the claims and the scope of their equivalents.

Claims (13)

1. A method for detecting branch points of a tree structure detects branch points in the tree structure through a deep learning network, and is characterized by comprising the following steps:

training a model for detecting the bifurcation points of the tree structure by using the bifurcation point topological atlas of the tree structure through the deep learning network;

and a reasoning step, namely strengthening the bifurcation point detected by the model trained in the training step by using the bifurcation point topological graph set.

2. The method of detecting a bifurcation point in a tree structure according to claim 1,

in the training step, a loss function used for the model training includes an item representing a difference between the bifurcation point topological graph set and an output value;

in the inference step, the detected bifurcation points are screened through the bifurcation point topological graph set, so as to perform the reinforcement.

3. The method of claim 2, wherein the tree bifurcation point is detected,

the loss function is represented by:

Loss _total ＝λ ₁ L(P，G)+λ ₂ L(P，A)

wherein,

Loss _total represents the total error value of the model shown;

l (P, G) represents the difference between the output value and the true value;

l (P, A) represents a difference between an output value and the bifurcation topology set;

λ ₁ and λ ₂ Are the weight coefficients.

4. The method of detecting a bifurcation point in a tree structure according to any one of claims 1 to 3,

the deep learning network is an hourglass network or a coupled U network.

5. An image registration method for registering images of a plurality of modalities of an organ having a tree structure using bifurcations of the tree structure of the organ, comprising the steps of:

an acquisition step of acquiring images of the plurality of modalities of the organ;

a tree-structure bifurcation point detection step of detecting a bifurcation point of the tree structure in the obtained image of the organ by the tree-structure bifurcation point detection method according to any one of claims 1 to 4; and

a registration step of registering images of the plurality of modalities using the detected bifurcation point.

6. The image registration method of claim 5,

a tree structure image generation step of generating a tree structure center line topology of the organ by using the branch point of the tree structure detected in the tree structure branch point detection step, and performing tree structure division of the organ according to the tree structure center line topology to generate a tree structure image;

the registering step further registers the images of the plurality of modalities according to the tree structure image.

7. The image registration method according to claim 5 or 6,

the images of the plurality of modalities including preoperative images of the organ and intraoperative images of the organ,

in the tree-structure bifurcation point detecting step, a tree-structure bifurcation point of the organ is detected in the preoperative image by the tree-structure bifurcation point detecting method, a tree-structure bifurcation point of the organ is detected in the intraoperative image by the tree-structure bifurcation point detecting method or a tree-structure bifurcation point of the organ is manually set in the intraoperative image,

in the step of detecting the branch point of the tree structure, the detection result of the branch point of the tree structure of the preoperative image is corrected according to the segmentation result of the organ.

8. The image registration method of claim 7,

the preoperative image is an MR image or a CT image, and the intraoperative image is an ultrasonic image.

9. The image registration method according to any one of claims 5 to 8,

the deep learning network is an hourglass network or a coupled U network.

10. An image segmentation method for segmenting an organ having a tree structure in a medical image, comprising the steps of:

an acquisition step of acquiring the medical image;

a tree-structure bifurcation point detection step of detecting a bifurcation point of the tree structure in the acquired medical image by the tree-structure bifurcation point detection method according to any one of claims 1 to 4;

a centerline generation step of generating a centerline of the tree structure using the detected bifurcation point; and

and a dividing step of dividing the tree structure according to the generated center line of the tree structure.

11. The utility model provides a branching point detecting system of tree structure, detects branching point in the tree structure through the deep learning network, its characterized in that possesses:

the training device is used for training a model for detecting the bifurcation points of the tree structure through the deep learning network by utilizing the bifurcation point topological atlas of the tree structure;

and the reasoning device is used for strengthening the bifurcation point detected by the model trained by the training device by utilizing the bifurcation point topological graph set.

12. An image registration system for registering images of a plurality of modalities of an organ having a tree structure using bifurcation points of the tree structure, the image registration system comprising:

an acquisition device that acquires images of the plurality of modalities of the organ;

a tree-structure bifurcation point detection device for detecting a bifurcation point of the tree structure in the obtained image of the organ by the tree-structure bifurcation point detection method according to any one of claims 1 to 4; and

and a registration device for registering the images of the plurality of modalities using the detected bifurcation point.

13. An image segmentation system for segmenting an organ having a tree structure in a medical image, comprising:

an acquisition device that acquires the medical image;

a tree-structure bifurcation point detection device for detecting a bifurcation point of the tree structure in the acquired medical image by the tree-structure bifurcation point detection method according to any one of claims 1 to 4;

a centerline generation means for generating a centerline of the tree structure using the detected bifurcation point; and

and a dividing device for dividing the tree structure according to the generated central line of the tree structure.

Images