CN118628667B

CN118628667B - Lung image three-dimensional reconstruction optimization method and device, electronic device and storage medium

Info

Publication number: CN118628667B
Application number: CN202411088957.6A
Authority: CN
Inventors: 李德轩; 茅蛟泽; 祖磊; 陈相儒; 谢晶
Original assignee: Hangzhou Zhuoxi Brain And Intelligence Research Institute; Hanyi Technology Hangzhou Co ltd
Current assignee: Hangzhou Zhuoxi Brain And Intelligence Research Institute; Hanyi Technology Hangzhou Co ltd
Priority date: 2024-08-09
Filing date: 2024-08-09
Publication date: 2025-01-21
Anticipated expiration: 2044-08-09
Also published as: CN118628667A

Abstract

The present invention discloses a method and device for optimizing the three-dimensional reconstruction of lung images, an electronic device, and a storage medium. The method uses a trained fracture repair model to perform vascular repair processing on a target area image in a lung image to obtain a repaired target area image; determines the number of adjacent nodes of each node in a first vascular skeleton line in the repaired target area image; determines the first type of nodes and the second type of nodes in the first vascular skeleton line according to the number of adjacent nodes of each node; determines the vascular skeleton line segments between adjacent first type nodes and second type nodes as vascular branches; calculates the branch length corresponding to the vascular branch; determines the vascular branch whose branch length is less than a preset length threshold as a target branch; and performs masking processing on the target branch to obtain a target area image. By masking the vascular branches whose branch length is less than a preset length threshold in the repaired lung image, the possibility of false positives is reduced, and the optimization of the lung image is achieved.

Description

Method and device for optimizing three-dimensional reconstruction of lung image, electronic equipment and storage medium

Technical Field

The disclosure relates to the technical field of data processing, and in particular relates to a lung image three-dimensional reconstruction optimization method and device, electronic equipment and a storage medium.

Background

In the field of modern medical imaging, accurate three-dimensional reconstruction of organ structures inside the human body plays an increasingly important role in clinical diagnosis, surgical planning and medical research. Among them, the three-dimensional reconstruction technique of pulmonary bronchus or pulmonary arteriovenous vessels has a critical position in early diagnosis of pulmonary diseases and formulation of personalized treatment schemes. The three-dimensional reconstruction of pulmonary bronchi or pulmonary arteriovenous vessels not only has important guiding function for clinicians in diagnosis and treatment decisions, but also provides opportunities for researchers to explore deeply the structural and functional association in the lungs.

The difficulty of three-dimensional reconstruction of the bronchus or the artery and vein vessels of the lung is that the computed tomography (Computed Tomography, CT) image of the bronchus or the artery and vein vessels of the lung generates more false yang and more faults after three-dimensional reconstruction. The false positive is mainly caused by poor performance of a segmentation model for segmenting CT images of bronchus or arteriovenous vessels of the lung and incapability of distinguishing image information of bronchus or arteriovenous vessels of the lung, so that the false positive is caused. In addition, the "burr" in the bronchus of the lung or the arteriovenous vessels of the lung is also another "false positive". If the pulmonary bronchi or pulmonary arteriovenous vessels are considered a large tree, then the "burr" is a myriad of bifurcated branches on the trunk of the large branch. The bifurcation is characterized by a large number and a significantly smaller length, and is not significant for analysis of the bronchi or the vessel trunk, and can cause visual disturbance, thus requiring removal. The reason for the large number of faults is that the three-dimensional reconstruction data of the bronchus of the lung or the artery and vein vessels of the lung reconstruct CT images, and if the thickness of the CT images is large, the data are easy to lose, so that the faults are caused. Secondly, the pixel threshold of the CT image has limitation, part of pulmonary bronchus or pulmonary artery and vein blood vessels are not obviously distinguished on the image, and if the segmentation model has low performance, missed detection occurs at some parts on the scanning data, so that the integral continuity is lost, and faults are also caused.

Because the existing three-dimensional reconstruction technology of the pulmonary bronchus or pulmonary arteriovenous blood vessels of the human lung CT image has poor effect, has more faults and false positive problems, can not accurately reconstruct the pulmonary bronchus or pulmonary arteriovenous blood vessels of the human body, and is easy to cause missed detection and false detection.

Disclosure of Invention

The disclosure provides a lung image three-dimensional reconstruction optimization method, a device, electronic equipment and a storage medium. The method mainly aims at solving the problems that the existing three-dimensional reconstruction technology of the human lung CT image pulmonary bronchus or pulmonary artery and vein is poor in effect, more faults and false positive exist, accurate three-dimensional reconstruction of the human lung bronchus or pulmonary artery and vein is not carried out, and missed detection and false detection are easy to cause.

According to a first aspect of the present disclosure, there is provided a method for optimizing three-dimensional reconstruction of a lung image, comprising:

Performing vascular repair treatment on the target area image in the lung image by adopting the trained fracture repair model to obtain a repaired target area image;

After extracting a first vascular skeleton line in the repaired target area image, determining the number of adjacent nodes of each node in the first vascular skeleton line;

Determining a first type node and a second type node in the first vascular skeleton line according to the number of adjacent nodes of each node, wherein the first type node is a node with only one adjacent node, and the second type node is a node with at least three adjacent nodes;

Determining a blood vessel skeleton line segment between adjacent first-class nodes and second-class nodes as a blood vessel branch;

The branch length corresponding to the blood vessel branch is calculated, and the blood vessel branch with the branch length smaller than a preset length threshold value is determined as a target branch;

And masking the target branch to obtain an optimized target area image.

Optionally, before performing the vascular repair process on the target area image using the trained fracture repair model, the method further includes:

invoking a central line extraction algorithm to extract a second vascular skeleton line in a training image, wherein the training image comprises an original label;

randomly marking the pixel points on the second blood vessel skeleton line to obtain marked pixel points;

performing morphological expansion operation on the marked pixel points to obtain morphological expanded pixel points;

deleting the morphologically expanded pixel points to obtain fracture masks corresponding to the training images;

And training the fracture repair model based on the fracture mask and the original label to obtain the trained fracture repair model.

Optionally, performing vascular repair processing on the target area image in the lung image by using the trained fracture repair model to obtain a repaired target area image, including:

Invoking a connected domain searching algorithm to search all the blood vessel connected domains in the target area image to obtain a blood vessel connected domain group;

Determining a largest vessel connected domain of the vessel connected domain group as a first vessel body;

respectively calculating first distances between other communicating domains except the blood vessel main body in the communicating domain group and the first blood vessel main body;

And inputting all other connected domains and the blood vessel main body into the fracture repair model in sequence according to the first distance order to perform blood vessel repair treatment, so as to obtain the repaired target area image.

Optionally, the determining the number of neighboring nodes of each node in the first vascular skeleton line includes:

Determining the number of target adjacent pixel points of each node by taking each node in the first vascular skeleton line as a center, wherein the target adjacent pixel points are adjacent pixel points positioned on the first vascular skeleton line;

and determining the number of target adjacent pixel points of the pixel points as the number of adjacent nodes of each node.

Optionally, the calculating the branch length corresponding to the blood vessel branch includes:

determining the number of pixel points constituting the blood vessel branch;

And determining the number of the pixel points as the branch length corresponding to the blood vessel branch.

Optionally, before performing the vascular repair process on the target region image in the lung image using the trained fracture repair model, the method further includes:

Carrying out partition processing on the lung image according to the characteristics of the lung blood vessels in the lung image to obtain a first region image, a second region image and the target region image;

After obtaining the optimized target area image, the method further comprises:

performing first median filtering processing on the first region image to obtain a filtered first region image;

Performing second median filtering processing on the second region image to obtain a filtered second region image;

performing third median filtering processing on the optimized target area image to obtain a filtered target area image, wherein the filter kernel sizes of the first median filtering processing, the second median filtering processing and the third median filtering processing are different;

And combining the filtered first area image, the filtered second area image and the filtered target area image to obtain a target image.

According to a second aspect of the present disclosure, there is provided a lung image three-dimensional reconstruction optimization apparatus comprising:

the repairing unit is used for performing vascular repairing treatment on the target area image in the lung image by adopting the trained fracture repairing model to obtain a repaired target area image;

A first determining unit configured to determine, after extracting a first vascular skeleton line in the repaired target area image, the number of neighboring nodes of each node in the first vascular skeleton line;

The second determining unit is used for determining a first type node and a second type node in the first vascular skeleton line according to the number of adjacent nodes of each node, wherein the first type node is a node with only one adjacent node, and the second type node is a node with at least three adjacent nodes;

a third determining unit, configured to determine a segment of a vascular skeleton between adjacent nodes of the first type and nodes of the second type as a vascular branch;

The calculating unit is used for calculating the branch length corresponding to the blood vessel branch;

A fourth determining unit, configured to determine a blood vessel branch with the branch length smaller than a preset length threshold as a target branch;

and the masking unit is used for masking the target branch to obtain an optimized target area image.

Optionally, the apparatus further comprises a training unit, the training unit comprising:

The extraction module is used for calling a central line extraction algorithm before performing vascular repair treatment on the target area image by adopting the trained fracture repair model to extract a second vascular skeleton line in the training image, wherein the training image comprises an original label;

The marking module is used for randomly marking the pixel points on the second blood vessel skeleton line to obtain marked pixel points;

the expansion module is used for carrying out morphological expansion operation on the marked pixel points to obtain morphological expanded pixel points;

The deleting module is used for deleting the morphological expanded pixel points to obtain a fracture mask corresponding to the training image;

and the training module is used for training the fracture repair model based on the fracture mask and the original label to obtain the trained fracture repair model.

Optionally, the repair unit includes:

The searching module is used for calling a connected domain searching algorithm to search all the blood vessel connected domains in the target area image so as to obtain a blood vessel connected domain group;

a first determining module configured to determine a largest vessel connected domain of the vessel connected domain group as a first vessel main body;

A first calculation module, configured to calculate first distances between the first vessel main body and other connected domains except the vessel main body in the connected domain group;

And the repair module is used for sequentially inputting all other connected domains and the blood vessel main body into the fracture repair model for performing blood vessel repair treatment according to the size sequence of the first distance to obtain the repaired target area image.

Optionally, the second determining unit includes:

the second determining module is used for determining the number of target adjacent pixel points of each node by taking each node in the first blood vessel skeleton line as a center, wherein the target adjacent pixel points are adjacent pixel points positioned on the first blood vessel skeleton line;

And the third determining module is used for determining the target adjacent pixel point number of the pixel points as the adjacent node number of each node.

Optionally, the computing unit includes:

a fourth determining module, configured to determine the number of pixels that constitute the blood vessel branch;

And a fifth determining module, configured to determine the number of pixel points as a branch length corresponding to the blood vessel branch.

Optionally, the apparatus further comprises a partition processing and filtering unit for:

before performing vascular repair treatment on a target area image in a lung image by adopting a trained fracture repair model, performing partition treatment on the lung image according to the characteristics of a lung blood vessel in the lung image to obtain a first area image, a second area image and the target area image;

The filtering unit includes:

The first filtering module is used for performing first median filtering processing on the first region image after the optimized target region image is obtained, so as to obtain a filtered first region image;

the second filtering module is used for carrying out second median filtering processing on the second region image to obtain a filtered second region image;

The third filtering module is used for carrying out third median filtering processing on the optimized target area image to obtain a filtered target area image, wherein the filter kernel sizes of the first median filtering processing, the second median filtering processing and the third median filtering processing are different;

And the merging module is used for merging the filtered first area image, the filtered second area image and the filtered target area image to obtain a target image.

According to a third aspect of the present disclosure, there is provided an electronic device comprising:

At least one processor, and

A memory communicatively coupled to the at least one processor, wherein,

The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of the first aspect.

According to a fourth aspect of the present disclosure, there is provided a non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of the preceding first aspect.

According to a fifth aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the method of the first aspect described above.

The lung image three-dimensional reconstruction optimization method, device, electronic equipment and storage medium comprise the steps of performing vascular repair processing on a target area image in a lung image by adopting a trained fracture repair model to obtain a repaired target area image, determining the number of adjacent nodes of each node in a first vascular skeleton line after the first vascular skeleton line in the repaired target area image is extracted, determining first type nodes and second type nodes in the first vascular skeleton line according to the number of adjacent nodes of each node, wherein the first type nodes are nodes with only one adjacent node, the second type nodes are nodes with at least three adjacent nodes, determining vascular skeleton line segments between the adjacent first type nodes and the second type nodes as vascular branches, calculating branch lengths corresponding to the vascular branches, determining the vascular branches with the branch lengths smaller than a preset length threshold as target branches, and performing masking processing on the target branches to obtain the optimized target area image. Compared with the related art, the three-dimensional reconstruction optimization method for the lung image has the advantages that the blood vessel faults in the lung image are repaired, and the blood vessel branches with the branch lengths smaller than the preset length threshold value in the repaired lung image are subjected to covering treatment, so that the possibility of false positive is reduced, the faults and the false positive problems in the lung image are solved, and the optimization of the lung image is realized.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the application or to delineate the scope of the application. Other features of the present application will become apparent from the description that follows.

Drawings

The drawings are for a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

Fig. 1 is a schematic flow chart of a three-dimensional reconstruction optimization method for lung images according to an embodiment of the disclosure;

FIG. 2 is a diagram of an example of a vascular fault provided by an embodiment of the present disclosure;

FIG. 3 is an exemplary diagram of a blood vessel "burr" provided by an embodiment of the present disclosure;

FIG. 4 is a flow chart of another lung image three-dimensional reconstruction optimization provided by an embodiment of the present disclosure;

FIG. 5 is a diagram of an example of a blood vessel of a lung image provided by an embodiment of the present disclosure;

FIG. 6 is a flow chart of another lung image three-dimensional reconstruction optimization provided by an embodiment of the present disclosure;

FIG. 7 is a flow chart of another lung image three-dimensional reconstruction optimization provided by an embodiment of the present disclosure;

fig. 8 is a schematic structural diagram of a three-dimensional reconstruction optimization device for lung images according to an embodiment of the present disclosure

FIG. 9 is a schematic structural view of another device for optimizing three-dimensional reconstruction of lung images according to an embodiment of the present disclosure;

fig. 10 is a schematic block diagram of an example electronic device 600 provided by an embodiment of the disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

In the presently disclosed embodiments, the term "exemplary" is used to mean "serving as an example, instance, or illustration. Any embodiment described as "exemplary" in this disclosure is not necessarily to be construed as preferred or advantageous over other embodiments. The prefix words "first", "second", etc. in the embodiments of the present disclosure are only for distinguishing different description objects, and do not limit the location, order, priority, number, content, etc. of the description objects, and the statement of the description object refers to the claims or the description of the embodiment context, and should not constitute unnecessary limitations due to the use of the prefix words. It will be apparent to one of ordinary skill in the art that the present application may be practiced without these specific details. In other instances, well-known structures and processes have not been described in detail so as not to obscure the description of the application with unnecessary detail. Thus, the present application is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.

It should be noted that, because the method of the embodiment of the present application is executed in the computing device, the processing objects of each computing device exist in the form of data or information, for example, time, which is substantially time information, it can be understood that in the subsequent embodiment, if the size, the number, the position, etc. are all corresponding data, so that the electronic device can process the data, which is not described herein in detail.

The lung image three-dimensional reconstruction optimization method and device, electronic equipment and storage medium according to the embodiments of the present disclosure are described below with reference to the accompanying drawings.

Fig. 1 is a schematic flow chart of a three-dimensional reconstruction optimization method for lung image, which is provided in an embodiment of the disclosure, and the method is applied to a computer, as shown in fig. 1, and includes the following steps:

and 101, performing vascular repair treatment on the target area image in the lung image by adopting a trained fracture repair model to obtain a repaired target area image.

In one embodiment provided in the present disclosure, the lung image is a lung image obtained by primarily segmenting arterial and venous blood vessels of a human lung by using a segmentation model and a tomographic image (CT), and the target area image is an image in which a large number of pulmonary capillary blood vessel areas are contained in the lung image, as shown in fig. 5, the pulmonary capillary blood vessels are characterized by dense and disordered structures, contain a large number of blood vessel branches, and have small blood vessel diameters, so that a blood vessel fault problem easily occurs in the target area image, and a false positive problem is caused by the existence of shorter blood vessel branches (i.e. "burrs"). For a better understanding of the vessel tomographic problem and vessel "burrs", please refer to fig. 2 and 3, and fig. 2 is a vessel tomographic illustration provided in an embodiment of the disclosure. Fig. 3 is an exemplary diagram of a blood vessel "burr" provided by an embodiment of the present disclosure.

Because impurities or newly added blood vessel branches are extremely easy to introduce in the repairing process of the blood vessel fault, false positive is caused, in the embodiment provided by the disclosure, blood vessel repairing treatment is needed to be carried out on the target area image first, breakpoint connection is carried out on the blood vessel fracture part, and then follow-up treatment is carried out on the repaired target area image.

In order to solve the problem of blood vessel fault in the target area image, the embodiment of the disclosure provides a trained fracture repair model for realizing blood vessel repair processing of the target area image, wherein the trained fracture repair model is used for realizing breakpoint connection of a blood vessel fracture in the target area image, so as to obtain the repaired target area image.

Step 102, after extracting the first vascular skeleton line in the repaired target area image, determining the number of adjacent nodes of each node in the first vascular skeleton line.

In one embodiment provided in the present disclosure, after obtaining the restored target area image, a first vascular skeleton line may be extracted from the restored target area image by using, but not limited to, an algorithm (such as morphology. Skeletonize_3d) of a skeletonized three-dimensional image in a picture processing library (such as scikit-IMAGE SCIKIT (toolkit for scipy), skimage), where the skeleton line is a 4-neighborhood or 8-neighborhood line extracted by a refinement algorithm (binary image skeleton extraction), and the end points and the intersections of the skeleton are particularly important. As shown in fig. 4, fig. 4 is a schematic diagram of a first vascular skeleton line provided in an embodiment of the present disclosure, where the first vascular skeleton line includes a plurality of nodes, and the nodes refer to connection points between any two adjacent vascular skeleton line segments.

Step 103, determining a first type node and a second type node in the first vascular skeleton line according to the number of adjacent nodes of each node, wherein the first type node is a node with only one adjacent node, and the second type node is a node with at least three adjacent nodes.

In some embodiments provided by the present disclosure, the categories of each node may be divided according to the number of neighboring nodes of the node. With continued reference to fig. 4, the first type node is a node having only one neighboring node, for example, the node B in fig. 4 has only one neighboring node a node, so the node B is a first type node, the second type node is a node having at least three neighboring nodes, and illustratively, the node H in fig. 4 is a node C, a node G, and a node F, and the number of neighboring nodes is 3, so the node H is the second type node. In addition, the nodes on the first vascular skeleton line are of a third type, and the third type refers to nodes with two adjacent nodes, for example, a node C in fig. 4, and the adjacent nodes are a node and an H node, so the node C is the third type of node.

And 104, determining a blood vessel skeleton line segment between the adjacent first type nodes and the second type nodes as a blood vessel branch.

In some embodiments provided by the present disclosure, the vascular skeleton line segment refers to all vascular skeletons between any two adjacent nodes in the first vascular skeleton line. For example, please continue to refer to fig. 4, wherein the line segments AC, CH, etc. are all vascular skeleton line segments. The G node and the F node are both first class nodes, and the H node is second class nodes, so that the line segment GH and the line segment FH are both branches of the blood vessel.

And 105, calculating the branch length corresponding to the blood vessel branch, and determining the blood vessel branch with the branch length smaller than a preset length threshold as a target branch.

In some embodiments provided by the present disclosure, since the blood vessel branches in the lung image are composed of one pixel point, the branch lengths corresponding to the blood vessel branches may be calculated by counting the number of pixel points between the first class node and the second class node of each blood vessel branch.

In some embodiments provided in the present disclosure, the preset length threshold may be flexibly adjusted according to actual application situations, and exemplary, the preset length threshold may be set to 10 pixels, or 15 pixels, etc., which is not limited in embodiments of the present disclosure.

And 106, masking the target branch to obtain an optimized target area image.

The lung image three-dimensional reconstruction optimization method comprises the steps of performing vascular repair processing on a target area image in a lung image by using a trained fracture repair model to obtain a repaired target area image, determining the number of adjacent nodes of each node in a first vascular skeleton line after the first vascular skeleton line in the repaired target area image is extracted, determining a first type node and a second type node in the first vascular skeleton line according to the number of the adjacent nodes of each node, wherein the first type node is a node with only one adjacent node, the second type node is a node with at least three adjacent nodes, determining a vascular skeleton line segment between the adjacent first type node and the second type node as a vascular branch, calculating the branch length corresponding to the vascular branch, determining the vascular branch with the branch length smaller than a preset length threshold as a target branch, and masking the target branch to obtain the optimized target area image. Compared with the related art, the three-dimensional reconstruction optimization method for the lung image has the advantages that the blood vessel faults in the lung image are repaired, and the blood vessel branches with the branch lengths smaller than the preset length threshold value in the repaired lung image are subjected to covering treatment, so that the possibility of false positive is reduced, the faults and the false positive problems in the lung image are solved, and the optimization of the lung image is realized.

To obtain a trained fracture repair model, a preset fracture repair model may be trained by using, but not limited to, a method of masking an automatic encoder (Masked Auto Encoder, MAE) to repair the target area image, and the disclosed embodiment further provides a flowchart of a training method of the fracture repair model, as shown in fig. 6, where the method includes the following steps:

Step 201, a center line extraction algorithm is invoked to extract a second vascular skeleton line in a training image, wherein the training image comprises an original label.

In one embodiment provided by the present disclosure, to simulate the real-world situation of a vessel rupture, a centerline extraction algorithm is first used to extract a second vessel line skeleton of the vessel. To obtain a training dataset for training the fracture repair model, a training dataset may be made using, but not limited to, a lung nodule public dataset (The Lung Image Database Consortium, LIDC-IDRI) consisting of chest medical image files (e.g., CT, X-ray) and corresponding diagnostic result lesion labels.

And 202, randomly marking the pixel points on the second vascular skeleton line to obtain marked pixel points.

And after the second vascular skeleton line is acquired, randomly marking pixel points belonging to capillary vessel parts on the second vascular skeleton line.

And 203, performing morphological expansion operation on the marked pixel points to obtain morphological expanded pixel points.

Morphological dilation is performed on the marked pixels, and the morphology is also called mathematical morphology, which is a very important research method in the image processing process. Morphology extracts mainly component information from the interior of an image, which is often of great importance for expressing and delineating features of an image, usually the most essential shape features used in image understanding. Morphological processing plays a very important role in the fields of visual inspection, word recognition, medical graphic processing and image compression coding. The morphological operations mainly comprise corrosion, expansion, open operation, close operation, morphological gradient operation, top cap operation and black cap operation. Among them, dilation and erosion are the most fundamental morphological operations in image processing, often combined to achieve some complex morphological operations of the image.

And 204, deleting the morphologically expanded pixel points to obtain a fracture mask corresponding to the training image.

And deleting the mask of the corresponding morphologically expanded pixel points on the vascular mask after the morphologically expanded pixel points are obtained, wherein the obtained vascular mask is a fracture mask after the fracture of the simulated blood vessel.

And step 205, training the fracture repair model based on the fracture mask and the original label to obtain the trained fracture repair model.

And forming a training sample by the fracture mask generated in the step 204 and the original label, and training the fracture repair model based on the training sample to obtain the trained fracture repair model.

By the aid of the training method for the fracture repair model, the fracture repair model is trained, so that the trained fracture repair model is obtained, and the blood vessel in the target area image is repaired.

In an embodiment provided in the present disclosure, after the trained fracture repair model is obtained, in order to perform a vascular repair process on the target area image in the lung image by using the trained fracture repair model, to obtain a repaired target area image, and better understand the repaired target area image, fig. 7 is a schematic flow diagram of fault repair provided in the embodiment of the present disclosure, as shown in fig. 7, the method includes the following steps:

Step 301, a connected domain searching algorithm is called, and all blood vessel connected domains in the target area image are searched to obtain a blood vessel connected domain group.

In one embodiment provided by the present disclosure, the connected Region (Connected Component) generally refers to an image Region (Region, blob) of foreground pixels in an image that have the same pixel value and are adjacent in position. Connected domain analysis (Connected Component Analysis, connected Component Labeling) refers to finding and labeling individual connected domains in an image. Connected domain analysis is a relatively common and basic method in CVPR and many application fields of image analysis processing. Such as character segmentation extraction (license plate recognition, text recognition, subtitle recognition, etc.) in optical character recognition Optical Character Recognition, OCR, motion foreground object segmentation and extraction in visual tracking (pedestrian intrusion detection, legacy object detection, vision-based vehicle detection and tracking, etc.), medical image processing (object region of interest extraction), etc.

Step 302, determining a largest vessel connected domain of the vessel connected domain group as a first vessel body.

After determining the blood vessel connected domain group, sorting the sizes of all connected domains in the blood vessel connected domain group, and determining the largest blood vessel connected domain as a blood vessel main body.

Step 303, calculating first distances between the other connected domains except the blood vessel main body and the first blood vessel main body in the connected domain group respectively.

After determining the blood vessel main body, respectively calculating first distances between other connected domains except the blood vessel main body in the connected domain group and the first blood vessel main body, and sequencing the first distances.

And step 304, inputting all other connected domains and the blood vessel main body into the trained fracture repair model in sequence according to the first distance order to perform blood vessel repair treatment, so as to obtain the repaired target area image.

After the sequence of the first distance is obtained, a vascular connected domain with the minimum first distance is selected to be combined with a vascular main body, the combination is put into the trained fracture repair model to repair, the obtained repaired second vascular main body is used as a new vascular main body, and then the rest vascular connected domain with the minimum first distance and the second vascular main body are input into the trained fracture repair model again until only one target vascular main body exists in the vascular connected domain group, and the repaired target region image is obtained.

By the method provided in the steps 301 to 304, the repair of the blood vessel fault problem existing in the target area image is realized.

As a further explanation of the above step 102, the determining the number of neighboring nodes of each node in the first vascular skeleton line may be, but is not limited to, first determining, with each node in the first vascular skeleton line as a center, a target number of neighboring pixels of the node, where the target number of neighboring pixels is located on the first vascular skeleton line, and determining the target number of neighboring pixels of the pixel as the number of neighboring nodes of each node. For example, with the node M as the center, calculating whether the corresponding pixels on 26 adjacent positions are on the first vascular skeleton line in the front-back, upper-lower, left-right and corresponding oblique angle directions, where the number of target adjacent pixels is equal to the number of adjacent nodes.

After obtaining the branch length of each blood vessel branch, comparing the branch length with a preset length threshold, and if the branch length is smaller than the preset length threshold, determining the corresponding blood vessel branch as a burr which causes a false positive problem, so that the burr is required to be masked. If the branch length is greater than the preset length threshold, the corresponding blood vessel branches into normal blood vessels, so that the problem of false positive is not caused, and the treatment is not needed.

By the method provided by the embodiment, the false positive problem in the repaired target area image is solved, and the three-dimensional reconstruction optimization of the target area image is realized.

In one embodiment provided by the disclosure, before performing a vascular repair process on a target area image in a lung image by using a trained fracture repair model, considering characteristics of a lung vessel, dividing the whole blood vessel into three parts according to the characteristics of the lung vessel in the lung image, respectively optimizing the lung image, and performing partition processing on the lung image to obtain a first area image, a second area image and the target area image, wherein the first area image is a blood vessel image close to a heart part, the second area image is a blood vessel image near a lung gate, and the target area image is a lung capillary vessel image.

First, the blood vessels are adjacent to the heart portion. This portion may be referred to as the vascular core. The blood vessel in the blood vessel core region is characterized by a large blood vessel whole body. The blood vessel in this section has little false positive and fault problems, nor any bifurcation. The whole presents a sphere. The main problem with this part of the vessel requiring optimization is the outer shell part of the vessel core region, which usually presents a step shape affected by the CT image layer thickness. So for this first region image, a median filter with a larger filter kernel can be applied to directly smooth the overall effect.

For the blood vessel in the second region image, the blood vessel of the part is an intermediate zone connecting the blood vessel core region and the capillary vessel, and the main characteristics are that the blood vessel is thick and the bifurcation is clear. Each bifurcation is connected to one lobe. The left lung branches off from the two portals, and the right lung branches off from the three portals. Since the blood vessels have a large thickness, median filtering in the median filtering kernel is performed for this portion.

And (3) for the optimized target area image, directly adopting median filtering with smaller filtering kernel to carry out smoothing treatment.

And respectively carrying out median filtering processing on the first region image, the second region image and the optimized target region image with different filter kernel sizes to obtain a filtered first region image, a filtered second region image and a filtered target region image, and then carrying out merging processing on the filtered first region image, the filtered second region image and the filtered target region image to obtain a target image.

Compared with the related art, the blood vessels in the distribution image are respectively processed in a partition processing mode, so that the blood vessel interruption layer problem and the false positive problem can be accurately processed, and the performance optimization of the lung image three-position reconstruction technology is improved.

In summary, the embodiments of the present disclosure have the following effects:

The method comprises the steps of repairing a blood vessel fault appearing in a lung image, and masking blood vessel branches with branch lengths smaller than a preset length threshold in the repaired lung image, so that the possibility of false positive is reduced, the faults and the false positive appearing in the lung image are solved, and the optimization of the lung image is realized.

Corresponding to the lung image three-dimensional reconstruction optimization method, the invention also provides a lung image three-dimensional reconstruction optimization method device. Since the device embodiment of the present invention corresponds to the above-mentioned method embodiment, details not disclosed in the device embodiment may refer to the above-mentioned method embodiment, and details are not described in detail in the present invention.

Fig. 8 is a schematic structural diagram of a three-dimensional reconstruction optimization device for lung image according to an embodiment of the present disclosure, and as shown in fig. 8, the device includes a repairing unit 41, a first determining unit 42, a second determining unit 43, a third determining unit 44, a calculating unit 45, a fourth determining unit 46, and a masking unit 47.

a repair unit 41, configured to perform a vascular repair process on the target area image in the lung image by using the trained fracture repair model, so as to obtain a repaired target area image;

A first determining unit 42 configured to determine, after extracting a first vascular skeleton line in the repaired target region image, the number of neighboring nodes of each node in the first vascular skeleton line;

A second determining unit 43, configured to determine a first type node and a second type node in the first vascular skeleton line according to the number of neighboring nodes of each node, where the first type node is a node with only one neighboring node, and the second type node is a node with at least three neighboring nodes;

A third determining unit 44, configured to determine a segment of the vascular skeleton between the adjacent first-type nodes and the second-type nodes as a vascular branch;

a calculating unit 45, configured to calculate a branch length corresponding to the blood vessel branch;

a fourth determining unit 46, configured to determine a blood vessel branch with the branch length smaller than a preset length threshold as a target branch;

and a masking unit 47, configured to perform masking processing on the target branch, so as to obtain an optimized target area image.

The lung image three-dimensional reconstruction optimization device comprises a training fracture repair model, a target area image, a first blood vessel skeleton line, a second blood vessel skeleton line, a first type node and a second type node, wherein the training fracture repair model is used for carrying out blood vessel repair processing on the target area image in the lung image to obtain a repaired target area image, the number of adjacent nodes of each node in the first blood vessel skeleton line is determined after the first blood vessel skeleton line in the repaired target area image is extracted, the first type node and the second type node in the first blood vessel skeleton line are determined according to the number of the adjacent nodes of each node, the first type node is a node with only one adjacent node, the second type node is a node with at least three adjacent nodes, a blood vessel skeleton line segment between the adjacent first type node and the second type node is determined to be a blood vessel branch, the branch length corresponding to the blood vessel branch is calculated, the blood vessel branch with the branch length smaller than a preset length threshold value is determined to be a target branch, and the target branch is subjected to masking processing to obtain an optimized target area image. Compared with the related art, the three-dimensional reconstruction optimization method for the lung image has the advantages that the blood vessel faults in the lung image are repaired, and the blood vessel branches with the branch lengths smaller than the preset length threshold value in the repaired lung image are subjected to covering treatment, so that the possibility of false positive is reduced, the faults and the false positive problems in the lung image are solved, and the optimization of the lung image is realized.

Further, in a possible implementation manner of this embodiment, as shown in fig. 9, the apparatus further includes a training unit 48, where the training unit 48 includes:

An extraction module 481, configured to invoke a center line extraction algorithm before performing a vascular repair process on the target area image by using the trained fracture repair model, to extract a second vascular skeleton line in the training image, where the training image includes an original label;

The marking module 482 is configured to randomly mark the pixel points on the second blood vessel skeleton line to obtain marked pixel points;

The expansion module 483 is configured to perform morphological expansion operation on the labeled pixel point to obtain a morphological expanded pixel point;

A deleting module 484, configured to delete the morphologically expanded pixel point to obtain a fracture mask corresponding to the training image;

And the training module 485 is configured to train the fracture repair model based on the fracture mask and the original label, and obtain the trained fracture repair model.

Alternatively, the repair unit 41 includes:

The searching module 411 is configured to invoke a connected domain searching algorithm to search all the blood vessel connected domains in the target area image, so as to obtain a blood vessel connected domain group;

a first determining module 412, configured to determine a largest vessel connected domain in the vessel connected domain group as a first vessel main body;

a first calculating module 413, configured to calculate first distances between the other connected domains except the blood vessel main body in the connected domain group and the first blood vessel main body, respectively;

and a repair module 414, configured to sequentially input all other connected domains and the vessel main body into the fracture repair model according to the order of the first distance to perform vessel repair processing, so as to obtain the repaired target area image.

Optionally, the second determining unit 43 includes:

A second determining module 431, configured to determine, with each node in the first vascular skeleton line as a center, a number of target adjacent pixels of the node, where the target adjacent pixels are adjacent pixels located on the first vascular skeleton line;

and a third determining module 432, configured to determine the target number of neighboring pixels of the pixel as the number of neighboring nodes of each node.

Optionally, the computing unit 45 includes:

A fourth determining module 451 for determining the number of pixels composing the blood vessel branch;

a fifth determining module 452, configured to determine the number of pixels as a branch length corresponding to the blood vessel branch.

Optionally, the apparatus further comprises a partition processing unit 49 and a filtering unit 410, wherein the partition processing unit 45 is configured to:

the filtering unit 410 includes:

a first filtering module 4101, configured to perform a first median filtering process on the first area image after obtaining the optimized target area image, to obtain a filtered first area image;

A second filtering module 4102, configured to perform a second median filtering process on the second area image, to obtain a filtered second area image;

A third filtering module 4103, configured to perform a third median filtering process on the optimized target area image to obtain a filtered target area image, where the filter kernel sizes of the first median filtering process, the second median filtering process, and the third median filtering process are different;

A merging module 4104, configured to merge the filtered first area image, the filtered second area image, and the filtered target area image to obtain a target image.

The foregoing explanation of the method embodiment is also applicable to the apparatus of this embodiment, and the principle is the same, and this embodiment is not limited thereto.

According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

Fig. 10 illustrates a schematic block diagram of an example electronic device 500 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 10, the apparatus 500 includes a computing unit 501 that can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 502 or a computer program loaded from a storage unit 508 into a RAM (Random Access Memory ) 503. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The computing unit 501, ROM 502, and RAM 503 are connected to each other by a bus 504. An I/O (Input/Output) interface 505 is also connected to bus 504.

The various components in the device 500 are connected to an I/O interface 505, including an input unit 506, e.g., a keyboard, a mouse, etc., an output unit 507, e.g., various types of displays, speakers, etc., a storage unit 508, e.g., a magnetic disk, optical disk, etc., and a communication unit 509, e.g., a network card, modem, wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.

The computing unit 501 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 501 include, but are not limited to, a CPU (Central Processing Unit ), a GPU (Graphic Processing Units, graphics processing unit), various specialized AI (ARTIFICIAL INTELLIGENCE ) computing chips, various computing units running machine learning model algorithms, a DSP (DIGITAL SIGNAL Processor ), and any suitable Processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as a lung image three-dimensional reconstruction optimization method. For example, in some embodiments, the lung image three-dimensional reconstruction optimization method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as the storage unit 508. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 500 via the ROM 502 and/or the communication unit 509. When the computer program is loaded into RAM 503 and executed by computing unit 501, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the aforementioned lung image three-dimensional reconstruction optimization method by any other suitable means (e.g. by means of firmware).

Various implementations of the systems and techniques described here above can be implemented in digital electronic circuitry, integrated Circuit System, FPGA (Field Programmable GATE ARRAY ), ASIC (Application-SPECIFIC INTEGRATED Circuit, application-specific integrated Circuit), ASSP (Application SPECIFIC STANDARD Product, application-specific standard Product), SOC (System On Chip ), CPLD (Complex Programmable Logic Device, complex programmable logic device), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include being implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be a special or general purpose programmable processor, operable to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, RAM, ROM, EPROM (ELECTRICALLY PROGRAMMABLE READ-Only-Memory, erasable programmable read-Only Memory) or flash Memory, an optical fiber, a CD-ROM (Compact Disc Read-Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid CRYSTAL DISPLAY) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user, for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback), and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include LAN (Local Area Network ), WAN (Wide Area Network, wide area network), the Internet, and blockchain networks.

The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service ("Virtual PRIVATE SERVER" or simply "VPS") are overcome. The server may also be a server of a distributed system or a server that incorporates a blockchain.

It should be noted that, artificial intelligence is a subject of studying a certain thought process and intelligent behavior (such as learning, reasoning, thinking, planning, etc.) of a computer to simulate a person, and has a technology at both hardware and software level. The artificial intelligence hardware technology generally comprises technologies such as a sensor, a special artificial intelligence chip, cloud computing, distributed storage, big data processing and the like, and the artificial intelligence software technology mainly comprises a computer vision technology, a voice recognition technology, a natural language processing technology, a machine learning/deep learning technology, a big data processing technology, a knowledge graph technology and the like.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel, sequentially, or in a different order, provided that the desired results of the disclosed aspects are achieved, and are not limited herein.

The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.

Claims

1. A method for optimizing three-dimensional reconstruction of lung images, comprising:

The trained fracture repair model is used to perform vascular repair processing on the target area image in the lung image to obtain a repaired target area image;

After extracting the first blood vessel skeleton line in the restored target area image, determining the number of adjacent nodes of each node in the first blood vessel skeleton line;

Determine, according to the number of adjacent nodes of each node, a first type of node and a second type of node in the first blood vessel skeleton line, wherein the first type of node is a node having only one adjacent node, and the second type of node is a node having at least three adjacent nodes;

Determine the blood vessel skeleton line segments between adjacent first-type nodes and second-type nodes as blood vessel branches;

Calculating the branch length corresponding to the blood vessel branch; and determining the blood vessel branch whose branch length is less than a preset length threshold as a target branch;

Performing masking processing on the target branch to obtain an optimized target area image;

The step of using the trained fracture repair model to perform vascular repair processing on the target region image in the lung image to obtain the repaired target region image includes: calling a connected domain search algorithm to search for all vascular connected domains in the target region image to obtain a vascular connected domain group;

Determine the largest vascular connection domain in the vascular connection domain group as the first vascular body;

respectively calculating first distances between other connected domains in the connected domain group except the blood vessel main body and the first blood vessel main body;

All other connected domains and the blood vessel body are sequentially input into the fracture repair model according to the order of magnitude of the first distances to perform blood vessel repair processing, so as to obtain the repaired target area image.

2. The method according to claim 1, characterized in that before using the trained fracture repair model to perform blood vessel repair processing on the target area image, the method further comprises:

Calling a centerline extraction algorithm to extract a second blood vessel skeleton line in a training image, wherein the training image includes an original label;

Randomly marking pixel points on the second blood vessel skeleton line to obtain marked pixel points;

Performing a morphological dilation operation on the marked pixel points to obtain morphologically dilated pixel points;

Deleting the pixel points after the morphological expansion to obtain a fracture mask corresponding to the training image;

The fracture repair model is trained based on the fracture mask and the original label to obtain the trained fracture repair model.

3. The method according to claim 1, wherein determining the number of adjacent nodes of each node in the first blood vessel skeleton line comprises:

Taking each node in the first blood vessel skeleton line as the center, determining the number of target adjacent pixel points of the node, wherein the target adjacent pixel points are adjacent pixel points located on the first blood vessel skeleton line;

The target number of adjacent pixels of the pixel is determined as the number of adjacent nodes of each node.

4. The method according to claim 1, characterized in that the calculating the branch length corresponding to the blood vessel branch comprises:

Determining the number of pixels constituting the blood vessel branch;

The number of pixels is determined as the branch length corresponding to the blood vessel branch.

5. The method according to claim 1, characterized in that before using the trained fracture repair model to perform vascular repair processing on the target area image in the lung image, the method further comprises:

According to the characteristics of the pulmonary blood vessels in the pulmonary image, the pulmonary image is partitioned to obtain a first region image, a second region image and the target region image;

After obtaining the optimized target area image, the method further includes:

Performing a first median filtering process on the first region image to obtain a filtered first region image;

Performing a second median filtering process on the second region image to obtain a filtered second region image;

Performing a third median filtering process on the optimized target area image to obtain a filtered target area image; wherein the filter kernel sizes of the first median filtering process, the second median filtering process and the third median filtering process are different;

The filtered first region image, the filtered second region image and the filtered target region image are merged to obtain a target image.

6. A lung image three-dimensional reconstruction optimization device, characterized by comprising:

A repair unit, used for performing blood vessel repair processing on a target area image in a lung image by using a trained fracture repair model to obtain a repaired target area image;

A first determining unit is used to determine the number of adjacent nodes of each node in the first blood vessel skeleton line after extracting the first blood vessel skeleton line in the restored target area image;

A second determining unit is used to determine a first type of node and a second type of node in the first blood vessel skeleton line according to the number of adjacent nodes of each node, wherein the first type of node is a node having only one adjacent node, and the second type of node is a node having at least three adjacent nodes;

A third determining unit is used to determine the blood vessel skeleton line segments between adjacent first-type nodes and second-type nodes as blood vessel branches;

A calculation unit, used to calculate the branch length corresponding to the blood vessel branch;

a fourth determining unit, configured to determine the blood vessel branch whose branch length is less than a preset length threshold as a target branch;

A masking unit, used for performing masking processing on the target branch to obtain an optimized target area image;

The repair unit includes: a search module, which is used to call a connected domain search algorithm to search for all blood vessel connected domains in the target area image to obtain a blood vessel connected domain group;

A first determination module, configured to determine the largest vascular connection domain in the vascular connection domain group as a first vascular body;

A first calculation module, used for respectively calculating first distances between other connected domains in the connected domain group except the blood vessel body and the first blood vessel body;

The repair module is used to input all other connected domains and the blood vessel body into the fracture repair model in order of the first distance to perform blood vessel repair processing and obtain the repaired target area image.

7. An electronic device, comprising:

at least one processor; and

a memory communicatively connected to the at least one processor; wherein,

The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 5.

9. A computer program product, characterized in that it comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.