WO2024169353A1

WO2024169353A1 - Intelligent delineation method for cone-beam ct image target volume based on regional narrow-band propagation

Info

Publication number: WO2024169353A1
Application number: PCT/CN2023/137193
Authority: WO
Inventors: 赵汉卿; 梁晓坤; 谢耀钦
Original assignee: 中国科学院深圳先进技术研究院
Priority date: 2023-02-16
Filing date: 2023-12-07
Publication date: 2024-08-22
Also published as: CN116071345A

Abstract

Disclosed in the present invention is an intelligent delineation method for a cone-beam CT image target volume based on regional narrow-band propagation. The method comprises: acquiring a planned CT image, and drawing the contour of a region of interest around the contour of a target organ as a narrow-band image of the planned CT image; and inputting the narrow-band image of the planned CT image into a pre-trained deep learning model to obtain a predicted narrow-band result in a corresponding cone-beam CT image, wherein the deep learning model is pre-trained with the optimization goal of maximizing the similarity between a predicted narrow-band image in the cone-beam CT image and the narrow-band image of the planned CT image, and a deformation vector field is obtained by means of pre-training. According to the present invention, the contour curved surface of the region of interest around the target organ can be automatically propagated, and the boundary of the target organ is focused, thereby improving the effect of adaptive radiotherapy.

Description

An intelligent delineation method for target area in cone-beam CT images based on regional narrow-band propagation

Technical Field

The present invention relates to the technical field of medical image processing, and more specifically, to a cone beam CT image target area intelligent delineation method based on regional narrowband propagation.

Background Art

Adaptive radiotherapy planning and online treatment planning for prostate cancer can reduce the additional dose to normal tissues, but they rely heavily on the accuracy of the target contours on daily cone-beam CT. The contours can be propagated from the planning CT to the cone-beam CT. Previous contour propagation methods were mainly based on two images with the same imaging mode, that is, from CT to CT. For adaptive radiotherapy, an essential step to achieve this adaptive replanning is the segmentation of the cone-beam CT images.

The accuracy of image registration and contour propagation can be adversely affected by image content in one image that has no correspondence with another, such as the presence or absence of intestinal balloon filling on a CT image. In addition, various image artifacts may disrupt the global consistency between planning CT and cone-beam CT. For example, metal artifacts can vary significantly from one CT scan to another, depending on the scanning method and reconstruction algorithm. Anatomical changes such as tumor shrinkage can occur during treatment. Therefore, the use of global registration methods may affect the accuracy of contour propagation, as the registration may be unnecessarily affected by image content far from the region of interest, which would otherwise be irrelevant to the contour mapping process. Typically, contour mapping is a regional problem, and global deformable image registration of CT images is neither necessary nor accurate. With the growing interest in online adaptive replanning of cone-beam CT, there is a need to introduce accurate contour propagation between planning CT and cone-beam CT images.

In the prior art, researchers proposed an automatic contour propagation method for online adaptive planning guided by orbital CT. They reported an average Dice coefficient of 0.84 for the seminal vesicle and prostate. However, the execution time of contour propagation of this method exceeded 1 minute.

Target contour mapping is an important factor affecting adaptive radiotherapy, and there are already some solutions. For example, patent application CN201810911894.8 proposes an adaptive radiotherapy structure automatic delineation method. This method preprocesses the patient's medical images; uses a first-level deep neural network to classify and locate the human organ structure in the patient's image; and uses a second-level deep neural network to segment and delineate the human organ structure based on the classification and positioning results. For another example, patent application CN202011437517.9 discloses a medical image segmentation model training method, which includes: inputting the medical image to be segmented into an initial medical image segmentation model to obtain a first image; receiving a modification trajectory of the first image by the user; using the first image and the modification trajectory as training samples, and using the standard medical segmentation image corresponding to the first image as a label, training the initial medical image segmentation model, and obtaining a target medical image segmentation model.

Technical issues

After analysis, the existing technology mainly has the following defects:

1) The most common image-guided radiotherapy technique currently used is prostate fiducial markers seen on portal x-ray and cone-beam CT imaging. This approach does not visualize the two most important organs at risk in pelvic radiotherapy: the bladder and rectum. Even if the prostate is well positioned, the filling of these two organs cannot be verified, resulting in planned dosimetry that may be very different from the delivered treatment, which is prone to significant impact on urinary and rectal toxicity.

2) Cone beam CT imaging provides the ability to visualize the prostate, bladder, and rectum, and this approach does not require the insertion of fiducials into any part of the patient's prostate prior to treatment planning. In this sense, it is less invasive, simpler, and more convenient for the patient. However, a disadvantage of cone beam CT imaging is that the images provided do not always allow for correct visualization of the prostate. If a significant deviation from the planned treatment is detected before treatment, the patient must be repositioned, for example after emptying the rectum, or the plan must be adjusted in almost real time. In order to do this, it is necessary to automatically delineate the prostate contour in cone beam CT. Existing methods require a certain treatment delay because it is not possible to perform this task manually for every patient every day.

Technical Solutions

The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a method for intelligent delineation of target areas in cone beam CT images based on regional narrowband propagation. The method comprises the following steps:

Acquire a planning CT image and outline the region of interest around the outline of the target organ as a narrow-band image of the planning CT;

Inputting the planned CT narrowband image into a pre-trained deep learning model to obtain a predicted narrowband result in a corresponding cone-beam CT image;

The deep learning model is pre-trained with maximizing the similarity between the predicted narrow band in the cone beam CT image and the narrow band image of the planned CT as the optimization goal, and the deformation vector field is obtained through pre-training.

Beneficial Effects

Compared with the prior art, the advantages of the present invention are that, aiming at the overall goal of the clinical implementation of adaptive radiotherapy, it provides a method for intelligent delineation of cone-beam CT image target areas based on regional narrowband propagation, integrates deep learning models into narrowband-based contour propagation, and uses unsupervised deep learning methods to propagate narrowbands from planning CT to cone-beam CT. The present invention can automatically propagate contour surfaces of regions of interest around target organs such as the prostate, focus on their boundaries, improve segmentation accuracy and reliability, and reduce the workload and time of manually annotating deep learning model training images, effectively improving adaptive radiotherapy and achieving better treatment effects.

Further features and advantages of the present invention will become apparent from the following detailed description of exemplary embodiments of the present invention with reference to the attached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

FIG1 is a flow chart of a method for intelligent delineation of a cone-beam CT image target region based on regional narrow-band propagation according to an embodiment of the present invention;

FIG2 is a schematic diagram of a process of a method for intelligent delineation of a cone-beam CT image target region based on regional narrow-band propagation according to an embodiment of the present invention;

FIG3 is a schematic diagram showing the effect of a deep unsupervised learning method for an example patient according to an embodiment of the present invention.

Embodiments of the present invention

Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present invention unless otherwise specifically stated.

The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

In all examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

The present invention first constructs a narrow band of the planned CT target area contour, and then enters it and the cone beam CT image into the deep learning model to learn the deformation vector field (or vector deformation field), and propagates the segmented contour from the planned CT to the cone beam CT through the deformation vector field, thereby realizing the automatic propagation of the contour. In the following, the prostate is described as the target organ, but it should be understood that it is also applicable to various other types of organs.

As shown in FIG1 , the provided cone beam CT image target area intelligent delineation method based on regional narrowband propagation includes the following steps:

Step S10, using an outward expansion method, constructing a contour extending to the planned CT image to depict the region of interest around the target organ as a narrow-band image of the planned CT.

For example, a narrow band extending to a 1.5 cm area around the manually drawn outline on the planning CT was constructed to exclude the volume around the prostate.

Step 20, using the set loss function to pre-train the deep learning model to map the narrow band, wherein the combination of the narrow band image from the planning CT and the cone beam CT image is used as the model input, and the deformation vector field is used as the model output.

The deep learning model can use various types of neural networks, such as convolutional neural networks. In one embodiment, as shown in FIG2 , the deep learning model uses a three-dimensional U-shaped network based on attention. The network is a U-shaped structure composed of an encoder and a decoder, and the feature map extracted by the encoder and the feature map corresponding to the decoder are spliced through an attention gate. In FIG2 , the attention mechanism is introduced into the U-shaped network, a jump connection is performed, and a sigmoid activation function is selected for normalization.

For example, the process of mapping narrowband through deep unsupervised learning model includes:

Step S21: The narrow-band image from the planning CT and cone beam CT images Merge input into the attention-based 3D U-network middle.

Step S22, the attention-based three-dimensional U-shaped network outputs a deformable vector field , maximizing cone-beam CT prediction and narrow-band planning CT For example, the loss function is expressed as:

(1)

Among them, the loss function Contains two items: Differences between narrow bands in penalized planning CT and predicted narrow bands in cone-beam CT. Penalize spatial variation, is the regularization parameter.

In one embodiment, It can be expressed as:

(2)

in, represents the number of voxels in the planning CT/cone beam CT image, x , y and z are the three directions of the volume, X, Y, Z are the upper limits of integration in the three directions, and represent the number of voxels in the corresponding directions. And the global smoothness constraint generates a deformation vector field while minimizing the image similarity.

Still in conjunction with FIG. 2 , the specific process of the provided cone beam CT image target area intelligent delineation method based on regional narrowband propagation includes:

Step S1: Perform contour propagation from planning CT to cone-beam CT.

Step S2: The cone beam CT image and the narrow band in the planning CT are passed into the encoder and enter the 3-time downsampling process. Each downsampling module contains a convolution layer with a convolution kernel size of 3×3×3, batch normalization, linear rectified unit activation function and a 2×2×2 maximum pooling layer.

Step S3: After passing through the encoder, it enters the upsampling process three times. Each upsampling module contains a 2×2×2 upsampling layer, a convolution layer with a convolution kernel size of 3×3×3, batch normalization, and a linear rectified unit activation function.

Step S4: Concatenate the feature map of the decoder part and its corresponding feature map of the encoder part through the attention gate.

Step S5: Output the vector deformation field through a 1×1×1 convolutional layer.

Step S6: Apply the vector deformation field to the narrow band in the planning CT through the space transformation module.

Step S7: Use similarity measurement and smoothness measurement to evaluate the narrowband propagation effect, and forward propagate the evaluation results to the feature map.

Step S30, using the deformation vector field obtained by the pre-trained deep learning model to realize narrowband propagation, and evaluating the narrowband propagation effect.

After pre-training the deep learning model, it can be used for narrowband propagation of real images. To evaluate the effect of narrowband propagation, 251 anonymous cone-beam CT data were analyzed. Each cone-beam CT has a corresponding planning CT from the same patient. Among them, 180/12/59 case studies were used for training/validation/testing. The test data was divided into Group 1 and Group 2. Group 1 has 50 cone-beam CTs, which were contoured by the same doctor. Group 2 consists of 9 cone-beam CTs, each of which was contoured independently by 4 doctors. The four manual contours can be merged into a consistent contour using a label fusion method. For Group 2, the contour metrics are calculated by comparing the model predictions with the single manual contour and the consistent contour.

To evaluate the quality of prostate contour propagation, the contour propagation results obtained by the present invention were compared with the contours manually outlined by doctors in cone-beam CT images. Several indicators were used to evaluate the consistency of the contours generated by deep unsupervised learning with the reference contours (clinician and/or consensus). To evaluate the accuracy of the propagated contours, the Dice similarity coefficient (DSC), Hausdorff distance (HD95), and Matthews correlation coefficient (MCC) were calculated. To analyze the displacement of the prostate between the deep unsupervised learning contour and the gold standard contour, the distance of the center of mass (COM) was calculated. See Table 1 for a summary of the target contour change assessment.

Table 1 Evaluation of target contour changes

The experimental results are shown in Figure 3. The first column shows two low-performance examples with a Dice similarity coefficient <0.78. The second column demonstrates two medium-performance examples with a Dice similarity coefficient ≈0.83. The last column demonstrates two high-performance examples with a Dice similarity coefficient >0.88. Rows 1-3 show the images in the axial view, coronal view, and sagittal view, respectively. Different colors are used to represent the contours automatically marked by the present invention and the reference contours manually marked in cone beam CT.

In Figure 3, high-density fiducial markers were implanted in prostate cancer patients (left column, first row), resulting in anatomical variability and severe image artifacts. These affect the performance of the results (Dice similarity coefficient ≈ 0.77). For moderate performance, the Dice similarity coefficient is 0.83, and larger errors occur at the prostate boundary. As can be seen from Figure 3, the manual reference prostate contour is not smooth, which may deviate from the true reference and hinder the accuracy of the assessment. For high performance, the Dice similarity coefficient is 0.88. In all examples, the automatically marked contours proposed by the present invention and the manually marked reference contours in cone beam CT have good consistency, and the contours obtained by the present invention are smoother.

In summary, compared with the prior art, the present invention has the following advantages:

1) Automatically propagates the contour surface of the region of interest around the target organ, focusing on its borders, enabling users to generate more reliable and precise segmentations.

2) In the process of high-precision contour propagation of cone-beam CT-guided adaptive radiotherapy achieved by the regional narrowband algorithm based on deep unsupervised learning, the present invention improves the efficiency of contour propagation and reduces the workload and time of manually annotating deep learning model training images.

3) The problem of anatomical changes in the region of interest in adaptive radiotherapy is solved. The characteristic of less interference in the narrow-band area is used to solve the problem of tumor shrinkage. Propagating the narrow band from the planning CT to the cone-beam CT can effectively improve adaptive radiotherapy and achieve better treatment effects.

The present invention may be a system, a method and/or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof. The computer readable storage medium used here is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (for example, a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing/processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and/or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and/or edge servers. The network adapter card or network interface in each computing/processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing/processing device.

The computer program instructions for performing the operation of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, Python, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect through the Internet). In some embodiments, by using the state information of the computer-readable program instructions to personalize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit may execute the computer-readable program instructions, thereby implementing various aspects of the present invention.

Various aspects of the present invention are described herein with reference to the flowcharts and/or block diagrams of the methods, devices (systems) and computer program products according to embodiments of the present invention. It should be understood that each box of the flowchart and/or block diagram and the combination of the boxes in the flowchart and/or block diagram can be implemented by computer-readable program instructions.

These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions/actions specified in one or more boxes in the flowchart and/or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and/or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions/actions specified in one or more boxes in the flowchart and/or block diagram.

Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions/actions specified in one or more boxes in the flowchart and/or block diagram.

The flowchart and block diagram in the accompanying drawings show the possible architecture, functions and operations of the system, method and computer program product according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and/or flowchart, and the combination of the boxes in the block diagram and/or flowchart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that it is equivalent to implement it by hardware, implement it by software, and implement it by combining software and hardware.

Embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the marketplace, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the present invention is defined by the appended claims.

Claims

A method for intelligent delineation of a cone beam CT image target area based on regional narrowband propagation comprises the following steps:

Acquire a planning CT image and outline the region of interest around the outline of the target organ as a narrow-band image of the planning CT;

Inputting the planned CT narrowband image into a pre-trained deep learning model to obtain a predicted narrowband result in a corresponding cone-beam CT image;

The deep learning model is pre-trained with maximizing the similarity between the predicted narrow band in the cone beam CT image and the narrow band image of the planned CT as the optimization goal, and the deformation vector field is obtained through pre-training.
The method according to claim 1 is characterized in that, during the deep learning model pre-training process, a narrow-band image from a planning CT is used and cone beam CT images The combination of is taken as input, the deformation vector field is taken as output, and the loss function is set as:

in, The difference between the narrow-band image of the penalized planning CT and the predicted narrow-band in the cone-beam CT image, is the spatial variation penalty term, is the regularization parameter, are the parameters of the deep learning model, is the deformation vector field.
The method according to claim 2, characterized in that the spatial variation penalty term It is expressed as:

in, represents the number of voxels in the planning CT image and cone beam CT image, x , y and z are the three directions of the voxel, and X, Y and Z are the upper limits of integration in the three directions.
The method according to claim 1 is characterized in that the deep learning model is a three-dimensional U-shaped network, including an encoder and a decoder, and the feature map extracted by the encoder and the feature map corresponding to the decoder are spliced through an attention gate, and then the deformation vector field is output through a 1×1×1 convolutional layer.
The method according to claim 4 is characterized in that the encoder and the decoder each include a convolution layer and a batch normalization layer, and a sigmoid activation function is selected for normalization processing.
The method according to claim 1, characterized in that the target organ is the prostate.
The method according to claim 6, characterized in that a narrow band of 1.5 cm area around the contour is depicted on the planning CT image.
A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 7.
A computer device comprises a memory and a processor, wherein a computer program that can be run on the processor is stored on the memory, and wherein the processor implements the steps of any one of the methods of claims 1 to 7 when executing the computer program.