KR20250056757A - Prognosis prediction system and method based on 3d deep learning using preoperative mri - Google Patents

Abstract

The present invention relates to a three-dimensional deep learning-based prognosis prediction system and method using preoperative MRI. According to the present invention, a three-dimensional deep learning-based prognosis prediction system using preoperative MRI includes an input unit which receives a plurality of three-dimensional MRI images taken of a brain region of a patient with glioma before surgery; a prediction unit which applies the plurality of three-dimensional MRI images to a pre-learned prognosis prediction model to perform spatial feature analysis of the images to predict a postoperative prognosis index of the patient with glioma; and an output unit which outputs and provides the postoperative prognosis index prediction result to a user terminal.
According to the present invention, by predicting the prognostic indicators after surgery based on a deep learning model using 3D MRI data of a patient with glioma before surgery, it is possible to help medical staff determine a treatment plan for the patient.

Description

{PROGNOSIS PREDICTION SYSTEM AND METHOD BASED ON 3D DEEP LEARNING USING PREOPERATIVE MRI}

The present invention relates to a three-dimensional deep learning-based prognosis prediction system and method using preoperative MRI, and more specifically, to a deep learning-based prognosis prediction system and method capable of predicting a postoperative prognosis score by analyzing preoperative three-dimensional whole brain MRI images of patients with glioma through deep learning.

Machine learning is a computer algorithm that can automatically learn from data without explicit programming. Machine learning has overcome the limitations of existing technologies and has been a breakthrough in various fields such as image analysis, bioinformatics, and natural language processing. In addition, several studies have proven that it can be useful in diagnosing musculoskeletal diseases and predicting the prognosis of diseases.

Deep learning technology is one of the advanced machine learning approaches. In particular, it builds an artificial neural network with a structure and function similar to the human brain by using multiple hidden layers. Deep learning technology can outperform existing machine learning technology and can learn unstructured perceptual data such as images and language.

Although technologies have been developed to interpret or predict brain diseases by analyzing brain MRI images taken from patients, there are not many technologies to predict the prognosis or survival rate of patients with brain diseases after surgery.

Previous studies have attempted to develop prognostic prediction models using deep learning, but the preprocessing process is complex, which limits its use when using external data, and since only segmented tumor areas are used, there are limitations in that spatial information in areas other than the tumor cannot be utilized.

Even for models that do not require such preprocessing or tumor segmentation, the use of spatial features in the height direction was limited because the models only used 2D slices of MRI.

In addition, there is currently no effective indicator for predicting the prognosis of patients with glioma in clinical practice other than the clinical index (Karnofsky performance scale; KPS) that evaluates the patient's function. In particular, in order to obtain genetic mutation information, which is an effective prognostic factor, it was necessary to obtain actual glioma tissue through an invasive surgical procedure.

Therefore, a technique is required that can accurately determine a patient's prognosis after surgery using only deep learning analysis of the patient's brain MRI images taken before surgery.

The technology underlying the present invention is disclosed in Korean Patent Publication No. 10-2021-0065768 (published on June 4, 2021).

The purpose of the present invention is to provide a 3D deep learning-based prognosis prediction system and method using preoperative MRI, which can predict the postoperative prognosis of a patient with glioma by analyzing the preoperative 3D brain MRI image based on deep learning.

The present invention relates to a 3D deep learning-based prognosis prediction system using preoperative MRI, comprising: an input unit for receiving a plurality of 3D MRI images taken of a brain region of a patient with glioma before surgery; a prediction unit for performing spatial feature analysis of the images by applying the plurality of 3D MRI images to a pre-learned prognosis prediction model to predict a postoperative prognosis index of the patient with glioma; and an output unit for outputting and providing the postoperative prognosis index prediction result to a user terminal.

In addition, the plurality of three-dimensional MRI images may include a T1w (T1 weight) image, a T2w (T2 weight) image, a T2 FLAIR image, and a CE-T1w (Contrast-enhanced T1 weight) image captured for a brain region of the patient with glioma.

Additionally, the above prognosis prediction model can be implemented including a CNN-based SE-ResNext50 model.

In addition, the prediction unit can perform preprocessing by sequentially performing skull region removal, brightness normalization, and resampling from an input 3D MRI image, and apply the preprocessed 3D MRI image to the prognosis prediction model.

In addition, the input unit additionally receives variable parameters including clinical information and molecular genetic information on the glioma patient, and the prediction unit can predict the post-surgical prognostic index of the glioma patient by applying the plurality of 3D MRI images and the variable parameters to a pre-learned prognostic prediction model.

In addition, the prognosis prediction model may include a first feature extraction module that receives the plurality of 3D MRI images as input and extracts a spatial feature vector, a second feature extraction module that receives variable parameters including clinical information and molecular genetic information as input and extracts a clinical genetic feature vector, and a result output module that receives feature vectors output from the first and second feature extraction modules and outputs a post-surgical prognosis index of a patient with glioma.

And, the present invention relates to a deep learning-based prognosis prediction method performed by a three-dimensional deep learning-based prognosis prediction system using preoperative MRI, comprising the steps of: receiving a plurality of three-dimensional MRI images taken of a brain region of a patient with glioma before surgery; performing spatial feature analysis of the images by applying the plurality of three-dimensional MRI images to a pre-learned prognosis prediction model to predict a post-operative prognosis index of the patient with glioma; and outputting and providing the post-operative prognosis index prediction result to a user terminal.

In addition, the input step additionally inputs variable parameters including clinical information and molecular genetic information on the glioma patient, and the predicting step applies the plurality of 3D MRI images and the variable parameters to a pre-learned prognosis prediction model to predict the post-surgical prognosis index of the glioma patient.

According to the present invention, it is possible to efficiently obtain prognostic predictors using only preoperative MRI image information, thereby helping to determine a treatment plan before surgery.

FIG. 1 is a diagram showing the configuration of a three-dimensional deep learning-based prognosis prediction system using preoperative MRI according to an embodiment of the present invention.
Figure 2 is a drawing detailing the configuration of the three-dimensional deep learning-based prognosis prediction system illustrated in Figure 1.
Figure 3 is a diagram exemplifying a three-dimensional deep learning-based prognosis prediction operation using the system of Figure 2.
FIG. 4 is a diagram exemplarily showing a CNN model applicable to an embodiment of the present invention.
Figure 5 is a diagram showing an example of a modified prognosis prediction model.
Figure 6 is a drawing explaining a three-dimensional deep learning-based prognosis prediction method using the system of Figure 1.

Hereinafter, with reference to the attached drawings, embodiments of the present invention will be described in detail so that those with ordinary skill in the art can easily implement the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In addition, in order to clearly describe the present invention in the drawings, parts that are not related to the description are omitted, and similar parts are assigned similar drawing reference numerals throughout the specification.

Throughout the specification, when a part is said to be "connected" to another part, this includes not only the case where it is "directly connected" but also the case where it is "electrically connected" with another element in between. Also, when a part is said to "include" a component, this does not mean that it excludes other components, but rather that it may include other components, unless otherwise stated.

FIG. 1 is a diagram showing the configuration of a three-dimensional deep learning-based prognosis prediction system using preoperative MRI according to an embodiment of the present invention.

As shown in Fig. 1, a 3D deep learning-based prognosis prediction system (100) using preoperative MRI can be connected to a user terminal (200) through a wired, wireless, or wired/wireless combination network to transmit and receive mutual information. The wireless network can include at least one of RF, WLAN, Wi-Fi, and Bluetooth methods, and various known wireless network methods can be utilized.

The 3D deep learning-based prognosis prediction system (100) can be implemented as an on/offline platform such as a web server or app server that provides a post-surgical prognosis prediction service based on deep learning of 3D MRI images of glioma patients before surgery to a network-connected user terminal (200), or can be implemented in the form of an application program, application, etc. on a user terminal, etc.

Therefore, the proposed 3D deep learning-based prognosis prediction system (100) can provide a tube result prediction service platform implemented as an app or web to a network-connected user terminal (200). The service platform can be an application program running in an app or web environment.

Here, of course, the 3D deep learning-based prognosis prediction system (100) may be a server itself that predicts the prognosis after surgery from the 3D MRI images of a patient with glioma before surgery, or it may correspond to an application program implemented in a user terminal (PC, desktop, smartphone, laptop, pad, etc.) of the medical staff. Each terminal can receive related services by being connected to the 3D deep learning-based prognosis prediction system (100) through a network while the related application program is running.

In an embodiment of the present invention, a user terminal (200) may include a device that can connect to a wired or wireless network and exchange information, such as a PC, a desktop, a smart phone, a tablet, a notebook, etc. The user terminal (200) may include a medical staff terminal, a general user terminal, etc.

This 3D deep learning-based prognosis prediction system (100) receives 3D MRI data collected in relation to a glioma patient from a user terminal (200), applies the input data to a deep learning-based pre-learned prognosis prediction algorithm, and can predict a prognosis index before the patient undergoes surgery. This prognosis prediction algorithm can be continuously learned and updated based on past big data related to glioma patients.

In the case of an embodiment of the present invention, by utilizing 3D MRI data of a patient with glioma before surgery and predicting the prognosis after surgery based on a deep learning model, it is possible to help medical staff determine a treatment plan for the patient.

FIG. 2 is a drawing detailing the configuration of the 3D deep learning-based prognosis prediction system illustrated in FIG. 1, and FIG. 3 is a drawing exemplarily showing the 3D deep learning-based prognosis prediction operation using the system of FIG. 2.

As shown in Fig. 2, a 3D deep learning-based prognosis prediction system (100) according to an embodiment of the present invention includes an input unit (110), a prediction unit (120), and an output unit (130). Here, the operation of each unit (110 to 130) and the data flow between each unit can be controlled by a control unit (not shown).

This 3D deep learning-based prognosis prediction system (100) may be implemented as a computer device physically configured to include a processor, memory, a user interface input/output device, a storage device, a network input/output unit, etc., or may be implemented as an application program running on a computer device or a user terminal.

The input unit (110) receives multiple 3D MRI images taken of the brain region of a patient with glioma before surgery. Here, the patient with glioma may include a group of patients with brain tumors, and may represent, for example, adult patients with glioma.

In an embodiment of the present invention, tumor biological characteristics of glioma can be more reliably identified by utilizing an appropriate MRI sequence.

In this embodiment of the present invention, the plurality of 3D MRI images used as inputs for deep learning may include a T1w (T1 weight) image, a T2w (T2 weight) image, a T2 FLAIR image, and a CE T1w (Contrast-enhanced T1 weight) image of a brain region of a patient with glioma.

Each 3D MRI image can be acquired from MRI equipment and has different strengths in feature acquisition. For example, the CE T1w image can show the enhancing tumor area, and the FLAIR image can show the nonenhancing tumor and peritumoral edema area. Since these areas are also infiltrated by microscopic tumor cells, even if the enhancing tumor is completely removed through surgery, recurrence is possible here. Therefore, it is desirable to include this area in the deep learning analysis as well.

When MRI images with different features are utilized in deep learning training, more reliable prognosis prediction becomes possible. Figure 3 shows two types of 3D MRI images as examples.

Next, the prediction unit (120) can predict a deep-learning based prognostic index (DPI) for a patient with glioma by applying multiple 3D MRI images to a pre-learned prognostic prediction model. The derived prognostic index can be provided in the form of a score in a set range, an index value, a status classification result, a survival probability, etc.

At this time, the prediction unit (120) sequentially processes skull region removal, brightness normalization, and resampling from the input 3D MRI image to perform image preprocessing, and can apply the 3D MRI image for which preprocessing is completed to the prognosis prediction model. When preprocessing is performed, feature extraction efficiency can be increased by removing unnecessary areas and making the data lighter.

In an embodiment of the present invention, the prediction unit (120) analyzes spatial features of multiple 3D MRI images acquired before surgery of a patient using a prognosis prediction model capable of 3D image learning based on deep learning, and can predict the prognosis index (DPI) of the patient after surgery based on the analyzed spatial features.

According to this, it is possible to extract features including not only the features of the extratumoral area but also the spatial features in the height direction, thereby improving the prognosis prediction performance.

In an embodiment of the present invention, a prognosis prediction model can be implemented by including a 3D CNN model capable of 3D spatial feature analysis.

Specifically, the prognosis prediction model can be implemented by including a CNN-based model, and can be implemented by CNN models (e.g., ResNet-50, DenseNet-50, EfficientNet-B0, SE-ResNext50), which are feature extractors capable of efficient spatial feature extraction for whole brain MRI.

FIG. 4 is a diagram exemplarily showing a CNN model applicable to an embodiment of the present invention, and exemplarily showing the structures of a ResNet model and a SE-ResNext model.

In this embodiment of the present invention, the prognosis prediction model can be implemented using the SE-ResNext50 model, which is a squeeze-excitation network that exhibits the best performance among the listed ResNet-50, DenseNet-50, EfficientNet-B0, and SE-ResNext50 models.

Such prognostic prediction models can be learned in advance based on big data related to preoperative 3D MRI data and survival analysis data according to the time elapsed after surgery for patients with past adult diffuse gliomas. At this time, the Cox proportional hazards model, a semi-parametric method used in survival analysis, can be utilized.

Of course, when utilizing such data, the prognosis prediction model can provide a prognosis index after a set period of time (e.g., 6 months, 1 year, 3 years) after surgery, and can provide the change in the prognostic index over time as a prediction result.

In this way, in the case of an embodiment of the present invention, a 3D CNN model can be used to extract prognostic spatial features from a whole brain MRI in a manner that is as generalizable as possible.

The output unit (130) can output and provide the prognostic index (DPI) result predicted by the prediction unit (120) to the user terminal (200). At this time, the output unit (130) can provide the 3D MRI image of the patient used as input data together with the prediction result. The prognostic prediction result can help the medical staff decide on the treatment plan for the patient before surgery.

In an embodiment of the present invention, the prediction unit (120) can perform prognosis prediction by additionally utilizing variable parameters including clinical information and molecular genetic information of the patient in addition to multiple 3D MRI data acquired from the patient.

To this end, the input unit (110) can additionally receive variable parameters including the patient's clinical feature information and molecular genetic feature information and transmit them to the prediction unit (120).

The prediction unit (120) can predict the post-operative prognosis index of a glioma patient by applying multiple 3D MRI images and variable parameters to a pre-learned prognosis prediction model. In this way, if clinical data and molecular genetic data of the patient are additionally considered in addition to the 3D MRI images acquired from the glioma patient before surgery, the prognosis prediction performance can be further improved. Here, the clinical data can include age, gender, medical history, etc., and the molecular genetic data can include genetic mutation information (IDH mutation).

In this case, the prognostic prediction model can be implemented by including multiple learning modules. Fig. 5 is a diagram showing an example of a modified prognostic prediction model.

As shown in Fig. 5, the prognosis prediction model can be implemented including a first feature extraction module (10) that receives a plurality of 3D MRI images as input and extracts spatial feature vectors, a second feature extraction module (20) that receives variable parameters including clinical and molecular genetic information as input and extracts clinical genetic feature vectors, and a result output module (30) that receives feature vectors output from the two feature extraction modules and outputs a post-surgical prognostic index (DPI) of a patient with glioma. At this time, the first feature extraction module (10) can be implemented as a CNN-based SE-ResNext50 model, and the remaining modules (20, 30) can be implemented as various known DNN models.

Figure 6 is a drawing explaining a three-dimensional deep learning-based prognosis prediction method using the system of Figure 1.

First, the 3D deep learning-based prognosis prediction system (100) receives multiple 3D MRI images taken of the brain region of a patient with glioma before surgery (S610).

Here, multiple 3D MRI images are obtained from MRI equipment, and may include four types of 3D MRI images (T1w, T2w, T2 FLAIR, CE-T1w).

And, the 3D deep learning-based prognosis prediction system (100) applies multiple 3D MRI images to a pre-learned prognosis prediction model to predict the post-surgical prognosis index of a brain glioma patient (S620). Here, the prognosis index can be output in the form of an index, a score, etc.

Next, the 3D deep learning-based prognosis prediction system (100) can output and provide the post-surgical prognosis index prediction result to the user terminal (200) (S630).

According to the present invention as described above, by utilizing 3D MRI data of patients with glioma prior to surgery and predicting prognosis in advance based on a deep learning model, it is possible to help medical staff determine a treatment plan for the patient.

Although the present invention has been described with reference to the embodiments shown in the drawings, these are merely exemplary, and those skilled in the art will understand that various modifications and equivalent other embodiments are possible therefrom. Accordingly, the true technical protection scope of the present invention should be determined by the technical idea of the appended claims.

100: 3D Deep Learning Based Prognosis Prediction System
110: Input section
120: Prediction Department
130: Output section
200: User Terminal

Claims (10)

In a 3D deep learning-based prognosis prediction system using preoperative MRI,
An input unit that receives multiple 3D MRI images of the brain region of a patient with glioma prior to surgery;
A prediction unit that applies the above multiple 3D MRI images to a pre-learned prognosis prediction model to perform spatial feature analysis of the images and predict the post-surgical prognosis index of the above glioma patient; and
A three-dimensional deep learning-based prognosis prediction system including an output section that outputs and provides the results of predicting the prognosis index after the above surgery to a user terminal. In claim 1,
The above multiple 3D MRI images are,
A three-dimensional deep learning-based prognosis prediction system including T1w (T1 weight) images, T2w (T2 weight) images, T2 FLAIR images, and CE-T1w (Contrast-enhanced T1 weight) images captured for the brain region of the above glioma patient. In claim 1,
The above prognosis prediction model is a 3D deep learning-based prognosis prediction system implemented including a CNN-based SE-ResNext50 model. In claim 1,
The above prediction part,
A 3D deep learning-based prognosis prediction system that sequentially performs preprocessing of skull region removal, brightness normalization, and resampling from an input 3D MRI image, and applies the preprocessed 3D MRI image to the prognosis prediction model. In claim 1,
The above input section,
Additional variable parameters including clinical information and molecular genetic information on the above brain tumor patients are input,
The above prediction part,
A 3D deep learning-based prognosis prediction system that predicts the postoperative prognosis index of a patient with glioma by applying the above-mentioned multiple 3D MRI images and the above-mentioned variable parameters to a pre-learned prognosis prediction model. In claim 5,
The above prognostic prediction model is,
A 3D deep learning-based prognosis prediction system, comprising: a first feature extraction module for receiving the above-described multiple 3D MRI images as input and extracting spatial feature vectors; a second feature extraction module for receiving variable parameters including clinical information and molecular genetic information and extracting clinical genetic feature vectors; and a result output module for receiving feature vectors output from the first and second feature extraction modules and outputting a post-surgical prognosis index for a patient with glioma. In a deep learning-based prognosis prediction method performed by a 3D deep learning-based prognosis prediction system using preoperative MRI,
A step of receiving multiple 3D MRI images of the brain region of a patient with glioma prior to surgery;
A step of applying the plurality of 3D MRI images to a pre-learned prognosis prediction model to perform spatial feature analysis of the images to predict the post-surgical prognosis index of the glioma patient; and
A three-dimensional deep learning-based prognosis prediction method including a step of outputting and providing the result of predicting the prognosis index after the above surgery to a user terminal. In claim 7,
The above multiple 3D MRI images are,
A three-dimensional deep learning-based prognosis prediction method including T1w (T1 weight) images, T2w (T2 weight) images, T2 FLAIR images, and CE-T1w (Contrast-enhanced T1 weight) images captured for the brain region of the above glioma patient. In claim 7,
The above prognosis prediction model is a 3D deep learning-based prognosis prediction method implemented including a CNN-based SE-ResNext50 model. In claim 7,
The above input receiving step is,
Additional variable parameters including clinical information and molecular genetic information on the above brain tumor patients are input,
The above predicting steps are:
A 3D deep learning-based prognosis prediction method for predicting the postoperative prognosis index of a patient with glioma by applying the above-mentioned multiple 3D MRI images and the above-mentioned variable parameters to a pre-learned prognosis prediction model.