WO2025018776A1 - Deep learning-based surgery plan system for reverse shoulder arthroplasty, and driving method thereof - Google Patents

Abstract

The method for generating a deep learning-based surgery plan, related to an embodiment of the present invention may include: a first step of acquiring X-ray image information; a second step of performing preprocessing for analysis of a predetermined region of the X-ray image information; a third step of analyzing the preprocessed X-ray image by using a convolutional neural network and extracting feature information from the result of the analysis and learning same; a fourth step of repeating the first step to the third step and generating template data for the predetermined region, on the basis of the repeatedly learned feature information; a fifth step of obtaining first X-ray image information of a user; a sixth step of extracting first feature information related to the user from the feature information by using the first X-ray image information and the template data; and a seventh step of generating a surgery plan related to the user, on the basis of the first feature information.

Description

Deep learning-based surgical planning system for retrograde total shoulder arthroplasty and its operating method

The present invention relates to a surgical planning system for reverse total shoulder arthroplasty based on deep learning and a method for operating the same. Specifically, the present invention proposes a surgical planning system that automatically establishes a surgical plan before surgery for reverse total shoulder arthroplasty, and a comprehensive surgical planning system for reverse total shoulder arthroplasty that can automatically design and produce a patient-specific surgical guide (Patient Specific Instrument) for performing a fast and accurate surgery according to the established surgical plan.

Performing a reverse total shoulder arthroplasty can present a number of challenges and potential complications, and without preoperative planning, it can be difficult for the surgeon to accurately determine the appropriate size and position of the prosthetic components.

Additionally, inadequate implant sizing can result in instability, limited range of motion, or joint inconsistency, which can lead to suboptimal results, potential complications, and challenges with intraoperative landmark identification, determining appropriate bone resection, and optimizing soft tissue balance.

Furthermore, this may result in longer surgical times and increased risk of complications, patient discomfort, and surgical fatigue.

Accordingly, the need for methods and systems that can solve this problem is increasing.

The present invention proposes a surgical planning system that automatically establishes a surgical plan prior to surgery for retrograde total shoulder arthroplasty that can solve the aforementioned problems.

The present invention proposes a comprehensive retrograde total shoulder arthroplasty surgical planning system that can automatically design and produce a patient-specific surgical guide (Patient Specific Instrument) for performing a fast and accurate surgery according to an established surgical plan.

Deep learning within the system applied to the present invention proposes a system that determines the cutting level of the acromion and recommends products from specific companies to proceed with templating.

Furthermore, the present invention proposes a system and method for designing a patient-specific surgical guide using a Boolean algorithm, thereby using 3D printing to produce a guide that can be used in surgery without a separate design work.

The technology proposed by the present invention can be summarized as follows.

1) Deep learning-based implant recommendation and resection range determination

2) 2D and 3D medical image processing technology

Meanwhile, the technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by a person having ordinary knowledge in the technical field to which the present invention belongs from the description below.

A deep learning-based surgical plan generation method related to an example of the present invention for realizing the above-described task may include: a first step of acquiring X-ray image information; a second step of performing preprocessing for analysis of a pre-designated area of the X-ray image information; a third step of analyzing the pre-processed X-ray image using a convolutional neural network (CNN) and extracting and learning feature information from the analysis result; a fourth step of repeating the first to third steps and generating template data for the pre-designated area based on the repeatedly learned feature information; a fifth step of acquiring first X-ray image information of a user; a sixth step of extracting first feature information related to the user from the feature information using the first X-ray image information and the template data; and a seventh step of generating a surgical plan related to the user based on the first feature information.

Additionally, the surgery involving the user may include a reverse total shoulder arthroplasty, and the pre-designated area may include a shoulder joint area.

Additionally, the preprocessing in the second step can be applied to enable analysis of anatomical information about the shoulder joint area.

Additionally, the feature information may include at least one of anatomical structure information related to the user, bone structure information, angle information of a pre-specified joint, muscle and/or tissue analysis information, and information related to the size and position of an implant structure to be applied to the shoulder joint.

In addition, the generated surgical plan may include at least one of information determining the type, size, and location of the implant to be applied to the shoulder joint, information on the cutting range and cutting angle for applying the retrograde rotator cuff arthroplasty, information on prediction of potential complications after the surgery, information on prediction of the user's range of motion related to the shoulder joint area after the surgery, and information on an evaluation score for the suitability of the established surgical plan determined based on pre-stored surgical data.

In addition, in the first step, a CT image on which a surgery has been completed in advance is acquired, and in the third step, a cutting level applied to the completed surgery is measured based on the CT image, a portion of the area related to the completed surgery is extracted from the X-ray image, and the learning can be performed after labeling the preprocessed X-ray image based on the cutting level and the extracted portion of the area.

In addition, in the fourth step, based on the area selected by the external user among the X-ray image information, a first shape is automatically determined according to the size of the selected area among a plurality of pre-stored component shapes, and the template data can be generated based on the first shape.

In addition, after the 7th step, the method may further include: an 8th step of acquiring CT image information of the user; a 9th step of automatically generating 3D image information by 3D-reconstructing the CT image information; and a 10th step of receiving the first X-ray image information and the template data, and generating STL information by using the automatically generated 3D image information, the first X-ray image information, and the template data together; an 11th step of inputting the generated STL data into a 3D printer; and a 12th step of outputting a surgical guide to which the surgical plan has been simulated in advance based on the STL data from the 3D printer.

In addition, in the ninth step, the PSI size related to the CT image information can be changed using the cutting level on the surgical plan, and the three-dimensional image information can be automatically generated by performing a two-dimensional Boolean operation on the image information with the changed PSI size to fit the user's skeleton.

Meanwhile, a deep learning-based surgical plan generation device related to another example of the present invention for realizing the above-described task includes: an acquisition unit for acquiring X-ray image information; a preprocessing unit for performing preprocessing for analyzing a pre-designated area of the X-ray image information; and a control unit for analyzing the preprocessed X-ray image using a convolutional neural network (CNN) and extracting and learning feature information from the analysis result; wherein, after repeating the operations of the acquisition unit, the preprocessing unit, and the control unit, the control unit generates template data for the pre-designated area based on the repeatedly learned feature information, and when the acquisition unit additionally acquires first X-ray image information of the user, the control unit can extract first feature information related to the user from the feature information by using the first X-ray image information and the template data, and generate a surgical plan related to the user based on the first feature information.

In the present invention, deep learning is used to make decisions with high accuracy on areas that are difficult for surgeons to determine, such as cutting levels, and to provide recommendations for the most suitable artificial joint product for a patient based on images alone, as well as templating, thereby resolving the overall aspects of surgical planning.

In addition, by providing designs for patient-tailored surgical guides, they can be produced using 3D printing without separate design work and applied directly to patient surgery, helping to ensure that the surgery can proceed according to the surgical plan.

In this case, the time required to establish a surgical plan and perform the surgical procedure can be drastically shortened, and the gap in the surgeon's experience and skills can be reduced, allowing even less experienced surgeons to perform the surgery accurately, while reducing the patient's pain due to the surgery.

Meanwhile, the effects obtainable from the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by a person having ordinary skill in the art to which the present invention belongs from the description below.

FIG. 1 illustrates an example of a block diagram of a deep learning-based surgical plan generation system in relation to the present invention.

FIG. 2 illustrates an example of a flowchart explaining a deep learning-based surgical plan generation method in relation to the present invention.

FIG. 3 illustrates an example of the UI configuration and function of a surgical planning system for retrograde total shoulder arthroplasty in relation to the present invention.

FIG. 4 illustrates an example of a cutting level determination deep learning model overview diagram related to the present invention.

FIG. 5 illustrates an example of a templated deep learning model overview diagram related to the present invention.

FIG. 6 illustrates an example of a Boolean-based PSI automatic design algorithm in relation to the present invention.

Reverse shoulder arthroplasty is an orthopedic surgical procedure used to treat certain shoulder conditions associated with severe arthritis, irreparable rotator cuff tears, or complex fractures.

Unlike conventional total shoulder arthroplasty, which replaces the acromion with an artificial ball and the joint space with an artificial socket, reverse total shoulder arthroplasty can reverse the anatomical structure of the shoulder joint.

Reverse shoulder arthroplasty aims to relieve pain, restore shoulder function, and improve range of motion in individuals with limited mobility or strength due to rotator cuff dysfunction or severe arthritis.

This procedure bypasses the need for rotator cuff function by utilizing the deltoid muscle as the primary exercise.

The need for a preoperative surgical planning system for retrograde arthroscopic shoulder arthroplasty lies in its ability to improve surgical outcomes, enhance patient safety, and optimize the overall surgical process.

Applying deep learning to these systems has several advantages.

Deep learning algorithms can analyze medical images and extract details with a high level of accuracy.

This allows for precise measurement of anatomical structures, implant size, and location, which can improve the accuracy of surgical planning.

Additionally, by learning from large data sets and previous surgical cases, we can provide personalized treatment plans based on the patient's specific anatomy, pathology, and desired outcomes.

This individualized approach leads to better surgical outcomes and patient satisfaction.

In this case, the system can quickly process large amounts of medical image data, enabling faster preoperative planning.

This saves surgeons valuable time, streamlines surgical workflow, helps reduce patient waiting times and improve hospital efficiency.

Surgical planning systems that use deep learning models can analyze complex patterns in medical images to provide surgeons with useful insights and recommendations.

This allows for informed decisions about implant selection, surgical technique, and potential complications, reducing the risk of error and improving patient safety.

Performing a reverse total shoulder arthroplasty can present many challenges and potential complications, and without preoperative planning, it can be difficult for the surgeon to accurately determine the appropriate size and position of the prosthetic components.

Additionally, inadequate implant sizing can result in instability, limited range of motion, or joint inconsistency, which can lead to suboptimal results, potential complications, and challenges with intraoperative landmark identification, determining appropriate bone resection, and optimizing soft tissue balance.

Furthermore, this may result in longer surgical times and increased risk of complications, patient discomfort, and surgical fatigue.

The present invention proposes a surgical planning system that automatically establishes a surgical plan prior to surgery for retrograde total shoulder arthroplasty that can solve these problems.

The present invention proposes a comprehensive retrograde total shoulder arthroplasty surgical planning system that can automatically design and produce a patient-specific surgical guide (Patient Specific Instrument) for performing a fast and accurate surgery according to an established surgical plan.

Deep learning within the system applied to the present invention proposes a system that determines the cutting level of the acromion and recommends products from specific companies to proceed with templating.

Furthermore, the present invention proposes a system and method for designing a patient-specific surgical guide using a Boolean algorithm, thereby proceeding with a guide that can be used in surgery using 3D printing without a separate design work.

The technology proposed by the present invention can be summarized as follows.

1) Deep learning-based implant recommendation and resection range determination

2) 2D and 3D medical image processing technology

FIG. 1 illustrates an example of a block diagram of a deep learning-based surgical plan generation system in relation to the present invention.

Referring to FIG. 1, a deep learning-based surgical plan generation system (1) may include an acquisition unit (10) that acquires X-ray image information, a preprocessing unit (20) that performs preprocessing for analysis of a pre-designated area among the X-ray image information, and a control unit (30) that analyzes the preprocessed X-ray image using a convolutional neural network (CNN) and extracts and learns feature information from the analysis results.

Here, after repeating the operations of the acquisition unit (10), preprocessing unit (20), and control unit (30), the control unit (30) generates template data for the pre-designated area based on the repeatedly learned feature information.

In addition, when the acquisition unit (10) additionally acquires the user's first X-ray image information, the control unit (30) uses the first X-ray image information and template data to extract the first feature information related to the user from among the feature information, and generates a surgical plan related to the user based on the first feature information.

Meanwhile, the control unit (30) can obtain the user's CT image information and automatically generate 3D image information by 3D reconstructing the CT image information.

Additionally, the control unit (30) or the 3D printing unit (2) may receive the first X-ray image information and the template data, and use the automatically generated 3D image information, the first X-ray image information, and the template data together to generate STL information.

When the generated STL data is input into the 3D printer section (2), the 3D printer section (2) can output a surgical guide that simulates the surgical plan in advance based on the STL data.

Meanwhile, FIG. 2 illustrates an example of a flowchart explaining a deep learning-based surgical plan generation method in relation to the present invention.

The surgery involved herein may include reverse rotator cuff arthroplasty, and the pre-specified area may include the shoulder joint area.

Referring to Fig. 2, first, a step (S1) of acquiring X-ray image information is performed.

Afterwards, a step (S2) of performing preprocessing for analysis of a pre-designated area of X-ray image information is performed.

Here, preprocessing can be applied to enable analysis of anatomical information about the shoulder joint area.

In addition, a step (S3) is performed in which the preprocessed X-ray image is analyzed using a convolutional neural network (CNN), and feature information is extracted and learned from the analysis results.

Here, the feature information may include at least one of anatomical structure information related to the user, bone structure information, angle information of a pre-specified joint, muscle and/or tissue analysis information, and information related to the size and position of an implant structure to be applied to the shoulder joint.

Thereafter, steps 1 to 3 are repeated, and a step (S4) of generating template data for a pre-designated area based on the repeatedly learned feature information is performed.

Meanwhile, in the case where a CT image of a previously completed surgery is acquired together in the first step, in the third step, a cutting level applied to the completed surgery is measured based on the CT image, a portion of the area related to the completed surgery is extracted from the X-ray image, and the learning can be performed after labeling the preprocessed X-ray image based on the cutting level and the extracted portion of the area.

In addition, based on an area selected by an external user from among X-ray image information, a first shape may be automatically determined based on the size of the selected area from among a plurality of pre-stored component shapes, and the template data may be generated based on the first shape.

Additionally, a step (S5) of acquiring the user's first X-ray image information is performed.

Thereafter, a step (S6) of extracting first feature information related to the user from among feature information is performed using the first X-ray image information and template data.

Additionally, a step (S7) of generating a surgical plan related to the user based on the first feature information is performed.

The surgical plan generated herein may include at least one of information determining the type, size, and location of an implant to be applied to the shoulder joint, information on the cutting range and cutting angle for applying the retrograde rotator cuff arthroplasty, information on prediction of potential complications after the surgery, information on prediction of the user's range of motion related to the shoulder joint area after the surgery, and information on an evaluation score for the suitability of the established surgical plan determined based on pre-stored surgical data.

Additionally, a step (S8) of acquiring the user's CT image information and automatically generating 3D image information by reconstructing the CT image information in 3D is performed.

Here, the PSI size related to the CT image information can be changed using the cutting level on the surgical plan, and the image information with the PSI size changed can be subjected to a two-dimensional Boolean operation to automatically generate the three-dimensional image information to fit the user's skeleton.

Thereafter, the first X-ray image information and template data are received, and STL information is generated using the automatically generated 3D image information, the first X-ray image information, and the template data together (S10), and the generated STL data is input to a 3D printer (S11).

Furthermore, a step (S12) of printing a surgical guide that simulates a surgical plan in advance based on STL data on a 3D printer can be performed.

The deep learning-based reverse rotator cuff arthroplasty preoperative surgical planning system proposed by the present invention described above is a cutting-edge technology solution that helps surgeons plan and optimize surgical procedures.

Reverse shoulder arthroplasty is a complex orthopedic surgery performed to treat shoulder conditions such as severe arthritis or rotator cuff tears. It involves reversing the normal anatomical structure of the shoulder joint to improve its function.

The present invention utilizes X-ray images to capture detailed anatomical information about a patient's shoulder joint, and these images are used as input data for a deep learning algorithm.

Deep learning algorithms, especially convolutional neural networks (CNNs), excel at analyzing preprocessed medical images and extracting meaningful features from them, and have proven effective in a variety of medical imaging tasks, extracting relevant features that are important for surgical planning.

These characteristics may include bone structure, joint angles, implant position, and muscle/tissue analysis.

Based on the extracted features, the system generates a detailed preoperative surgical plan, which may include determining the optimal implant type, size and position, cutting range and angle, predicting potential complications, predicting postoperative range of motion, and assessing the overall feasibility of the surgery.

The present invention proposes a comprehensive surgical planning system for retrograde total shoulder arthroplasty that automatically establishes a surgical plan prior to surgery for retrograde total shoulder arthroplasty and automatically designs and produces a patient-specific surgical guide (Patient Specific Instrument) according to the established surgical plan for fast and accurate surgery.

Deep learning within the system can determine the cutting level of the shoulder and even recommend products from specific companies to proceed with templating.

Furthermore, by using a Boolean algorithm to design a patient-specific surgical guide, a guide that can be used in surgery can be produced using 3D printing without separate design work.

Meanwhile, FIG. 3 illustrates an example of the UI configuration and function of a surgical planning system for retrograde total shoulder arthroplasty in relation to the present invention.

Figure 3 (a) illustrates examples of UI configurations in which image information (110) and surgical guides are designed (120) according to the final surgical plan.

Additionally, Fig. 3 (b) illustrates examples of steps S1 to S9 described above.

In relation to step S10 of receiving first X-ray image information and template data and generating STL information by using automatically generated 3D image information, first X-ray image information, and template data together, steps S10a and S10b are shown separately.

Additionally, steps S11 and S12 are displayed over time.

In addition, FIG. 4 illustrates an example of a cutting level determination deep learning model overview diagram related to the present invention.

Referring to (a) and (b) of FIG. 4, an example of the operation of the control unit (30) for measuring the cutting level by configuring a CT image on which surgery has been completed in advance in three dimensions and extracting and labeling a portion of an X-ray image of the same patient and then learning it is illustrated in relation to step S3.

Figure 4 (a) illustrates an example of the results of a cutting level determination deep learning model.

FIG. 5 illustrates an example of a templated deep learning model overview diagram in relation to the present invention.

Referring to (a) and (b) of FIG. 5, in relation to step S3, an example of the operation of the control unit (30) for automatically determining the size and model of each shoulder joint image and templating the prepared component shape is illustrated.

Figure 5 (a) illustrates an example of the application result of a templating deep learning model.

FIG. 6 illustrates an example of a Boolean-based PSI automatic design algorithm in relation to the present invention.

Figure 6 illustrates the application of the Boolean-based PSI automatic design algorithm related to steps S8, S9, and S10.

Each step of (a) to (J) represents an example of a process of changing a previously stored basic PSI size using a cutting level determined from deep learning, performing 2D Boolean to fit the patient's skeleton, and reconstructing it in 3D.

When applying the system and method of the present invention described above, the present invention can accurately determine a part that is difficult for a surgeon to determine, such as a cutting level, by using deep learning, and can resolve the overall part of the surgical plan by recommending the most suitable artificial joint product for the patient and even performing templating using only images.

In addition, by providing designs for patient-tailored surgical guides, they can be produced using 3D printing without separate design work and applied directly to patient surgery, helping to ensure that the surgery can proceed according to the surgical plan.

In this case, the time required to establish a surgical plan and perform the surgical procedure can be drastically shortened, and the gap in the surgeon's experience and skills can be reduced, allowing even less experienced surgeons to perform the surgery accurately, while reducing the patient's pain due to the surgery.

Meanwhile, the effects obtainable from the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by a person having ordinary skill in the art to which the present invention belongs from the description below.

In addition, the system and its control method described above are not limited to the configuration and method of the embodiments described above, and the embodiments may be configured by selectively combining all or part of each embodiment so that various modifications can be made.

Claims (10)

Step 1: Obtaining X-ray image information; A second step of performing pre-processing for analysis of a pre-designated area among the above X-ray image information; A third step of analyzing the above preprocessed X-ray image using a convolutional neural network (CNN) and extracting and learning feature information from the analysis results; A fourth step of repeating the first to third steps and generating template data for the pre-designated area based on the repeatedly learned feature information; Step 5 of acquiring the user's first X-ray image information; A sixth step of extracting first feature information related to the user from among the feature information using the first X-ray image information and the template data; and A deep learning-based surgical plan generation method comprising a seventh step of generating a surgical plan related to the user based on the first feature information. In paragraph 1, The surgery associated with the above user includes retrograde total shoulder arthroplasty, The above pre-designated area is a deep learning-based surgical plan generation method including the shoulder joint area. In the second paragraph, The preprocessing in the second step is as follows: A method for generating a deep learning-based surgical plan that can analyze anatomical information about the above shoulder joint area. In the third paragraph, The above feature information is, A method for generating a deep learning-based surgical plan, comprising at least one of anatomical structure information related to the user, bone structure information, angle information of a predetermined joint, muscle and/or tissue analysis information, and information related to the size and position of an implant structure to be applied to the shoulder joint. In paragraph 4, In the above generated surgical plan, A deep learning-based surgical plan generation method including at least one of information determining the type, size, and location of the implant to be applied to the shoulder joint, information on the cutting range and cutting angle for applying the retrograde total shoulder arthroplasty, information predicting potential complications after the surgery, information predicting the user's range of motion related to the shoulder joint area after the surgery, and information evaluating the suitability of the established surgical plan based on pre-stored surgical data. In paragraph 5, In the above step 1, CT images of previously completed surgery are acquired together, In the third step above, Based on the above CT image, the cutting level applied to the completed surgery is measured, Some areas related to the completed surgery are extracted from the above X-ray image, A deep learning-based surgical plan generation method for labeling the preprocessed X-ray image based on the above-described cutting level and the extracted partial area, and then performing the learning. In paragraph 6, In the above 4th step, A deep learning-based surgical plan generation method, wherein a first shape is automatically determined based on the size of the selected region among a plurality of pre-stored component shapes based on an area selected by an external user from the above X-ray image information, and the template data is generated based on the first shape. In Article 7, After step 7 above, Step 8 of obtaining CT image information of the above user; Step 9 of automatically generating 3D image information by reconstructing the above CT image information in 3D; and A 10th step of receiving the first X-ray image information and the template data, and generating STL information by using the automatically generated 3D image information, the first X-ray image information, and the template data together; Step 11 of inputting the above generated STL data into a 3D printer; and A method for generating a deep learning-based surgical plan, further comprising a 12th step of outputting a surgical guide that has applied the surgical plan in advance based on the STL data from the 3D printer. In Article 8, In the above step 9, The PSI size related to the CT image information is changed using the cutting level in the above surgical plan, A deep learning-based surgical plan generation method that automatically generates three-dimensional image information by performing two-dimensional Boolean operations on image information with the PSI size changed to fit the user's skeleton. An acquisition unit for acquiring X-ray image information; A preprocessing unit that performs preprocessing for analysis of a pre-designated area among the above X-ray image information; and A control unit that analyzes the above-mentioned preprocessed X-ray image using a convolutional neural network (CNN) and extracts and learns feature information from the analysis results; After repeating the operations of the above acquisition unit, preprocessing unit and control unit, the control unit generates template data for the pre-designated area based on the repeatedly learned feature information. If the above acquisition unit additionally acquires the user's first X-ray image information, A deep learning-based surgical plan generation device, wherein the control unit extracts first feature information related to the user from among the feature information using the first X-ray image information and the template data, and generates a surgical plan related to the user based on the first feature information.

Images