CN116737031A - A tooth root information visualization system and method based on mixed reality - Google Patents

Abstract

The invention belongs to the technical field of orthodontic treatment, in particular to a tooth root information visualization system and method based on mixed reality, wherein the system comprises a tracking rigid body 1, a tracking rigid body 2, a probe, optical tracking equipment, mixed reality glasses, a checkerboard calibration plate, an orthodontic anchorage nail implantation navigation system and a visualization system; the method comprises an optical tracking device-mixed reality glasses calibrating method and a complete dentition virtual model-patient reality space registering method; the optical tracking device-mixed reality glasses calibrating method is used for unifying internal coordinate systems of the optical tracking device and the mixed reality glasses; the complete dentition virtual model-patient reality space registration method is used for unifying the complete dentition virtual model with a coordinate system of a reality space where a patient is located. The invention solves the problem that doctors cannot directly observe the information limitation of the anatomical structure of the tissues around the implantation position in the process of implanting the orthodontic anchorage nails, and improves the accuracy and efficiency of implanting the orthodontic anchorage nails.

Description

A tooth root information visualization system and method based on mixed reality

Technical field

The present invention relates to the technical field of oral orthodontics, specifically a mixed reality-based tooth root information visualization system and method.

Background technique

Anchorage control is the key to successful orthodontic treatment. Accurately and safely implanting orthodontic anchorage nails into the patient's alveolar bone is an important part of anchorage control. Currently, the existing clinical anchorage nail implantation operation mainly involves doctors observing the patient's imaging data to determine the relationship between the implantation site, the tooth root position and the maxillary sinus, and perform anchorage nail implantation based on clinical experience. . This clinical operation requires the surgeon to have a clear understanding of the anatomical structure of the surrounding tissues and requires high experience, otherwise it will easily cause damage to the tooth root and surrounding tissues. In addition, for complex or difficult-to-operate cases, the number of roots, the close distance between adjacent roots, and the large variation in root furcations often lead to an increased risk of anchorage nail implantation, which discourages a large number of young doctors. Studies have shown that whether the anchorage nail touches or damages the tooth root during implantation, as well as the safety and long-term stability after implantation, are the keys to evaluating the success of the implantation operation. Therefore, the high requirements for clinical implantation, the high hazards of improper operation, and the high failure rate have seriously hindered the application and popularization of implant anchorage technology in orthodontic clinical practice.

The anchorage nail implantation technology based on image-guided surgical navigation tracks the real-time positions of the patient and the anchorage nail implantation tool (joystick) and displays the navigation information on a fixed monitor. During the operation, the doctor's field of view needs to switch back and forth between the monitor and the operating field, which not only affects the efficiency and consistency of the surgical operation, but also increases the possibility of surgical risks.

Therefore, we propose a mixed reality-based tooth root information visualization system and method to solve the above problems.

Contents of the invention

(1) Technical problems solved

In view of the shortcomings of the existing technology, the present invention provides a mixed reality-based tooth root information visualization system and method to solve the problem of low implantation accuracy of conventional anchorage nail implantation methods, reliance on doctors' clinical experience, and image-guided surgical navigation. During the anchorage nail implantation technology operation, the doctor's field of view needs to switch back and forth between the monitor and the operating field, which affects the efficiency and continuity of the surgical operation and increases the risk of surgery.

(2) Technical solutions

In order to solve the limitation of being unable to directly observe the anatomical structure information of tissues around the implantation site during orthodontic anchorage nail implantation, the present invention proposes a mixed reality-based tooth root information visualization system that applies artificial intelligence technology to image-guided surgical navigation. In the anchorage nail implantation technology, based on mixed reality technology, with the help of mixed reality glasses and optical tracking equipment, a virtual model of the dentition with complete tooth information is accurately displayed in the doctor's surgical field; allowing the doctor to perform anchorage nail implantation During the insertion process, the surrounding root structure of the anchorage nail implantation location can be observed, guiding the doctor to accurately implant the anchorage nail at the target location to avoid damage to other surrounding tissues due to mistakes.

In order to achieve the above purpose, the present invention specifically adopts the following technical solutions: a mixed reality-based tooth root information visualization system. The system includes a tracking rigid body 1, a tracking rigid body 2, a probe, an optical tracking device, mixed reality glasses, and a checkerboard calibration board. , Orthodontic anchorage nail implantation navigation system and visualization system;

The tracking rigid body 1 is rigidly connected to the outer headband of the mixed reality glasses, and is used to obtain the real-time position and posture of the mixed reality glasses;

The tracking rigid body 2 is rigidly connected to the buccal side of the crown of the patient's upper (lower) jaw teeth, and is used to obtain the real-time position and posture of the patient's upper (lower) jaw dentition;

The probe is manually held by the doctor and used to obtain the position coordinates of the patient's dental anatomical landmark points in the coordinate system of the optical tracking device;

The optical tracking device uses binocular vision technology to collect in real time the real-time positions and postures of the tracking rigid body 1 and the tracking rigid body 2, as well as the real-time position of the probe tip;

The mixed reality glasses are used to fuse and display the virtual model of the complete dentition with the real scene of the patient's oral cavity in real time;

The checkerboard calibration board is used to achieve calibration between the tracking rigid body 1 and the coordinate system of the mixed reality glasses;

The orthodontic anchorage nail implantation navigation system is used to send navigation information to mixed reality glasses;

The visualization system is installed in mixed reality glasses and is used for three-dimensional visualization of a complete dental virtual model.

A method applied to a mixed reality-based tooth root information visualization system according to claim 1, which method includes an optical tracking device-mixed reality glasses calibration method and a complete dental virtual model-patient real space registration method; The optical tracking device-mixed reality glasses calibration method is used to unify the internal coordinate systems of the optical tracking device and the mixed reality glasses; the complete dentition virtual model-patient real space registration method is used to combine the complete dentition virtual model with the patient. The coordinate system of the real space is unified;

The optical tracking device-mixed reality glasses calibration method includes the following implementation steps:

Step 1: Install the tracking rigid body 1 on one side of the headband outside the mixed reality glasses. The installation position should ensure that the tracking rigid body continues to be in the field of view of the optical tracking device during the calibration process. The following mixed reality glasses refer to mixed reality with tracking rigid body 1 installed. Glasses;

Step 2: Place the checkerboard calibration plate on the workbench. Make sure it is completely still and at any angle. Make sure the mixed reality glasses can smoothly shoot it;

Step 3: Set the mixed reality glasses to a posture to ensure that they can successfully capture the checkerboard calibration board;

Step 4: Set the optical tracking device at a position where tracking rigid body 1 can be observed;

Step 5: Collect the images captured by the mixed reality glasses and transfer them to the computer for storage. At the same time, transfer the rigid body position and attitude data collected by the optical tracking system to the computer for storage;

Step 6: Change the posture of the mixed reality glasses in step 3 and repeat step 5 no less than 20 times;

Step 7: Process all the data in step 6, in which the checkerboard calibration plate coordinate system is obtained through the camera calibration algorithm To the mixed reality glasses coordinate system/> The spatial transformation matrix/> , use it as "hand" data to track the spatial transformation matrix of rigid body 1 to the optical tracking coordinate system/> , treat it as "eye" data, and save "hand" data and "eye" data separately;

Step 8: Use the hand-eye calibration algorithm to calibrate the data obtained in step 7 to obtain the tracking rigid body 1 coordinate system To the mixed reality glasses coordinate system/> The spatial transformation matrix/> ;

The complete dental virtual model-patient real space registration and visualization method includes the following implementation steps:

Step 1: Use medical image processing software to segment the complete dentition from the patient's CBCT image and export the STL data of the complete dentition model;

Step 2: Import the STL data in step 1 into the mixed reality glasses, and record the data as , three-dimensional display through the visualization system;

Step 3: Fix the tracking rigid body 2 on the buccal side of the tooth crown on the side of the patient’s upper (lower) jaw where the anchorage nail is to be implanted;

Step 4: Set up the optical tracking device on the side of the patient. The position should ensure that Rigid Body 1 and Rigid Body 2 continue to be in the field of view of the optical tracking device during the tracking process;

Step 5: Set the probe tip in the "V"-shaped concave cone tip, hold the probe in spherical motion, and calibrate the probe tip position;

Step 6: Use a probe to select 4 distinctive anatomical landmark points on the occlusal surface of the first molar and second molar on the side of the patient's upper (lower) jaw where the anchorage nail is to be implanted, and record them as ;

Step 7: Use a probe to trace the buccal surface of the central incisor, lateral incisor and canine on the side of the patient's upper (lower) jaw where the anchorage nail is to be implanted, and save the probe tip data collected by the optical tracking device to The computer is the buccal surface point cloud data of the upper (lower) mandibular central incisors, lateral incisors and canines, and the data is recorded as ;

Step 8: The operating doctor wears the mixed reality glasses, and in the visualization system, uses the mouse to click on the anatomical landmark points at the same positions as in step 7 in the same order, which are recorded as ;

Step 9: Use the marker point registration algorithm to calculate the and in step 8/> The spatial transformation relationship between them achieves rough registration between the virtual model and the real model;

Step 10: Use the iterative closest point algorithm to convert the Same as step 7/> Perform precise registration to obtain the transformation matrix/> , At this point, high-precision registration of the complete dental model-patient's real space is achieved;

Step 11: Virtual model coordinate system and mixed reality glasses coordinate system transformation matrix array Calculated by the following formula:

;

Step 12: According to step 11, the position and posture of the complete dentition virtual model in the visualization system are transformed in real time, so as to complete the visualization of the complete dentition virtual model in the patient's real mouth.

(3) Beneficial effects

Compared with the existing technology, the present invention provides a mixed reality-based tooth root information visualization system and method, which has the following beneficial effects:

This invention applies artificial intelligence technology to image-guided surgical navigation anchorage nail implantation technology, based on mixed reality technology, with the help of mixed reality glasses and optical tracking equipment, to accurately display a virtual model of the dentition with complete tooth information. In the doctor's surgical field, the doctor can observe the surrounding root structure of the anchor nail implantation position during the anchorage nail implantation process, and guide the doctor to accurately implant the anchorage nail at the target location to avoid damage to other surrounding tissues due to mistakes. Damage, solves the problem of being unable to directly observe the limitations of information on the anatomical structure of tissues around the implantation site during orthodontic anchorage nail implantation, and improves surgical accuracy and efficiency.

Description of drawings

Figure 1 is a schematic diagram of the implementation steps of the optical tracking device-mixed reality glasses calibration method of the present invention;

Figure 2 is a schematic diagram of the implementation steps of the complete dentition virtual model-patient real-space registration and visualization method of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

Example

As shown in Figures 1 and 2, an embodiment of the present invention proposes a mixed reality-based tooth root information visualization system and method. The system includes a tracking rigid body 1, a tracking rigid body 2, a probe, an optical tracking device, mixed reality glasses, Checkerboard calibration board, orthodontic anchorage nail implantation navigation system, visualization system; methods include optical tracking equipment-mixed reality glasses calibration method and complete dental virtual model-patient real space registration method;

The tracking rigid body 1 is rigidly connected to the outer head ring of the mixed reality glasses, and is used to obtain the real-time position and posture of the mixed reality glasses;

The tracking rigid body 2 is rigidly connected to the buccal side of the crown of the patient's upper (lower) jaw teeth, and is used to obtain the real-time position and posture of the patient's upper (lower) jaw dentition;

The probe is manually held by the doctor and used to obtain the position coordinates of the patient's dental anatomical landmark points in the coordinate system of the optical tracking device;

The optical tracking equipment uses binocular vision technology to collect the real-time positions and postures of tracking rigid body 1 and tracking rigid body 2 in real time, as well as the real-time position of the probe tip;

Mixed reality glasses are used to integrate and display the virtual model of the complete dentition with the patient's oral real-life scene in real time;

The checkerboard calibration board is used to achieve calibration between the tracking rigid body 1 and the coordinate system of the mixed reality glasses;

Orthodontic anchorage nail implantation navigation system is used to send navigation information to mixed reality glasses;

The visualization system is installed in mixed reality glasses and is used to three-dimensionally visualize the complete dental virtual model;

Optical tracking device-mixed reality glasses calibration method is used to unify the internal coordinate systems of the optical tracking device and mixed reality glasses;

The complete dentition virtual model-patient real space registration method is used to unify the complete dentition virtual model with the coordinate system of the patient's real space.

As shown in Figure 1, in some embodiments, the optical tracking device-mixed reality glasses calibration method includes the following implementation steps:

Step 1: Install the tracking rigid body 1 on one side of the headband outside the mixed reality glasses. The installation position should ensure that the tracking rigid body continues to be in the field of view of the optical tracking device during the calibration process. The following mixed reality glasses refer to mixed reality with tracking rigid body 1 installed. Glasses;

Step 2: Place the checkerboard calibration plate on the workbench. Make sure it is completely still and at any angle. Make sure the mixed reality glasses can smoothly shoot it;

Step 3: Set the mixed reality glasses to a posture to ensure that they can successfully capture the checkerboard calibration board;

Step 4: Set the optical tracking device at a position where tracking rigid body 1 can be observed;

Step 5: Collect the images captured by the mixed reality glasses and transfer them to the computer for storage. At the same time, transfer the rigid body position and attitude data collected by the optical tracking system to the computer for storage;

Step 6: Change the posture of the mixed reality glasses in step 3 and repeat step 5 no less than 20 times;

Step 7: Process all the data in step 6, in which the checkerboard calibration plate coordinate system is obtained through the camera calibration algorithm To the mixed reality glasses coordinate system/> The spatial transformation matrix/> , use it as "hand" data to track the spatial transformation matrix of rigid body 1 to the optical tracking coordinate system/> , treat it as "eye" data, and save "hand" data and "eye" data separately;

Step 8: Use the hand-eye calibration algorithm to calibrate the data obtained in step 7 to obtain the tracking rigid body 1 coordinate system To the mixed reality glasses coordinate system/> The spatial transformation matrix/> .

As shown in Figure 2, in some embodiments, the complete dentition virtual model-patient real space registration method includes the following implementation steps:

Step 1: Use medical image processing software to segment the complete dentition from the patient's CBCT image and export the STL data of the complete dentition model;

Step 2: Import the STL data in step 1 into the mixed reality glasses, and record the data as , three-dimensional display through the visualization system;

Step 3: Fix the tracking rigid body 2 on the buccal side of the tooth crown on the side of the patient’s upper (lower) jaw where the anchorage nail is to be implanted;

Step 4: Set up the optical tracking device on the side of the patient. The position should ensure that Rigid Body 1 and Rigid Body 2 continue to be in the field of view of the optical tracking device during the tracking process;

Step 5: Set the probe tip in the "V"-shaped concave cone tip, hold the probe in spherical motion, and calibrate the probe tip position;

Step 6: Use a probe to select 4 distinctive anatomical landmark points on the occlusal surface of the first molar and second molar on the side of the patient's upper (lower) jaw where the anchorage nail is to be implanted, and record them as ;

Step 7: Use a probe to trace the buccal surface of the central incisor, lateral incisor and canine on the side of the patient's upper (lower) jaw where the anchorage nail is to be implanted, and save the probe tip data collected by the optical tracking device to The computer is the buccal surface point cloud data of the upper (lower) mandibular central incisors, lateral incisors and canines, and the data is recorded as ;

Step 8: The operating doctor wears the mixed reality glasses, and in the visualization system, uses the mouse to click on the anatomical landmark points at the same positions as in step 7 in the same order, which are recorded as ;

Step 9: Use the marker point registration algorithm to calculate the and in step 8/> The spatial transformation relationship between them achieves rough registration between the virtual model and the real model;

Step 10: Use the iterative closest point algorithm to convert the Same as step 7/> Perform precise registration to obtain the transformation matrix/> , At this point, high-precision registration of the complete dental model-patient's real space is achieved;

Step 11: Virtual model coordinate system and mixed reality glasses coordinate system transformation matrix array Calculated by the following formula:

;

Step 12: According to step 11, the position and posture of the complete dentition virtual model in the visualization system are transformed in real time, so as to complete the visualization of the complete dentition virtual model in the patient's real mouth.

The present invention uses mixed reality technology to directly present the three-dimensional anatomical structure information of the doctor's area of interest in the doctor's operating field, assisting the doctor to perform more accurate surgical operations, thereby alleviating the problems of doctor perspective switching and information misreading in image-guided surgical navigation. problems, thereby improving surgical accuracy and efficiency.

Finally, it should be noted that the above are only preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, for those skilled in the art, it is still The technical solutions described in the foregoing embodiments may be modified, or some of the technical features may be equivalently replaced. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

Claims (2)

1. A tooth root information visualization system based on mixed reality, characterized in that: the system includes a tracking rigid body 1, a tracking rigid body 2, a probe, an optical tracking device, mixed reality glasses, a checkerboard calibration plate, and an orthodontic anchorage nail. Implanted navigation system and visualization system; The tracking rigid body 1 is rigidly connected to the outer headband of the mixed reality glasses, and is used to obtain the real-time position and posture of the mixed reality glasses; The tracking rigid body 2 is rigidly connected to the buccal side of the crown of the patient's upper (lower) jaw teeth, and is used to obtain the real-time position and posture of the patient's upper (lower) jaw dentition; The probe is manually held by the doctor and used to obtain the position coordinates of the patient's dental anatomical landmark points in the coordinate system of the optical tracking device; The optical tracking device uses binocular vision technology to collect in real time the real-time positions and postures of the tracking rigid body 1 and the tracking rigid body 2, as well as the real-time position of the probe tip; The mixed reality glasses are used to fuse and display the virtual model of the complete dentition with the real scene of the patient's oral cavity in real time; The checkerboard calibration board is used to achieve calibration between the tracking rigid body 1 and the coordinate system of the mixed reality glasses; The orthodontic anchorage nail implantation navigation system is used to send navigation information to mixed reality glasses; The visualization system is installed in mixed reality glasses and is used for three-dimensional visualization of a complete dental virtual model. 2. A method applied to a mixed reality-based dental root information visualization system according to claim 1, characterized in that: the method includes an optical tracking device-mixed reality glasses calibration method and a complete dental virtual model-patient Real space registration method; the optical tracking device-mixed reality glasses calibration method is used to unify the internal coordinate systems of the optical tracking device and mixed reality glasses; the complete dentition virtual model-patient real space registration method is used to unify the complete dentition virtual model-patient real space registration method The virtual model of the dentition is unified with the coordinate system of the patient's real space; The optical tracking device-mixed reality glasses calibration method includes the following implementation steps: Step 1: Install the tracking rigid body 1 on one side of the headband outside the mixed reality glasses. The installation position should ensure that the tracking rigid body continues to be in the field of view of the optical tracking device during the calibration process. The following mixed reality glasses refer to mixed reality with tracking rigid body 1 installed. Glasses; Step 2: Place the checkerboard calibration plate on the workbench. Make sure it is completely still and at any angle. Make sure the mixed reality glasses can smoothly shoot it; Step 3: Set the mixed reality glasses to a posture to ensure that they can successfully capture the checkerboard calibration board; Step 4: Set the optical tracking device at a position where tracking rigid body 1 can be observed; Step 5: Collect the images captured by the mixed reality glasses and transfer them to the computer for storage. At the same time, transfer the rigid body position and attitude data collected by the optical tracking system to the computer for storage; Step 6: Change the posture of the mixed reality glasses in step 3 and repeat step 5 no less than 20 times; Step 7: Process all the data in step 6, in which the checkerboard calibration plate coordinate system is obtained through the camera calibration algorithm To the mixed reality glasses coordinate system/> The spatial transformation matrix/> , use it as "hand" data to track the spatial transformation matrix of rigid body 1 to the optical tracking coordinate system/> , treat it as "eye" data, and save "hand" data and "eye" data separately; Step 8: Use the hand-eye calibration algorithm to calibrate the data obtained in step 7 to obtain the tracking rigid body 1 coordinate system To the mixed reality glasses coordinate system/> The spatial transformation matrix/> ; The complete dental virtual model-patient real space registration method includes the following implementation steps: Step 1: Use medical image processing software to segment the complete dentition from the patient's CBCT image and export the STL data of the complete dentition model; Step 2: Import the STL data in step 1 into the mixed reality glasses, and record the data as , three-dimensional display through the visualization system; Step 3: Fix the tracking rigid body 2 on the buccal side of the tooth crown on the side of the patient’s upper (lower) jaw where the anchorage nail is to be implanted; Step 4: Set up the optical tracking device on the side of the patient. The position should ensure that Rigid Body 1 and Rigid Body 2 continue to be in the field of view of the optical tracking device during the tracking process; Step 5: Set the probe tip in the "V"-shaped concave cone tip, hold the probe in spherical motion, and calibrate the probe tip position; Step 6: Use a probe to select 4 distinctive anatomical landmark points on the occlusal surface of the first molar and second molar on the side of the patient's upper (lower) jaw where the anchorage nail is to be implanted, and record them as ; Step 7: Use a probe to trace the buccal surface of the central incisor, lateral incisor and canine on the side of the patient's upper (lower) jaw where the anchorage nail is to be implanted, and save the probe tip data collected by the optical tracking device to The computer is the buccal surface point cloud data of the upper (lower) mandibular central incisors, lateral incisors and canines, and the data is recorded as ; Step 8: The operating doctor wears the mixed reality glasses, and in the visualization system, uses the mouse to click on the anatomical landmark points at the same positions as in step 7 in the same order, which are recorded as ; Step 9: Use the marker point registration algorithm to calculate the and in step 8/> The spatial transformation relationship between them achieves rough registration between the virtual model and the real model; Step 10: Use the iterative closest point algorithm to convert the Same as step 7/> Perform precise registration to obtain the transformation matrix/> , At this point, high-precision registration of the complete dental model-patient's real space is achieved; Step 11: Virtual model coordinate system and mixed reality glasses coordinate system transformation matrix Calculated by the following formula: ; Step 12: According to step 11, the position and posture of the complete dentition virtual model in the visualization system are transformed in real time, so as to complete the visualization of the complete dentition virtual model in the patient's real mouth.