CN114404039B - Tissue drift correction method and device for three-dimensional model, electronic equipment and storage medium - Google Patents

Abstract

The embodiment of the application discloses a tissue drift correction method and device of a three-dimensional model, electronic equipment and a storage medium. The method comprises the following steps: establishing a three-dimensional model based on the acquired medical image data, and extracting a plurality of mark points of a tissue structure in the skull in the three-dimensional model; acquiring spatial position information of a plurality of body surface feature points by using a tracking system, performing rigid registration with a three-dimensional model, and establishing a space-to-model conversion matrix; extracting a plurality of mark points of a tissue structure in the skull in the three-dimensional model, acquiring first space position information of the plurality of mark points, and matching second space position information of the plurality of mark points with the first space position information according to a conversion matrix; calculating the displacement relation between the intra-operation position and the pre-operation position of each mark point to obtain a non-rigid matching relation for drift correction of the three-dimensional model; and correcting the three-dimensional model by using the non-rigid matching relation. The technical scheme of the application can effectively solve the problem that tissue deformation causes three-dimensional digital model drift in operation.

Description

Three-dimensional model tissue drift correction method, device, electronic equipment and storage medium

technical field

The present application relates to the field of medical image processing, in particular to a method, device, electronic device and storage medium for tissue drift correction of a three-dimensional model.

Background technique

As the processing power and efficiency of computer systems have improved, the use of computer-assisted medical procedures has become possible. With the support of computer software and hardware systems, combined with cutting-edge technologies such as digital scanning technology, microsurgery technology and stereotaxic detection technology, the existing technology can realize the precise positioning, feedback and guidance of the position of the scalpel during the operation. Surgical navigation system, this technology is of great significance to minimally invasive surgery or fine surgery (such as neurosurgery).

Taking brain surgery as an example, the existing surgical navigation system generally performs a complete scan of the patient's brain before the operation to establish a three-dimensional digital model in advance. During the operation, through the detection and positioning of the scalpel, the The 3D digital model displays the position of the scalpel and the surrounding brain tissue in real time to help doctors observe the operation process and guide follow-up actions. However, in the actual operation process, when the external protection of the brain tissue (such as dura mater, etc.) is opened, the position and shape of the brain tissue will change due to factors such as changes in intracranial pressure, gravity, and loss of cerebrospinal fluid. The three-dimensional structure of midbrain tissue is quite different from that before operation, and the three-dimensional model established before operation cannot continue to accurately guide the position of the scalpel.

In order to solve this problem, some registration correction methods have appeared in the existing technology. Through the three-dimensional automatic segmentation algorithm, the target tissue (brain tissue) is obtained, and then the segmented brain tissue is gridded, based on the linear elasticity theory. By assigning corresponding biomechanical properties to each grid unit, a physical model of the brain tissue is established, combined with the physical model for finite element calculations, the deformation of any position of the entire brain tissue is obtained, and the three-dimensional model is corrected to reduce errors. Or without considering the cost, intraoperative imaging can also be used to reconstruct the 3D model in real time to achieve high-precision surgical navigation.

However, the inventor found in the process of realizing the related technical solutions of the embodiments of the present invention that at least the following problems exist in the prior art: for the marker points in the 3D model, although someone has already formed the marker points by collecting the sulcus features of the cerebral cortex; But in fact, due to the lack of a clear and clear structure of the gray matter of the cerebral cortex, coupled with individual differences, clinicians report that some people have very shallow brain sulci, and even experienced experts often cannot identify important structures and sulci. It is difficult to obtain reliable landmarks by scanning the sulcus as the target surface; furthermore, when the existing technology uses a biomechanical model for correction, the sulcus landmarks are all located on the surface of the brain tissue due to the large volume and uneven structure of the brain tissue It also cannot provide effective registration support, resulting in poor correction effect. For real-time intraoperative imaging, on the one hand, the cost of related equipment is high, and on the other hand, multiple real-time imaging and modeling during surgery require a lot of time, which seriously affects the efficiency of surgery and increases the risk of surgery, and the actual effect is not ideal.

Contents of the invention

Aiming at the above-mentioned technical problems in the prior art, the embodiment of the present application proposes a tissue drift correction method, device, electronic equipment and computer-readable storage medium for a three-dimensional model to solve the problem of fast and reliable registration of three-dimensional digital images due to intraoperative brain tissue deformation. model problem.

The first aspect of the embodiments of the present application provides a tissue drift correction method for a three-dimensional model, including:

Build a 3D model based on the collected medical image data;

Under the assistance of the tracking system, use the first data acquisition unit to collect the spatial position information of at least three body surface feature points, perform rigid registration with the three-dimensional model, and establish a conversion matrix from real space to the three-dimensional model; the body surface features The point can be the corner of the eye, the tip of the nose, etc., or it can be a bone nail or a marker pasted on the skin surface;

Selecting at least four marker points of the intracranial tissue structure in the three-dimensional model, recording the spatial position information of the marker points in the three-dimensional model as the first spatial position information, and using it with the assistance of a tracking system after the brain tissue is deformed The second data acquisition unit acquires the spatial position information of at least three said marker points again, and converts the spatial position information of said marker points in the three-dimensional model through said transformation matrix as the second spatial position information, and converts said marker points pairing the second spatial position information with the first position information to obtain a non-rigid matching relationship;

The three-dimensional model is calibrated by using the non-rigid matching relationship to obtain a calibrated three-dimensional model.

In some embodiments, the process of acquiring and recording the first spatial position information may be performed after the transformation matrix of the 3D model is established and before the second data acquisition unit acquires the spatial position information of at least three of the marker points conduct.

In some embodiments, the medical imaging data includes one or more of the following: computerized tomography (CT, Computed Tomography), magnetic resonance imaging (MRI, Magnetic Resonance Imaging), X-ray, C-arm, and positive Electron emission computed tomography (PET, Positron Emission Tomography).

In some embodiments, in the method, the three-dimensional model is a three-dimensional model of the brain, and the body surface feature points include facial feature points, such as nose tip, eye corner, etc., markers pasted on the surface, and fixed to the skull. Connected structures, such as bone nails, etc., the internal tissue structures are brain tissue structures, including blood vessel structures.

In some embodiments, the extracting the marker points of the internal tissue structure in the three-dimensional model includes:

The three-dimensional model is processed according to a deep learning algorithm, and features are segmented and extracted to obtain a vascular tissue structure and a plurality of vascular landmarks.

In some embodiments, the method also includes:

The calibrated three-dimensional model is verified using a probe and at least one marker point not used in the process of calibrating the three-dimensional model.

In some embodiments, in the method, the second data acquisition unit uses at least one of a probe, a laser point cloud, and a non-contact laser ultrasonic wave to acquire the spatial position information of the plurality of marker points.

In some embodiments, the vascular structure includes the deep vascular structure of the brain tissue, and the spatial position information of the deep blood vessels in the brain tissue is obtained by using non-contact ultrasound, and then the conversion matrix obtained by registering the body surface feature points is converted into the deep blood vessel Second spatial location information.

The second aspect of the embodiment of the present application provides a three-dimensional model tissue drift correction device, including:

A three-dimensional modeling module for establishing a three-dimensional model based on collected medical imaging data;

The tracking module is used to collect spatial position information under the coordinate system of the tracking system;

A registration module, using the spatial position information of at least three body surface feature points collected by the first data acquisition unit assisted by the tracking module to perform rigid registration with the 3D model, and establish a registration relationship from the actual space to the 3D model;

The model correction module selects at least four marker points of the intracranial tissue structure in the three-dimensional model, records the spatial position information of the marker points in the three-dimensional model as the first spatial position information, and uses the second data acquisition unit to deform the brain tissue After obtaining at least three spatial position information of the marker points again, and converting the spatial position information of the marker points in the three-dimensional model through the transformation matrix as the second spatial position information, the first spatial position information of the marker points The second spatial position information is matched with the first position information to obtain a non-rigid matching relationship, and then the 3D model is corrected using the non-rigid matching relationship.

In some embodiments, in the device, the three-dimensional model is a three-dimensional model of the brain, the body surface feature points are fixed structures and/or devices at the skull and/or face, and the internal tissue structure is a blood vessel structure .

In some embodiments, the three-dimensional modeling module includes:

The marker point extraction module is used to process the three-dimensional model according to the deep learning algorithm, segment and extract features, and obtain the vascular tissue structure and multiple vascular marker points.

In some embodiments, the device also includes:

A verification module, configured to use the probe and/or at least one marker point other than the plurality of marker points to verify the correction matrix and/or the corrected three-dimensional model.

In some embodiments, in the device, at least one of a probe, a laser point cloud, and a non-contact laser ultrasonic wave is used to acquire the spatial position information of the plurality of marker points.

A third aspect of the embodiments of the present application provides an electronic device, including:

memory and one or more processors;

Wherein, the memory is connected in communication with the one or more processors, the memory stores instructions executable by the one or more processors, and the instructions are executed by the one or more processors , the electronic device is used to implement the methods described in the foregoing embodiments.

The fourth aspect of the embodiments of the present application provides a computer-readable storage medium on which computer-executable instructions are stored. When the computer-executable instructions are executed by a computing device, they can be used to implement the above-mentioned embodiments. Methods.

A fifth aspect of the embodiments of the present application provides a computer program product, the computer program product includes a computer program stored on a computer-readable storage medium, the computer program includes program instructions, and when the program instructions are executed by the computer , it can be used to implement the methods described in the foregoing embodiments.

A sixth aspect of the embodiments of the present application provides a surgical navigation system, which includes the above-mentioned three-dimensional model tissue drift correction device, a data acquisition unit, an image processing unit, and a display unit.

In some embodiments, the data acquisition unit includes at least one of X-ray, CT, MRI, conventional ultrasound, probe, laser point cloud, and non-contact laser ultrasound.

The technical solution of the embodiment of the present application improves the consistency of the 3D digital model through two registrations and corrections of internal and external dual tissue landmarks, thereby effectively solving the problem of drift of the 3D digital model caused by intraoperative brain tissue deformation.

Description of drawings

The features and advantages of the present application will be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the application in any way. In the accompanying drawings:

Fig. 1 is a schematic flowchart of a tissue drift correction method for a three-dimensional model according to some embodiments of the present application;

Fig. 2 is a structural block diagram of a three-dimensional model tissue drift correction device according to some embodiments of the present application;

Fig. 3 is a schematic diagram of an electronic device according to some embodiments of the present application.

Detailed ways

In the following detailed description, numerous specific details of the application are set forth by way of example in order to provide a thorough understanding of the relevant disclosure. It will be apparent, however, to one skilled in the art that the application may be practiced without these details. It should be understood that the terms "system", "device", "unit" and/or "module" used in this application are used as a means to distinguish between different components, elements, parts or assemblies at different levels in a sequential arrangement. method. However, these terms may be replaced by other expressions if the same purpose can be achieved by other expressions.

It will be understood that when a device, unit or module is referred to as being "on," "connected to" or "coupled to" another device, unit or module, it can be directly on the other device, unit or module. connected or coupled to or communicate with other devices, units or modules, or intervening devices, units or modules may be present, unless the context clearly suggests an exception. For example, as used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.

The terminology used in the present application is for describing specific embodiments only, and does not limit the scope of the present application. As shown in the specification and claims of this application, words such as "a", "an", "an" and/or "the" do not refer to the singular, and may also include the plural, unless the context clearly indicates an exception. Generally speaking, the terms "comprising" and "comprising" only suggest the inclusion of clearly identified features, integers, steps, operations, elements and/or components, and such expressions do not constitute an exclusive list, other features, Integrals, steps, operations, elements and/or assemblies may also be included.

These and other features and characteristics, methods of operation, functions of relevant elements of structure, combinations of parts, and economies of manufacture of the present application may be better understood with reference to the following description and drawings, which form a part of the manual. However, it can be clearly understood that the drawings are only for the purpose of illustration and description, and are not intended to limit the protection scope of the present application. It is understood that the drawings are not drawn to scale.

Various structural diagrams are used in this application to illustrate various modifications of the embodiments according to this application. It should be understood that the preceding or following structures are not intended to limit the present application. The protection scope of the present application shall be determined by the claims.

The existing surgical navigation system cannot effectively solve the problem that intraoperative tissue deformation affects the accuracy of the pre-built 3D digital model, especially because of the large volume and uneven structure of the brain tissue, the existing sulcus feature mark point method cannot achieve a good result. correction effect. In view of this, the embodiment of the present application provides a tissue drift correction method for a 3D model, which improves the consistency of the 3D digital model through two registrations and corrections of internal and external dual tissue landmarks, thereby effectively solving the problem of intraoperative brain tissue drift. Deformation causes the problem of 3D digital model drift. In one embodiment of the present application, as shown in FIG. 1, the method for correcting tissue drift of a three-dimensional model includes steps:

S101, building a three-dimensional model based on medical imaging data (CT, MRI, PET, etc.) containing brain information;

S102, using a tracking system (probe, laser point cloud) to collect spatial position information of at least three body surface feature points, registering with the three-dimensional model, and establishing a transformation matrix from real space to the three-dimensional model;

S103, select at least four marker points of the intracranial tissue structure in the three-dimensional model; the selected point cloud may also be a point cloud of the intracranial tissue structure, and record the first marker point in the tracking system or the three-dimensional model A spatial position information, after the brain tissue is deformed, use a tracking system (probe, laser point cloud) to obtain at least three three-dimensional spatial position information of the marker points again, and convert the marker points into The spatial position information in the three-dimensional model is used as the second spatial position information, and the second spatial position information of the marker point is paired with the first position information in the tracking system or the three-dimensional model to obtain a non-rigid matching relationship;

S104. Calibrate the 3D model by using the non-rigid matching relationship to obtain a calibrated 3D model. Wherein, in the embodiments of the present application, the three-dimensional model is preferably a three-dimensional model of the brain, and the body surface feature points are preferably feature structures and/or devices on the skull and/or face, such as the corners of the eyes, nose or ( implanted) bone nails, etc., the internal tissue structure is preferably a blood vessel structure. Therefore, the technical solution of this application is preferably applied to the surgical navigation of brain surgery. By using two registrations and corrections of external tissue and internal tissue to improve the consistency between the three-dimensional digital model and the actual tissue structure, it solves the problem of brain surgery during surgery. The large deformation of the three-dimensional structure of the tissue affects the accuracy of the three-dimensional digital model and reduces the performance of surgical navigation.

Optionally, the non-rigid registration adopts the position information method based on the three-dimensional model for registration, so that the information corresponding to the blood vessel structure corresponding to the three-dimensional model is consistent with the information corresponding to the blood vessel structure after the brain drift,

Optionally, the non-rigid registration process includes:

Extract the feature points of the brain vascular structure, obtain the first spatial position information of the feature points, use these feature points as the source points, and form the source surface with the edges,

Then obtain the second spatial position information of these feature points as sampling points to form a target surface;

Use a local affine transformation on the source surface to align it with the blood vessels after brain drift.

Optionally, the expression mode of the non-rigid registration result is a deformation graph, specifically including: an affine matrix corresponding to each vertex.

Optionally, in the process of affine transformation of the deformation map, factors that need to be considered include: alignment error, transformation regularization of the entire vascular structure, and a deviation between the transformation matrix and the rotation matrix,

The impact result of the affine transformation process corresponding to the alignment error is registration validity;

The influence result of the affine transformation process corresponding to the transformation regularization of the overall vascular structure is local consistency;

The influence result of the affine transformation process corresponding to the deviation between the transformation matrix and the rotation matrix is local rigidity.

Optionally, the method for obtaining the affine transformation result of the deformation map includes: performing weighting processing on the consideration factors except the alignment error according to actual requirements, and summing up all the weighted consideration factors and the alignment error , the resulting minimum value is taken as the affine transformation result of the transformed graph,

The considerations other than alignment error include: transformation regularization of the source surface as a whole and deviations between transformation matrices and rotation matrices.

Optionally, the source surface supports a local affine transformation process, and the specific operation process is as described above.

Further, in a preferred embodiment of the present application, in the above step S101, the three-dimensional model is processed according to a deep learning algorithm, and features are segmented and extracted to obtain a vascular tissue structure and a plurality of vascular landmarks. In the preferred embodiment of the present application, the brain blood vessel landmarks are mainly used as the basis for correction. Since blood vessels have the characteristics of wide distribution and high degree of differentiation from surrounding tissues, the method in the embodiment of the present application is highly robust.

Optionally, in the above step S101, the process of obtaining the vascular tissue structure and multiple vascular marker points can also be performed in the above step S103,

The execution order of the process of obtaining the vascular tissue structure and the plurality of vascular marker points is selected according to the actual situation.

Optionally, the process of segmenting and extracting features uses a deep learning model based on 3D Attention U-Net to segment blood vessels;

3D Attention U-Net uses skip-connection in structure, so that more detailed features (low-level) in the final prediction map are preserved and fused, and the quality of image details is preserved;

And through the attention mechanism, it is possible to better focus on blood vessels rather than the background (air or other tissues, etc.), so that the result of the prediction map is more accurate.

Optionally, the Models Genesis model is selected for pre-training, and the Models Genesis improves the learning effect of the spatial features of the vascular structure, so that the model adapts to the vascular segmentation task and improves the effect.

Optionally, the pre-training process includes:

The Models Genesis performs image transformation on the input 3D image, and inputs the 3DAttention U-Net model;

The 3D Attention U-Net restores the transformed 3D image and uses it for training.

Optionally, the 3D image content includes brain tissue anatomy, grayscale distribution and spatial structure of blood vessels.

Optionally, the image transformation content of the 3D image includes: nonlinear transformation, local pixel reorganization, and image edge blurring.

Optionally, the 3D Attention U-Net training content includes: brain tissue anatomy, grayscale distribution, and spatial structure of blood vessels.

Preferably, in the embodiment of the present application, at least one of a probe and a laser point cloud is used to acquire the spatial position information of the plurality of marker points. For body surface feature points, its spatial position information is relatively easy to collect, which can be obtained by real-time detection on the surface using probes and laser point clouds; for internal tissue structure mark points, if they are only located on the surface, laser point clouds can also be used to detect them on the surface. Surface detection, and then obtain landmarks and their spatial location information (because human tissue itself has radians, the spatial location information of these surface landmarks can also include information in multiple directions).

Optionally, the feature recognition of the blood vessel is realized by scanning the impact, and the recognition feature of the blood vessel includes: blood vessel color, blood vessel continuity, and blood vessel boundary.

Optionally, the mask of the blood vessel is obtained through image processing, so as to obtain the point cloud of the blood vessel.

However, in the preferred embodiment of the present application, the marker points can also be deep in the tissue. These surface detection methods cannot obtain ideal spatial position information in the deep tissue. If large-scale equipment (such as X-ray machines, CT machines, MRI etc.), it is very expensive during the operation; in order to avoid adverse effects on the operation, even connected contact detection means (such as B-ultrasound, color Doppler ultrasound, etc.) should not be used. In order to solve this problem, in a preferred embodiment of the present application, non-contact laser ultrasound is used to collect spatial position information of deep tissue marker points. Among them, the principle of the laser ultrasonic method is as follows. It sends a laser pulse of a specific wavelength to the human body, penetrates the skin and allows the blood vessel tissue to absorb; the blood vessel tissue is rapidly expanded and relaxed by the laser heating, and then quickly recovered by body temperature cooling, and repeats when the next pulse arrives. This process, and the resulting mechanical vibrations form sound waves. On this basis, another laser wave of the same wavelength is further designed to receive the acoustic signal returned from the human body at a certain distance to complete the imaging. In this way, laser ultrasonic technology can scan from within half a meter of the human body to obtain information on tissues such as muscles, fat, and bones inside the human body. This method can penetrate at least 6 cm below the skin and can be used wirelessly The method of contact can effectively detect the structure of internal deep tissue, thus making it possible to correct the deformation of deep brain tissue.

In addition, in order to further improve the accuracy of the surgical navigation system, in a preferred embodiment of the present application, the correction matrix and/or correction The final 3D model is verified. Wherein, using a probe to verify includes: using a probe to detect at least one verification point on the surface or superficial surface of the internal tissue structure (any point except the mark point that has been used for registration), according to the verification The consistency of the point's spatial position information and its position information in the corrected three-dimensional model verifies the reliability of the calibration matrix and/or the corrected three-dimensional model. Checking using at least one marker point other than the plurality of marker points includes: extracting at least one marker point that has not yet been used, detecting its spatial position information, and comparing the position information in the corrected three-dimensional model according to its spatial position information. The consistency of the real position checks the reliability of the correction matrix and/or the corrected three-dimensional model.

In the embodiment of this application, the initial registration of body surface feature points is rigid registration, which has less deformation before and during operation, and is easier to identify and process. The initial registration is mainly used to determine the preoperative registration relationship . However, the internal tissue structure landmarks such as blood vessels have large deformations before and during the operation, and there is uncertainty. The secondary registration to calculate the displacement relationship is non-rigid registration, which is used to determine the degree of tissue drift of the real deformation. The 3D model is transformed using the correction matrix T2 to obtain a new 3D model, so that the transformation relationship T1 can be used for navigation.

Fig. 2 is a schematic diagram of a device for correcting tissue drift of a three-dimensional model according to some embodiments of the present application. As shown in Figure 2, the device 200 includes:

A three-dimensional modeling module 210, configured to establish a three-dimensional model based on collected medical image data;

The tracking module 220 is configured to collect spatial position information in the tracking system coordinate system; the registration module 230 uses the spatial position information of at least three body surface feature points collected by the first data acquisition unit with the assistance of the tracking module and the three-dimensional The model is rigidly registered, and the registration relationship between the actual space and the 3D model is established;

The model correction module 240 selects at least four marker points of the intracranial tissue structure in the three-dimensional model, records the spatial position information of the marker points in the three-dimensional model as the first spatial position information, and uses the second data acquisition unit to record the spatial position information in the brain tissue Obtaining the second spatial position information of at least three marker points again after deformation, matching the second spatial position information of the marker points with the first position information to obtain a non-rigid matching relationship, and then using the non-rigid Rigid matching relations are used to modify the three-dimensional model. Among them, the tissue drift correction method and device of the 3D model in the embodiment of the present application are mainly used in the surgical navigation system, and the surgical navigation system obviously also includes some hardware units such as collection, processing, projection and imaging modules of medical image data, These hardware units can generally be realized by using existing hardware units in the prior art. Since the technical solution of the present application mainly improves the correction process of the 3D model, these existing hardware units will not be described one by one here.

In some embodiments, in the device, the three-dimensional model is a three-dimensional model of the brain, the body surface feature points are fixed structures and/or devices at the skull and/or face, and the internal tissue structure is a blood vessel structure .

In some embodiments, the three-dimensional modeling module includes:

The marker point extraction module is used to process the three-dimensional model according to the deep learning algorithm, segment and extract features, and obtain the vascular tissue structure and multiple vascular marker points.

In some embodiments, the device also includes:

A verification module, configured to use the probe and/or at least one marker point other than the plurality of marker points to verify the correction matrix and/or the corrected three-dimensional model.

In some embodiments, in the device, at least one of a probe, a laser point cloud, and a non-contact laser ultrasonic wave is used to acquire the spatial position information of the plurality of marker points.

Referring to FIG. 3 , it is a schematic diagram of an electronic device provided by an embodiment of the present application. As shown in Figure 3, the electronic device 500 includes:

memory 530 and one or more processors 510;

Wherein, the memory 530 is communicatively connected with the one or more processors 510, the communication bus 540, and the memory 530 stores program instructions 532 executable by the one or more processors, and the program instructions Step 532 is executed by the one or more processors 510, so that the one or more processors 510 execute the steps in the foregoing method embodiments. Further, the electronic device 500 can also interact with external devices through the communication interface 520 .

In some embodiments, the positioning navigation system may be an optical surgical navigation system, an electromagnetic navigation system, etc.;

The optical surgical navigation system mainly includes: an infrared surgical navigation system and a visible light surgical navigation system;

The infrared optical surgical navigation system includes: a host computer, an infrared tracking camera system, and a probe with a positioning marker.

In some embodiments, the infrared optical surgical navigation system can be used for three-dimensional model correction, specifically including:

The host receives the medical image and builds a three-dimensional model;

Using a probe to collect at least three body surface feature points fixedly connected to the skull surface, and obtaining the spatial position of the body surface feature points in the coordinate system of the infrared tracking camera system;

The infrared optical surgical navigation system registers the spatial position of the bone nail with the three-dimensional model, and establishes a one-to-one conversion matrix between the actual space and the three-dimensional model;

Selecting at least four bifurcations of blood vessels in the brain tissue as landmark points, and displaying their positions in the three-dimensional model as the first spatial position information;

After the brain tissue is deformed, the probe collects the spatial positions of at least three marker points after the brain tissue deformation, and converts them into the three-dimensional model according to the transformation matrix, and the deformed brain tissue Converting the marker point to the coordinate position in the three-dimensional model as the second spatial position information;

The host establishes a non-rigid matching relationship according to the first spatial position information and the second spatial position information of the marker point;

The host then calibrates the three-dimensional model through the established non-rigid matching relationship, so that the three-dimensional model is consistent with the deformed brain tissue structure.

Optionally, the body surface feature points include: corners of eyes, tip of nose, bone nails, and markers pasted on the surface of the skin.

Optionally, it is also possible to use other marker points not used in establishing the non-rigid matching relationship for verification, record the theoretical coordinate positions of the marker points in the three-dimensional model, and then use the probe to track the infrared camera Collect the spatial position of the marker point under the tracking of the system, use the transformation matrix to calculate the position in the three-dimensional model coordinate system as the actual coordinate position, and then calculate the difference between the theoretical position coordinate position and the actual coordinate position, as Evaluate the parameters of the non-rigid matching relationship.

An embodiment of the present application provides a computer-readable storage medium, wherein computer-executable instructions are stored in the computer-readable storage medium, and each step in the foregoing method embodiments is executed after the computer-executable instructions are executed.

Those skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the above-described devices and modules can refer to the corresponding descriptions in the foregoing method and/or device embodiments, and details are not repeated here.

Although the subject matter described herein is presented in the general context of being executed in connection with the execution of operating systems and application programs on a computer system, those skilled in the art will recognize that other types of program modules can also be used to execute other programs. accomplish. Generally, program modules include routines, programs, components, data structures, and other types of structures that perform particular tasks or implement particular abstract data types. Those skilled in the art will appreciate that the subject matter described herein may be practiced using other computer system configurations, including handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, etc. , can also be used in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

Those skilled in the art can appreciate that the units and method steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present application.

If the functions described above are realized in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned computer-readable storage medium includes physically volatile and non-volatile, removable and non-removable media implemented in any manner or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Indong medium. Computer-readable storage media specifically include, but are not limited to, U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), erasable programmable read-only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Flash or other solid-state memory technology, CD-ROM, Digital Versatile Disk (DVD), HD-DVD, Blue-Ray or other optical storage device, tape, disk storage or other magnetic storage device, or any other medium that can be used to store the desired information and that can be accessed by a computer.

In summary, the present disclosure proposes a tissue drift correction method, device, electronic device and computer-readable storage medium for a three-dimensional model. Through the technical solution of the embodiment of the present application, the consistency of the three-dimensional digital model is improved through two registrations and corrections of internal and external double tissue marker points, thereby effectively solving the problem of drift of the three-dimensional digital model caused by intraoperative brain tissue deformation.

It should be understood that the above specific implementation manners of the present application are only used to illustrate or explain the principle of the present application, but not to limit the present application. Therefore, any modification, equivalent replacement, improvement, etc. made without departing from the spirit and scope of the present application shall fall within the protection scope of the present application. Furthermore, the claims appended to this application are intended to embrace all changes and modifications that come within the scope and metes and bounds of the appended claims, or equivalents of such scope and metes and bounds.

Claims (13)

1. A computer-readable storage medium, characterized in that computer-executable instructions are stored on the computer-readable storage medium, and when the computer-executable instructions are executed by a computing device, they can be used to realize the organization of a three-dimensional model Drift correction methods, including: Build a 3D model based on medical imaging data including head information; Using the first data collection unit to collect spatial position information of at least three body surface feature points with the assistance of the tracking system, registering with the three-dimensional model, and establishing a transformation matrix from real space to the three-dimensional model; Selecting at least four marker points of the intracranial tissue structure in the three-dimensional model, recording the spatial position information of the marker points in the three-dimensional model as the first spatial position information, and using it with the assistance of a tracking system after the brain tissue is deformed The second data acquisition unit acquires the spatial position information of at least three marker points again, and converts the spatial position information of the marker points in the three-dimensional model through the conversion matrix as the second spatial position information, and The second spatial position information of the marker point is paired with the first spatial position information to obtain a non-rigid matching relationship; The three-dimensional model is calibrated by using the non-rigid matching relationship to obtain a calibrated three-dimensional model. 2 . The computer-readable storage medium according to claim 1 , wherein in the method for correcting tissue drift of the three-dimensional model, the intracranial tissue structure includes a blood vessel structure. 3 . 3 . The computer-readable storage medium according to claim 1 , wherein the landmarks include: intracranial specific anatomical structures and blood vessel landmarks that can be identified by doctors. 4 . 4. The computer-readable storage medium according to claim 1, wherein the method further comprises: The calibrated three-dimensional model is verified using a probe and at least one marker point not used in calibrating the three-dimensional model. 5. The computer-readable storage medium according to claim 1, wherein the second data acquisition unit uses at least one of a probe, a laser point cloud, and a non-contact laser ultrasonic wave to acquire the marker points spatial location information. 6 . The computer-readable storage medium according to claim 2 , wherein the vascular structure comprises deep brain tissue vascular structure, and non-contact laser ultrasound is used to obtain position information of marker points of deep brain tissue vascular structure. 7 . 7. A tissue drift correction device for a three-dimensional model, characterized in that it comprises: A three-dimensional modeling module for establishing a three-dimensional model based on medical image data including head information; The tracking module is used to collect the spatial position information under the coordinate system of the tracking system, The registration module uses the spatial position information of at least three body surface feature points collected by the first data acquisition unit assisted by the tracking module to perform rigid registration with the 3D model, establishes a conversion matrix from the actual space to the 3D model, and records the markers Point spatial location information; The model correction module selects at least four marker points of the intracranial tissue structure in the three-dimensional model, records the spatial position information of the marker points in the three-dimensional model as the first spatial position information, and uses the second data acquisition unit to deform the brain tissue After obtaining at least three second spatial position information of the marker points again, matching the second spatial position information of the marker points with the first spatial position information to obtain a non-rigid matching relationship, and then using the non-rigid Rigid matching relations are used to modify the three-dimensional model. 8. The device according to claim 7, further comprising: The marker point extraction module is used to process the three-dimensional model according to the deep learning algorithm, segment and extract features, and obtain the vascular tissue structure and the vascular marker points. 9. The device according to claim 7, further comprising: A verification module, configured to use the probe and/or at least one marker point that has not been used by the registration module to verify the calibration matrix and/or the corrected three-dimensional model. 10 . The device according to claim 7 , wherein, in the device, at least one of a probe, a laser point cloud, and a non-contact laser ultrasonic wave is used to obtain the second spatial position information of the marker point. 11 . 11 . The device according to claim 7 , wherein the vascular structure further includes the deep vascular structure of the brain tissue, and the spatial position information of the marker points of the deep vascular structure of the brain tissue is obtained by using non-contact laser ultrasound. 12. A surgical navigation system, characterized in that the surgical navigation system comprises the three-dimensional model tissue drift correction device according to any one of claims 7-11. 13. A surgical robot system, characterized in that the surgical robot system comprises the three-dimensional model tissue drift correction device according to any one of claims 7-11.

Images