CN114652443A - Ultrasonic operation navigation system and method, storage medium and device - Google Patents

Abstract

The embodiment of the invention discloses an ultrasonic operation navigation system and method, a storage medium and equipment, wherein the ultrasonic operation navigation system comprises: the system comprises an imaging system, a tracking and positioning system, a surgical instrument and a first marker for contacting with the body surface. The preoperative image data and the preoperative physical space position are used for obtaining a registration result before operation, the preoperative image data is used for obtaining a target planning path, and then the target operation path and the registration result are used for intraoperative operation guidance. Through the ultrasonic operation navigation system, the real-time position of the surgical instrument in the operation can be guided by using the target operation path obtained by the preoperative image data, the space position of the surgical instrument is determined without carrying out multiple times of X-ray perspective imaging in the operation, the radiation injury can be reduced, the consumed time is reduced, the accuracy is improved, the real-time relative position relation between the surgical instrument and the surrounding key anatomical structures can be determined by adopting the mode for guiding.

Description

Ultrasound surgical navigation system and method, storage medium and device

technical field

The present invention relates to the technical field of medical devices, in particular to an ultrasonic surgical navigation system and method, a storage medium and equipment.

Background technique

Existing puncture needle tracking technologies in interventional operations are generally divided into two categories: one is to use medical image processing methods to extract relevant features from fluoroscopic images obtained by X-rays to calculate the position of puncture interventional instruments. In order to maintain the real-time and validity of information, it is usually necessary to collect X-ray images multiple times, which results in high radiation dose and time-consuming, resulting in insufficient surgical accuracy. One of the reasons is that the system is difficult to adapt to the complex nonlinear deformation of puncture needles and other instruments. The second reason is that the X-ray image is two-dimensional image data, resulting in the lack of three-dimensional information of the extracted puncture needle, which cannot accurately reflect the spatial relationship between the interventional device and the surrounding important tissues and organs.

The other is to integrate a tracking locator on the puncture needle or ultrasonic probe to directly obtain the real-time position of the interventional instrument during surgery. This method only obtains the real-time position of the interventional instrument, and does not provide the real-time relative positional relationship between it and the surrounding blood vessels and other key anatomical structures, so it cannot provide an accurate surgical reference.

SUMMARY OF THE INVENTION

The main purpose of the present invention is to provide an ultrasonic surgical navigation system and method, storage medium and equipment, which can solve the technical problem of tracking the puncture needle in interventional surgery in the prior art

In order to achieve the above object, a first aspect of the present invention provides an ultrasonic surgical navigation system, the ultrasonic surgical navigation system includes: an imaging system, a tracking and positioning system, a surgical instrument, and a first marker for contacting the body surface;

The imaging system is used to acquire preoperative image data, so as to display the preoperative image, and acquire the preoperative image space point set corresponding to the first marker;

The tracking and positioning system is used to obtain preoperative tracking data of the first marker, so as to obtain the preoperative physical space position of the first marker in the preoperative tracking data;

The ultrasonic surgical navigation system is used for registering the preoperative image space and the preoperative physical space according to the preoperative image space point set and the preoperative physical space position, and determining the registration result; using the preoperative image The data is used for surgical path planning, and the target surgical path is determined; the surgical guidance of the surgical instrument is performed according to the target surgical path, the registration result, the intraoperative image data and the intraoperative physical space position.

In a feasible implementation manner, the imaging system includes an ultrasound probe, and the ultrasound probe is provided with a second marker;

The ultrasonic probe is used for acquiring intraoperative image data;

The tracking and positioning system is also used to obtain the intraoperative physical space position of the second marker;

The ultrasonic navigation system is further configured to perform image fusion display using the registration result and the intraoperative physical space position of the second marker, so as to determine the intraoperative ultrasonic imaging plane corresponding to the ultrasonic probe in the preoperative image space. The image fusion display includes the fusion display of the intraoperative ultrasound image and the preoperative image.

In a feasible implementation manner, the surgical instrument is provided with a third marker;

The tracking and positioning system is also used to obtain the intraoperative physical space position of the third marker;

The ultrasound navigation system is further configured to use the registration result and the intraoperative physical spatial position of the third marker to determine a preoperative image spatial position corresponding to the intraoperative physical spatial position, so as to display the preoperative image in the preoperative image. The intraoperative spatial posture of the surgical instrument is determined in space.

In a feasible implementation manner, the ultrasonic surgical navigation system determines the registration result, including:

Using the iterative closest point algorithm, the preoperative image space point set and the preoperative physical space position are registered, and the target transformation matrix corresponding to the preoperative image space and the preoperative image space registration is determined;

The target transformation matrix is used as the registration result.

In one possible implementation, the marker comprises a magnetic marker or an optical marker.

In a feasible implementation manner, the tracking and positioning system includes a magnetic tracking system or an optical tracking system.

In a feasible implementation manner, the preoperative images include tomography images or magnetic resonance images.

In order to achieve the above object, a second aspect of the present invention provides an ultrasonic surgical navigation method, the method is applied to the ultrasonic surgical navigation system according to the first aspect and any feasible implementation manner, and the method includes:

Obtain the preoperative image space point set corresponding to the first marker;

obtaining the preoperative physical space position corresponding to the first marker;

According to the preoperative image space point set and the preoperative physical space position, the registration of the preoperative image space and the preoperative physical space is performed to determine the registration result;

Use the preoperative image data to perform surgical path planning to determine a target surgical path;

The surgical guidance of the surgical instrument is performed according to the target surgical path, the registration result, the intraoperative image data and the intraoperative physical space position.

To achieve the above object, a third aspect of the present invention provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the processor causes the processor to perform the steps shown in the second aspect.

In order to achieve the above object, a fourth aspect of the present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes as described in Section 1. Steps shown in the second aspect.

Adopting the embodiment of the present invention has the following beneficial effects:

The invention provides an ultrasonic surgical navigation system. The ultrasonic surgical navigation system includes: an imaging system, a tracking and positioning system, surgical instruments, and a first marker for contacting the body surface; the imaging system is used for acquiring preoperative image data to The preoperative image is displayed, and the preoperative image space point set corresponding to the first marker is obtained; the tracking and positioning system is used to obtain the preoperative tracking data of the first marker, so as to obtain the first marker in the preoperative tracking data the preoperative physical space position; the ultrasonic surgical navigation system is used to register the preoperative image space with the preoperative physical space according to the preoperative image space point set and the preoperative physical space position to determine the registration result; use the preoperative image data Carry out surgical path planning to determine the target surgical path; perform surgical guidance of surgical instruments according to the target surgical path, registration results, intraoperative image data and intraoperative physical space position. Through the above-mentioned ultrasonic surgical navigation system, the target surgical path using the preoperative image data can be used to guide the real-time position of the surgical instrument during the operation, and there is no need to perform multiple X-ray fluoroscopic imaging during the operation to determine the spatial position of the surgical instrument. It can greatly reduce radiation damage to patients and doctors, and does not need to perform intermittent multiple imaging, reducing time-consuming and improving surgical accuracy, using the target surgical path, registration results, intraoperative image data and intraoperative physical space. The position is guided and displayed, and the real-time relative positional relationship between the surgical instrument and the surrounding key anatomy can also be determined.

Description of drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts.

in:

1 is an application environment diagram of an ultrasonic surgical navigation system in an embodiment of the present invention;

2 is a structural block diagram of an ultrasonic surgical navigation system according to an embodiment of the present invention;

3 is a flowchart of an ultrasonic surgical navigation method according to an embodiment of the present invention;

4 is another structural block diagram of an ultrasonic surgical navigation system according to an embodiment of the present invention;

5 is another flowchart of an ultrasonic surgical navigation method according to an embodiment of the present invention;

FIG. 6 is a structural block diagram of a computer device in an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Please refer to FIG. 1 . FIG. 1 is an application environment diagram of an ultrasonic surgical navigation system according to an embodiment of the present invention. As shown in FIG. 1 , the application environment includes: a patient 10 , an operating bed 11 , a first marker 12 , and a tracking and positioning system 13 , an imaging workstation 14 , an ultrasound probe 15 , a second marker 16 , a surgical instrument 17 , and a third marker 18 . The imaging workstation 14 and the ultrasound probe 15 constitute the imaging system in the embodiment of the present invention, and the imaging workstation 14 may be a visual electronic device for performing real-time imaging display according to image data. The ultrasound probe is used to scan the body surface of the patient 10 for ultrasound imaging. The first marker is used to contact the body surface of the patient; the second marker 16 is arranged on the ultrasound probe 15 to reflect the spatial position of the ultrasound probe; the third marker is arranged on the surgical instrument to reflect the position of the surgical instrument. Spatial position; the tracking and positioning system 13 is used to track each marker. It should be noted that the first marker, the second marker, and the third marker have different identification marks, so as to distinguish multiple markers located at different positions. markers to differentiate. The surgical instrument may be an interventional instrument, such as a puncture needle.

Please refer to FIG. 2 . FIG. 2 is a structural block diagram of an ultrasonic surgical navigation system according to an embodiment of the present invention, which provides an ultrasonic surgical navigation system 20 . The system can be applied to both the terminal and the server, and this embodiment is described by taking the application to the terminal as an example. The ultrasonic surgical navigation system 20 specifically includes: an imaging system 201, a tracking and positioning system 202, a surgical instrument 203, and a first marker 204 for contacting the body surface;

The imaging system 201 is used to acquire preoperative image data, so as to display the preoperative image, and acquire the preoperative image space point set corresponding to the first marker; it should be noted that the preoperative image is a three-dimensional image, and the preoperative image is Image data is obtained by performing tomography or MRI scans on patients, or other imaging modalities suitable for application scenarios. The preoperative impact data can be tomography data or nuclear magnetic resonance data. The preoperative image may be a three-dimensional tomographic image obtained by using tomography data, or a three-dimensional magnetic resonance scan image obtained by using nuclear magnetic resonance data. And obtain the preoperative image space point set corresponding to the first marker in the obtained preoperative image, that is, the spatial position (coordinate) of the first marker in the preoperative image coordinate system. For the image data, the UNet deep learning neural network is used to perform network training on the preoperative image data including the first marker, so as to realize the automatic extraction of the spatial position of the marker, and obtain the preoperative image space point set corresponding to the first marker. The preoperative image space point set includes real-time preoperative image space position coordinates of several first markers.

Wherein, the tracking and positioning system 202 is used to obtain the preoperative tracking data of the first marker 204, so as to obtain the preoperative physical space position of the first marker in the preoperative tracking data; wherein, the preoperative physical space position is the first marker The preoperative physical space position (coordinates) of the object in the coordinate system corresponding to the tracking and positioning system, for example, the patient is lying on the operating bed in the operating room, while keeping the position of the first marker 204 pasted on the body surface unchanged ( The same as in three-dimensional imaging); the doctor turns on the tracking and positioning system, and the navigation software system will automatically record the physical space position of the first marker 204 on the patient's body surface, so as to obtain the preoperative physical space position of the first marker. The physical space position is a coordinate set, and the coordinate set includes real-time preoperative physical space position coordinates of several first markers. Wherein, at least three first markers 204 for contacting with the body surface are provided.

Further, the ultrasonic surgical navigation system is used to perform the registration of the preoperative image space and the preoperative physical space according to the preoperative image space point set and the preoperative physical space position to determine the registration result; use the preoperative image data to plan the surgical path, Determine the target surgical path; perform surgical guidance of the surgical instrument 203 according to the target surgical path, the registration result, the intraoperative image data, and the intraoperative physical space position.

It should be noted that the ultrasonic surgical navigation system is interconnected with the imaging system and the tracking and positioning system, which may be wireless or wired. The ultrasonic surgical navigation system can be implemented by an electronic device with processing capabilities. Not limited. Specifically, the registration of the preoperative image space and the preoperative physical space refers to aligning the preoperative image space with the preoperative physical space, that is, converting the two coordinate system spaces into the same coordinate system space. In this embodiment, It can be the conversion of the preoperative physical space into the preoperative image space, or the conversion of the preoperative image space into the preoperative physical space, which is not limited here. And use the preoperative image data to plan the operation path to determine the target operation path. The operation path planning includes the needle entry point and the puncture target point, and then define the puncture intervention path, that is, the target operation path according to the needle entry point and the target point. Finally, the surgical guidance of the surgical instrument 203 can be performed according to the target surgical path, the registration result, the intraoperative image data, and the intraoperative physical space position, so as to provide the doctor with a reference for the details of the intraoperative operation. The registration result is used to reflect the mapping relationship between the two coordinate systems for automatic alignment of the two coordinate spaces.

Exemplarily, to make the implementation of this implementation clearer, the following process is used for description:

First, medical imaging tomography equipment (imaging system) (for example: Computed Tomography (CT), Nuclear Magnetic Resonance Imaging (MRI), etc.) is used to obtain the tomographic image of the patient's lesion; The three-dimensional spatial position of the previous image, the first marker with a magnetic sensor is pasted on the surface of the patient's chest and abdomen; the obtained preoperative tomographic images are segmented, three-dimensional surface reconstruction, rendering and visualization are performed on the surrounding organs of the puncture target. And according to the results of the three-dimensional reconstruction, the detailed planning of the preoperative surgical path (target surgical path) is carried out. For the acquired diagnostic image data (preoperative image), the UNet deep learning neural network is used to train the image of the magnetic marker, so as to realize the automatic extraction of the image space position of the first marker, and obtain the preoperative image space point set. Denoted as: X _Image ={x _i ,i∈n}, X _Image is the preoperative image space point set, _xi is the position coordinate of the first marker in the preoperative image space at a certain moment before the operation, and n is point in time.

Secondly, the patient is lying on the operating bed in the operating room, while keeping the position of the magnetic marker (hundredth marker) pasted on the body surface unchanged; the doctor turns on the magnetic tracking system (tracking and positioning system), and the navigation software system will automatically record The preoperative physical space position of the magnetic marker on the patient's body surface: Y _Physical ={y _i ,i∈n}, where Y _Physical is the preoperative physical space position (that is, the set of physical space positions), and y _i is the preoperative physical space position The coordinates of the position corresponding to the first marker in the preoperative physical space at a certain moment, and n is the time point.

Then, through the obtained image of the magnetic marker on the patient's body surface and the two sets of point sets in the physical space, the registration process is performed to obtain the registration result, and the automatic registration of the two sets of point sets is automatically completed, so as to realize the preoperative operation. - Fusion guidance of intraoperative images. Finally, the surgical guidance of the surgical instrument 203 can be performed through the target surgical path, the registration result, the intraoperative image data, and the intraoperative physical space position, so as to guide the surgeon to perform safe and accurate surgery.

The invention provides an ultrasonic surgical navigation system. The ultrasonic surgical navigation system includes: an imaging system, a tracking and positioning system, surgical instruments, and a first marker for contacting the body surface; the imaging system is used for acquiring preoperative image data to The preoperative image is displayed, and the preoperative image space point set corresponding to the first marker is obtained; the tracking and positioning system is used to obtain the preoperative tracking data of the first marker, so as to obtain the first marker in the preoperative tracking data the preoperative physical space position; the ultrasonic surgical navigation system is used to register the preoperative image space with the preoperative physical space according to the preoperative image space point set and the preoperative physical space position to determine the registration result; use the preoperative image data Carry out surgical path planning to determine the target surgical path; perform surgical guidance of surgical instruments according to the target surgical path, registration results, intraoperative image data and intraoperative physical space position. Through the above-mentioned ultrasonic surgical navigation system, the target surgical path using the preoperative image data can be used to guide the real-time position of the surgical instrument during the operation, and there is no need to perform multiple X-ray fluoroscopic imaging during the operation to determine the spatial position of the surgical instrument. It can greatly reduce radiation damage to patients and doctors, and does not need to perform intermittent multiple imaging, reducing time-consuming and improving surgical accuracy, using the target surgical path, registration results, intraoperative image data and intraoperative physical space. The position is guided and displayed, and the real-time relative positional relationship between the surgical instrument and the surrounding key anatomy can also be determined.

As shown in FIG. 3 , in one embodiment, an ultrasonic surgical navigation method is provided, and the ultrasonic surgical navigation method is applied to an ultrasonic surgical navigation system. The system can be applied to both a terminal and a server. This embodiment Take the application to the terminal as an example. , the method specifically includes the following steps:

301. Obtain a preoperative image space point set corresponding to the first marker;

302. Obtain the preoperative physical space position corresponding to the first marker;

It should be noted that steps 301 and 302 are to obtain the preoperative image space point set and the preoperative physical space position corresponding to the first marker, and through steps 301 and 302, the coordinate data corresponding to the two preoperative coordinate systems can be obtained , the acquisition method can be processed by the imaging system and the tracking and positioning system respectively, and directly obtained by the ultrasonic surgical navigation system. Then, the preoperative image space point set and the preoperative physical space position corresponding to the first marker are extracted, which are not limited here. It can be understood that the imaging system and the tracking and positioning system respectively have communication links with the ultrasonic surgical navigation system, so data sharing can be realized.

The preoperative image space point set is the set of position coordinates at each moment corresponding to the first marker in the preoperative image data in the preoperative three-dimensional image space, and the preoperative image data may be tomography data or nuclear magnetic resonance data. The pre-image data may be obtained by performing a tomography scan or an MRI scan of the patient. Specifically, the preoperative image space point set corresponding to the first marker can be expressed as X _Image ={x _i ,i∈n}, where X _Image is the preoperative image space point set, and _xi is the point set at a certain moment before the operation. The position coordinates of a marker in the preoperative image space, and n is the time point. The preoperative image data including the first marker can be automatically extracted by using the UNet deep learning neural network to perform network training on the acquired preoperative image data, and the preoperative image corresponding to the first marker can be obtained. Image space point set. The preoperative image space point set includes real-time preoperative image space position coordinates of several first markers.

Wherein, the preoperative physical space position is the preoperative physical space position (coordinates) of the first marker in the coordinate system corresponding to the tracking and positioning system. Exemplarily, the patient is lying on the operating bed in the operating room, while maintaining the sticking on the body. The position of the first marker 204 on the watch remains unchanged (same as in 3D imaging); the doctor opens the tracking and positioning system, and the navigation software system will automatically record the physical space position of the first marker 204 on the patient's body surface, thereby obtaining the first marker The preoperative physical space position of the object is a set of coordinates, and the coordinate set includes the real-time preoperative physical space position coordinates of several first markers. Wherein, at least three first markers 204 for contacting with the body surface are provided.

303. Perform registration of the preoperative image space and the preoperative physical space according to the preoperative image space point set and the preoperative physical space position, and determine a registration result;

Further, in step 303, the preoperative image space and the preoperative physical space are registered according to the preoperative image space point set and the preoperative physical space position, and the registration result is determined, and the registration result is used to compare the preoperative image space with the preoperative physical space. The preoperative physical space is converted to the same coordinate system, and the data of the two coordinate systems are fused and displayed on the same visual terminal interface. The registration of the preoperative image space and the preoperative physical space refers to aligning the preoperative image space with the preoperative physical space, that is, converting the two coordinate system spaces into the same coordinate system space. It is the conversion of the preoperative physical space into the preoperative image space, or the conversion of the preoperative image space into the preoperative physical space, which is not limited here. The registration method can be through the obtained preoperative image space point set and the preoperative physical space position, which are two sets of point sets, and the image registration algorithm of Iterative Closest Point (ICP) is used to compare the two sets of points. The spatial transformation relationship T _{Physical→Image} of the point set pair to be solved is solved; the automatic registration of the two sets of point set pairs is automatically completed by the ICP algorithm, and the transformation relationship of the two Zobo Asi is obtained. The spatial transformation relationship T _{Physical→Image} is the registration result.

Exemplarily, the spatial transformation relationship T _{Physical→Image} can be obtained by the following formula:

Correspondence:

Transformation matrix:

Among them, x _i is the preoperative image space point set, _yi is the preoperative physical space position, and T ^k+1 is the spatial transformation relationship.

这一步的结果为一组对应点对的集合(x_i，

)。在变换矩阵估算步骤，我们寻找最能描述或解释这组对应关系的变换矩阵，以得到配准结果。采用上述方式还可以将二二维影像空间以及三围影像空间进行融合显示。It should be noted that, if the transformation matrix T corresponding to the two point sets is known, then ICP can be used to determine the one-to-one correspondence between the two point sets. One of the most striking features of the ICP algorithm is that it does not require that the points in the point set X and Y are in exact one-to-one correspondence. Conversely, if the transformation matrix T corresponding to the two point sets is known, then ICP can be used to determine the one-to-one correspondence between the two point sets. Mathematically, the solution process of the ICP algorithm can be regarded as an iterative process of the above two equations. In the process of minimization, in the corresponding relationship estimation step, we transform each point of the x _i set through the current transformation matrix T ^k , and then find the closest point to T ^k ( _xi ) in the _yi set. We mark this point as the corresponding point at the kth iteration

The result of this step is a set of corresponding point pairs (x _i ,

). In the transformation matrix estimation step, we find the transformation matrix that best describes or explains the set of correspondences to get the registration result. In the above manner, the two-dimensional image space and the three-dimensional image space can also be fused and displayed.

304. Perform surgical path planning using the preoperative image data to determine a target surgical path;

It should be noted that surgery planning can be performed based on preoperative image data to determine the target surgical path. Effective path planning can be achieved through preoperative 3D image data. The target surgical path is used to guide the operation. The surgical path planning is carried out on the preoperative image based on the preoperative image data, including selecting the needle entry point on the body surface and puncturing the target point to obtain the target surgical path.

305. Perform surgical guidance of the surgical instrument according to the target surgical path, the registration result, the intraoperative image data, and the intraoperative physical space position.

Finally, the surgical guidance of the surgical instrument 203 can be performed through the target surgical path, the registration result, the intraoperative image data, and the intraoperative physical space position, so as to guide the surgeon to perform safe and accurate surgery. The registration results, intraoperative image data, and intraoperative physical space positions are displayed in a single visual device, providing doctors with visual reference information.

The present invention provides an ultrasonic surgical navigation method, which is applied to an ultrasonic surgical navigation system. The method includes: acquiring a preoperative image space point set corresponding to a first marker; acquiring a preoperative physical space position corresponding to the first marker ;According to the preoperative image space point set and the preoperative physical space position, register the preoperative image space with the preoperative physical space to determine the registration result; use the preoperative image data to plan the surgical path to determine the target surgical path; according to the target The surgical path, registration result, intraoperative image data and intraoperative physical space position are used for surgical guidance of surgical instruments. Through the above-mentioned ultrasonic surgical navigation system, the target surgical path using the preoperative image data can be used to guide the real-time position of the surgical instrument during the operation, and there is no need to perform multiple X-ray fluoroscopic imaging during the operation to determine the spatial position of the surgical instrument. It can greatly reduce radiation damage to patients and doctors, and does not need to perform intermittent multiple imaging, reducing time-consuming and improving surgical accuracy, using the target surgical path, registration results, intraoperative image data and intraoperative physical space. The position is guided and displayed, and the real-time relative positional relationship between the surgical instrument and the surrounding key anatomy can also be determined.

As shown in FIG. 4 , FIG. 4 is another structural block diagram of an ultrasonic surgical navigation system in an embodiment of the present invention. The ultrasonic surgical navigation system 40 can be applied to both a terminal and a server. This embodiment is applied to Terminal example. The ultrasonic surgical navigation system 40 specifically includes: an imaging system 401, a tracking and positioning system 402, a surgical instrument 403, a first marker 404 for contacting the body surface, and the imaging system 40 includes an ultrasonic probe 4011, which is provided on the ultrasonic probe 4011 There is a second marker 405, and a third marker 406 is set on the surgical instrument 403; the imaging system 401 is used to acquire preoperative image data to display the preoperative image and acquire the preoperative image corresponding to the first marker 404 Spatial point set; the tracking and positioning system 402 is used to obtain the preoperative tracking data of the first marker 404 to obtain the preoperative physical spatial position of the first marker 404 in the preoperative tracking data; the ultrasonic surgical navigation system 40 is used to The preoperative image space point set and the preoperative physical space position are used to register the preoperative image space with the preoperative physical space to determine the registration result; use the preoperative image data to plan the surgical path to determine the target surgical path; according to the target surgical path , the registration result, the intraoperative image data and the intraoperative physical space position for surgical guidance of the surgical instrument 403 .

It should be noted that the above-mentioned imaging system 401 , tracking and positioning system 402 , and surgical instrument 403 , as well as the first marker 404 for contacting the body surface, are the same as the imaging system 201 , the tracking and positioning system 202 and the surgical instrument shown in FIG. 2 . 203, and the content of the first marker 204 for contacting the body surface are similar, so to avoid repetition, the description will not be repeated here, and reference may be made to the imaging system 201, the tracking and positioning system 202, the surgical instrument 203, and the device shown in FIG. 2. on the content of the first marker 204 in contact with the body surface.

In one possible implementation, the marker comprises a magnetic marker or an optical marker. Corresponding tracking and positioning systems include magnetic tracking systems or optical tracking systems. Exemplarily, if the marker is a tracking and positioning system corresponding to a magnetic marker, it is a magnetic tracking system, and if the marker is a tracking and positioning system corresponding to an optical marker, it is an optical tracking system, which can be configured as required, which is not limited again.

In a feasible implementation manner, the ultrasonic surgical navigation system 30 determines the registration result, which may include: using an iterative closest point algorithm, registering the preoperative image space point set and the preoperative physical space position, determining the preoperative image space and The target transformation matrix corresponding to the preoperative image spatial registration; the target transformation matrix is used as the registration result. The target transformation matrix can be written as: T _{Physical→Image} .

Exemplarily, taking the puncture intervention of the chest and abdomen as an example, the surgical intervention guided by intraoperative two-dimensional real-time ultrasound images is the most extensive solution at present. In the process of traditional puncture intervention, due to the lack of three-dimensional spatial information in the two-dimensional real-time ultrasound images, doctors need to frequently perform X-ray imaging of the patient's lesion, so as to grasp the relationship between the interventional instruments such as the puncture needle for performing the intervention and the puncture target. The relative positional relationship between the puncture and interventional devices ensures that the puncture interventional device can reach the lesion correctly, and avoids the interventional device from damaging key anatomical structures such as blood vessels and nerves, so as to avoid complications. However, X-ray imaging will cause radiation damage to patients and doctors; in order to observe the anatomical structure of blood vessels, it is often necessary to apply contrast agents to patients repeatedly and continuously; the interventional process is often interrupted by X-ray imaging, and the operation is time-consuming and serious. Rely on doctor's experience. Ultrasound image guidance has the advantages of real-time, non-radiation, and portability. It has specific advantages in the imaging guidance of soft tissue in the chest and abdomen, but there are also problems such as lack of three-dimensional spatial positioning information in ultrasound images, and free scanning relying on doctors' experience.

Therefore, according to the characteristics of vascular interventional surgery, the present invention intends to locate the real-time position of the surgical instrument for puncture intervention by combining the marker with the surgical instrument and the imaging system, and display it through the fusion of the real-time ultrasound image and the preoperative three-dimensional surgical planning path. , in order to reduce the frequency of X-ray angiography imaging, reduce radiation damage, and improve the efficiency of puncture intervention. Therefore, please continue to refer to the following content.

In a feasible implementation manner, the ultrasonic probe 4011 is used to acquire intraoperative image data, wherein the intraoperative and intraoperative image data is two-dimensional image data, and ultrasonic imaging can be performed. In this embodiment, intraoperative ultrasound imaging is used to reduce radiation damage, and there is no need to apply contrast agents multiple times; further, the tracking and positioning system 402 is also used to obtain the intraoperative physical space position of the second marker 405; the ultrasound navigation system 40 is also used to perform image fusion display using the registration result and the intraoperative physical space position of the second marker, so as to determine the spatial posture of the intraoperative ultrasound imaging plane corresponding to the ultrasound probe in the preoperative image space, and the image fusion display includes The ultrasound images were fused with the preoperative images.

In a possible implementation, the tracking and positioning system 402 is also used to obtain the intraoperative physical space position of the third marker 406 ; the ultrasound navigation system 40 is also used to utilize the registration result and the intraoperative physical space of the third marker 406 . The position determines the preoperative image spatial position corresponding to the intraoperative physical spatial position, so as to determine the intraoperative spatial posture of the surgical instrument in the preoperative image space.

In order to facilitate the description of this embodiment, the working principle of the system shown in FIG. 4 will be described in detail with reference to FIG. 1 and FIG. 5 . For details, please refer to the following:

(1) The system setting and navigation operation procedures include:

1.1) Preoperative preparation: Referring to FIG. 1 , the patient 10 is lying on the operating bed 11 to prepare for anesthesia, disinfection and other interventional liver and abdomen operations, and a magnetic marker 12 is pasted on the attachment of the intervention site of the patient 10 .

1.2) Preoperative image data preparation: After the preoperative preparation is completed, the patient is pushed into the CT or MR imaging equipment, and the CT/MRI image scanning of the interventional site is performed to obtain the preoperative 3D image scanning data.

1.3) Extraction of preoperative image space position of magnetic marker 12: UNet deep learning network is used to automatically extract the image space position of magnetic marker in the image data, and the preoperative image space point set X _Image = {x _i , i ∈n}.

1.4) Automatic registration of preoperative image space and patient physical space: Turn on the magnetic tracking system 13, and the navigation system software will automatically record the current preoperative physical space position of the magnetic marker 12; and through the feature registration algorithm, automatically complete the image space and Patient physical space registration, feature registration algorithm is ICP, registration is to obtain the registration and calibration of two coordinate systems.

1.5) Intraoperative ultrasound image real-time guidance: The ultrasound imaging system is used as the intraoperative image, and the surface of the ultrasound probe 15 is affixed with a positioning marker 16, so that the tracking and positioning system 13 can pass the positioning marker 16 to the ultrasound probe 15 in three dimensions. The position in the physical space is tracked and positioned in real time, and the real-time intraoperative image acquired by the ultrasound probe 15 is transmitted to the image workstation 14 for display through the data line.

1.6) Interventional puncture guide needle: In order to facilitate the spatial positioning and real-time tracking of the interventional surgical instrument 17, the system installs the magnetic marker 18 on the puncture and interventional instrument 17, which can realize the puncture and interventional surgical instrument 17 under the unified magnetic tracking coordinate system. 3D real-time tracking and visualization. The surgical instrument 17 of the present invention may select different surgical instruments according to specific surgical operations, which is not limited herein.

1.7) In interventional surgery, first collect real-time ultrasound images combined with the spatial pose of the corresponding scanning plane to obtain the three-dimensional spatial pose of the ultrasound image for preoperative diagnostic imaging data such as CT/MR. The imaging workstation will also perform a real-time fusion display of the ultrasound image and the preoperative tomographic image according to the preoperatively positioned coordinate mapping relationship, and give the position of the puncture interventional surgical instrument 17 in the patient image coordinate system in real time. Doctors mainly perform operations based on the 3D real-time rendering results of the imaging workstation.

(2) Intraoperative navigation process, as shown in Figure 5 and Figure 1:

Step 1, preparation for initialization of the navigation system: the marker 12 of the wireless or wired magnetic positioning sensor is pasted on the patient's body surface, and the position of the marker 12 is kept fixed throughout the interventional operation. At the same time, ensure that the ultrasound probe 15 and the penetrating interventional instrument 17 with the markers 16 and 18 of the wireless or wired magnetic localization sensor mounted are ready in place.

Step 2, preoperative three-dimensional image data acquisition: acquiring CT or MR tomographic images of the patient's interventional site, acquiring angiographic images of the patient's lesion site, and transmitting to the imaging workstation 14 .

Step 3, intervention path planning: after acquiring the preoperative tomographic image, the body surface needle entry point and the puncture target point can be defined according to the preoperative three-dimensional tomographic image; the puncture intervention path can be defined according to the needle entry point and the target point to obtain the target surgical path.

Step 4, image location extraction of the magnetic marker 12: Using the UNet deep learning network, the image of the magnetic marker 12 in the three-dimensional image space is automatically extracted, denoted as: X _Image ={ _xi ,i∈n}. The training of the UNet network can collect multiple image data of magnetic markers before surgery and obtain them through training.

Step 5, automatic registration: after the patient 10 is transferred to the operating bed 11, obtain the real-time spatial position coordinates of the magnetic positioning sensor 12 on the body surface of the patient 10 at the current moment: Y _Physical ={y _i ,i∈n}, and transmit it to the imaging workstation 14. According to the two sets of spatial location data, the least squares solution is performed, and feature point registration is performed, so as to realize the automatic registration of patient image space and physical location space.

Step 6, three-dimensional real-time display of the intraoperative ultrasound image: using the registered spatial transformation matrix T, the spatial position of the intraoperative ultrasound real-time imaging device can be transformed into the spatial coordinate system where the preoperative CT/MR image data is located. Since the magnetic marker 16 is installed on the ultrasound probe, the spatial posture of the ultrasound imaging plane can be determined in the preoperative image coordinate system after registration.

Step 7, real-time tracking of the puncture needle 17 in the preoperative three-dimensional surgical planning during the interventional operation: Since the magnetic marker 18 is installed on the puncture interventional surgical instrument 17, the registered space transformation matrix can be used to convert the intraoperative puncture intervention The spatial position of the surgical instrument is transformed into the spatial coordinate system where the preoperative CT/MR image data is located. Real-time display of the puncture needle model and the patient model in the same 3D scene, to complete the tracking process of the navigation operation.

Step 8, according to the doctor's judgment on the process of interventional surgery, repeat steps 6 and 7 when necessary to update the spatial position of the ultrasound imaging plane and the spatial position of the puncture interventional instrument, that is, the doctor needs to adjust the ultrasound 15 probe in real time according to the visual display. The detection position and the intervention position of the surgical instrument 17 are used to realize surgical guidance more in line with the target surgical path.

The method adopted in the present invention does not need to determine the spatial position of the puncture needle from the X-ray fluoroscopic image, so during the puncture interventional operation, it is not necessary to perform X-ray fluoroscopic imaging on the patient in the whole process, which can greatly reduce the radiation received by the patient and the doctor. injury, and has better precision and stability, improving surgical efficiency. The method adopted in the present invention can not only obtain the real-time position of the puncture needle, but also display it in the ultrasonic two-dimensional image, the preoperative high-resolution image and the operation planning path, and can obtain the real-time relationship between the puncture needle and the blood vessels and key anatomical structures in real time. relative positional relationship.

Figure 6 shows an internal structure diagram of a computer device in one embodiment. Specifically, the computer device may be a terminal or a server. As shown in Figure 6, the computer device includes a processor, memory and a network interface connected by a system bus. Wherein, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system, and also stores a computer program. When the computer program is executed by the processor, the processor can implement the above method. A computer program may also be stored in the internal memory, and when the computer program is executed by the processor, the processor may execute the above method. Those skilled in the art can understand that the structure shown in FIG. X is only a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer equipment to which the solution of the present application is applied. Include more or fewer components than shown in the figures, or combine certain components, or have a different arrangement of components.

In one embodiment, a computer device is proposed, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the program as shown in FIG. 3 . steps of the method.

In one embodiment, a computer-readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, causes the processor to execute the steps of the method shown in FIG. 3 .

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium , when the program is executed, it may include the flow of the above-mentioned method embodiments. Wherein, any reference to memory, storage, database or other medium used in the various embodiments provided in this application may include non-volatile and/or volatile memory. Nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Road (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, all It is considered to be the range described in this specification.

The above-mentioned embodiments only represent several embodiments of the present application, and the descriptions thereof are relatively specific and detailed, but should not be construed as a limitation on the scope of the patent of the present application. It should be pointed out that for those skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which all belong to the protection scope of the present application. Therefore, the scope of protection of the patent of the present application shall be subject to the appended claims.

Claims (10)

1. An ultrasonic surgical navigation system, comprising: the system comprises an imaging system, a tracking and positioning system, a surgical instrument and a first marker for contacting with the body surface;

the image system is used for acquiring preoperative image data to display preoperative images and acquiring a preoperative image space point set corresponding to the first marker;

the tracking and positioning system is used for acquiring preoperative tracking data of the first marker so as to acquire a preoperative physical space position of the first marker in the preoperative tracking data;

the ultrasonic operation navigation system is used for registering the preoperative image space and the preoperative physical space according to the preoperative image space point set and the preoperative physical space position, and determining a registration result; planning a surgical path by using the preoperative image data, and determining a target surgical path; and performing surgical guidance of the surgical instrument according to the target surgical path, the registration result, the intra-operative image data and the intra-operative physical space position.

2. The ultrasonic surgical navigation system of claim 1, wherein the imaging system includes an ultrasonic probe having a second marker disposed thereon;

the ultrasonic probe is used for acquiring intraoperative image data;

the tracking and positioning system is also used for acquiring the intraoperative physical space position of the second marker;

the ultrasonic navigation system is further configured to perform image fusion display by using the registration result and the intraoperative physical spatial position of the second marker to determine a spatial posture of an intraoperative ultrasonic imaging plane corresponding to the ultrasonic probe in a preoperative image space, where the image fusion display includes fusion display of an intraoperative ultrasonic image and a preoperative image.

3. The ultrasonic surgical navigation system of claim 1, wherein a third marker is disposed on the surgical instrument;

the tracking and positioning system is also used for acquiring the intraoperative physical space position of the third marker;

the ultrasonic navigation system is further used for determining a preoperative image space position corresponding to the intraoperative physical space position by using the registration result and the intraoperative physical space position of the third marker so as to determine the spatial posture of the surgical instrument in the operation in the preoperative image space.

4. The ultrasonic surgical navigation system of claim 1, wherein the ultrasonic surgical navigation system determines a registration result, comprising:

registering the preoperative image space point set and the preoperative physical space position by using an iterative closest point algorithm, and determining a target conversion matrix corresponding to registration of the preoperative image space and the preoperative image space;

and taking the target conversion matrix as the registration result.

5. The ultrasonic surgical navigation system of any one of claims 1-4, wherein the marker comprises a magnetic marker or an optical marker.

6. The ultrasonic surgical navigation system of claim 5, wherein the tracking and positioning system comprises a magnetic tracking system or an optical tracking system.

7. The ultrasonic surgical navigation system of claim 5, wherein the pre-operative image comprises a tomographic image or a magnetic resonance image.

8. An ultrasonic surgical navigation method applied to the ultrasonic surgical navigation system according to any one of claims 1 to 7, the method comprising:

acquiring a preoperative image space point set corresponding to a first marker;

acquiring a preoperative physical space position corresponding to the first marker;

registering the preoperative image space and the preoperative physical space according to the preoperative image space point set and the preoperative physical space position, and determining a registration result;

planning a surgical path by using the preoperative image data, and determining a target surgical path;

and performing surgical guidance of surgical instruments according to the target surgical path, the registration result, the intraoperative image data and the intraoperative physical space position.

9. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, causes the processor to carry out the steps of the method as claimed in claim 8.

10. A computer arrangement comprising a memory and a processor, characterized in that the memory stores a computer program which, when executed by the processor, causes the processor to carry out the steps of the method as claimed in claim 8.

Images