KR20170030688A - Guiding method of interventional procedure using medical images and system for interventional procedure for the same - Google Patents

Abstract

Disclosed is a medical image-applying interventional procedure system. More specifically, the present invention relates to an interventional procedure system which comprises: a medical tool; a control part storing an insertion path of the medical tool; an image acquisition device acquiring real-time images of a patient; and a display displaying the real-time images. The control part is capable of displaying both positional information of the medical tool and an operation place image onto the display.

Description

TECHNICAL FIELD [0001] The present invention relates to an interventional procedure guide method using a medical image, and an interventional procedure system for the same. BACKGROUND OF THE INVENTION 1. Field of the Invention < RTI ID =

Disclosure relates to an interventional procedure guide method using a medical image as a whole, and to an interventional procedure system for the same, and more particularly, to an integrated interventional procedure system for positioning a robot arm according to a procedure plan and confirming insertion conditions of a medical instrument , And an interventional treatment system for the same.

Herein, the background art relating to the present disclosure is provided, and these are not necessarily meant to be known arts.

1 shows an example of a system for supporting percutaneous intervention procedures as disclosed in U.S. Patent No. 8,386,019. The system disclosed herein comprises a CT imaging system, a robot that is matched to the device, and a device for sensing the patient's movement. According to the method of the present invention, the robot is equipped with an interventional device and is matched with the imaging device, and the system detects the movement of the patient. The system simultaneously transmits the patient's movement to the robot. The system is designed to match the 3D image of the demonstration field with the demonstrative 3D image created before the procedure and prevent the insertion of the intervention device of the robot unless the matching is done. The robot 106 is equipped with a mounting device 108 for a puncture needle at the end of the robot arm 107, and is installed on the ceiling. The patient wears a respiratory belt (110) to allow the system to sense internal and external movements. The C-arm X-ray system 101, the control 109 of the robot 106, and the respiratory belt 110 are connected to the processing unit 11. This processing unit is recorded by an input interface 112, a four-dimensional image data set and a C-arm for storage of a four-dimensional image data set before the procedure, and a three-dimensional image data set A matching module 113 for generating a target path for the surgical tool and transmitting the information to the control program of the control 109 for moving and controlling the robot arm 107. [

FIG. 2 is a view for explaining an example in which an operator is exposed to radiation during interventional procedures. In a needle insertion type intervention such as a biopsy, a minimally invasive procedure has been rapidly increasing recently. Such interventional procedures are generally performed under radiographic imaging guidance. These interventions are highly dependent on the practitioner experience, and the radiation exposure of the practitioner and patient is a problem.

A medical device such as a biopsy needle (eg, a biopsy needle), a lead (eg, a lead for Deep Brain Stimulation), a probe, a catheter, It is important that interventional procedures, such as insertion or implantation, are performed so that vessels or anatomically important structures are not damaged or minimally invasive. Biopsy is an intervention that minimizes the damage to the surrounding normal tissue and extracts the specimens necessary for the pathological diagnosis of the target. The biopsy is based on the retroperitoneal membrane of the adrenal, pancreas, It is widely applied to areas of the lungs, spine, and limb.

In such a medical image-based biopsy, the insertion route of the biopsy needle is generally planned in advance in the diagnosis image (pre-operation image) due to problems such as exposure to radiation.

As a medical image-based biopsy, the CT-based biopsy can localize the lesion in a delicate three-dimensional region using a high-resolution image and view the biopsy needle that has entered the tissue, It is easy to detect lesions. CT-based biopsies are superior to lesions in which tissue is superimposed over ultrasound or x-ray fluoroscopy. In addition, the CT-based biopsy shows the relationship with surrounding tissues, so that the clinician can set the trajectory to the lesion and can perform the operation at various patient positions.

In the CT-based biopsy, the initial angle of entry of the biopsy needle to the patient's body is important. In CT-based biopsy, the surgeon adjusts the biopsy needle, and the assistant can use the protractor to guide the biopsy needle to the surgeon in the on-the-spot biopsy, or CT or C-arm fluoroscopy, In some cases, the insertion path of the biopsy needle may be guided by the image, but in this case, the operator is highly dependent on the experience of the operator, so that the operation is performed with the operator exposed to the radiation, and the exposure time can be changed according to his experience.

Therefore, there is a need to develop an interventional robot in order to solve problems such as an increase in the exposure time of the operator and the patient to the radiation and the accuracy of the operation. The use of such an interventional robot can reduce the radiation dose of the patient by shortening the procedure time, and it is expected to reduce the complication and maximize the safety. In addition, it is possible to reduce or eliminate the radiation exposure of the practitioner and to improve the safety of the operator through the automation system.

Fig. 3 is a diagram showing an example of an arbitration procedure robot shown in U.S. Published Patent Application No. 2010-0250000, in which an arbitration procedure robot called product da Vinci is presented. The interventional procedure robot has a plurality of robot arms (201, 202, 203, 204). Each arm 201, 202, 203, 204 has end effectors 211, 212, 213, 214. The end effectors 211, 213, and 214 are mainly in the form of a forceps for laparoscopic surgery, and the end effector 212 is an endoscope. A display 220 is also provided for indicating the target.

However, conventional techniques using such an interventional robot have limitations in achieving the automation of the intervention procedure and the accuracy, safety, and convenience of the intervention using medical instruments such as a biopsy needle. The system is too heavy, the movement and the installation are inconvenient, and the medical expenses are increased. Also, there is a possibility that the operator and the patient may not be conscious of the radiation exposure and may expose them to the radiation for a long time.

This will be described later in the Specification for Implementation of the Invention.

SUMMARY OF THE INVENTION Herein, a general summary of the present disclosure is provided, which should not be construed as limiting the scope of the present disclosure. of its features).

According to one aspect of the present disclosure, in an interventional treatment system using a medical image, a control unit in which an insertion path of a medical instrument and a medical instrument is stored, An image acquisition device, and a display capable of displaying a real time image, and the control unit is provided with an arbitration procedure system using a medical image to simultaneously display the procedure field image and the position information of the medical instrument on the display.

This will be described later in the Specification for Implementation of the Invention.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a view showing an example of an arbitration procedure system disclosed in U.S. Patent No. 8,386,019,
FIG. 2 is a view for explaining an example in which an operator is exposed to radiation when an intervention is performed;
FIG. 3 is a diagram showing an example of an arbitration procedure robot presented in U.S. Published Patent Application No. 2010-0250000,
4 and 5 are views showing an example of an interventional treatment system using a medical image according to the present disclosure,
6 is a view for explaining an example of a robot arm according to the present disclosure,
7 is a view for explaining an example of a multifunctional end effector,
8 is a view for explaining an example in which a system of an intervention procedure using a medical image uses a camera,
9 is a view for explaining examples of a master console,
10 is a view for explaining an example of a process of controlling a biopsy needle mounted on an end effector by a master console,
11 is a view for explaining an example of an intervention procedure guide method using a medical image according to the present disclosure;
12 to 14 are diagrams for explaining an example of a procedure plan generation method,
15 is a view for explaining an example of the operation of the interventional treatment system,
16 is a diagram for explaining an overall process of an example of an intervention procedure guide method,
17 is a diagram showing an example of a display screen in a split mode,
18 is a view showing an example of a display screen of the plan mode,
19 and 20 are views showing an example of a screen of a display in a matching mode,
21 is a view for explaining an example of a matching method,
22 and 23 are views for explaining an example of a window in which a treatment plan can be modified,
24 to 26 are views showing an example of a display screen of the navigation mode,
27 and 28 are views for explaining an example of a display screen in the insertion mode.

The present disclosure will now be described in detail with reference to the accompanying drawings.

FIGS. 4 and 5 are diagrams showing an example of a configuration and a configuration method of an arbitration procedure system using the medical image according to the present disclosure, wherein an arbitration procedure system using a medical image (hereinafter referred to as an arbitration procedure system) A control unit 500 (e.g., a computer) for controlling a robot arm 400 in real time based on a pre-operation image or acquiring an image of a body or an animal and a medical tool And an apparatus 700 (e.g., Optical Tracker (NDI)) for responding to an emergency by monitoring the position or posture of the patient or the treatment device.

The robot arm 400 may be used for biopsy and treatment for reducing the radiation exposure and for improving the accuracy of the procedure, and for biopsy and treatment of a 1 cm level lesion in the abdomen, chest, and the like. The robot arm 400 may be part of a slave robot and an end-effector 460 may be mounted on the robot arm 400. The medical tool 470 is mounted to the end effector 460 and the end effector 460 may be powered directly to the medical tool 470 to drive the medical tool 470 for automated procedures, 470), and may have a structure in which a plurality of medical tools 470 can be performed at one time. The medical tool 470 may have a very small sensor and may have a configuration capable of transmitting information to the control unit.

The medical tool 470 may be a medical device, such as a biopsy needle (e.g., a biopsy needle), a lead (e.g., lead for Deep Brain Stimulation), a probe, a catheter, . Of course, the medical instrument 470 mounted on the robot arm 400 is not limited thereto. Hereinafter, the biopsy needle 470 will be mainly described as an example in this embodiment. The biopsy needle 470 may be composed of a tissue sampling needle (e.g., inner stylet; see Figure 1) and a guide needle (e.g., Sheath; see Figure 1) It can be constituted by only a guide needle.

The image acquisition apparatus 600 may be a CT apparatus, a C-arm fluoroscopy apparatus, or the like. For example, as shown in FIG. 5A, the table 620 moves the patient 5 into and out of the CT apparatus 600. The robot arm 400 is mounted on the robot base next to the table 620 and moves partly into and out of the CT apparatus 600. 5B, the control unit 500 includes a CT apparatus 600 and a master console 310 interlocked with the robot arm 400 and performing real-time image registration and used for controlling the robot arm 400, And a display 350 on which an image is displayed. When the biopsy needle 470 is inserted into the target 100 and the obstacle is encountered or a signal indicating a certain point needs to be given when the haptic is applied by the haptic, The master console 310 may be provided with a vibration response for a force or a signal at a specific point on the master console 470. It can also be used for interventional procedures training.

FIG. 6 is a view for explaining an example of a slave robot according to the present disclosure. The slave robot is composed of a robot arm 400 and a base 410. The robot arm 400 is fixed in position by the base 410 in the periphery of the patient 5 according to a control signal applied from the control unit 500. The robot arm 400 may have a multi-joint high precision (e.g., a positional accuracy of 1 mm), and preferably has a material and structure that reduces X-ray interference. The entire robot arm 400 may be an X-ray permeable material, or only a part or a part of the portion (for example, the end effector 460) that enters the CT and does not affect the image of the patient may be X- Lt; / RTI >

The robot arm 400 includes a base 410, a sliding portion 420 that slides relative to the base 410, a first arm 430 connected to the sliding portion 420, a second arm 430 connected to the first arm 430, A fourth arm 440, and a third arm 450 coupled to the second arm 440. The end effector 460 may be mounted on the third arm 450, and the type of the end effector 460 may be changed according to the kind of the intervention. The end effector 460 may be regarded as being included in the robot arm 400. The end effector 460 is movable relative to the third arm 450. The base 410, the sliding portion 420, the first arm 430, the second arm 440, and the third arm 450 function as a positioning portion of the end effector 460, and the end effector 460 Can be regarded as an attitude determining unit for determining the attitude of the biopsy needle 470. The sliding part 420, the first arm 430 and the second arm 440 are rotatably connected to each other to determine the height of the end effector 460. The third arm is connected to the first arm 430, 2 is rotatably mounted on the second arm in a direction perpendicular to the plane in which the two arms are connected. At this point, the third arm will be mounted in a direction that allows the end effector 460 to approach the image acquisition device 600. Again, the end effector 460 is rotatably mounted on the third arm. The end effector 460 can approach the image acquisition device 600 by the sliding portion 420. Normally, the positioning unit may be located outside the image obtaining apparatus 600, and the attitude determining unit may hold the attitude (orientation, distance, etc.) with respect to the target within the image acquiring apparatus 600.

5B, the robot arm 400 is interlocked with the control unit 500, and the control unit 500 can calculate the coordinates of the end of the biopsy needle 470 using a kinematic model . The positional relationship between the base 410 and the table 620, the angle between the sliding portion 420 and the first arm 430, the angle between the first arm 430 and the second arm 440, The angle of the third arm 450 and the end effector 460, the length of the arms 430, 440 and 450, the length of the biopsy needle 470, The coordinates on the space of the end can be calculated. The angle information can be obtained using an encoder mounted on each motor that drives the arms 430, 440, and 450, or a sensor capable of indicating displacement. This information is transmitted to the control unit so that the control unit can calculate the kinematic model of the robot arm 400 and calculate the position of the end of the biopsy needle 470.

6 (c) is a view showing the structure of the base 410. FIG. The base 410 allows the robot arm 400 to be mounted thereon and is movable. The robot arm 400 may be located on the side of the base 410. To enable the base 410 to move, a wheel 411 is attached at the bottom and has a base fixing device 412 for immovably fixing at a specific position. After the position is fixed once in the treatment field, if it randomly or unintentionally moves, there is a problem that the spatial coordinates of the medical tool 470 calculated by the control unit 500 may be distorted. The base fixing device 412 may be formed at one or more sides of the base 410 so that the base 410 can be stably fixed even during the operation of the robot arm 400 and the control unit 500 or the base 410 It is possible to fix the base 410 in contact with the ground by an electronic signal of itself. Such a fixing device 412 may serve to adjust the level of the slave robot. In addition, a brake system may be applied to the wheels to allow the base 410 to be more firmly secured against movement.

In the treatment field, the base 410 is positioned next to the table 620, and the robot arm 400 mounted thereon sends the end effector 460 on the patient in the direction of the CT. In this case, the position of the center of gravity of the slave robot deviates from the center of the base 410, which may hinder the stability of movement of the robot arm 400. Accordingly, there is a moving weight 413 inside the base 410, which can change the center of gravity of the slave robot in accordance with the movement of the robot arm 400. As the position of the end effector 460 moves away from the center of the slave robot, the moving weight 413 also moves to the opposite side. According to an embodiment of the present disclosure, the moving weight 413 may move in both directions on the directional axis of the table 620 on which the patient is lying. The moving weight driving unit 414 may be positioned below the moving weight 413 so that the moving weight driving unit 414 may automatically move in accordance with the movement of the robot arm 400. [

6 (d) is a view for explaining the end effector 460. FIG. The end effector 460 is a needle-inserted end effector 460 capable of repetitive needle insertion operation, and enables rotation of the needle to improve the steering and insertion function of the biopsy needle 470. End effector 460 is configured to include an insertion device 462 for inserting biopsy needle 470, a fixation device 461 for fixation, and a needle rotation device 463 for rotating biopsy needle 470 do. When insertion of the biopsy needle 470 is required, the fixing device 461 releases the two bars holding the biopsy needle 470 in both directions to release the fixation, and the insertion device 462 holds the biopsy needle 470 The gear or the toothed device is rotated so that the biopsy needle 470 is moved. When rotation of the needle is required for smooth insertion of the biopsy needle 470, the needle rotation device 463 operates to rotate the biopsy needle 470. The insertion device 462 and the needle rotation device 463 may be operated simultaneously.

 It is desirable to have a function of releasing the biopsy needle 470 at the time of an emergency or after the insertion of the biopsy needle 470 is completed. The end effector 460 is also equipped with a medical instrument for treatment 470 such as dilation and RF ablation so that the end effector 460 can perform related procedures.

The structure of the first arm 430, the second arm 440, and the third arm 450 has an advantageous structure in that the biopsy needle 470 is pulled back as it is along the insertion path when an emergency occurs. When the control unit 500 recognizes the emergency situation itself or the operator pushes the emergency button of the control unit 500, the slave robot first causes the end effector 470 to place the biopsy needle 470, So that the robot arm 400 moves to the safe area. This may be performed automatically by the control unit 500 or may prevent the motor brake of the sliding unit 420 from operating so that the operator can move the robot arm 400 to a desired position directly. And may have a function of stopping the operation when the robot arm 400 comes into contact with an obstacle (CT device or the like) for safety.

Figures 7 (a) and 7 (b) show a multifunctional end effector 480 that allows multiple biopsies at a time. The multifunctional end effector 480 includes a housing 481 capable of receiving a plurality of biopsy needles 470, a drive transmission portion 33 for rotating the housing 481, a mover 35, and a multifunctional end effector 480. [ And a support portion 482 for supporting the support portion. The support portion 482 may include a guide rack to adjust the orientation of the biopsy needle 470. A guide hole (not shown) is formed in the housing 481 to accommodate a plurality of biopsy needles 470. An outlet of the biopsy needle 470 is formed in the lower portion of the housing 481. The biopsy needle 470 may have a sheath. The biopsy needle 470 has a structure movable within the sheath. The guide hole of the housing 481 may further include a guide needle for guiding the biopsy needle 470 to the target. In this case, when the procedure is started, the guide needle is inserted first, and other biopsy needles reach the target through the guide needle so that a sample can be taken.

The drive transmitting portion 33 may be provided with teeth or gears to rotate the housing 481 or to rotate the biopsy needle 470. The slider 35 is provided with a holder for moving the biopsy needle 470 to transmit the force of the motor transmitted to the slider 35 to the biopsy needle 470 to drive the biopsy needle 470 up and down (See FIG. 7B).

The end effector 460 may further include an arm engagement portion 483 coupled to the support portion 482. The female engaging portion 483 is coupled to the robot arm 400 and can be mounted and detached, for example, to the third arm 450 shown in Fig. The female engagement 463 in this example may have a mechanism 484 including a plurality of links and a motor that pushes or pulls the support 462 to rotate the support 482 The biopsy needle 470 attached to the housing 481 can be tilted so that the biopsy needle 470 is horizontally moved as shown in FIG. 7 (a) or when the height of the biopsy needle 470 is constant. The biopsy needle 470 may be controlled by a combination of the operation of the robot arm 400 and the operation of the end effector 460. The end effector 460 may be fixed to the robot arm 400 , And the robot arm 400 only controls the posture.

The end effector 460 may be directly coupled to the third arm 450 without passing through the female engaging portion 480 as shown in FIG. 6 (d). At this time, a driving motor for rotating the end effector 460 is mounted on the third arm or end effector 460 so that the direction of the medical tool 470 can be steered in the pitch itch direction (see FIG. 10).

8 is a view for explaining an example in which additional equipment is added to the guide system of the intervention procedure using the medical image. In this example, the robot arm 400 is provided with a camera 480 (optical camera or thermal, infrared camera) A laser pointer device is installed. The sliding portion 420, the first arm 430, the second arm 440 and the third arm 450 are moved from the control unit 500 to the patient 5 on the table 620 , The sliding portion 420 may slide on the base 410 to allow the end effector 460 to move horizontally into the CT device 600. Only the end effector 460 may be inserted into the CT apparatus 600 or a part of the third arm 450 and the end effector 460 may be inserted into the CT apparatus 600. Thereafter, the end effector 460 is posture controlled, and the biopsy needle 470 is aligned along the insertion path 475 (see FIG. 15F). The biopsy needle 470 is aligned on the insertion point 471 indicated by laser pointers L1 and L2 on the skin of the patient 5 (see FIG. 15F). The image of the camera 480 is displayed on the display 350 of the control unit 500 (917; see FIG. 24).

The camera 480 is preferably installed at a position that does not enter the irradiation area 615 and can show the biopsy needle 470 and the insertion point 20 (see FIG. 15F). 8 (b), the end effector 460 may enter the radiation irradiation area 615 of the CT apparatus 600, but only the biopsy needle 470 mounting portion 455 protrudes from the end effector 460 Thereby minimizing the portion of the end effector 460 entering the irradiation region 615. [ Or the portion of the end effector 460 which enters the irradiation region 615 may be formed of a transparent material. In this example, the camera 480 is installed at a portion of the end effector 460 where the radiation does not reach. It is also conceivable to install a radiation detection sensor in conjunction with the camera 480 so that when the camera 480 enters the radiation application area, an alarm is sounded or displayed so that the camera 480 avoids the radiation application area.

9 is a diagram for explaining the master console 310. FIG. The control unit 500 includes a master console 310 for controlling the robot arm 400. 9A, the master console 310 includes a clutch 313 for controlling the mode switching, an insertion button 311 for controlling the insertion of the biopsy needle 470, and a rotation controlling rotation of the biopsy needle 470 Button 312, as shown in FIG. In addition, the master console 310 may include an emergency stop device, and may include a motor and a sensor.

The mechanism portion supporting the master console 310 has a structure capable of expressing the rolling and pitching movements of the biopsy needle 470 by the master console 310. Fig. 9 (b) shows a structure in which six encoders are provided so as to represent three different axes on both sides, and the motion information of the master console is converted. In another embodiment 9 (c), encoders can be mounted on two axes orthogonal to the circular support surrounding the master console to signal-convert the rolling and pitching information of the console. The structure (not shown) for moving the robot in the x-y coordinates is connected to these rolling-pitching mechanisms.

The control unit may be divided into a positioning step for moving the robot arm 400 and a biopsy needle 470 for controlling the end effector 460 to be driven. Depending on the situation, the practitioner can operate these steps separately, as shown in Figure 9A. In the positioning step, the master console 310 is manipulated in a desired direction to tilt the biopsy needle 470 forward, backward, left and right, tilts in various directions, and moves to a desired position. In the posture control step, ) Of the biopsy needle 470 can be controlled by tilting the biopsy needle 470 forward, backward, leftward, and rightward, or tilting the biopsy needle 470 in various directions (see FIG. 10).

After the biopsy needle 470 is aligned according to the insertion path 475 (see FIG. 15F), the practitioner presses the insertion button 311 to switch the system to the needle insertion mode. When the system is switched to the needle insertion mode, the control unit 500 does not accept any other motion signals other than the command by the insertion button 311 and the rotation button 312 of the end effector 460. The rotating button 312 can be turned to control the rotation of the biopsy needle 470 (see FIG. 10). Thereafter, when the insertion button 311 is pressed in a state where the clutch 313 is depressed, the biopsy needle 470 penetrates the insertion point 471. Simultaneous command of the rotary button 312 during the insertion of the biopsy needle 470 is also made possible so that the biopsy needle 470 can be rotated while entering the patient's body.

At this time, as described above, the biopsy needle 470 includes only the sheath, and the practitioner can take the tissue using the sheath as a guide. To this end, the robot arm 400 exits the CT apparatus 600 after inserting the sheath. Alternatively, in the state where the robot arm 400 keeps fixing the sheath, tissue collection can be performed with the sheath as a guide by the operator. Alternatively, it is also possible that the biopsy needle 470 attached to the end effector 460 includes both the inner stylet and the sheath, and the tissue can be harvested by the robot arm 400.

10 is a view for explaining an example of a process of controlling the biopsy needle 470 attached to the end effector 460 by the master console 310. After the robot arm 400 is positioned, In the posture adjusting step of the needle 470, the biopsy needle 470 can be posture controlled by the end effector 460 and the third arm 450. 10A, the biopsy needle 470 may be rolled or rotated about the X axis by tilting the master console 310 back and forth, side to side, or tilting in many other directions, as shown in FIG. 10B And can be rotated and pitched about the Y-axis, as shown in Fig. 7, the end effector 460 may be rolled or punched relative to the third arm 450, such that the biopsy needle 470 may be rolled and pinched. Further, the biopsy needle 470 may be rotated by the end effector 460 by turning the rotation button 312 to be yawed.

FIG. 11 is a view for explaining an example of a guide method of an intervention procedure using a medical image according to the present disclosure. The guide method of an intervention procedure using a medical image may be applied to an organ such as lung, kidney, liver, , It is not excluded to apply to other parts of the organ.

In an intervention procedure guide method using a medical image (hereinafter referred to as an intervention procedure guide method), a pre-operation image is acquired (S210). The pre-operation image is acquired using the image acquisition device 600. [ As the image obtaining apparatus 600, a medical three-dimensional image obtaining apparatus 600 such as a CT apparatus 600, a Cone-beam CT (CBCT) apparatus, or an MRI apparatus may be used. Pre-treatment images (e.g., 811, 812, 813; see FIG. 17) for the lung, liver, etc. acquired by the CT device are loaded into the display 350. Where the display 350 may be the display 350 of the interventional treatment system or the display 350 for a separate treatment plan. The surgical site is checked and reviewed in the displayed pre-procedure images, and the pre-procedure images are segmented. As a result of the segmentation, for example, skin, bones, blood vessels, organs to be treated, protected organs, critical structures, and targets (targets, lesions, or lesions) are divided and defined as respective data. The divided preoperative images can be stored in the interventional system or transmitted from outside the interventional system during the procedure.

Thereafter, an insertion path (e.g., 475; see FIG. 17) of the medical instrument 470 is generated based on the divided pre-operation images (S220). For example, the direction of the patient 5 is determined, the insertion point 471, the insertion direction, and the insertion distance are determined, the type of the biopsy needle 470 and the end effector 460 are selected, path) is displayed and fine adjustment is made to create a treatment plan including the insertion path. The insertion path is preferably selected so as to minimize invasion by the biopsy needle 470. When the insertion route is instructed or designated or selected by the practitioner through the user interface, the procedure plan is automatically generated by the computer and visualized on the display device and displayed. The treatment plan is stored or transmitted to the interventional system using TCP / IP or a dedicated communication protocol. Or a treatment plan may be created in the interventional procedure system.

12 to 14 are diagrams for explaining an example of a procedure plan generation method. First, the anatomical structures (eg, blood vessels, bones, etc.) included in the pre-operation image are obtained as a three-dimensional set of voxels as a result of division of the pre-operation images. For example, after acquiring volumetric chest CT images (lung images), the lung images are divided into divided lung images. For example, anatomical structures (eg, blood vessels, ribs, airways, lung boundaries, etc.) included in a lung image are segmented by a segmentation technique (eg, adaptive threshold) )do. As a result of the segmentation, anatomical structures such as blood vessels are extracted into a three-dimensional set of voxels. 12 shows an axial view of a lung image in which an anatomical structure such as a blood vessel is divided. Anatomical structures such as blood vessels, ribs, and airways, which are divided from lung images, may be used as a lung mask, a vessel mask, a rib mask, an airway mask, etc. .

Thereafter, using a lung mask, a vessel mask, a rib mask, an airway mask, and the like, a distance map of a lung boundary, a distance map map of rib, a distance map of pulmonary vessel, and a distance map of airway.

The generation process of the pulmonary blood vessel distance map may include a process of giving distance information to all the voxels in the pulmonary image from the blood vessel boundary to all the voxels. The generation process of the lung boundary distance map, lip distance map and airway distance map may likewise include processes in which distance information from the lung boundary, distance from the rib boundary, and distance information from the airway boundary are imparted to the voxels, respectively. Using such distance maps, the distance of the insertion path or the distance between the insertion path and the anatomical structure can be calculated. Thus, an anatomical structure that intersects the insertion path of the biopsy needle 470 can be found.

The above-described distance map can be used in the process of calculating the distance of the penetration amount and the insertion path. In calculating the distance between the invasive volume and the insertion path, a method of using a pulmonary vein tree may be considered in addition to the method of using the distance map. Using the pulmonary vein tree, the number of blood vessels meeting the insertion path and the extent to which the blood vessels are invasive can be calculated.

The distance to the anatomical structures, such as the vessels that meet the insertion path, is calculated using a distance map by 3D casting or using a pulmonary vascular tree. Although the entire 360 degrees may be laid to find the insertion path, a user (e.g., a practitioner) may define the range 20 of the entry point to avoid unnecessary computation (see FIG. 12). The range 20 of the insertion point at this time can be widely selected except for the area where insertion of the biopsy needle 470 is not allowed from a medical viewpoint. When the range (20) of the insertion point is determined, the insertion path is set from the insertion point to the target while the computer continuously changes the insertion point automatically within the range (20) of the insertion point. And the distance of the insertion path can be calculated. For example, a certain range is given as the insertion point range based on the insertion point 471 of the shortest distance insertion path 300 (see FIG. 12), and as the insertion point is changed within this range, . Although a 2D excise section is illustrated in Fig. 12, the extraction of the insertion path can be performed in three dimensions. Here, the amount of invasion is the number and thickness (or area) of the insertion path with the anatomical structure such as the blood vessel, and the distance of the insertion path is the distance from the insertion point of the lung boundary to the target 100. The actual insertion path is determined between these two methods. For example, an insertion path having an infiltration amount less than a tolerance value is extracted, and an insertion path having a minimum insertion path distance can be extracted as an optimal insertion path.

FIG. 13 is a diagram showing an insertion path reduced by a safety margin, and such a plurality of insertion paths can be represented by a three-dimensional insertion area 230. The insertion region 230 may have a cone shape with a reduced cross-sectional area from the insertion point 271 to the target 100. Alternatively, the insertion region may have a cylindrical shape. Empirically and theoretically, the safety margin of the insertion area of the biopsy needle 470 is determined. The safety margin may be a distance from the invading structure of the blood vessel 140 or the virtual wall 120 (e.g., other organs other than the lung). Therefore, it is preferable that the insertion path in the safety margin in the insertion area 230 is removed. As a result of the removal of the insertion path in the safety margin, a reduced insertion area 235 can be generated.

Fig. 14 is a diagram showing an example in which the insertion path described in Fig. 13 is actually implemented. In Fig. 14, a cone-shaped insertion region 235 and a selected insertion path 475 are visualized three-dimensionally between the ribs, the ribs . In order to more surely confirm the insertion path 475 and the three-dimensional visualized insertion area 235, an insertion area 235 (see FIG. 3) is formed on a multiplanar reconstruction (e.g., axial view, coronal view, sagittal view) ), The optimal insertion path and the selected insertion path 475 may be overlaid and displayed. In this way, the biopsy needle 470 is guided along the insertion path confirmed on the MPR, so that the biopsy and other necessary procedures can be performed. If the system for this purpose is configured separately from the control unit 500, the final confirmed insertion path may be transmitted to the interventional system using TCP / IP or a dedicated communication protocol to assist in the procedure.

15A is a view for explaining an example of the operation of the interventional treatment system. In the process of acquiring the procedure field image and generating the insertion path of the biopsy needle 470, the robot arm 400, Is set to the standby state. At this time, the biopsy needle 470 is attached to the robot arm 400. The treatment plan is loaded together with the pre-treatment image (Pre-CT) and displayed on the display 350 (see FIG. 17), and the posture of the patient 5 can be adjusted. When the robot arm 400 is powered on, additional calibration may be performed. The robot arm 400 is ready to monitor the respiration and movement of the patient 5, and the matching between the robot arm 400 and the CT apparatus 600 is performed.

Thereafter, the immediately preceding procedure procedure image Ref-CT is acquired. 15B, the patient 5 is placed on the table 620 of the treatment field, the patient 5 enters the CT apparatus 600 by the table 620, and the CT apparatus 600 is operated , A procedure field image of the patient 5 is acquired (S230).

The pre-operation image and the treatment field image can be displayed on the display 350 (e.g., see Fig. 19). Thereafter, the pre-operation image and the treatment field image are matched (S240; see, for example, FIGS. 19 and 20). For example, the procedure field image is matched to the pre-operation image using the coordinate system of the table 620, and then converted into the procedure field image scale.

As the matching method, a method of rigid registration and non-rigid registration may be used together. Such a mutual information based rigid registration matches the pre-operation image with the operation field image. In complementary information-based rigid body matching, it is assumed that similar tissue regions with similar shades in one image will correspond to regions of similar shading in other images. Alternatively, other known methods of matching can be used. As a result of the matching, the insertion path is mapped to the procedure field image (e.g., see Fig. 20), and the coordinate system of the patient 5, the robot arm 400, and the CT apparatus 600 is matched.

After the images are registered, the robot arm 400 moves according to the control signal from the control unit 500 to hold the initial position, and the biopsy needle 470 moves to the position just before the entry point on the skin of the patient 5 (S250). The robot arm 400 moves over the patient 5 in accordance with the control signal transmitted from the control unit 500, as shown in Fig. 15C. Thereafter, as shown in Fig. 15D, the end effector 460 enters the CT apparatus 600 by the robot arm 400. [

Thereafter, the biopsy needle 470 is aligned along the insertion path 475 by the end effector 460 as shown in 15e and 15f. Biopsy needle 470 is rolled, punched, and yawed by robot arm 400 and end effector 460 and aligned along insertion path 475, as described in Fig. The tip of the biopsy needle 470 is positioned at about 1 cm from the entry point of the skin of the patient 5. The user or practitioner can visually confirm this process and confirm the alignment status. Alternatively, the controller 500 may automatically confirm whether the biopsy needle 470 provided in the end effector 460 of the robot arm 400 coincides with the insertion path 475 on the currently displayed procedure image.

If there is a difference in level between the procedure field image and the pre-operation image, an offset may occur when the planned insertion path 475 in the pre-operation image is mapped to the procedure field image through matching. Accordingly, the control unit 500 can compare the offsets immediately in real time to check how many offsets have occurred and calculate them. In this example, the control unit 500 calculates the offset and instructs the robot arm 400 so that the robot arm 400 moves the table 620 to the offset In order to remove the < / RTI >

Also, a process may be added in which the insertion path 475 is modified on the matched procedure field after being matched. For example, the insertion path 475 shown in the matched procedure field image can be modified using a user interface (e.g., a mouse). At this time, the amount of penetration by the modified insertion path 475 and the distance of the insertion path 475 may be automatically calculated and displayed on the display 350. For example, the display 350 may display an indicator (e.g., a number by vessel size) that helps the operator determine the insertion path 475 when adjusting or modifying the insertion path 475. Thereafter, the biopsy needle 470 can be precisely rearranged along the insertion path 475 by the robot arm 400 and the end effector 460.

Thereafter, in this example, a process for reducing the error due to breathing may be added between the confirmation of the alignment of the biopsy needle 470 and the insertion of the biopsy needle 470 into the insertion point (S260). For example, the CT device 600 is reactivated to display real-time video or breath monitoring information on the display 350. When the pre-treatment image is acquired at a specific respiratory level, the treatment plan mapped to the treatment field image through the matching is performed, so that the real-time breathing level of the patient 5 located on the table 620 is controlled by breathing It is preferable to adjust it to the level. Alternatively, it is possible that the procedure field image is acquired at a specific breathing level, and the real-time breathing level of the patient 5 is matched to this breathing level. When the breathing level is equal to the breathing level before the operation, the patient 5 temporarily breathes and a biopsy needle 470 is inserted into the body of the patient 5 to perform a biopsy. If the level is different between the real-time image at the moment of breathing and the procedure field mapped with the insertion path 475, the controller 500 calculates the difference and sets the difference (E.g., translation, rotation, operation of the end effector, etc.) of the robot arm 400 so as to remove the robot arm 400.

Thereafter, the alignment of the respiration and insertion path 475 with the biopsy needle 470 is confirmed, and the biopsy needle 470 is driven and moved by the motor from the end effector 460 according to the transmitted instruction, or The biopsy needle is fired by the triggering device to penetrate the insertion point of the skin to reach the target and to collect the tissue or to insert the biopsy needle for guide into the vicinity of the target (S270). The display 350 of the interventional treatment system may display a procedure image (e.g., a three-dimensional image) matched with a real-time image (e.g., a 2D image). The biopsy needle 470 appears on the real-time image, and the biopsy needle 470 is replaced with the biopsy needle image on the matched procedure field image together with the procedure field image. An insertion depth gauge bar 560 may be displayed on the display 350 to more accurately visualize the insertion depth of the biopsy needle 470.

The insertion path 475 is selected from among a plurality of insertion paths 475 in the cone-shaped insertion area, and the insertion path 475 can be changed in preparation of the procedure field. The insertion path may be arbitrarily selected from among a plurality of insertion paths 475 of the cylindrical insertion region, or may be inclined insertion. The insertion of the biopsy needle 470 may be performed as it is along the original insertion path 475 or when the haptic device of the master console 310 vibrates or approaches the target 100 is reached, the biopsy needle 470 may automatically stop. On the other hand, in a dangerous situation, the robot arm 400 retreats on the biopsy needle 470, and the robot arm 400 can automatically come out of the CT apparatus 600.

The sampling of the tissue by the biopsy needle 470 may be performed a plurality of times according to the structure of the end effector 460 and may include a process of rotating the biopsy needle 470 before and / or after insertion, It is also possible to perform biopsy in several places without completely removing the body 470 from the body of the patient 5. [ The biopsy needle 470 is pulled out by the second arm 440 or the end effector 460 and moved vertically by the second arm 440 and the second arm 440 and the sliding portion 460 are moved The robot arm 400 can be removed from the CT apparatus 600 by the controller 420 (S280). The master console 310 may be used for controlling the biopsy needle 470 by the robot arm 400 and the end effector 460.

16 is a diagram for another example illustrating the overall process of the intervention procedure guide method. The presented process is displayed at the top of the display 350 so that the operator can know at which stage the operator is at.

The system is divided into a planning stage (Navigation Stage) and a navigation stage (Navigation Stage). First, before the planning stage, the patient is acquired and segmented before the procedure. A high resolution image is obtained, a three-dimensional image of the patient is created, and the organs are divided on the image so that the control unit 500 can recognize the target and the dangerous organs. In the planning stage, the practitioner creates an insertion path for safely inserting the medical instrument into the body using the pre-procedure image. The patient's pre-scan image is obtained (pre-scan) in the procedure field, and the pre-operation image and the operation field image are registered. After obtaining the 3-D image according to the patient's procedure condition by matching, if necessary, correct the matched insertion path and make a final confirmation (Confirm). Thereafter, the control unit proceeds to a navigation step of guiding the robot arm (e.g., 400 in Figs. 4 to 10). The navigation step may include a positioning mode, a breathing level comparison confirmation mode, and a needle insertion mode. The robot arm 400 is driven to move the end effector to the insertion position and the posture control of the end effector 460 is performed so that the biopsy needle 470 is aligned along the insertion path. Thereafter, a real time image is acquired in the insertion mode of the biopsy needle 470, and when the respiration level and the pre-operation breathing level are compared and confirmed, the biopsy needle 470 reaches the target. The real-time image can visualize the current position of the biopsy needle 470.

Each process will be described in detail below.

17 is a diagram illustrating an example of a display screen in a segmentation mode in which a segmentation mode is selected in an upper menu bar 551 of the display 350 and a pre- And the pre-operation images 811, 812 and 813 are segmented and displayed. As a result of the segmentation, the target 100 included in the pre-operation image and anatomical structures (e.g., blood vessels, bones, organs) can be obtained as a three-dimensional set of voxels. A variety of segmentation techniques can be used, including an adaptive threshold. The display 350 may include pre-treatment images 811, 812, 813, e.g., an axial view, a sagittal view 812, and a coronal view 813 at different angles Can be displayed. In addition, the three-dimensional image 814 is displayed in another window, and the three-dimensional image 814 can be rotated and displayed at a required angle. Pre-treatment images 811, 812, and 813 are acquired by the CT apparatus 600 included in the interventional treatment system, or pre-treatment images 811, 812, and 813 are acquired by an image acquisition device that is separate from the interventional treatment system, 350 < / RTI >

Next, FIG. 18 shows an example of a display screen of a plan mode. In FIG. 18, an insertion path 475 is created with the plan mode selected in the upper menu bar 551 of the display 350, The generated insertion path 475 can be modified. In this example, the pre-treatment images 811, 812, and 813 in the segmentation mode and the three-dimensional image 814 are displayed on the display 350 of the plan mode. The insertion path 475 is generated separately from the interventional procedure system and is loaded on the display 350 of the interventional procedure system with the pre-operation images 811, 812, 813 or after the pre-operation images 811, Can be created on the system. In the method of setting the insertion path, a target point on the target 100 is selected as a user interface (UI) such as a mouse, and an insertion point is selected so that an insertion path 475 is generated. The insertion path 475 can adjust the orientation to the user interface according to the review of the practitioner. When the insertion path 475 is modified, the insertion path 475 in the images 811, 812, and 813 at different angles also automatically reflects the correction contents.

The boundary around the target 100 can be viewed three-dimensionally while rotating the three-dimensional image 814. [ If desired, the user may select skin or other non-critical structures while viewing the three-dimensional image 814. In addition, a three-dimensional image 815 of the target 100 is displayed, so that it can be referred to when generating the insertion path 475.

An obstacle such as a blood vessel is checked while simulating the biopsy needle 470 ahead of and backward along the insertion path 475 using the insertion simulation image 817 (first confirmation window 816; second confirmation window) . For example, it can be seen that blood vessels appear on the insertion path and disappear. This will be further described below.

19 and 20 are views showing an example of a screen of a display in the matching mode. 19 shows that the insertion path 475 described in FIGS. 17 and 18 is created in the upper windows of the display 350 with the registration mode selected in the upper menu bar 551 of the display 350 Pre-treatment images 811, 812, and 813 are displayed, and the procedure window images 911, 912, and 913 are displayed on the lower side windows of the display 350. FIG. The procedure field images 911, 912, and 913 are data obtained immediately before the operation, and may have lower image quality than the pre-operation image. According to the matching command, the pre-operation images 811, 812, and 813 and the operation field images 911, 912, and 913 are matched with each other to obtain three-dimensional data matching the current patient condition.

As a matching method, a level-set motion registration method can be used. The pre-treatment images 811, 812, and 813 are displayed with images 811, 812, and 813 (e.g., an axial view, a cervical view, a coronal view) in different directions, and a target 100 (shown in green) is shown in each of the images. The procedure field images 911, 912, and 913 are also displayed in different directions (eg, 911, 912, and 913, eg, an axial view, a cervical view, and a coronal view) corresponding to the pre-operation images 811, The levels of the procedure field images 911, 912 and 913 corresponding to the pre-operation images 811, 812 and 813 are found. At this level, matching occurs. When it is matched, an insertion path 475 appears in the procedure field images 911, 912, and 913.

On the other hand, if the patient 5 is placed in a specific posture such as lying down or lying down in the process of generating the insertion path 475 of the pre-operation images 811, 812, and 813, the patient 5 may take the specific posture . If the procedure is planned in a prone position, if the patient (5) enters the treatment field and lies in a different posture, there may be slight differences in the position of the blood vessels or organs. To eliminate the difference, preoperative images (811,812,813) A process of matching the procedure field images 911, 912, and 913 may be added.

Matching may be Rigid Transformation and Level-set Registration, which is non-rigid matching. Fig. 21 shows the contents related to this.

22 and 23 are diagrams for explaining an example of a window in which a treatment plan can be modified. Even if the matching is performed, as shown in FIG. 22 and FIG. 23, when the levels of the pre-operation images 811, 812, and 813 and the procedure field images 911, 912, and 913 are slightly different, Can be switched. Simultaneously rotating the 3D image 814 in this window or simulating the forward and backward movement of the biopsy needle 470 along the insertion path 475 in the insertion simulation images 816 and 817, Whether the patient is passing through a blood vessel, and the like, and correct the insertion path 475 according to the treatment plan.

The insertion simulation images 816 and 817 are controlled by the simulation module of the control unit 500. [ The insertion simulation images 816 and 817 show obstacles that the biopsy needle 470 is encountered in the process of inserting the biopsy needle 470. The target 100 and the insertion path 475 are displayed in the first confirmation window 817 of the insertion simulation image and the end 476 of the biopsy needle 470 to be displayed by the simulation is the virtual marker Is shown. When the tip 476 of the biopsy needle 470 is moved forward and backward along the insertion path 475 with the mouse, an insertion corresponding to the end 476 of the biopsy needle 470 is performed in the second confirmation window 816 of the insertion simulation image. Sectional image 478 perpendicular to the path 475 varies along the virtual end 476 of the biopsy needle. The image of the first confirmation window may show an image rotating around the insertion path 475. [ In this section, the parts classified as dangerous organs can be displayed separately (for example, in red). In addition, the distance from the insertion route to the dangerous organ is automatically displayed, so that the practitioner can be informed of the probability of actual invasion irrespective of the screen magnification. Alternatively, if the distance is within the error range, the simulation module may display a warning alarm message.

23 in which the insertion path 475 is slightly modified as compared to the insertion path 475 shown in Fig. 22, when the end 476 of the biopsy needle 470 approaches the target 100 in the first confirmation window, 2 confirmation window, it is possible to confirm that the blood vessel 105 invading the biopsy needle 470 is present. In this case, the insertion path 475 is modified so that the invasion to the blood vessel does not occur, and the end 476 of the biopsy needle 470 is advanced and retracted again with the mouse to confirm the infiltrating blood vessel or structure, (475).

FIGS. 24 to 26 are views showing examples of a display screen in the navigation mode. The insertion path 475 is finally confirmed by the practitioner after the insertion and insertion path correction process of the interventional system is performed in the above process. Thereafter, a navigation mode is selected in the upper menu bar 551 of the display 350, and the position of the robot arm 400 is controlled. For example, when the move entry position menu is selected in the menu bar 555 on the right side of the display 350, the biopsy needle 470 is moved by the robot arm 400 along the insertion path 475 (See Fig. 15D). The biopsy needle 470 moves to the vicinity of the insertion point 471 by the robot arm 400 and moves along the insertion path 475 by an operation such as rolling, pitching, and yawing of the end effector 460, ) Are aligned (see Fig. 15E).

The optical camera 480 may be installed in the robot arm 400 or the end effector 460. [ A camera image 917 of the skin of the patient 5 obtained by the camera 480 and the biopsy needle 470 aligned thereon is displayed on the display 350. [ Through the camera image 917, the practitioner can visually confirm the procedure situation in a space not exposed to radiation. The insertion point 471 can be directly displayed on the patient 5 by the laser beams L1 and L2 and displayed on the display of the control unit 500 through the image of the camera 917. [

The display 350 displays the matched procedure field image 911 with the insertion path 475 confirmed and the three-dimensional images 915 and 916 are displayed. The target 100 and the insertion path 475 are displayed on the matched procedure field image 911. [ And shows the alignment state of the insertion path 475 and the actual biopsy needle 470. [ The orientation of the master console 310 is displayed in the three-dimensional image 916. Thereafter, when ready is selected in the right menu bar 555, the current orientation of the master console 310 is automatically adjusted to the orientation of the biopsy needle 470, as shown in FIG. 25, 916, the indication of the master console 310 is fitted to the insertion path 475. [

Referring to FIG. 26, the CT apparatus is operated in a ready state, and a real-time image 918 is displayed on the display 350. FIG. Thereafter, a process of matching the breathing level is operated to reduce the error due to breathing. As an example of a method of comparing respiratory levels, in the case where the target 100 is affected by the respiration of the patient 5, in order for the procedure to proceed as planned, the patient 5 is required to acquire the pre- It is preferable that the breathing level A be equal to the breathing level B in the real-time image 918 (breathing level after alignment). Therefore, the patient 5 is allowed to breathe at a specific breathing level (A: the patient 5 is maximally breathed or the breath is maximally breathed out), and then the target CT apparatus 600 is used to set the target 100 to obtain the respiration level B of the patient 5 with the breathing level A. [ For example, after the breathing level (A) is acquired at the acquisition of the pre-treatment image (811), the respiration level of the patient (5) is checked through the breathing level checking device in the procedure field, When the respiration level B of the patient 5 of the user matches the respiration level A of the matched image, the display 350 may be displayed (919; If necessary, the biopsy needle 470 can be configured to be automatically activated upon breathing level confirmation. For the measurement of respiratory levels (A, B), a respiratory measurement method using a separate breath level measuring instrument (eg, a pressure belt type, a respiratory meter using an InfraRed marker, etc.) may be applied. Also, a method of matching the breathing level using a 2D CT fluoroscopy real-time image obtained in a procedure field can be used.

The control unit may have a section acquisition module for generating a plurality of two-dimensional sectional images related to the target 100 in advance from the three-dimensional image or the matching three-dimensional image in the procedure field before the operation. The generated multiple 2D cross-sectional images can be compared with real-time 2D CT fluoroscopy screens in the breathing level matching process. A two-dimensional sectional image closest to the real-time image is selected, and the patient can be guided to breath in a real-time image according to the breathing level of the two-dimensional sectional image.

27 and 28 are diagrams for explaining an example of a display screen in the insertion mode, in which an insertion mode is selected when an indication or an alarm that the respiration levels described above are provided, and the patient 5 The biopsy needle 470 is inserted into the target 100 within the breath taking time. The selection of the insertion mode is possible before the breathing level is confirmed and can be done by the insertion button 311 of the master console 310. The control unit 500 may limit the insertion of the biopsy needle 470 if the respiration level matching is not performed in the insertion mode. For this purpose, the procedure can be performed in the order of the operator's insertion mode selection, breathing level matching, insertion of the biopsy needle 470 (drive of the end effector 460).

The interventional procedure system can guide the insertion of the biopsy needle 470 in real time. 29, when the CT apparatus 600 is operated, the clutch 313 of the master console 310 is depressed, and when the insertion button 311 is pressed, the biopsy needle 470 is inserted into the insertion point 471). In this process, an insertion gage bar 560 indicating the insertion amount of the biopsy needle 470 with respect to the entire insertion path 475 is displayed on the procedure field image 911 matched with the real-time image 918. The control unit 500 can transmit the information about the position of the biopsy needle 470 more accurately to the operator by causing the insertion depth gauge bar 560 to appear together with the real time image 918. Through this information, .

When the insertion gauge bar 560 reaches the target line 557, insertion of the biopsy needle 470 can be automatically stopped. Of course, the practitioner can see the insertion gauge bar 560 and stop the insertion by experience. The target line 557 may be displayed on the insertion gauge bar 560, the real-time image 918, or the matched procedure field image 911. [ In this example, the target line 557 is displayed on the insertion gauge bar 560 side. In the camera image 917, the skin of the patient 5 and a biopsy needle 470 penetrating the insertion point 471 are displayed.

On the other hand, as shown in FIGS. 27 and 28, the real-time image 918 displays in real time that the biopsy needle 470 advances toward the target. However, there may be a time delay until it is acquired by the CT device 600 and displayed on the display 350. [ For example, there may be a time delay of about one second such that the position of the biopsy needle 470 shown in the real-time image 918 may be the position of the biopsy needle 470 one second before the current time. Thus, if the biopsy needle 470 in the real-time image 918 determines the end of insertion at the time of reaching the target, there is a risk of striking deeper than the planned position. To solve this problem, the insertion gauge bar 560 complements the time delay of the biopsy needle 470 displayed in the real-time image 918. The control unit 500 simultaneously displays the accurate current position of the current biopsy needle 470 in the real time screen. The position of the actual biopsy needle 470 may be shown using an insertion gauge bar. At this time, the controller may judge whether the locus of the medical instrument matches the locus of the medical instrument of the real-time image and display it on the display. The location information of the biopsy needle 470 displayed through the real-time image, the velocity information, the position information received through the slave robot, the position information of the biopsy needle 470, and the velocity information can be displayed on the screen. When the insertion gauge bar 560 reaches the target line 557, the biopsy needle 470 can be automatically stopped. For example, the control unit recognizes the initial spatial coordinates of the end of the biopsy needle 470 mounted on the end effector 460 and determines the position of the end of the biopsy needle 470 from the motion information of the robot arm and the end effector driving unit motor The spatial coordinates can be calculated. Alternatively, an external optical instrument may be used to indicate the location of the biopsy needle 470.

A shape (e.g., a shape of a needle) related to the positional information of the biopsy needle 470 and related information may be superimposed on the real time image (Overlay) on the display using the augmented reality technique. The controller may judge whether the locus of the medical instrument matches the locus of the medical instrument of the real-time image and display it on the display. The location information of the biopsy needle 470 displayed through the real-time image, the velocity information, the position information received through the slave robot, the position information of the biopsy needle 470, and the velocity information can be displayed on the screen.

Alternatively, a stop line 559 may be set in front of the target 100 on the real-time image 918 to stop the biopsy needle 470 when the biopsy needle 470 reaches the stop line 559 ). Of course, in the real-time image 918, the biopsy needle 470 is stopped at the stop line 559 located in front of the target 100, but actually reaches the target 100 due to the above-described time delay. It is also possible to control the progress of the biopsy needle 470 by a skilled practitioner.

Also, as described above, in the interventional procedure method and system of the present example, the biopsy needle 470 includes only a guide needle (e.g., a cis), or a needle for tissue and / or tissue retrieval (e.g., an inner stylet) It is possible to include both. Therefore, by using the insertion gauge bar 560, the target line 557, the real-time image 918, the stop line 559, or the like, the position where the end of the sheath reaches is guided, 100 of the tissue. As such, the interventional procedure system guides the interventional procedure using the insertion gauge bar 560 that shows the biopsy needle 470 in real time and considers the time delay of the display 350, thereby improving the accuracy, safety, and convenience do.

The biopsy needle 470 attached to the end effector 460 is used to perform a biopsy by the robot arm 400 or the end effector 460 after the release mode is selected on the display 350. [ The needle 470 is withdrawn from the patient 5 and the end effector 460 exits the CT apparatus 600. [ For example, if a practitioner presses the clutch 313 of the master console 310 and raises the insertion button 311, the end effector 460 pulls the biopsy needle 470 out of the patient 5, The arm 440 and the third arm 450 are moved so that the biopsy needle 470 can be retracted from the patient 5 as it is. Then, the robot arm 400 comes out of the CT apparatus 600. As another example, the guide needle may remain inserted in the patient 5, and the needle for tissue collection may be extracted. As another example, in the insertion mode, only the guide needle is inserted into the patient 5, and when the release mode is selected, the end effector 460 places the guide needle and the robot arm 400 can come out of the CT apparatus 600 have. In this case, a biopsy can be performed by inserting a biopsy needle for tissue sampling into the guide needle inserted into the patient 5 by the practitioner.

The interventional treatment system may set a virtual wall around the target to signal the practitioner at each step as the biopsy needle 470 enters. First, a virtual wall is set a certain distance from the target in the pre-procedure stage. The virtual wall may be displayed in spherical or related graphics on the three-dimensional image of the treatment target. When the procedure is started, the practitioner starts inserting the biopsy needle 470 using the insertion button 311 of the master console 310. When the biopsy needle 470 reaches a position corresponding to the virtual wall, a warning message is displayed on the display of the control unit 500 or the master console 310 is vibrated to notify the operator that the specific area has been reached. This allows the practitioner to proceed with the procedure in a step-by-step manner.

Various embodiments of the present disclosure will be described below. The practice of the invention is possible by various combinations thereof.

(1) An interventional treatment system using medical images, including a medical instrument, a control unit for storing an insertion path of a medical instrument, an image acquisition apparatus capable of acquiring real-time images of a patient, and a display capable of displaying real- And the control unit uses the medical image to simultaneously display the procedure image and the position information of the medical instrument on the display.

(2) Interventional system using medical image which is 2D CT fluoroscopy image in real time.

(3) a slave robot to which a medical tool is attached, and the control unit receives the motion information from the slave robot and generates the position information of the medical tool.

(4) The slave robot includes a robot arm having a joint driving part, and an end effector mounted on the robot arm and including a medical tool driving part. The control part controls the position of the medical tool from the movement of the joint driving part and the medical tool driving part Is calculated using a medical image.

(5) The control unit compares the position of the medical tool with the insertion path, and uses the medical image to stop the medical tool driving unit when the position of the medical tool reaches the end of the insertion path.

(6) The position of the medical instrument is an interventional system using a medical image, which is a gauge bar indicating the position of the current medical instrument relative to the length of the insertion path.

(7) An interventional procedure system using a medical image that superimposes the position of a medical tool on a real-time image and displays it on a display by an augmented reality technique.

(8) The control unit judges whether the locus of the medical instrument matches the trajectory of the medical instrument of the real-time image and displays it on the display.

(9) An intervention procedure guide method using a medical image, comprising the steps of generating a trajectory of a medical instrument in a pre-procedure image including a target, acquiring a procedure field image, A step of aligning the medical instrument with the insertion path by moving the robot arm equipped with the medical tool to the vicinity of the entry point, And acquiring a real-time image including an image of the medical instrument, and supplementing the time delay of displaying the medical instrument to the real-time image, and visualizing the current position of the medical instrument on the display. .

(10) A method for guiding an intervention procedure using a medical image, characterized in that, at the step of visualizing the current position of the medical instrument, the operation depth of the robot arm is calculated to calculate the insertion depth of the end of the medical instrument.

(11) A method of guiding an intervention procedure using a medical image, characterized by displaying a current insertion depth of a medical instrument in an inserted depth field image and a real-time image in an insertion depth gauge bar.

(12) A step of indicating or expressing the post-alignment respiration level and the pre-treatment respiration level before visualizing the current position of the medical instrument after aligning the medical instrument according to the insertion path A method of guiding intervention procedures using medical images.

According to the guide method and the intervention procedure system using the medical image according to the present disclosure, the automation, accuracy, stability, and convenience of the intervention guide method using medical images are improved.

Table (620) Image acquisition device (600) Master console (310) Clutch (313)
Insert button 311 Rotate button 312 Display 350 Control unit 500
Insertion depth gauge bar (560) Upper menu bar (551)
The robot arm 400 includes a base 410, a sliding portion 420, a first arm 430,
The second arm 440, the third arm 450, the end effector 460, the medical instrument 470,
Insertion path 475 housing 481 support portion 482 mover drive portion 35,

Claims (12)

1. An interventional treatment system using medical images,
Medical tools;
A control unit for storing an insertion path of the medical instrument;
An image acquisition device capable of acquiring a real time image of a patient; And,
And a display capable of displaying a real-time image,
The control unit uses the medical image to simultaneously display the procedure image and the position information of the medical instrument on the display. The method according to claim 1,
The real-time image is a 2D CT fluoroscopic image, which is a medical image using an interventional system. The method according to claim 1,
And a slave robot to which the medical instrument is mounted,
The control unit uses the medical image to receive the motion information from the slave robot and generate the position information of the medical instrument. The slave robot according to claim 3,
A robot arm having a joint drive part; And,
An end effector mounted on the robot arm and including a medical tool drive,
The control unit uses the medical image to calculate the position information of the medical tool from the motion of the joint driving unit and the medical tool driving unit. The method of claim 4,
The control unit compares the position of the medical tool with the insertion path, and uses the medical image to stop the medical tool driving unit when the position of the medical tool reaches the end of the insertion path. The method according to claim 1,
The position of the medical instrument is an interventional system using a medical image, which is a gauge bar indicating the position of the current medical instrument relative to the length of the insertion path. The method according to claim 1,
Wherein the control unit superimposes the position of the medical tool on the real time image by the augmented reality technique and displays it on the display. The method according to claim 1,
Wherein the control unit judges whether the trajectory of the medical instrument matches the trajectory of the real-time image with the trajectory of the medical instrument, and displays it on the display. In an intervention procedure guide method using a medical image,
Generating a trajectory of a medical instrument in a pre-procedure image including a target;
Obtaining a procedure field image;
A step of matching the pre-operation image with the operation field image and displaying the insertion path on the matched procedure field image;
Moving the robot arm equipped with the medical tool around the entry point and aligning the medical tool along the insertion path; And
Acquiring a real-time image including an image of the medical instrument, supplementing the time delay of the medical instrument display to the real-time image, and visualizing the current position of the medical instrument on the display, . The method of claim 9,
In the step of visualizing the current position of the medical instrument,
And calculating the insertion depth of the end of the medical instrument by calculating the motion of the robot arm. The method of claim 9,
Wherein the current insertion depth of the medical instrument is displayed by an insertion depth gauge bar in the matched procedure field image and the real time image, respectively. The method of claim 9,
After the step of aligning the medical tool according to the insertion path, prior to the step of visualizing the current position of the medical tool,
And displaying or comparing the respiration level after the alignment and the respiration level before the operation.

Images