JP5784388B2 - Medical manipulator system - Google Patents

Description

  The present invention relates to a medical manipulator system.

  Conventionally, a medical manipulator system including a master manipulator that is operated by an operator and a slave manipulator that performs a treatment based on the operation of the master manipulator is known as a surgery support system.

For example, Patent Document 1 discloses a medical robot system that includes a master manipulator and a slave manipulator, and a display screen that displays an image of a treatment object.
The medical robot system described in Patent Document 1 includes two master input devices operated by an operator's hand and two slave arms connected to each master input device, and the operation of the master input device. The slave arm that operates in response to can be switched using a switch.
The medical robot system described in Patent Document 1 includes a mode that operates as a pointing device that moves a cursor that specifies an input position and coordinates on a display screen as an operation mode of the master manipulator.

Special table 2009-512514 gazette

  However, in the medical robot system described in Patent Document 1, since the operation feeling of the slave arm and the operation feeling of the cursor are extremely different, the master input device is used as a mode for operating the slave arm by the master input device and the pointing device. When the mode to be switched is switched, the operator may feel uncomfortable until the operator gets used to the operational feeling in the switched mode.

  The present invention has been made in view of the above-described circumstances, and an object thereof is to provide a medical manipulator system that can be operated without a sense of incongruity.

In order to solve the above problems, the present invention proposes the following means.
One aspect of the present invention acquires a master manipulator that issues an operation command, a slave manipulator to which a surgical instrument for performing a treatment on a treatment target is attached, and a first image including a part of the slave manipulator and the treatment target. A first image composing device, a second image composing device constituting an image different from the first image and including a virtual manipulator image corresponding to a part of the slave manipulator, and the first image wherein the display device for displaying either one of said second image, and selecting means for one of the images from said first image and the second image is selected, and the first image configuration device virtual a selection unit for setting the operation target by the master manipulator one of the manipulators is switched based on the selection by the selection means, Conversion that converts the operation command issued by the master manipulator into a new command so that the coordinate system on the display device of the image selected by the selection means matches the coordinate system on the operation of the master manipulator and means, and a driving means for operating the slave manipulator according to the new command in the case where the selection unit by with the first image configuration device is set to the operation target, the second image configuration device, the An image for generating the second image so that the image of the virtual manipulator becomes an image reflecting the posture and motion corresponding to the operation by the master manipulator when the virtual manipulator is set as an operation target by the selection unit. having a processing unit, a medical manipulator system, wherein a.

  Further, the first image construction device is an endoscope device, and the second image construction device generates a virtual manipulator image that is displayed as if the slave manipulator is operating according to the new command, It is preferable that the second image is generated by combining the virtual manipulator image with a base image in which the treatment target is imaged.

  The second image construction device may change the viewpoint by updating the base image based on the operation command from the master manipulator.

Further, the medical manipulator system of the present invention is used together with an inspection apparatus that images the treatment target during the treatment work on the treatment target, and the second image construction device serves as the base image during the treatment work. An image imaged by the inspection apparatus may be used.
Moreover, the said inspection apparatus may be provided in the said slave manipulator.

  ADVANTAGE OF THE INVENTION According to this invention, the medical manipulator system which can be operated without discomfort can be provided.

1 is an overall view showing a medical manipulator system according to a first embodiment of the present invention. It is a block diagram of the medical manipulator system. It is a figure for demonstrating operation | movement at the time of use of the medical manipulator system. It is a block diagram which shows the structure of a part of medical manipulator system of 2nd Embodiment of this invention.

(First embodiment)
A medical manipulator system 1 according to a first embodiment of the present invention will be described. FIG. 1 is an overall view showing the medical manipulator system of the present embodiment. FIG. 2 is a block diagram of a medical manipulator system.
As shown in FIGS. 1 and 2, the medical manipulator system 1 includes a master manipulator 2, a slave manipulator 7, a display device 10, and a control device 20.

  The master manipulator 2 is provided to transmit the movement of the operator Op to the slave manipulator 7, and a switch 6 (selection means) for switching between the master arm 3 moved by the operator Op and an image displayed on the display device 10. ).

  The master arm 3 has a multi-axis structure that can move according to the movable range of the slave manipulator 7 and the surgical instrument 110. Further, the master arm 3 includes a displacement sensor 4 that generates a change in position and orientation of the master arm 3 as an electrical signal, and an arm drive unit 5 that operates the master arm 3 according to a drive signal from a master control unit 22 described later. Is attached.

  The electrical signal generated by the displacement sensor 4 is output to the control device 20 and serves as an operation command for operating the slave manipulator 7. As described above, the master manipulator 2 receives an operation by the operator Op and issues an operation command based on the operation by the operator Op.

  The switch 6 is configured to output a predetermined switching signal to the control device 20 when pressed. In FIG. 1, the switch 6 is disposed on the floor near the manipulator, but the input mechanism is not particularly limited, and various switches may be appropriately selected.

  The slave manipulator 7 includes a slave arm 8 to which the endoscope apparatus 100 (first image forming apparatus) and the surgical instrument 110 are respectively attached, and an actuator that operates the endoscope apparatus 100, the surgical instrument 110, and the slave arm 8, respectively. 9 (driving means). The endoscope device 100, the surgical instrument 110, and the actuator 9 that operates the slave arm 8 all operate according to a drive signal output from the control device 20.

  The endoscope apparatus 100 provided in the slave manipulator 7 includes an image acquisition unit 101 that acquires an image (first image) including a part of the slave manipulator 7 and a treatment target. In the present embodiment, a part of the slave manipulator 7 included in the first image refers to a part of the surgical instrument 110 and a part of the slave arm 8. That is, the image acquisition unit 101 can acquire an image in a state where a treatment target is being treated using the slave manipulator 7. In the present embodiment, the endoscope apparatus 100 acquires two images that can be viewed stereoscopically and outputs them to the display apparatus 10 as electrical signals.

  The surgical instrument 110 provided in the slave manipulator 7 is provided for the purpose of performing a treatment on a treatment target within the field of view of the image acquisition unit 101. The type of the surgical instrument 110 is not particularly limited, and a known surgical instrument can be appropriately employed depending on the content of the treatment. The surgical instrument 110 is not limited to a surgical instrument that can be attached to the slave manipulator 7, and may be a surgical instrument that cooperates with the slave manipulator 7.

  The display device 10 is attached to the same base as the master manipulator 2 and is installed in front of the operator Op. The display device 10 includes an image processing unit 11 that generates a stereoscopically viewable image based on the two images acquired by the image acquisition unit 101, and a display panel 12 made of, for example, a liquid crystal panel or an organic EL panel. Two images that can be configured can be displayed. In the present embodiment, the display device 10 is attached with 3D glasses 13 that distribute the two images acquired by the image acquisition unit 101 to both eyes. The operator Op of the medical manipulator system 1 can wear the 3D glasses 13 and view the image displayed on the display panel 12 as a stereoscopic image through the 3D glasses 13. The display panel 12 may be a panel that supports autostereoscopic viewing. In this case, the 3D glasses 13 do not need to be attached to the display device 10.

  As shown in FIGS. 1 and 2, the control device 20 includes a main control unit 21, a switching unit 24, and an inspection image search system 30 (second image construction device).

  The main control unit 21 is electrically connected to the master manipulator 2, the switching unit 24, and the inspection image search system 30 via signal lines, and includes a master control unit 22 and a slave control unit 23.

  An operation command issued from the master manipulator 2 is input to the master control unit 22. Further, the master control unit 22 outputs the input operation command to the slave control unit 23. Further, the master control unit 22 outputs a driving signal to the arm driving unit 5 in accordance with the postures of the slave manipulator 7 displayed on the display device 10 and the virtual manipulator 40 to be described later. The posture of the (gripping part) is converted.

  The slave control unit 23 receives an operation command issued from the master control unit 22. Further, the slave control unit 23 outputs a drive signal for operating the slave manipulator 7 to the switching unit 24 in accordance with the operation command. Although details will be described later, the drive signal output from the slave control unit 23 may be output to the slave manipulator 7 after the content (coordinate system) is converted in the switching unit 24.

  The switching unit 24 includes a selection unit 25 that selects any one of an endoscope apparatus 100 provided in the slave manipulator 7 and a virtual endoscope 41 provided in a virtual manipulator 40 described later as an operation target, and a selection unit Conversion means 26 for converting the coordinate system of the drive signal for the operation target selected by 25, and a signal output unit 27 for outputting the drive signal whose coordinate system has been converted by the conversion means 26 to the operation target. The virtual manipulator 40 is a virtual manipulator that is generated by the examination image search system 30 and displayed as an image on the display device 10.

  The selection unit 25 is electrically connected to the switch 6 and switches between the endoscope apparatus 100 and the virtual endoscope 41 each time the switch 6 is pressed, and sets the operation target. When the endoscope device 100 is selected as an operation target in the selection unit 25, an image acquired by the image acquisition unit 101 of the endoscope device 100 is output to the display device 10. When the virtual endoscope 41 is selected as an operation target in the selection unit 25, an image configured by the examination image search system 30 is output to the display device 10.

The conversion unit 26 drives the drive signal so that the coordinate system on the display screen in a state where the selected operation target is displayed on the display panel 12 of the display device 10 matches the coordinate system on the operation of the master manipulator 2. Convert the coordinate system of. The coordinate system on the display screen is a three-dimensional coordinate system in a stereoscopic image formed when an image displayed on the display panel 12 is viewed using the 3D glasses 13.
The drive signal whose coordinate system has been converted by the conversion means 26 becomes a new command having a coordinate system different from the operation command issued from the master manipulator 2.

  When the operation target is the endoscope apparatus 100, a drive signal is output from the signal output unit 27 to the endoscope apparatus 100 and the surgical instrument 110. When the operation target is the virtual endoscope 41, a drive signal is output from the signal output unit 27 to the examination image search system 30.

The examination image search system 30 is a system that images information obtained when a patient to be treated is examined and outputs the information together with the image of the virtual manipulator 40 to the display device 10.
As shown in FIG. 2, the examination image retrieval system 30 configures a data server 31 in which information obtained when examining a patient or the like is stored, and a base image based on the information stored in the data server 31. The image forming unit 32 includes an image processing unit 33 that processes an image formed by the image forming unit 32, and an image output unit 34 that outputs the image processed by the image processing unit 33 to the display device 10.

  The data server 31 stores images (ultrasonic images) obtained by ultrasonic image diagnosis performed on a patient to be treated before surgery.

The image constructing unit 32 constructs a base image composed of a three-dimensional image based on the ultrasound image stored in the data server 31. Further, the drive signal output from the switching unit 24 is input to the image configuration unit 32.
The image construction unit 32 updates the base image in response to the drive signal output from the switching unit 24. For example, when the operator Op moves the virtual endoscope 41 or changes the posture of the virtual endoscope 41 using the master manipulator 2, the image configuration unit 32 changes the viewpoint of the virtual endoscope 41. An image that appears as if it has been changed to the position or angle of is set as a new base image.

The image processing unit 33 selects, from a predetermined database, a virtual manipulator image (virtual manipulator image) that can be stereoscopically viewed corresponding to the posture and motion defined in the drive signal from the conversion means 26. The virtual manipulator image is an image including a part of the slave arm 8 and a part of the surgical instrument 110 or an image imitating them, and an image reflecting the posture and movement corresponding to the operation by the master manipulator 2. is there.
The image processing unit 33 generates an image (second image) that appears as if the virtual manipulator 40 is operating based on the operation of the master manipulator 2 on the base image by superimposing the virtual manipulator image on the base image. To do.
Further, the image processing unit 33 outputs the generated second image to the image output unit 34.

  The image output unit 34 temporarily stores the second image output from the image processing unit 33 and sequentially outputs the temporarily stored second image to the display device 10 at a predetermined frame rate. As a result, the display device 10 displays a three-dimensional moving image in which the virtual manipulator 40 appears to operate according to the drive signal (new command) converted by the conversion means 26.

The operation of the medical manipulator system 1 of the present embodiment will be described. FIG. 3 is a diagram for explaining an operation during use of the medical manipulator system 1.
As shown in FIG. 1, when using the medical manipulator system 1, the operator Op wears 3D glasses 13 and performs a treatment on a treatment object while viewing an image displayed on the display panel 12 of the display device 10. . By depressing the switch 6, an image of the patient during operation (first image, see FIG. 1) acquired by the image acquisition unit 101 (see FIG. 2) of the endoscope apparatus 100 provided in the slave manipulator 7, An image (second image, see FIG. 3) based on the ultrasonic image obtained by the preoperative examination can be switched and displayed on the display panel 12.

The first image includes an actual image of the slave manipulator 7, and the second image includes the image of the virtual manipulator 40 selected by the image processing unit 33.
When the first image is displayed, the coordinate system of the slave manipulator 7 on the display device 10 and the coordinate system for operation of the master manipulator 2 are the same. Furthermore, when the second image is displayed, the coordinate system of the virtual manipulator 40 on the display device 10 and the coordinate system in operation of the master manipulator 2 are the same. For this reason, the operator Op can switch and operate the slave manipulator 7 and the virtual manipulator 40 without a sense of incongruity.

  Whether the virtual endoscope 41 is moved to a desired position in the second image, which is a stereoscopically viewable image, by updating the base image corresponding to the operation of the master manipulator 2 in the image construction unit 32. Thus, the operator Op can recognize the preoperative examination image by the same operation as that of the master manipulator 2 when the real endoscope apparatus 100 is used. As a result, the information displayed on the first image can be intuitively compared with the image displayed on the second image, and work can be performed efficiently.

(Modification 1)
Next, a modified example of the first embodiment will be described with reference to FIGS.
In this modification, the configuration of the inspection image search system 30 is different.
The data server 31 stores images (CT images) obtained by computed tomography (CT) using X-rays performed on a patient to be treated before surgery.
The image construction unit 32 constructs a base image composed of a three-dimensional image based on the CT images stored in the data server 31.
This modification is the same as the above-described embodiment except that the base image is based on the CT image.
Even with such a configuration, the same effects as those of the above-described embodiment can be obtained.

(Modification 2)
Next, another modification of the first embodiment will be described with reference to FIGS.
In this modification, the configuration of the inspection image search system 30 is different.
The data server 31 stores images (MRI images) obtained by nuclear magnetic resonance imaging (MRI) performed on a patient to be treated before surgery.
The image construction unit 32 constructs a base image by searching for one slice image selected by the operator Op from the MRI images stored in the data server 31. As a method for selecting a base image, for example, a method of displaying a plurality of slice images as candidates for a base image on the display device 10 and selecting one slice image by moving the master arm 3 can be cited.
This modification is the same as the above embodiment and modification 1 except that the base image is a slice image of an MRI image.
Even with such a configuration, the same effects as those of the above-described embodiment can be obtained.

  Further, the MRI image is not limited to the slice image (two-dimensional), and a three-dimensional image may be configured by the same method as in the first modification. Even with such a configuration, the same effects as those of the above-described embodiment can be obtained.

  In addition, the data server 31 in the examination image search system 30 may incorporate not only patient examination information but three-dimensional (or two-dimensional) CG data such as a general human anatomical model. Even with such a configuration, the same effects as those of the above-described embodiment can be obtained.

(Second Embodiment)
Next, the medical manipulator system 1 according to the second embodiment of the present invention will be described. FIG. 4 is a block diagram illustrating a partial configuration of the medical manipulator system according to the present embodiment.
As shown in FIG. 4, the medical manipulator system 1 </ b> A of the present embodiment is different in that an intraoperative inspection device 35 is connected to the inspection image search system 30.

  The intraoperative examination apparatus 35 is an apparatus such as an ultrasonic endoscope, a fluorescence endoscope, a real-time tomography apparatus, or a real-time MRI apparatus, and is attached to the slave manipulator 7 or both of the slave manipulators 7 for a patient to be treated. It is arranged in the vicinity. The intraoperative inspection device 35 performs observation and inspection on the patient during the operation, and stores information obtained as a result of the observation and inspection in the data server 31. As a result, the examination image search system 30 constructs a base image in the image construction unit 32 based on the information acquired by the intraoperative examination apparatus 35, and the virtual manipulator converts the base image into the base image in the image processing unit 33 as in the first embodiment. A second image is formed by superimposing the images.

  According to the medical manipulator system 1A of the present embodiment, patient information is acquired by the intraoperative inspection device 35, which is a device different from the endoscope device 100, during treatment of the patient. The second image configured based on the information can be immediately displayed on the display device 10.

As mentioned above, although embodiment of this invention was explained in full detail with reference to drawings, the concrete structure is not restricted to this embodiment, The design change etc. of the range which does not deviate from the summary of this invention are included.
For example, preoperative and intraoperative inspection devices or medical images used in the first and second embodiments are not limited to X-rays and MRI, but include scintigraphy, positron tomography (PET), fluoroscopy, infrared imaging, and blood imaging. The method is not limited as long as it is a known inspection device such as a system. In these cases, the same effects as those of the above-described embodiment can be obtained.

  The display device 10 also includes an operation screen including buttons and icons for setting the operation mode of the medical manipulator system 1, 1 A, an image such as a document or a figure, or medical information such as an electrocardiogram. Two images can be displayed. Also in this case, the second image can be operated by operating the master arm 3.

  A head mounted display may be adopted as the display device.

  The inspection image search system 30 may also include a gesture input function that determines that a specific operation input has been made when the master arm 3 performs a specific operation.

  In each of the above-described embodiments, an example in which an image displayed on the display panel 12 is switched using the switch 6 has been described. However, a display device including a plurality of display panels 12 is employed, and the first image and the second image are displayed. May be displayed on another display panel 12. In this case, the same switch 6 as in the first embodiment may be provided, but the switching unit 24 determines which display panel 12 the operator Op is looking at instead of the switch 6. It is preferable that a determination unit is provided. The determination means preferably includes a sensor for detecting the face direction and the line-of-sight direction of the operator Op.

In the above-described first embodiment, an example in which the arm drive unit 5 for adjusting the posture of the distal end portion (gripping unit) of the master arm 3 is provided is shown. A configuration in which a person can manually adjust the posture of the master arm 3 may be employed.
Further, the input device may be a motion sensor such as an optical tracking sensor that detects the position / posture of the master gripper without having a master arm. Also in this case, the operator manually adjusts the posture of the master gripper.

  In addition, the constituent elements shown in the above-described embodiments and modifications can be combined as appropriate.

DESCRIPTION OF SYMBOLS 1, 1A Medical manipulator system 2 Master manipulator 3 Master arm 4 Displacement sensor 5 Arm drive part 6 Switch (selection means)
7 Slave manipulator 8 Slave arm 9 Actuator (drive means)
DESCRIPTION OF SYMBOLS 10 Display apparatus 11 Image processing part 12 Display panel 20 Control apparatus 21 Main control part 22 Master control part 23 Slave control part 24 Switching part 25 Selection part 26 Conversion means 27 Signal output part 30 Inspection image search system (2nd image structure apparatus)
DESCRIPTION OF SYMBOLS 31 Data server 32 Image structure part 33 Image processing part 34 Image output part 35 Intraoperative inspection apparatus 40 Virtual manipulator 41 Virtual endoscope 100 Endoscope apparatus (1st image structure apparatus)
101 image acquisition unit 110 surgical tool Op operator

Claims (5)

A master manipulator that issues an operation command;
A slave manipulator to which a surgical instrument for performing treatment on a treatment target is attached;
A first image construction device for obtaining a first image including a part of the slave manipulator and the treatment target;
A second image composing device that configures a second image that is different from the first image and includes an image of a virtual manipulator corresponding to a part of the slave manipulator;
A display device for displaying either of said first image and said second image,
Selecting means for selecting one of the first image and the second image;
A selection unit configured to switch one of the first image construction device and the virtual manipulator based on selection by the selection unit and set the operation target by the master manipulator;
Conversion that converts the operation command issued by the master manipulator into a new command so that the coordinate system on the display device of the image selected by the selection means matches the coordinate system on the operation of the master manipulator Means,
Drive means for operating the slave manipulator according to the new command when the first image forming apparatus is set as an operation target by the selection unit ;
With
In the second image construction device, when the virtual manipulator is set as an operation target by the selection unit, an image of the virtual manipulator is an image in which a posture and an operation corresponding to an operation by the master manipulator are reflected. Having an image processing unit for generating the second image,
A medical manipulator system characterized by that. The medical manipulator system according to claim 1,
The first image construction device is an endoscope device,
The second image construction device generates a virtual manipulator image that is displayed as if the slave manipulator is operating in accordance with the new command, and the virtual manipulator image is displayed on a base image in which the treatment target is imaged. A medical manipulator system, wherein the second image is generated by synthesizing. A medical manipulator system according to claim 2,
The second image construction device updates the base image based on the operation command from the master manipulator and changes the viewpoint to change the viewpoint. A medical manipulator system according to claim 2,
Used with an inspection device for imaging the treatment object during a treatment operation on the treatment object;
The second image construction apparatus uses, as the base image, an image imaged by the inspection apparatus during the treatment operation. A medical manipulator system according to claim 4,
The medical manipulator system, wherein the inspection device is provided in the slave manipulator.

Images