CN103815972A - Automatic tracking system for operative region of laparothoracoscope and method - Google Patents

Abstract

The invention discloses an automatic tracking system for an operative region of a laparothoracoscope. The automatic tracking system comprises a controller, an operative instrument, an image collecting module, an image caching module, an operative region identifying module and an image display module, wherein the controller is used for monitoring a minimally invasive surgery process in enterocoelia and thoracic cavity; the operative instrument is arranged in an instrument channel of the laparothoracoscope, is connected with the controller and is used for performing the minimally invasive surgery in the enterocoelia and thoracic cavity; the image collecting module is arranged in a shooting channel of the laparothoracoscope, is connected with the controller and is used for shooting the images of the organs and the minimally invasive surgery process in the enterocoelia and thoracic cavity; the image caching module is connected with the controller and is used for storing the images in the enterocoelia and thoracic cavity; the operative region identifying module is connected with the controller and the image caching module and is used for identifying the image of the operative instrument in the shot images; the image display module comprises a whole image display module and an operative region image display module and is used for displaying the image of the minimally invasive surgery process. The automatic tracking system for the operative region of the laparothoracoscope can automatically track the local image of the operative region in real time while realizing the panorama image display and also can freely amplify the local image.

Description

System and method for automatic tracking of surgical area in laparoscopy

technical field

The invention relates to an abdominal thoracoscopic system for minimally invasive surgical medicine used for internal inspection, diagnosis and treatment of a human body, and specifically designs an automatic tracking system for an operation area of the abdominal thoracoscope. The invention also relates to an automatic tracking method for the operation area of the abdominal thoracoscope.

Background technique

As shown in Figure 1 and Figure 2, the Chinese invention patent application number 201110166358.8 discloses an OCT electronic abdominal thoracoscope, including a display 1, a cold light source 2, a controller 3, a computer 4, a printer 5, an OCT connector 6, Control hand wheel 7, pincer crossing 8, rotating probe 9, rigid probe rod 10, host 11, control button 12, image acquisition channel 13, OCT channel 14, instrument channel 15 and lighting channel 16; the abdominal thoracoscope has a convenient Hand-held host 11 for holding operation; said host 11 is respectively connected with cold light source 2 and OCT joint 6 on controller 3 through wires; said controller 3, display 1, computer 4 and printer 5 are connected through wires; One side of the main machine 11 is provided with a control hand wheel 7; the lower part of the main machine 11 is provided with a jaw 8 for inserting surgical instruments, and a push-type control button 12 is installed on the top; the front end of the main machine 11 is sequentially connected with a long round rod Shaped rigid probe rod 10 and rotating probe 9, the rotating probe 9 is connected with the control handwheel 7 through a steel wire rope, and the direction and angle of rotation of the rotating probe 9 can be conveniently controlled by manipulating the handwheel 7, so as to realize omnidirectional observation; the rotating probe The front end of 9 has a lighting channel 16, an instrument channel 15, an OCT channel 14 and an image acquisition channel 13.

The operation tracking method of existing abdominal thoracoscopy comprises the following steps:

(1) Put a camera in the shooting channel;

(2) Move the camera to track and shoot images of the operation area;

Because most of the current abdominal thoracoscopes use ordinary cameras, the shooting accuracy and field of view are limited, so the doctor needs to manually adjust the shooting angle and position during the operation, which increases the pain for the patient. And for doctors, it has the following defects:

1. It is impossible to preview the panoramic images of the abdominal and thoracic cavity, it is inconvenient to find the surgical area for the lesion, and there is insufficient understanding of the macroscopic nature of the operation;

2. The local image of the operation area cannot be automatically tracked and extracted, which increases the number of personnel and complexity of the operation;

3. The local image of the operation area cannot be enlarged freely, and the observation of the image operation process.

To sum up, the existing abdominal thoracoscopic surgery tracking method needs to be operated by a doctor, which is not smart enough, and the operation time of the patient is relatively long, and the degree of pain is great. Therefore, an automatic abdominal thoracoscopic surgical area tracking system and method are very necessary.

Contents of the invention

The technical problem to be solved by the present invention is to provide an automatic tracking system and method for abdominal thoracoscopic surgery area, which can realize the panoramic image display and automatically track the partial image of the surgical area in real time, and can freely zoom in on the local image.

In order to solve the above technical problems, the present invention provides an automatic tracking system for abdominal thoracoscopic surgery area, including:

A controller for monitoring the minimally invasive surgery process in the abdominal thoracic cavity;

Surgical instruments, arranged in the instrument channel of the abdominal thoracoscope and connected to the controller, for performing minimally invasive surgery in the thoracoabdominal cavity;

The image acquisition module is arranged in the shooting channel of the abdominal thoracoscope and connected to the controller, and is used to take images of organs in the thoracoabdominal cavity and the minimally invasive surgery process;

The image buffer module is connected with the controller and is used to store images in the chest and abdominal cavity, and its output end is connected with the overall image display module of the image display module;

An operating area identification module, connected to the controller and the image buffer module, is used to identify the surgical instrument image in the captured image, and its output terminal is connected to the operating area image display module of the image display module;

The image display module includes the overall image display module and the operating area image display module, which are used to display images of the minimally invasive surgery process.

Preferably, the automatic tracking system of the abdominal thoracoscopic operation area of the present invention also includes an image synthesis module connected to the controller and the image buffer module, and the image acquisition module includes 2 or more than 2 cameras , the image synthesis module is used to synthesize the images captured by the various cameras and output them to the image buffer module.

Furthermore, in the automatic tracking system for the surgical area of abdominal thoracoscopic surgery of the present invention, the cameras are high-definition cameras with the same resolution and the same frame rate when shooting.

Further, in the automatic tracking system for the operation area of the abdominal thoracoscope of the present invention, the depth of field of the camera is 0-200mm.

Further, the automatic tracking system for the operation area of abdominal thoracoscopic surgery of the present invention also includes an image magnification control module connected with the controller and the operation area image display module, which is used to magnify the local operation area image, and the Zoom in for continuous zoom.

Furthermore, the automatic tracking system of the abdominal thoracoscopic operation area of the present invention also includes a voice acquisition module, which is respectively connected to the controller, the image magnification control module and the operation area identification module, It is used to adjust the enlargement or reduction of the partial image of the operation area according to the voice command, and correct the position of the image of the operation area according to the voice command.

Preferably, in the automatic tracking system for the surgical area of abdominal thoracoscopic surgery of the present invention, the operating area identification module includes a surgical instrument identification module and a moving object identification module, and the surgical instrument identification module is used to identify surgical instruments in the panoramic image, so The mobile object identification module is used to identify the movement of the surgical instrument position.

Furthermore, in the automatic tracking system for the surgical area of the abdominal thoracoscopic surgery of the present invention, the surgical instrument adopts a color different from that of the organs in the thoracoabdominal cavity.

Further, in the automatic tracking system for the surgical area of the abdominal thoracoscopic surgery of the present invention, the surgical instruments use marked color blocks or color circles.

The invention also discloses a method for automatically tracking the surgical area of abdominal thoracoscopic surgery, including:

(1) The image acquisition module shoots the panoramic image of the minimally invasive surgery process in the thoracoabdominal cavity, and stores the panoramic image in the image buffer module;

(2) identifying the surgical instrument in the panoramic image, and extracting a partial image including the surgical instrument;

(3) The image display module simultaneously displays the panoramic image of step (1) and the partial image of step (2) including surgical instruments.

Preferably, in the method for automatic tracking of the surgical region by abdominal thoracoscopic surgery of the present invention, the image acquisition module described in step (1) uses a high-definition camera for fixed shooting, and the depth of field of the high-definition camera is 0-200mm.

Further, in the method for automatic tracking of the surgical area of abdominal thoracoscopic surgery of the present invention, step (1) uses 2 or more than 2 high-definition cameras, and the resolution and shooting frame rate of each high-definition camera are the same, and each The images captured by the high-definition camera are synthesized into panoramic images and stored in the image buffer module.

Preferably, in the method for automatic tracking of an abdominal thoracoscopic surgical area according to the present invention, step (2) includes identifying the surgical instrument according to the marked color block or color circle of the surgical instrument, and extracting a partial image containing the surgical instrument.

Further, in the method for automatic tracking of the surgical area of the abdominal thoracoscopic surgery of the present invention, the step (2) also includes identifying whether the position of the surgical instrument has moved, and if so, extracting a new local image of the surgical area; A frame contains a partial image output of the surgical instrument.

Preferably, in the method for automatic tracking of an abdominal thoracoscopic surgical region of the present invention, step (3) marks the position and range of the partial image in the panoramic image when displaying the partial image including surgical instruments in step (2).

Preferably, in the method for automatic tracking of the surgical region by laparoscopic surgery of the present invention, step (3) enlarges the partial image including the surgical instruments in step (2) when displaying the partial image.

Further, in the method for automatic tracking of the surgical region of the abdominal thoracoscopic surgery of the present invention, the step (3) continuously enlarges the local image.

The system and method for automatic tracking of the surgical area of the abdominal thoracoscope of the present invention can display the panoramic image in the thoracoabdominal cavity and automatically track the partial image of the surgical area, and the partial image can be freely enlarged, which improves the intelligence of the thoracic and laparoscopic surgery and facilitates the operation It is very convenient for the doctor to operate and reduce the pain of the patient in the operation.

Description of drawings

The present invention will be described in further detail below in conjunction with the accompanying drawings and specific embodiments.

Figure 1 is a schematic diagram of the abdominal thoracoscopy system;

Fig. 2 is a schematic diagram of the probe channel of the abdominal thoracoscope;

Fig. 3 is the structural diagram of the automatic tracking system of the operation area of the abdominal thoracoscope of the present invention;

Fig. 4 is a schematic diagram of an image acquisition channel of an abdominal thoracoscope with a high-definition camera;

Fig. 5 is a schematic diagram of an image acquisition channel of an abdominal thoracoscope with two cameras;

Fig. 6 is a schematic diagram of an image acquisition channel of an abdominal thoracoscope with multiple cameras.

The reference signs in the figure are: 1, display, 2, cold light source, 3, controller, 4, computer, 5, printer, 6, OCT connector, 7, control hand wheel, 8, pincer crossing, 9, rotation Probe, 10, rigid probe rod, 11, host, 12, control button, 13, image acquisition channel, 14, OCT channel, 15, instrument channel, 16, lighting channel, 17, high-definition camera.

Detailed ways

Embodiment one

As shown in Figure 3, the present invention provides an automatic tracking system for an abdominal thoracoscopic operation area, including:

(1) The controller is used for monitoring the minimally invasive surgery process in the abdominal thoracic cavity.

(2) Surgical instruments, arranged in the instrument channel of the abdominal thoracoscope and connected to the controller, for performing minimally invasive surgery in the thoracoabdominal cavity.

(3) An image acquisition module, which is arranged in the imaging channel of the abdominal thoracoscope and connected to the controller, and is used to capture images of organs in the thoracoabdominal cavity and the minimally invasive surgery process. As shown in Figure 4, it is an example of adopting a high-definition camera 17 in the image acquisition module, in order to fix the position of the camera in the operation and take a panoramic image, it is necessary to use a high-definition camera (the standard high-definition camera is 1080P, and a high-definition camera can also be used) The 4k/8k or higher level of the standard high-definition camera, so that the micron level can be seen after the magnification level reaches a certain level, which is equivalent to microscopic surgery), and a camera with a relatively deep focal length (focal length) is required. The depth of field is in the range of 0-50mm, and the preferred depth of field in the present invention is in the range of 0-200mm.

(4) An image buffer module, connected to the controller, used to store images in the thoracic and abdominal cavity, and its output terminal is connected to the overall image display module of the image display module.

(5) An operating area identification module, connected to the controller and the image buffer module, used to identify surgical instrument images in the captured images, and its output terminal is connected to the operating area image display module of the image display module. The operating area identification module includes a surgical instrument identification module and a moving object identification module, the surgical instrument identification module is used to identify the surgical instrument in the panoramic image, and the moving object identification module is used to identify the movement of the surgical instrument. In order to facilitate identification, the surgical instrument may be in a color different from that of the organs in the thoracoabdominal cavity, or marked color blocks or color rings (such as black and white or multiple different colors) may be used on it.

(6) The image display module includes the overall image display module and the operating area image display module, which are used to display images of the minimally invasive surgery process.

(7) A preferred embodiment also includes an image synthesis module, connected to the controller and the image buffer module, including 2 or more cameras (3 cameras are shown in Figure 3, and each camera is for resolution The high-definition camera with the same rate, and the same frame rate when shooting, Fig. 5 and Fig. 6 are respectively the example that adopts 2 and 4 high-definition cameras 17, need to arrange the LED light source that illumination is used around each high-definition camera 17 accordingly) , the image synthesis module is used to synthesize the images captured by the various cameras and output them to the image buffer module. The number of cameras used is different, and the shooting angle range of each camera is also different. For example, if two cameras can be used to shoot, it is necessary to select each camera with a shooting angle of 120-140 degrees. If three cameras are selected, the shooting angle of each camera More than 100 degrees can achieve panorama.

(8) A preferred embodiment further includes an image magnification control module connected to the controller and the operating area image display module, for enlarging the partial operating area image, and the enlarging process is continuous. In addition, the system may also include a voice acquisition module, which is respectively connected to the controller, the image magnification control module and the operation area recognition module, and is used to adjust the enlargement or reduction of the partial image of the operation area according to the voice command, And correct the position of the operation area image according to the voice command. For example, during the operation, the doctor controls the local image of the control system through voice commands such as "zoom in", "zoom out", "left" and "right".

Preferably, the automatic tracking system of the surgical area of the abdominal thoracoscope of the present invention, further, the automatic tracking system of the surgical area of the abdominal thoracoscope of the present invention

Based on the system shown in Fig. 3, the present invention also discloses a method for automatically tracking the surgical area of abdominal thoracoscopic surgery, including:

(1) The image acquisition module shoots the panoramic image of the minimally invasive surgery process in the thoracoabdominal cavity, and stores the panoramic image in the image buffer module; the image acquisition module adopts a high-definition camera for fixed shooting, and the depth of field of the high-definition camera 0-200mm. Preferably, two or more HD cameras are used, and the resolution and frame rate of each HD camera are the same. During shooting, the images captured by each HD camera are synthesized into a panoramic image and then stored in the image buffer module.

(2) Identify the surgical instrument in the panoramic image, and extract the partial image containing the surgical instrument; identify the surgical instrument according to the marked color block or color circle of the surgical instrument, and then identify the position of the surgical instrument Whether there is movement compared with the previous frame, if so, extract a new partial image of the operation area, if not, output the partial image containing the surgical instrument in the previous frame.

(3) The image display module simultaneously displays the panoramic image of step (1) and the partial image of the surgical instrument in step (2), marks the position and range of the partial image in the panoramic image, and Capable of continuous zooming.

The system and method for automatic tracking of the surgical area of the abdominal thoracoscope of the present invention can display the panoramic image in the thoracoabdominal cavity and automatically track the partial image of the surgical area, and the partial image can be freely enlarged, which improves the intelligence of the thoracic and laparoscopic surgery and facilitates the operation It is very convenient for the doctor to operate and reduce the pain of the patient in the operation.

The above descriptions are all preferred embodiments of the present invention, and are not intended to limit the present invention. For those skilled in the art, various modifications and changes are possible. Any modification, equivalent replacement, improvement, etc. made within the spirit and scope of the present invention shall be included in the protection scope of the present invention.

Claims (17)

1. an operative region automatic tracking system for laparothoracoscope, is characterized in that, comprising:

Controller, for monitoring the intrathoracic minimal invasive surgical procedures of abdomen;

Operating theater instruments, is arranged in the instrument channel of laparothoracoscope and is connected with described controller, for carry out Minimally Invasive Surgery in splanchnocoel;

Image collecting module, is arranged in the shooting passage of laparothoracoscope and is connected with described controller, for taking organ in splanchnocoel and the image of described minimal invasive surgical procedures;

Image cache module, is connected with described controller, and for storing the image in splanchnocoel, its outfan is connected with the overall image display module of image display module;

Field of operation identification module, is connected with described controller and described image cache module, and for identifying the operating theater instruments image of filmed image, its outfan is connected with the field of operation image display module of described image display module;

Described image display module, comprises described overall image display module and field of operation image display module, for showing the image of described minimal invasive surgical procedures.

2. the operative region automatic tracking system of laparothoracoscope as claimed in claim 1, it is characterized in that, also comprise an image synthesis module being connected with described controller and described image cache module, described image collecting module comprises 2 or 2 above photographic head, and described image synthesis module is used for the image of described each photographic head shooting being synthesized and outputing to described image cache module.

3. the operative region automatic tracking system of laparothoracoscope as claimed in claim 2, is characterized in that, described photographic head is the high-definition camera that resolution is identical, and take time frame speed identical.

4. the operative region automatic tracking system of laparothoracoscope as claimed in claim 2, is characterized in that, the depth of field of described photographic head is 0-200mm.

5. the operative region automatic tracking system of laparothoracoscope as claimed in claim 1, it is characterized in that, also comprise the image zoom control module being connected with described controller and described field of operation image display module, for by local field of operation image zoom, described in be enlarged into continuous amplification.

6. the operative region automatic tracking system of laparothoracoscope as claimed in claim 5, it is characterized in that, also comprise voice acquisition module, be connected with described controller, described image zoom control module respectively and described field of operation identification module connects, for regulating zooming in or out of field of operation local image according to phonetic order, and according to the position of phonetic order corrective procedure district image.

7. the operative region automatic tracking system of laparothoracoscope as claimed in claim 1, it is characterized in that, described field of operation identification module comprises operating theater instruments identification module and mobile object identification module, described operating theater instruments identification module is for identifying the operating theater instruments of full-view image, and described mobile object identification module is for identifying the movement of operating theater instruments position.

8. the operative region automatic tracking system of laparothoracoscope as claimed in claim 7, is characterized in that, described operating theater instruments adopts the color different from organ in splanchnocoel.

9. the operative region automatic tracking system of laparothoracoscope as claimed in claim 7, is characterized in that, uses color lump or the colour circle of labeled in described operating theater instruments.

10. an operative region automatic tracking method for laparothoracoscope, is characterized in that, comprising:

(1) image capture module is taken the full-view image of the minimal invasive surgical procedures in splanchnocoel, and stores described full-view image into image cache module;

(2) operating theater instruments in full-view image described in identification, and the local image that comprises described operating theater instruments is extracted;

(3) the described full-view image of image display module while step display (1) and the local image that comprises operating theater instruments of step (2).

The operative region automatic tracking method of 11. laparothoracoscopes as claimed in claim 10, is characterized in that, the image capture module described in step (1) adopts high-definition camera to be fixed shooting, and the depth of field of described high-definition camera is 0-200mm.

The operative region automatic tracking method of 12. laparothoracoscopes as claimed in claim 11, it is characterized in that, step (1) is used 2 or 2 above described high-definition cameras, and the resolution of each high-definition camera is identical with photographed frame speed, the image of in shooting, each high-definition camera being taken stores described image cache module into after synthesizing full-view image.

The operative region automatic tracking method of 13. laparothoracoscopes as claimed in claim 10, it is characterized in that, step (2) comprises according to the labeled color lump of operating theater instruments or colour circle identification operating theater instruments, and extracts the local image that comprises described operating theater instruments.

The operative region automatic tracking method of 14. laparothoracoscopes as described in claim 113, it is characterized in that, step (2) also comprises whether the position of operating theater instruments is moved described in identification, if it is extract new field of operation local image, if otherwise the local image that previous frame is comprised to described operating theater instruments output.

The operative region automatic tracking method of 15. laparothoracoscopes as claimed in claim 10, it is characterized in that, when the local image that comprises operating theater instruments of step (3) step display (2) in described full-view image position and the scope of labelling local image.

The operative region automatic tracking method of 16. laparothoracoscopes as claimed in claim 10, is characterized in that, when the local image that comprises operating theater instruments of step (3) step display (2), described local image is amplified.

The operative region automatic tracking method of 17. laparothoracoscopes as claimed in claim 16, is characterized in that, step (3) is amplified continuously to described local image.

Images