DE10225077B4 - Object tracking device for medical operations - Google Patents

Abstract

Property Follow arrangement (1) for Medical operations with a large number of optical sensors (2, 7), of which a first subset of the sensors (2) for detection arranged from the direction of the sensors (2) illuminated object (3) is, and an evaluation means (4) for determining an object position in response to the sensor signals, characterized in that the first Subset of the sensors (2) in an illuminating the site to be observed Operating light (6) is arranged for site survey and that another subset of the surgical light alignment sensors (7) is arranged, which evaluation means (4) for the cascaded determination the object position in response to the signals of both subsets the sensors (2, 7) is formed.

Description

The The present invention relates to the term claimed above. Thus, the present invention is concerned with tracking to be detected objects, that is it concerns a tracking system.

Property Follow arrangements are required and known to the position of certain objects to pursue. This is gaining increasing importance for instance in surgical Intervened where the position of a surgical instrument is accurate must be recorded in order to document it or, beyond that, to using automatic or semi-automatic robotic systems to support the surgeon.

From the DE 196 39 615 C2 is a reflector referencing system for surgical and medical instruments, and a marker system for neurosurgical parts of the body to be treated known. It is proposed there that the reflector torenreferenzierungssystem for surgical and medical instruments or treatment apparatus with at least two cameras and a computer connected to the computer unit is provided with a graphics screen output, wherein a radiation source for infrared radiation and a reflector group with at least two reflectors for this infrared radiation is provided, wherein the reflectors are mounted in a characteristic only for this group of reflectors arrangement on the instruments or treatment apparatus.

The Cameras capture the area of the operating table where the treatment takes place and can three-dimensional spatial coordinates of surgical instruments and Detecting treatment apparatus, which are provided with reflectors, which Reflect infrared rays.

Out GB / A / 2 345 538 is an optical tracker with a monocular Videocamera, with a video image capture card, a computer and a dedicated software for The 3D and 2D tracking of incoming images is known that uses is used to control the computer cursor, that is, in place of a mouse.

A visual real-time server system for laparoscopic surgery is from G.-Q. WEI and K. Arbter have been published, cf. DE 195 29 950 C1 ,

Out US Patent Publication US 2001/0008561 A1 is a real-time computer vision system Known to be one or more objects that are in a scene move, using a multi-location technique, where does not have to be searched.

The Object tracking in video sequences is known from WO 94/25930 A1.

From the not pre-published EP 1 340 470 A1 a system is known for combined illumination of a predefinable area and for referencing three-dimensional spatial coordinates, in particular of surgical or medical instruments, comprising a surgical light for illuminating a region, a radiation source for emitting infrared rays and at least two cameras for detecting infrared optical signals, wherein the surgical light and the camera are held together so that the optical infrared signals for referencing three-dimensional spatial coordinates in the illuminated area are evaluable.

Of the EP 1 340 470 A1 it can not be deduced to determine an absolute position of the situs cascaded.

Out The US-PS 6,188,777 B1 is a method and an apparatus for Personnel detection and tracking known.

The generic document US 6,288,785 B1 includes a system for determining the spatial position and / or the orientation of one or more objects. The system includes an optical subsystem and a non-optical subsystem. The optical subsystem includes subsystem optical light sources mounted on one or more of the objects and an optical subsystem sensor configured to measure the energy of the subsystem optical light sources. The optical subsystem has an optical coordinate system that is related to the optical subsystem sensor. The optical subsystem sensor produces position and / or orientation signals relative to the optical subsystem coordinate system in response to the measured energy of the subsystem optical light source. The non-optical subsystem has a non-optical subsystem coordinate system and is configured to produce position and / or orientation signals of one or more of the objects relative to the non-optical subsystem coordinate system. A coupling arrangement is provided for providing position and / or orientation signals indicative of the position and / or orientation of the selected optical or non-optical subsystem relative to the coordinate system of the other optical or non-optical subsystem. A processor is responsive to the signals produced by the coupling arrangement and the signals the individual sensors for determining the position and / or orientation of one or more objects relative to the suitably defined coordinate system.

If Objects in critical applications such as surgical procedures be tracked, about the position of an electric scalpel determine its excitement exactly, so the position detection done quickly and accurately. This is true in all circumstances, too then if the person involved in the operation or another Member of the operating team in the detection field between sensor and to the object to be detected. In addition, it is desirable to create a permanent calibration that will take care of that wearing, that object tracking during guaranteed vibration safe throughout the duration of the operation is. It is desirable at least partially better at least some of these requirements suffice to be able to.

The Object of the present invention is an object tracking while Ensure the entire duration of surgery safely.

in the Claim 1 will be an object tracking arrangement for medical operations specified, with which this task is solved. Preferred embodiments the object tracking arrangement can be found in the subclaims.

In order to beats the present invention according to a first principles an object tracking arrangement for medical Operations with a variety of optical sensors, of which a first subset for detecting one from the direction of the sensors irradiated object is arranged, and an evaluation means for determining an object position in response to the sensor signals which is provided that the first subset in a to observational situs lighting surgical light for site survey is arranged, and a further subset of optical sensors for Detecting the surgical light alignment is arranged, wherein the evaluation means for the cascaded determination of the object position is designed in response to the signals of both subsets.

One The first essential aspect of the invention is thus to be seen in that the sensor means themselves are movable and their position is also evaluated. In this way, on the one hand achieved that the arrangements against shakes, for example, against the sensor means such as cameras, pushing people is secured in its usability, and it becomes on the other hand allows the sensor means always close to the objects to be detected hold. This in turn makes it possible to arrange the sensor means so that nothing between the sensor means and the object to be detected and, moreover, one precise Determining the object position relative to the sensor is possible.

The Object tracking arrangement of the present invention is therefore preferred in closed rooms, especially in operating theaters used, wherein the reference position can be fixed in space. In alternative Way would be also possible, to refer to a movable operating table or the like, which makes sense when a patient is on a table fixed, be it at a radiation, where he is relative must be moved to a source of radiation, be it with others Occasions.

In In a preferred variant, several identical per se sensor means and at least one of them is relative to others Sensor means movable. It should be noted, however, that certain Sensor means which detect objects in an identical way per se, where appropriate, other interpretations are possible, such as when Cameras have different focal lengths for different elements be and / or especially during the object tracking are adjustable. In particular, it is preferable when two or more sensors are movable together and relative to several other sensors. In such a Case can the movable sensors are placed close to the object and have preferred very good view of this, while with the remaining sensors as Sensormittelposi tion detecting means the position of the movable sensors is detected.

Prefers it is when the sensors optically detect the objects, as in one such case with comparatively inexpensive position sensitive sensors can be worked, such as CCD cameras, CMOS fields or others photosensitive semiconductors. The optical object detection becomes preferably at least partially in the visible range (VIS range), in particular, the object colors can be taken into account. This allows to operate in the operating field without additional IR sources, which is particularly preferred because of the always required bright illumination of the surgical field anyway a strong heat load of the area is given. The object color consideration makes it possible at the same time that different objects based on their different Colors are detected or easily marks on objects be applied and thus a very fast assignment and detection possible is.

In a preferred variant of the invention, several of the object-detecting sensors are present seen with which, in particular by triangulation, the position of the object can be determined. The provision of three or more sensors then has the advantage that redundant information can be obtained, which also facilitates the determination of the position with higher precision and / or improves the robustness. In a preferred variant, it is sufficient if not the position of the entire object is determined, but if only the position of a predetermined object part is detected. That in this case the object part may be a reference mark, a luminous point or a luminous area and / or a color point may be mentioned. The reference to such reference marks facilitates the identification in the image data evaluation.

If, as preferred, the object position with a number of sensors or sensor means is detected, which is an over-determination allow, so it is preferred, the most probable object position by weighting the sensor means signals or the thereof to get evaluated sizes. The weighting may be, for example, taking into account the distance between object and sensor means, with typically lower distances to a higher one Weighting be and / or can Partial Obscurations detected on certain sensor means become. Is about a circular Reference mark before, at a certain time due to a partial Obfuscation is only detected as a semicircle, so while per If object positioning is still possible, it will not be as accurate as there, where the view of the object is unhindered is.

Especially it is preferred if the object tracking arrangement of the present Invention provides the relative sensor means mobility by the movable sensor means in or on a movable Lighting arrangement can be arranged. In particular, it is possible the moving Sensor means such as CCD elements or the like in an operation lamp to arrange. It is then possible to detect the position of the movable sensor means in that about the back the operating light with the remaining sensor means detected becomes. This is advantageous because typi cal surgical lights a comparatively large Possess extension for Purpose of triangulation a sufficiently large sensor center distance attached sensors, because the surgical field is not typical shaded is illuminated, so an excellent view the surgical field and thus the instruments to be followed guaranteed is, and because moreover These instruments are also optimally illuminated.

The Sensor means position detecting means, in particular, when the Sensor means are placed even in or on a surgical light, be implemented by decoder, in particular angle and / or linear position decoder, with which the angle or linear positions of the surgical light are detected. In such a case, it is not mandatory, more Cameras outside to provide the surgical light. Alternatively and / or in addition but it is also possible a variety of cameras or other sensor means in the operating room or another room in which an object is to be tracked and then on the one hand with these the position of the movable sensor means capture and / or simultaneously track the objects.

By se it is sufficient if cascaded with a first set of sensors used as sensor means position detection means is detected where the sensors directly directed to the object are located, and then the object is tracked with them. Prefers is, however, if, at least for consistency control and / or for possibly only a low-weighted correction or as a backup system the sensor means position detecting means at the same time to the actual Objekterfassung be used. In such a case will typically, such as when the sensor means position detecting means used sensors on walls and / or blankets are placed, a higher weight on the near placed on the object movable sensors to the object position determine.

It it should be noted that, if necessary, a sensor means position detection be provided with more than two stages in cascade arrangement can. This is useful if a room surveillance is given with a variety of fixed and fixed aligned cameras happens and within the room a larger area is formed movable, such as a mechanically movable treatment table, on which in turn movable parts are arranged. It would also be conceivable to integrate the sensor means for object detection in a spectacle assembly for the surgeon in neurosurgical procedures or the like.

The Multiple cascading is also useful in very large spaces, such as factory buildings etc..

The Invention will now be described below by way of example with reference to the drawing described. In this shows:

1 the arrangement of sensors of an object tracking arrangement according to the invention;

2 a schematic representation of the signal evaluation;

3 a schematic arrangement of the sensor means in a surgical light.

To 1 includes a general with 1 designated object tracking arrangement 1 a variety of sensor means 2 for the creation of an object 3 from different directions and an evaluation means 4 for determining the position of the object 3 relative to a given reference 5 in response to signals from the sensor means 2 , wherein the sensor means 2 movable for reference 5 on a surgical light 6 are arranged and sensor means position detecting means 7 are provided to the position of the sensor means 2 to capture, and wherein the evaluation means 4 adapted to the object position in response to the signals from the sensor means 2 and to the signals from the sensor means position detecting means 7 to determine.

The object tracking arrangement 1 serves as the basis of the operating table 8th and the patient on it 9 indicated, for the pursuit of medical instruments as objects 3 during an operation. It is accordingly arranged in an operating room.

The sensor means 2 comprise a plurality of color-sensitive CCD cameras of sufficiently high resolution and are fixedly spaced from each other in a surgical light 6 arranged. They are with this surgical light 6 over the patient 9 movably mounted, with the lamp 6 about the patient 9 to be lowered, as indicated by arrow 10a indicated to be rotated about several axes of rotation, as indicated by arrows 10b and 10c indicated as well as can be tilted, as by arrow 10d indicated.

To 3 are the sensor means 2 as three separate CCD fields 2a . 2 B . 2c in the peripheral areas of the surgical light 6 arranged on positions between the actual illuminant carriers 6a . 6b . 6c , The signals of the sensors 2a . 2 B . 2c are suitably conditioned and shielded to the evaluation unit 4 guided.

The object 3 as a scalpel 3 is shown with color dots 3a . 3b . 3c which allows its identification and the identification of its spatial position and orientation.

To the evaluation device 4 become signals from the sensor means 2 and the sensor means position detecting means 7 fed there to, as later with respect to 2 will be described, to be evaluated.

As a given reference 5 serves in the illustrated embodiment, a stationary point 5 in the operating room, based on the X, Y and (not shown) Z coordinates, ie the reference need not be defined by a physically detectable, fixed mark, but may serve as a fixed, fixed reference point in coordinate measurement, which can be arbitrarily redefined, in particular for each basic calibration. It should be noted that while in the 1 and 2 reference only to coordinates X and Y, in typical applications three spatial coordinates are determined for the detection of each object point. If a plurality of object points to be detected can be located on an object of greater spatial extent, its spatial angular position can thus also be detected. It should be mentioned that other coordinates can be used instead of Cartesian coordinates.

At the operating light 6 it is a conventional surgical light, which, however, as in 3 Darge presents, is provided with sensor means and reference marks in the housing on the side facing away from the luminous field 6d . 6e . 6f which are particularly good from the sensor means position detecting means 7 are detectable.

The sensor means position detecting means 7a to 7f are fixed to the walls and ceiling of the operating room and generally point towards the operating table 8th and capture this as well as the surgical light 6 , wherein by suitable arrangement of the sensor means position detecting means 7 on the walls and ceiling of the operating room is ensured that the reference marks 6d . 6e . 6f on the operating light 6 in each possible position of the same in each case by at least two sensor means position detecting means is detectable, such as for 7a . 7b for the reference mark 6d shown. This is due to the solid line of sight lines provided by the sensor means 7a . 7b to the reference mark 6d go out, visible. Moreover, the sensor means position detecting means at least partially also detect the reference marks 3a . 3b . 3c on the object 3 , This is at least some of the sensor means position detecting means 7 possible. The sensor means position detecting means 7 are just like the actual sensor means in the surgical light 6 designed as CCD cameras and have the same, high resolution and color sensitivity like this.

2 shows the wiring of the evaluation arrangement 4 , the suitably conditioned signals from the sensor means 7 and the sensor means 2 receives.

The reference marks 3a . 3b . 3c on the object 3 or the reference marks 6d . 6e . 6f on the operating light 6 result on the array of CCD sensors dedicated, with suitable means such as FPGA cards recognizable signatures, by means of which the position of the dot image on the sensor can be determined and on the one hand can be assigned to which object the mark belongs, so for example to a scalpel or the operating light, and on the other hand it can be seen how well the mark on the sensor means is detected, that is, with what size and possibly what partial shading. The evaluation level 4 is able to ascertain this by suitable evaluation per se in the prior art. It is then by triangulation and / or other methods known per se in the evaluation stage for sensor pairs and sensor means detecting means using a once performed for the fixed sensor means position detecting means calibration and calibration or determination of the imaging properties of the movable sensor means or pairs of sensor respectively one position x, y , z and the associated weighting quality q of the respective detection determined, as far as possible. If an object with a sensor means is not detected, as for the object 3 and the sensor means 7c If so, the quality of the associated acquisition is set to zero.

By evaluating the signals from the CCD fields and the pairwise linking, the evaluation stage thus obtains on the one hand a set of coordinates of the object x_Obj_abs_ (i), y_Obj_abs_ (i) and z_Obj_abs_ (i) with associated weighting q_Obj_abs_ (i) obtained from the signals the sensor means position detection signals are derived. Furthermore, coordinates of the operating light reference marks and thus of the movable with this sensor means are determined in 2 x_Lamp_abs_ (i), y_Lamp_abs_ (i), z_Lamp_abs_ (i), q_Lamp_abs_ (i). These coordinates correspond to absolute positions relative to the reference point 5 because the wall and ceiling mounted sensor means 7 are stationary. Typically, however, the associated quality will be low, since many sensor means do not detect the objects or not completely and moreover only from a great distance. It should be pointed out, however, that the overdetermination for object detection supplies redundant information in particular in the medical environment by means of more sensors, which leads to improved measured values and / or protects against object loss during shading and / or improves the robustness and is not regarded as known per se , which is why even here a worthy invention is seen, regardless of a cascading and the like.

From the signals of the mobile sensors 2 On the other hand, coordinate pairing x_object_relativ_ (j), y_object_relativ_ (j), z_object_relativ_ (j) along with associated qualities q_object_relativ_ (j) are then likewise determined by pairwise triangulation of the evaluated CCD array signals. The quality of the coordinates x_object_relativ_ (j), y_object_relativ_ (j), z_object_relativ_ (j) will typically be high because the sensors are located close to the object and will typically have a clear view of it.

In the evaluation stage are now from the absolute positions of the lamp, as they are from the sensors 7 is determined and the resulting sensor means positions of the sensors 2 using the relative coordinates x_object_relativ_ (j), y_object_relativ_ (j), z_object_relativ_ (j) and the determined respective weighting 9 determines the weighted averaged coordinates Ø_x-object, Ø_y-object, Ø_z-object. On the one hand, the very precisely determined coordinates relative to the movable sensor means enter into these middle coordinates, the precision resulting from the low shading, the good illumination and the small spacing and, on the other hand, the absolute position of the lamp relative to the sensor means are arranged in the space, wherein here the precision and accuracy results from the consideration of a plurality of sensor means, so that an error of the absolute determination of the position of the movable sensor means by averaging over a plurality of different pairs of measured values is also reduced. The overall precision of the arrangement is thus very high and there is nevertheless no danger that a complete determination, for example by a surgeon, makes a position determination impossible. Thus, a fast, accurate and, in view of the comparatively low prices of CCD devices still inexpensive position detection is possible, which also offers the required precision in fine surgical procedures and the like.

It should be noted that the scalpel, other than shown, may in particular be a robot moving scalpel or other automatically or semi-automatically moved surgical instrument such as a saw, a drill, an endoscope, a catheter or the like, wherein the robot in Response to signals from the controller 4 can be mobile. In such a case, the precision of the robot movement depends on the precision and speed of the object detection.

It should be noted that coordinates can also be determined by the formation of triangulation groups comprising at least one movable and one stationary sensor means, both for Objects as well as sensor center positions. It should be mentioned that in order to determine different coordinates of a point, completely or partially different sensor means can be used.

It be mentioned that instead the CCD cameras described also CMOS cameras o. Ä. For all or some of the sensor means can be used.

It be further mentioned that it possible is, first first sensors relative to other temporarily fixed second sensors to move, then using the sensor signals of the second sensors to determine where the first sensors are positioned and then others, especially the second sensors to move around so one successive change between moving and fixed sensors.

Claims (18)

Object tracking arrangement ( 1 ) for medical operations with a plurality of optical sensors ( 2 . 7 ), of which a first subset of the sensors ( 2 ) for detecting one from the direction of the sensors ( 2 ) illuminated object ( 3 ), and an evaluation means ( 4 ) for determining an object position in response to the sensor signals, characterized in that the first subset of the sensors ( 2 ) in a surgical light illuminating the site to be observed ( 6 ) is arranged for site survey and that a further subset of the sensors ( 7 ) is arranged for detecting the operation light orientation, which evaluation means ( 4 ) for the cascaded determination of the object position in response to the signals of both subsets of the sensors ( 2 . 7 ) is trained. Object tracking arrangement ( 1 ) according to claim 1, characterized in that the subset of the optical sensors ( 7 ) is arranged at least partially fixed space for detecting the operation light alignment. Object tracking arrangement ( 1 ) according to claim 1 or 2, characterized in that at least in the operating light ( 6 ) arranged sensors ( 2 . 7 ) relative to the sensors arranged for operating light alignment detection ( 2 . 7 ) are movable. Object tracking arrangement ( 1 ) according to one of the preceding claims, characterized in that the optical object tracking is carried out completely in the VIS area. Object tracking arrangement ( 1 ) according to the preceding claim, characterized in that the optical object tracking is designed for object color consideration. Object tracking arrangement ( 1 ) according to one of the preceding claims, characterized in that the sensors ( 2 . 7 ) Comprise camera elements, photosensitive semiconductors, CMOS cameras and / or CCD arrays. Object tracking arrangement ( 1 ) according to one of the preceding claims, characterized in that at least two the object ( 3 ) are provided sensors. Object tracking arrangement ( 1 ) according to one of the preceding claims, characterized in that at least three the object ( 3 ) are provided for the delivery of redundant information detecting sensors. Object tracking arrangement ( 1 ) according to one of the preceding claims, characterized in that the evaluation means ( 4 ) is designed to determine the object position based on the position of an object part and / or make the determination based on the definition of an object part. Object tracking arrangement ( 1 ) according to one of the preceding claims, characterized in that the evaluation means ( 4 ) is adapted to detect the object position by triangulation. Object tracking arrangement ( 1 ) according to one of the preceding claims, characterized in that the evaluation means ( 4 ) is adapted to the object position with reference to reference marks ( 3a . 3b . 3c . 6a . 6b . 6c ) capture. Object tracking arrangement ( 1 ) according to the preceding claim, characterized in that it is suitable for detecting reference marks ( 3a . 3b . 3c . 6a . 6b . 6c ) is formed, the at least one of color and / or luminous points or surfaces on the object ( 3 ) and / or on the operating light ( 6 ). Object tracking arrangement ( 1 ) according to one of the preceding claims, characterized in that the evaluation means ( 4 ) is adapted to the object position with an over-determination permitting number of sensors ( 2 . 7 ) capture. Object tracking arrangement ( 1 ) according to the preceding claim, characterized in that the evaluation means ( 4 ) is adapted to detect a probable object position from the signal of the overdetermined number of sensors ( 2 . 7 ) under weighting. Object tracking arrangement ( 1 ) according to the preceding claim, characterized in that the evaluation means ( 4 ) is adapted to the weighting taking into account the object Sen sor distance and / or partial object occlusion at certain sensors ( 2 . 7 ). Object tracking arrangement ( 1 ) according to one of the preceding claims, characterized in that at least some of the sensors ( 7 ) for detecting the Operationsleuchtenausrich device are at the same time at least temporarily connected to the object detection. Object tracking arrangement ( 1 ) according to one of the preceding claims, characterized in that the evaluation means ( 4 ) is designed to determine the object position with respect to a spatially fixed reference. Object tracking arrangement ( 1 ) according to one of the preceding claims, characterized in that the evaluation means ( 4 ) is designed to determine different objects based on different colors.