JP6878435B2 - Medical device tracking - Google Patents

Description

The present invention relates to the field of medical device tracking, specifically to medical device tracking systems, medical device tracking methods, computer program elements and computer readable media.

Medical devices such as mobile devices, flexible catheters or stiffer arms or shafts are used for minimally invasive procedures in the diagnosis and treatment of medical conditions. In some applications and situations, it is important to know the location and orientation of the medical device in imaging-assisted surgical interventions in order to perform delicate procedures without damaging important surrounding structures such as arteries and nerves. is there. For example, International Patent Publication WO2015 / 032676A1 describes an electromagnetic tracking sensor that determines the position and shape of an intervention device.

Some tracking methods require information about the dimensions of the medical device stored in the database. For example, U.S. Patent Application Publication No. 2012/0259204A1 describes providing a device with an axial visibility element. As a result, the positions of the elements that characterize a specific device are stored in the database.

Therefore, only a limited number of known medical devices can be tracked.

Medical devices need to be tracked easily.

An object of the present invention is solved by the subject matter of the independent claims. Further embodiments are incorporated into the dependent claims. It should be noted that the aspects of the invention described below apply equally to medical device tracking systems, medical device tracking methods, computer program elements and computer readable media.

According to the first aspect of the present invention, there is provided a system for tracking an elongated medical device within a field of interest. The system includes a tracking device and a processing unit.

The tracking device acquires an image of the medical device, which image includes a device marker placed on the medical device at a marker position away from the tip of the medical device.

The processing unit i) detects at least one line segment and device marker in the acquired image, and ii) selects the line segment corresponding to the detected device marker from at least one detected line segment. Identify the medical device by iii) Determine the offset of the position of the tip of the medical device based on the medical device in contact with the reference marker.

In one example, a straight, elongated medical device is used. The device marker is used as a reference during the medical procedure in which the medical device is tracked. The offset between the marker and the tip position is determined to indicate the tip position of the medical device even if part of the medical device is inside the subject. For this purpose, a reference marker on the subject is contacted by the medical device tip before inserting the medical device into the subject.

The contact of the reference marker by the tip of the device is detected by the tracking device. For example, if the reference marker is visible in the image provided by the tracking device, contact between the device tip and the medical device is detected in the provided image. In the context of this application, "contact" is not only a physical contact, but also a virtual contact (ie, the medical device is close enough to the reference marker, and thus one or more of the tracking images. In the tracking image of, the device representation and the marker representation overlap).

Therefore, contact is detected even if the position of the tip of the device itself is not sufficiently reliable and established in the tracking image.

In one example, at least one reference marker is provided on the subject. For example, a set of reference markers may be provided externally to the patient. In this case, the tracking device tracks the patient's movements in addition to tracking the device position.

In this way, it is not necessary to create a database that stores geometric information of a plurality of medical devices. Even with unknown devices (as opposed to known medical devices in the database), users simply use the device markers on the medical device to track the medical device even if some of the medical device is within the subject. can do. Therefore, a variety of medical devices can be tracked.

The expression "device marker placed on a medical device" includes that the device marker is another marker attached to the medical device and that the device marker is an integral part of the medical device. ..

In one embodiment, a single device marker is sufficient to track the medical device. The offset between a single marker and the tip of the device is reliably determined by the contact interaction between the tip of the device and the reference marker.

According to one example, the device marker comprises at least one of a group of biocompatible reflective paints, fluorescent paints, materials having a unique reflex reaction and forming medical devices and adhesive tapes.

The device marker may be attached to or sprayed on the medical device.

According to one example, biocompatible reflective paints and / or fluorescent paints have bandwidth-selective luminescence.

In this way, the excitation / emission of the reflective or fluorescent paint is bandwidth-selected so that the video image of the scene remains undisturbed while the filtered light source and / or camera is used for positioning. Is the target.

Alternatively, or in addition, the equipment landmark itself may be used as the equipment marker. In one example, the boundary between two parts of a medical device is used as a device marker. For example, the boundary between the shaft and the handle of the device (the boundary is perpendicular to the shaft direction) is sufficiently clearly visible in the tracking image to allow device tracking. By detecting such a transition part, it becomes possible to track the device without attaching a separate (artificial) marker to the device.

To track a medical device, in one example, an optical tracking device with at least one camera is used. Therefore, by using images from at least one camera, the three-dimensional position of the medical device is determined. In one example, at least two optical cameras are used to allow accurate 3D tracking. As an alternative, or as an input for 3D tracking, a single depth resolution camera may be used to produce an improved optical image with depth measurements.

Alternatively, by using at least two cameras and a light source, a three-dimensional model of the medical device can be generated. For example, a medical device can be rotated in front of a camera system to calculate a 3D model from a series of images. Cameras and light sources may have specific wavelength filters suitable for the type of reflective material used. By using a variety of reflective materials and a set of filters of a particular wavelength, one or more instruments used in surgery can be identified.

A second aspect of the invention provides a method of tracking a medical device within a field of interest. The method is
a) A step of acquiring an image of a medical device including a device marker placed on the medical device at a marker position away from the tip of the medical device,
b) In the image, the step of detecting at least one line segment in the field of interest, and
c) Steps to detect device markers and
d) A step of identifying a medical device by selecting a line segment corresponding to the detected device marker from at least one detected line segment.
f) Includes the step of determining the offset of the position of the tip of the medical device by contacting the reference marker with the medical device.

These and other aspects of the invention will become apparent from the embodiments described below and will be described with reference to those embodiments.

An exemplary embodiment of the present invention will be described below with reference to the following drawings.

FIG. 1 schematically shows an example of a medical device tracking system. FIG. 2 schematically shows an example of a field of interest. FIG. 3 shows the basic steps of an example of a medical device tracking method. FIG. 4 shows a further example of the method. FIG. 5 shows a further example of the method.

The drawings are only schematically illustrated and are not at exact scale. The same reference numerals refer to the same or similar features throughout the drawing.

FIG. 1 schematically shows a system 100 that tracks a medical device 10 in a field of interest 12.

The system 100 includes a tracking device 16 and a processing unit 18. The device marker 14 is placed on the medical device 10 at marker position 20 to allow tracking of the device marker 14.

The term "field of interest" refers to fields in which medical devices are used, such as surgical fields, surgical fields and the like.

The term "medical device" refers to a specially designed tool or device that performs a surgical procedure, such as altering biological tissue, or a specific action that performs a desired effect during surgery.

In the example shown in FIG. 1, the medical device 10 includes a straight, elongated device shaft 30 and a handle 38. The device marker 14 is mounted on the device shaft 30 away from the medical device tip 28. Alternatively, or further, the boundary between the shaft 30 and the handle 38 may be used as the equipment marker 14'.

Therefore, even if the medical device tip 28 is blocked, for example, when inserted into the subject, the device markers 14, 14'can provide a reference indicating the position of the medical device tip 28.

In one example, the device marker 14 comprises at least one of a group of biocompatible reflective paints, fluorescent paints, materials having an intrinsic reflex reaction and forming part of a medical device and adhesive tapes.

A fluorescent dye similar to the fluorescent dye used in tumor surgery may be used as the fluorescent paint. The fluorescent paint should be biocompatible. Examples of fluorescent dyes include indocyanine green (ICG), fluorescein, photofurin, 5-aminolevulinic acid (ALA), patent blue, methylene blue and the like.

The biocompatible reflective or fluorescent paint is coated or sprayed onto a medical device such as a surgical or laparoscopic device used, for example during a minimally invasive surgical procedure.

The reflective coating or optical brightener coating is reapplied after several uses, based on the efficiency of the dye or coating material. The coating material can be pressure sterilized or disinfected for repeated use.

Alternatively, a portion of the medical device, such as a portion of the shaft 30, may be made of a material that has a unique reflection or fluorescence response. That is, a part of the medical device may be used as a device marker indicating the position of the medical device tip 28.

In a further example, the device marker 14 is, for example, a piece of adhesive tape that is wrapped around the central axis of the medical device until it is securely fastened.

Adhesive tapes are made from materials that include, for example, silicone rubber sleeves, heat shrinkable tubes or permanently painted, etched or engraved markers that slide on medical device shafts.

The device marker may optionally have a pattern extending in a direction surrounding the medical device.

In the example shown in FIG. 1, the device marker 14 has a line pattern. Line patterns, such as the number of lines and / or line thickness, are changed to distinguish between different medical devices. In a further example, medical devices with the same pattern are distinguished using, for example, the thickness of the shaft of the medical device.

Other examples of the pattern include, for example, a two-dimensional bar code, a color code, and the like. In this way, the medical device is detected or tracked in all directions.

The medical device 10 is tracked by the tracking device 16. The tracking device 16 may be an optical tracking device, an ultrasonic tracking device or any other suitable tracking device, depending on the application and circumstances.

In the example shown in FIG. 1, the tracking device 16 is an optical tracking device that includes at least two optical sensors 32, such as the two cameras shown in FIG. 1, and can track a plurality of medical devices at the same time. The positions and orientations of the optical sensors 32 relative to each other can be determined, for example, through a rigid mechanical configuration. Alternatively or additionally, the optical tracking device includes a depth resolution camera that provides an optical image with depth measurements.

As described in International Patent Publication WO2010 / 067281A1, for example, an optical sensor is supported by a determined spatial relationship to an X-ray device in a medical imaging system.

When a reflective or fluorescent material is used, the optical sensor 32 includes a wavelength filter suitable for the type of reflective or fluorescent material used. One or more medical devices used in surgery are identified by using various reflective materials and a set of filters of a particular wavelength.

The tracking system 16 is incorporated into the surgical setup as part of a medical imaging system (eg, the C-arm X-ray system described above) or as a separate tracking unit.

Hereinafter, for the sake of facilitating the description of the present technology, the optical tracking device will be generally described in the present specification, but of course, other tracking devices are also within the scope of the present technology.

FIG. 2 schematically shows an example of a field of interest 12 in which a medical device 10 is located.

The tracking system 100 detects the line segment 22 (shown by the solid line) in the field of interest 12 and detects the instrument marker 14 or the marker 14'shown in FIG. The processing unit 18 selects the line segment 24 (indicated by a broken line) corresponding to the detected device marker as the medical device 10, and determines the marker position 20 as the offset 26 of the tip position of the medical device tip 28. Used as a reference for.

The term "offset" refers to spatial distance.

The tip position of the medical device includes a possible extension of the medical device, for example the length of the (tightly) connected portion of the vertebral pedicle screw attached to the screwdriver.

Next, referring to both FIGS. 1 and 2, during surgery, the tracking system 16 images the field of interest 12 and acquires image data of the field of interest 12. The image is corrected for lens distortion so that a straight line in the real world looks like a straight line in the image. However, if a low distortion lens is used, this step is omitted.

The processing unit 18 detects a straight line in the acquired image data, for example, through a Hough transform.

The processing unit 18 further determines the endpoints of each detected straight line. For each detected straight line, the plane extending between the end points of the line and the center of the camera are stored.

The processing unit 18 further determines the intersection of each plane pair for each detected straight line. This results in a 3D line description with a start point and an end point.

Along the detected straight line (or on its extension), the device marker 14 is detected on at least one camera image, for example by pattern recognition. The straight line on which the device marker 14 mounted on the straight line is detected is identified and tracked as a medical device.

For example, when an overlap or intersection between the tracked medical device 10 and the reference marker 34 is detected in the tracking image, that is, when the device 10 has a distance of less than a predetermined threshold to the reference marker 34, the operator The tip position can be initialized. Reference marker 34 is typically a marker within or near the field of interest and is used to track patient movement.

The reference marker 34 is provided so as to be in contact with the medical device tip 28 and is visible in the image obtained by the tracking device. The reference marker 34 is, for example, circular, but other shapes may be used to more reliably establish contact with the marker 34 by the medical device tip 28. For example, the marker 34 shown in FIG. 1 has a circular body from which the tail portion extends to further improve the visibility of the marker 34 in the tracking image when the device 10 is adjacent to the marker. To do.

The processing unit further detects the distance between the device marker 14 and the reference marker 34 in contact with the selected line segment 24, and inputs the detected distance to determine the offset 26 between the marker 14 and the tip position 28. Used as.

Alternatively, the tracking device 16 acquires the image data of the field of interest 12. The processing unit 18 generates a three-dimensional model from the acquired image data, and calculates an offset 26 to the tip position based on the marker position 20.

In one embodiment (not shown), the tracking image is made visible to the user on the screen of the display. In this case, feedback on the contact interaction between the device tip 28 and the reference marker 34 is given in the augmented reality view. This gives the user visual feedback that the instrument has been successfully calibrated, that is, the tip offset has been determined. Based on this, the user then initiates an intervention procedure in which the tip of the device is inserted into the subject.

Further, when the tip offset is initialized, a virtual device (not shown) is displayed on the patient data using the stored distance between the device marker 14 and the tip position 28 along the line segment. To. Patient data may be a 3D tomographic data set registered prior to minimally invasive intervention. The three-dimensional tomography dataset may be, for example, a CT / MR (Computed Tomography / Magnetic Resonance) dataset or an XperCT / VasoCT / angiography dataset.

Alternatively, the patient data may be live X-ray data presented to the user in addition to one or more tracking camera images of the tracking camera images.

FIG. 3 shows the basic steps of an example of the device tracking method 200. The method includes the following steps.

In the first step 202 (also referred to as step a)), a medical device image including a device marker placed on the medical device at a marker position away from the tip of the medical device is acquired.

In the second step 204 (also referred to as step b)), at least one line segment in the field of interest is detected in the image.

In the third step 206 (also referred to as step c)), the device marker is detected.

In the fourth step 208 (also referred to as step d)), the medical device is detected by selecting the line segment corresponding to the detected device marker from at least one detected line segment.

In optional step 210 (also referred to as step e)), the orientation of the medical device is determined based on the orientation of the selected line segment.

In the fifth step 212 (also referred to as step f)), the offset of the position of the tip of the medical device is determined by touching the reference marker with the medical device.

FIG. 4 shows that step f) of method 200 is further selected in steps 224, f3) of using the medical device tip to contact the next substep: f1) reference marker providing step 222, f2) reference marker. Further examples are shown, including step 226 to detect the distance between the device marker along the line segment and the contacted reference marker, and f4) step 228 to use the detected distance as an offset.

In another exemplary embodiment of the invention, a computer program or computer program element characterized by being adapted to perform the steps of the method according to one of the above embodiments on a suitable system. Provided.

Therefore, computer program elements may be stored in the computer unit. The computer unit may also be part of an embodiment of the present invention. The computer unit performs the steps of the above method or guides the execution of the steps. Further, the computer unit operates the components of the above device. The computer unit operates automatically and / or executes the user's instructions. The computer program may be loaded into the working memory of the data processor. Therefore, the data processor has the ability to carry out the methods of the invention.

This exemplary embodiment of the invention covers both a computer program that uses the invention from the beginning and a computer program that, by updating, transforms an existing program into a program that uses the invention.

In addition, the computer program element can provide all the necessary steps to meet the procedures of the exemplary embodiments of the above method.

According to a further exemplary embodiment of the invention, a computer readable medium such as a CD-ROM is presented. Computer program elements are stored on a computer-readable medium, which are described in the section above.

The computer program is a suitable medium (specifically, but not necessarily, a non-temporary medium) such as an optical storage medium or a solid medium supplied with or as part of other hardware. ) Are stored and / or distributed, but may be distributed in other forms, such as via the Internet or other wired or wireless communication systems.

However, the computer program may be presented via a network such as the World Wide Web and downloaded from the network into the working memory of the data processor. A further exemplary embodiment of the invention provides a medium that makes a computer program element available for download, the computer program element being a method according to one of the above embodiments of the invention. Configured to do.

The embodiments of the present invention are described with reference to various subjects. Specifically, some embodiments are described with reference to method type claims, while others are described with reference to device type claims. However, one of ordinary skill in the art will appreciate any combination of features related to various subjects, in addition to any combination of features belonging to one type of subject, from the above and below description, unless otherwise specified. It can be understood that it is considered to be disclosed by. However, all features can be combined as long as they provide a synergistic effect that goes beyond the mere addition of features.

Although the present invention has been exemplified and described in detail in the drawings and the above description, the examples and description should be viewed as exemplary and not limited. The present invention is not limited to the disclosed embodiments. Other variations of the disclosed embodiments will be understood and implemented by those skilled in the art who will practice the claimed invention from the drawings, disclosure content and examination of the dependent claims.

In the claims, the term "contains" does not exclude other elements or steps, nor does the indefinite definite article "a" or "an" exclude the plural. A single processor or other unit may perform the function of some of the items cited in the claims. The fact that specific means are described in different dependent claims does not indicate that the combination of these means cannot be used in an advantageous manner. Any reference code in the claims should not be construed as limiting the scope.

Claims (13)

A system that tracks elongated medical devices within a field of interest,
Tracking device and
With the processing unit
Including
The tracking device acquires an image of the medical device, the image comprising a device marker placed on the medical device at a marker position away from the tip of the medical device.
The processing unit i) detects a plurality of line segments and the device marker in the acquired image, and ii) from the detected plurality of line segments, a line segment corresponding to the detected device marker is obtained. A system that identifies the medical device by selection and determines the offset of the position of the tip of the medical device based on the medical device in contact with the iii) reference marker. The processing unit further detects the distance between the equipment marker and the contacted reference marker along the selected line segment and uses the detected distance in determining the offset. The system according to 1. The device marker is
Biocompatible reflective paint,
Fluorescent paint,
Materials that have a unique reflex reaction and form the medical device, and
The system of claim 1 or 2, comprising at least one of a group of adhesive tapes. The system of claim 3, wherein the biocompatible reflective coating and / or the fluorescent coating has bandwidth-selective luminescence. The system according to any one of claims 1 to 4, wherein the device marker has a pattern extending tangentially along the surface of the medical device. The system of claim 1 or 2, wherein the device marker comprises a boundary between two parts of the medical device. The system of claim 6, wherein the device marker is a boundary between the shaft of the needle and the handle. The tracking device acquires image data of the field of interest and
The processing unit according to any one of claims 1 to 7, wherein the processing unit generates a three-dimensional model from the acquired image data, and calculates the offset to the position of the tip based on the marker position. system. The system according to any one of claims 1 to 8, wherein the tracking device is an optical tracking system including at least one camera. A way to track medical devices in a field of interest,
a) A step of acquiring an image of the medical device including a device marker placed on the medical device at a marker position away from the tip of the medical device.
b) In the image, a step of detecting a plurality of line segments in the field of interest, and
c) The step of detecting the device marker and
d) A step of identifying the medical device by selecting a line segment corresponding to the detected device marker from the plurality of detected line segments.
f) A step of determining the offset of the position of the tip of the medical device by contacting the reference marker using the medical device, and
Including methods. Step f) further
f1) The step of providing the reference marker and
f2) The step of using the tip of the medical device so as to come into contact with the reference marker, and
f3) A step of detecting the distance between the device marker and the contacted reference marker along the selected line segment, and
f4) A step of using the detected distance as the offset, and
10. The method of claim 10. A computer program comprising instructions that, when executed by a computer, cause the computer to perform the steps of the method according to claim 10 or 11. A computer-readable medium storing the computer program according to claim 12.

Images