CN105934215B - The robot of imaging device with optic shape sensing controls - Google Patents

Abstract

A kind of system for tracking equipment image includes imaging system (110) in the art of the probe (146) with the image for being configurable to generate region.Shape senses enabled instrument (102) and is configured in the enabled instrument of shape sensing relative to the orientable part in the region.Shape senses enabled instrument with the coordinate system with the co-registration of coordinate systems used of imaging system in art.Robot is configured as coordinating the movement between the probe and the enabled instrument of shape sensing, makes probe be moved to maintain shape sensing enabled instrument in described image relative to the movement in the region so that shape senses enabled instrument.

Description

Robotic Control of Imaging Devices with Optical Shape Sensing

technical field

The present disclosure relates to medical instrumentation, and more particularly to robotic control systems and methods using optical shape sensing technology.

Background technique

Robotic control of real-time imaging devices, such as ultrasound imaging or any type of optical imaging, aims at simplifying the positioning of imaging devices during surgical and interventional procedures. The operator is allowed to perform remote control or image-based control of the equipment.

A disadvantage of remote control using traditional input devices (eg, joysticks) is that the mapping of the robot coordinate system and the coordinate system of the imaging device to the output image is not explicitly known to the operator. This mapping is typically learned during the process and can result in extended operating times.

Image guidance has other problems. These issues can include image guidance being able to track targets that are only within the imaging device's field of view, while the most challenging targets, those not in the field of view, remain inaccessible. Image guidance also requires the use of image processing methods to track targets, which can fail due to obscuring the field of view or poor image quality. For example, in ultrasound imaging, devices such as catheters are difficult to track due to poor visualization of the catheter tip and low signal-to-noise ratio in ultrasound images. Also, in 2D ultrasound, the device can be moved out of the viewing plane. For example, in optical imaging with an endoscope, the field of view can be significantly smaller than the anatomical region of interest. Also, in laparoscopic surgery, equipment can be out of the field of view, which can cause damage to tissue when the operator requires visual feedback using the operating site.

US Publication No. 2010/274087 and EP Patent No. 1779802 disclose medical robotic systems.

Contents of the invention

According to the present principles, a system for tracking an image of a device includes an intraoperative imaging system having a probe configured to generate an image of a region. The shape sensing enabled instrument is configured to have a portion of the shape sensing enabled instrument that is positionable relative to the area. The shape sensing enabled instrument has a coordinate system registered with a coordinate system of the intraoperative imaging system. The robot is configured to coordinate movement between the probe and the shape-sensing-enabled instrument such that movement of the shape-sensing-enabled instrument relative to the area causes the probe to be moved to sense the shape. The test enables the instrument to remain within the image.

Another system for tracking images of a device includes an ultrasound imaging system having a probe configured to generate images of a region. The shape sensing enabled instrument is configured to have at least a portion of the shape sensing enabled instrument that is positionable relative to the area. A robot is configured to coordinate movement between the probe and the shape sensing enabled instrument. The robotic control system includes a nested control loop comprising a first feedback loop and a second feedback loop, wherein the first feedback loop employs shape sensing feedback from the shape sensing enabled instrument, the second feedback loop A loop employs robotic encoder information as motion feedback for the probe, wherein the control system maintains a spatial relationship between the shape-sensing-enabled instrument and the probe such that the shape-sensing-enabled instrument is relatively Movement in the region causes the probe to be moved to maintain the shape sensing enabled instrument within the image.

A method for tracking an image of a device includes positioning a shape sensing enabled instrument at an interior region to be imaged; imaging the interior region of a subject with a probe for an intraoperative imaging system to generate a region-specific image ; registering the coordinate system of the shape-sensing-enabled instrument with the coordinate system of the intraoperative imaging system; robotically positioning the probe relative to the shape-sensing-enabled instrument such that the shape-sensing A measurement-enabled instrument is positioned within the image; and, the probe is robotically repositioned in accordance with movement of the shape-sensing-enabled instrument.

These and other objects, features and advantages of the present disclosure will become apparent from the following detailed description of illustrative embodiments read in conjunction with the accompanying drawings.

Description of drawings

The present disclosure will be described in detail in the following description of preferred embodiments with reference to the following drawings, in which:

1 is a block/flow diagram illustrating a system for image tracking configured to track a shape sensing enabled device or instrument with an imaging device using a robot, according to one embodiment;

2 is a cross-sectional view of a subject showing a built-in transducer following a shape sensing enabled device or instrument to image a volume or plane, according to one embodiment;

Figure 3 is a schematic diagram illustrating nested control loops for a robot control system, according to one embodiment;

Fig. 4A is a diagram illustrating an initial image illustrating a possible range of motion of a probe in accordance with the present principles;

FIG. 4B is a diagram illustrating an image of an object illustrating a possible range of motion of a probe in accordance with the present principles; and

5 is a flowchart showing a method for using a robot to physically track a shape sensing enabled device or instrument and an imaging probe, according to an illustrative embodiment.

Detailed ways

In accordance with the present principles, systems, devices, and methods are described that provide control of robots using optical shape sensing techniques. Image guidance of robotically controlled imaging devices solves the aforementioned mapping problem by allowing the operator to select a target from the most intuitive frame (the image frame). According to the present principles, robotic control of an ultrasound probe is provided to intelligently track an optical shape sensing (OSS) enabled catheter or other device. A system and method are provided to visualize devices during medical procedures by sensing the shapes of the devices, registering the shapes with a robot coordinate frame, and guiding a robot to bring those devices into the field of view of an imaging device. The present principle allows repositioning of the imaging device to reach targets that are not visible in the current view.

An optical shape sensing interventional device can be used to close the feedback control loop of a robotically activated imaging probe. The robot uses the known position of the shape sensing device with respect to the ultrasound volume to track the shape sensing device and maintain the device within the field of view during the procedure. Smart Tracking is also able to automatically select the best slice or plane to show the shape sensing device in the image. Furthermore, the tracking can take into account how to optimize the image resolution for the selected plane. Potential ultrasound image sources may include transesophageal echocardiography (TEE) probes, pediatric TEEs, micro-TEEs, surface imaging probes (eg, C5-2), or optical imaging probes (endoscope, bronchoscope, etc.). Other devices and processes can also benefit from the present principles.

In one embodiment, optical shape sensing is used to create virtual ultrasound images. This can be done using, for example, known transformations between the ultrasound probe and the catheter or shape sensing enabled device to reformat the ultrasound image so that the image appears as if the transducer aperture was on the catheter or device. The optical shape sensing catheter can then allow the aperture to translate and rotate about or relative to the catheter as long as the virtual aperture remains within the ultrasound data set. Additionally, the ultrasound delivery sequence can be adapted to optimize the virtual ultrasound image based on the location of the catheter in the ultrasound volume. To maintain the catheter at a position within or relative to the ultrasound imaging volume, the ultrasound imaging probe can be robotically controlled using the known position from the shape sensing catheter. Robotic control can improve the generation of virtual images by aligning the imaging volume with the device and allowing repositioning of the probe used to generate the virtual volume.

Potential sources of ultrasound images may include internal ultrasound probes, such as TEE; transrectal ultrasound (TRUS); single surface probes (eg, linear, curved, fanned, matrix); multi-surface probes (simultaneously or sequentially or both), etc. Potential registration between an ultrasound probe (e.g., head position) and a shape-sensing-enabled device (e.g., catheter) may include shape-based sensing of the ultrasound probe (shape-to-shape registration prior to deployment) Image-based registration using ultrasound probes such as EchoNav ^™ (e.g., TEE probe heads), model-based approaches, x-ray-based registration of shape sensing devices, etc.; using electromagnetic sensors such as TEE probe heads Tracking (EM to shape registration prior to deployment), optical tracking of hand-held probes, alternative tracking of probes with ultrasound image-based recognition, etc.

The known position and plane of the device can be used (by mechanical positioning of the robot) to alter the ultrasound transmission profile. Alternatively, ultrasound images can be used as input to drive the device in the direction being visualized/targeted (eg, for intravascular ultrasound (IVUS) withdrawal). The present principles allow any shape sensing enabled device to be converted into an intracardiac echocardiography (ICE) or IVUS device with the addition of an external ultrasound probe. For navigation purposes, any device already enabled for optical shape sensing can be repurposed to perform virtual IVUS with the addition of a standard ultrasound imaging probe.

The present principles apply robotic control of an external or internal probe (eg ultrasound) to move an ultrasound volume relative to the position of an optical shape sensing device, which can be used to define an aperture for the volume. Optical shape sensing devices may include wires or catheters, but can be extended to endoscopes, bronchoscopes, and other such devices or applications.

It should be understood that the present invention will be described in terms of medical instruments; however, the teachings of the present invention are broader and apply to any robotically controlled instrument using OSS enabled devices. In some embodiments, the present principles are employed in tracking or analyzing complex biological or mechanical systems. In particular, the present principles apply to internal tracking processes of biological systems and/or processes in all regions of the body (eg lungs, gastrointestinal tract, excretory organs, blood vessels, heart), etc. Elements depicted in the drawings can be realized in various combinations of hardware and software, and provide functions that can be combined in a single element or in a plurality of elements.

The functions of the various elements shown in the figures can be provided by the use of dedicated hardware as well as hardware capable of executing software in conjunction with appropriate software. When provided by a processor, the functions can be provided by a single dedicated processor, a single shared processor, or multiple individual processors (some of which can be shared). Furthermore, explicit use of the terms "processor" or "controller" should not be read to refer solely to hardware capable of executing software and can implicitly include, but is not limited to, digital signal processor ("DSP") hardware, Read Only Memory ("ROM"), Random Access Memory ("RAM"), Non-Volatile Memory, etc. for storing software.

Moreover, all recitations herein reciting principles, aspects, and embodiments of the invention, as well as specific examples thereof, are intended to encompass both structural and functional equivalents. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future (ie, any elements developed to perform the same function, regardless of structure). Thus, for example, it will be appreciated by those skilled in the art that block diagrams presented herein represent conceptual views of illustrative system components and/or circuits embodying the principles of the invention. Similarly, it will be appreciated that any flow charts, flowcharts, etc. represent various processes that may substantially be represented in a computer-readable storage medium and executed by a computer or processor, whether or not such computer or processor is explicitly shown. processor.

Furthermore, embodiments of the present invention can take the form of a computer program product accessible from a computer-usable storage medium or a computer-readable storage medium providing information for use by a computer or any instruction execution system. program code for or in conjunction with it. For the purposes of this description, a computer-usable storage medium or a computer-readable storage medium can be any means that can include storing, communicating, propagating, or transporting for use by or in connection with an instruction execution system, apparatus, or device the program used. The medium can be an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system (or device or device) or a propagation medium. Examples of computer readable media include semiconductor or solid state memory, magnetic tape, removable computer floppy disk, random access memory (RAM), read only memory (ROM), hard disk, and optical disk. Current examples of optical discs include Compact Disc - Read Only Memory (CD-ROM), Compact Disc - Read/Write (CD-R/W), Blu-Ray ^™ , and DVD.

Referring now to the drawings, wherein like numerals represent like or similar elements, and initially to FIG. system 100 . System 100 may include a workstation or console 112 from which the process is supervised and managed. Workstation 112 preferably includes one or more processors 114 and memory 116 for storing programs and applications. Memory 116 may store an optical sensing module 115 configured to interpret optical feedback signals from shape sensing device or system 104 . The optical sensing module 115 is configured to use optical signal feedback (and any other feedback, e.g., electromagnetic (EM) tracking, ultrasound, etc.) , optical shape sensing (OSS) devices, OSS catheters, catheters, etc.) associated deformations, deflections, and other changes. In some embodiments, the medical instrument 102 may include catheters, wires, probes, endoscopes, robots, electrodes, filter devices, balloon devices, or other medical components, among others.

The shape sensing system or device 104 on the instrument 102 includes one or more optical fibers (or cores or channels) 126 that are coupled to the instrument 102 in a setup mode or modes. Optical fiber 126 is connected to workstation 112 by cable 127 . The cable 127 may include optical fibers, electrical connections, other instruments, etc. as required.

The shape sensing system 104 with optical fiber may be based on a Fiber Bragg Grating sensor. A Fiber Bragg Grating (FBG) is a small length of optical fiber that reflects a specific wavelength of light and transmits all other wavelengths. This is achieved by adding a periodic variation of the refractive index in the core, which generates a wavelength-specific dielectric mirror. Therefore, fiber Bragg gratings can be used as inline optical filters to block certain wavelengths, or as wavelength-specific mirrors.

The basic principle behind the operation of fiber Bragg gratings is Fresnel reflection at each interface where the refractive index changes. For some wavelengths, multiple cycles of reflected light are in phase, whereby there is constructive interference for reflection and thus destructive interference for transmission. The Bragg wavelength is sensitive to strain and temperature. This means that Bragg gratings can be used as sensing elements in fiber optic sensors (as discrete or continuous elements along the fiber). In FBG sensors, the measured variable (eg, strain) causes a shift in the Bragg wavelength.

One advantage of this technique is the ability to distribute multiple sensor elements over the length of the fiber. Incorporating three or more cores into multiple sensors (meters) along the length of the fiber embedded in the structure allows the three-dimensional form of such structures to be precisely determined, typically with an accuracy better than 1 mm. At multiple locations along the length of the fiber, multiple FBG sensors (eg, 3 or more fiber sensing cores) can be positioned. From the strain measurements of each FBG, the curvature of the structure at that location can be deduced. Based on multiple measured positions, a complete three-dimensional form is determined.

As an alternative to fiber Bragg gratings, the inherent backscattering in conventional optical fibers can be exploited. One such approach is to use Rayleigh scattering in standard single-mode communication fibers. Rayleigh scattering occurs as a result of random fluctuations in the refractive index in the fiber core. These random fluctuations can be modeled as a Bragg grating with random variations in amplitude and phase along the length of the grating. By using this effect in three or more cores running within a single length of a multi-core fiber, it is possible to follow the 3D shape and dynamics of the surface of interest.

Optical shape sensing (OSS) uses light along multi-core fibers for device positioning and navigation during surgical interventions. The principles involved make use of distributed stress measurements in optical fibers using characteristic Rayleigh backscattering or controlled grid patterns. The shape along the fiber starts at a specific point along the sensor, called the firing region 156 or z=0, and subsequent shape positions and orientations are relative to that point. Optical shape sensing fibers integrated into medical devices such as catheters and guide wires provide live guidance of the device during minimally invasive procedures and can provide the position and orientation of the entire instrument 102 .

In one embodiment, the workstation 112 includes an image generation module 148 configured to receive feedback (position data) from the shape sensing device 104 as to where the sensing device 104 is or has been within the object 160 . Image volume (or data set) 131 is imaged within subject 160 using imaging system 110 , such as an ultrasound imaging system, although other intraoperative imaging systems may be employed. One or more frames of the data set are collected from the imaging system 110 using one or more internal or external probes or transducers 146 (also referred to as probes, probe heads, ultrasound probe heads, TEE probe heads, etc.). Image 134 to mark image volume 131 . An image 134 can be displayed on the display device 118 . Image 134 may be overlaid on, blended with, or otherwise drawn along with other preoperative or intraoperative images.

Workstation 112 includes display 118 for viewing internal images of subject (patient or other object) 160 or volume 131 . Display 118 may also allow a user to interact with workstation 112 and its components and functions or any other element within system 100 . This process is facilitated through interface 120 , which may include a keyboard, mouse, joystick, haptic device, or any other peripheral device or control modality to allow user feedback from and interaction with workstation 112 .

In accordance with present principles, shape sensing enabled instrument 102 and ultrasound probe or transducer 146 may have their movements coordinated using robotic system 108 . An ultrasound probe 146 may be coupled to the ultrasound imaging system 110, which may be part of the console 112 or may be a separate unit. The ultrasound image 131 can be generated from a single transducer, e.g., a transesophageal echocardiography (TEE) transducer, a nasal TEE probe, or an external surface probe (such as a C5-2 probe, etc.), and may include volumetric (3D image ), planes or slices (2D images). Ultrasound probe 146 may be an external probe or an internal probe. Although ultrasound imaging is described herein, other imaging modalities can also be used.

In a particularly useful embodiment, OSS-enabled instrument 102 is manipulated within subject 160 to perform functions during a surgical procedure. The robotic system 108 tracks the portion of the instrument 102 (or instruments) while the OSS-enabled instrument 102 is positioned within the subject. Different shapes or features on the distal portion (or other portion) of instrument 102 may be used as targets for imaging probe 146 . OSS enabled instrument 102 and imaging probe 146 may be registered such that movement of OSS enabled instrument 102 causes robotic system 108 to move probe 146 accordingly. The probe 146 is moved along the surface of the subject 160 and pressure and acoustic coupling to the subject needs to be monitored to ensure no injury and proper imaging of the subject 160 . Robotic system 108 may be controlled using control system 140 . Control system 140 may also allow for manual user control and/or image-based guidance. Control system 140 may include hardware, software, or a combination thereof.

Registration between the probe 146 and the OSS-enabled instrument 102 can be accomplished in a variety of ways. A registration module 158 may be included to handle registration operations for the system. One method of registration may include having the robotic system 108 control motion for the two devices. Other methods include registering the coordinate frames of the probe 146 and OSS-enabled instrument 102 to a common coordinate frame, and the like. Other registration methods may also be used to establish registration between probe 146 and OSS-enabled instrument 102 . For example, such a registration technique to register the ultrasound probe 146 (head position) with the OSS-enabled instrument 102 (e.g., shape sensing catheter) may include employing: EchoNav ^™ registration of the probe head (shape sensing device x-ray based registration of the probe head); electromagnetic tracking of the probe head (EM to shape registration before deployment), optical shape sensing tracking of the probe head (shape to shape registration before deployment), etc.

In another embodiment, a fluoroscopy-based registration may be performed. For example, the TEE probe tip 146 can be registered to the x-ray image (e.g., as in EchoNav ^™ ), and the OSS catheter 102 can also be registered to the x-ray image, providing the TEE probe tip 146 with the OSS can switch between instruments 102. The TEE probe head 146 would need to be dynamically tracked via x-ray (eg, by the robotic system 108 ), while the OSS enabled instrument 102 would only need to be registered to the x-ray image once. In yet another embodiment, an alternative tracking transducer 146 and shape sensing catheter 102 may be employed. The transducer or probe head 146 can be tracked using other tracking techniques (e.g., electromagnetic tracking or optical tracking for external transducers), and the triggering device 156 of the optical shape sensing enabled instrument 102 can be registered to the Alternative Tracking Solutions. Other registration techniques and methods are possible and contemplated in light of the present principles.

In one embodiment, to allow proper generation of views in the correct plane of the catheter or instrument 102 , the control scheme of the robotic control system 140 can be modified such that the US probe 146 is guided by the shape of the proximal end of the catheter 102 . In this embodiment, the shape of the conduit 102 is fitted to a plane. The orientation of the plane to a point in the probe workspace is optimized to maximize the instrument 102 in the physical image plane. In yet another embodiment, the robot 108 can move to an optimal position to generate an ultrasound volume, and then return to the original position to visualize some other feature (eg, another device or an anatomical feature). The position of such features can be "marked" in the ultrasound volume (and thus the probe coordinate system) by saving the position (or other position) of the distal tip of the instrument 102 at discrete instants in time. Even when the shape sensing instrument 102 has been navigated away from the location, the location can then be re-imaged by the probe 146 .

An optimal plane can be determined through the distal portion of the instrument 102 to determine the optimal position of the probe 146 . Since the motion will be known from the robot encoders and the robot workspace, the spatial relationships in the frame of reference are known and can be visualized. The physical volume can be updated and in an "offline" volume visualization scenario can show those parts of the volume that are visible in the new probe position. The robotic system 108 can move between two different positions to update the entire volume, cycle between images, and the like.

In three-dimensional imaging, the ultrasound volume 131 may be resampled to extract only the slice (or subset of the volume 131 ) that includes the instrument 102 . In particular, this can be performed by extracting the plane that best shows the distal portion of the instrument 102 . In two-dimensional imaging, when the robotic system 108 is moved, the 2D slice can be adjusted to best capture the plane of the instrument 102 . Extracting the image of the instrument 102 may be performed by employing pattern recognition or other image processing techniques (eg, using the image processor 148 ) to identify the instrument 102 in the image. The instrument 102 position can also be determined using the registration information, and based on user preferences an optimal slice or position can be determined. Other approaches can also be employed and contemplated.

In one embodiment, to allow proper generation of views in the correct plane of catheter 102 , the control scheme can be modified such that US probe 146 is guided by the shape of the proximal end of catheter 102 . In this embodiment, the shape of the conduit 102 is fitted to a plane. The orientation of the plane to a point in the probe workspace is optimized such that the plane is aligned to the desired view of the instrument 102, eg, the angle between the plane of the shape of the instrument 102 and the physical image plane may be minimized. In yet another embodiment, the robotic system 108 can move to an optimal position to generate an ultrasound volume, and then return to an original position to visualize some other feature (eg, another device or an anatomical feature). An optimal plane can be determined through the distal portion of the instrument 102 to determine the optimal position of the probe head 146 . Since the motion will be known from the robot encoders and the robot workspace, the spatial relationships in the frame of reference are known and can be visualized. From the physical volume can be updated and in an "offline" volume visualization scenario show those parts of the volume that are visible in the new probe position. The robotic system 108 is able to move between two different positions to update the target or the entire volume.

Referring to FIG. 2 , a cross-sectional illustration shows a patient 202 with a TEE transducer 204 on a TEE probe 206 . Transducer 204 passes through the esophagus of patient 202 and creates ultrasound imaging volume 208 within patient 202 . The imaging volume 208 overlaps one or more regions in which a shape sensing enabled instrument 214 (eg, a catheter) has an optical shape sensing fiber 212 therein. Instrument 214 may be provided through a port or through a natural orifice in the patient. Based on instrumentation 214, imaging volume 208 is positioned at a selected location and orientation. TEE probe 206 is robotically controlled and used to visualize shape sensing enabled instrument 214, which is depicted as an intravascular catheter. An optimal plane 220 for the three-dimensional imaging volume 208 is selected based on the shape of the distal end of the catheter 214 . The shape of the distal end of catheter 214 is used as a mechanism for defining viewing plane 220 which, in particularly useful embodiments, includes visualization of tools within viewing plane 220 . That is, the intravascular shape sensing catheter 214 is visualized using the robotically controlled TEE probe 206 with an optimal slice or plane 220 configured for display within the 3D imaging volume 208 according to the shape of the distal portion of the instrument 214 . Examples of pre-curved intravascular catheter shapes, which can be considered to define a viewing plane, may include, for example, the shapes provided by instruments such as Cobra ^™ , Berenstein ^™ , SIM2 ^™ , and the like. Custom shapes are also available.

During a clinical procedure, the operator may need to maintain the instrument 214 within the ultrasound field of view (FOV). During the interventional procedure, the instrument 214 may be moved continuously or intermittently as the operator navigates between relevant clinical goals. Examples of switching between multiple positions may include when an interventional cardiologist navigates the catheter towards the mitral valve of the heart to deploy a device, such as a trim, or the like. There is a periodic motion caused by heartbeat or respiration that moves the instrument 214 in and out of the imaging plane. In some clinical applications, multiple targets may need to be visualized during a procedure. For example, in the case of a paravalvular leak, the interventionalist will want to switch views between the leak itself and the many catheters that can be used to close the leak. Because the field of view is limited, ultrasound volumes cannot visualize all areas at once.

In accordance with the present principles, shape sensing system 212 is configured to interrogate shape in a plurality of locations or areas. There may also be multiple shape sensing systems 212 employed on or with one or more tools or instruments used during the procedure. These tools or instruments may include catheters or any other equipment. Shape sensing system 212 preferably includes one or more optical fibers embedded therein. The probe 206, eg, an ultrasound probe, such as a TEE probe, is configured to acquire 2D or 3D images. The robotic system 226 is configured to move the probe according to commands from the robotic control and/or user interaction system 228 .

In one embodiment, TEE probe 206 is inserted into patient 202 to visualize the heart. A shape sensing enabled instrument (catheter) 214 is introduced into the heart to perform the procedure. A user selects multiple targets (catheters or other devices with shape sensing fibers integrated therein) to have their coordinate system 232 registered to the robot coordinate frame 230 and then brought into the field of view of the ultrasound probe 206 .

During the process, the user is able to select different modes of robot control. These may include a continuous mode selected by the user such that the robot continuously tracks the device 214 in order to maintain the device 214 in the field of view. Another mode includes a static mode, where the user can select this mode so that the robot moves on explicit command to visualize a specific target without continuously updating the position. Another mode can include a toggle mode, where the user can select this mode to cause the robot to visualize all objects in the circuit (eg, display each view in sequence). This mode can be used to check the position and status of all equipment and anatomical structures. Other modes are also contemplated, which may include the described modes or other modes. For example, split screen mode can be used instead of toggle mode to view multiple objects at the same time.

Referring to FIG. 3 , a nested closed loop feedback control system 300 is shown according to an illustrative embodiment. The control system 300 may be used to control the robot (108, FIG. 1; 226, FIG. 2). In this control system 300 , a higher level control loop 302 uses the known position of the catheter distal tip (shape sensing feedback 304 ) with respect to the ultrasound probe head 306 as an input to a lower level controller 303 . The lower level controller 303 monitors the encoder position (encoder feedback) 308 and updates the motor position 310 accordingly. The control scheme of Figure 3 can be used with any type of robot and imaging device. The illustrative nested control loop 300 utilizes shape sensing feedback 304 to robotically control the ultrasound volume (probe head 306) to maintain a fixed relationship with respect to the shape sensing interventional device (102, FIG. 1; 214, FIG. 2). The ultrasound probe position is adjusted by the robot to bring the catheter into the ultrasound field of view, either at some discrete times or in a continuous loop.

Referring again to FIG. 2 , during tuning, the robot control 228 needs to consider multiple factors simultaneously as feedback data. For example, movement of probe head 206 needs to result in moving imaging volume 208 or 2D slice 220 such that device 214 is within the field of view. The robotic control 228 needs to monitor the contact force between the probe head 206 and the tissue surface to ensure that excessive force is not applied, while maintaining sufficient force to ensure acoustic coupling. Other physical constraints of the ultrasound probe 206 also need to be considered (eg, the TEE probe 206 may have a bounded range of possible locations). In the case of a robotically controlled TEE probe 206 , control of the robot 226 may include control of two dials for varying the position of the curved probe tip of the probe 206 .

4A and 4B with continued reference to FIG. 2, an example of the field of view of the TEE probe (206) can be adjusted to respond to Object 402 is imaged. The textured area 408 on the surface of the sphere 410 represents the allowable range of motion of the TEE probe (206) when controlled via the dial. Each point on the sphere 410 has an intrinsic orientation 407 of the probe head 206, ie, a corresponding image volume. The points of the sphere 410 and the volume orientation 407 associated with those points are characteristics of the probe 206 and can be stored in a lookup table or other data storage structure in the robot control system 228 . Once the target 402 is selected in the coordinate frame 230 of the robot 226 , an ideal orientation of the US volume 208 is calculated such that the target 402 is in the middle of the volume 208 . This orientation is matched to the closest orientation in the lookup table, which is matched to the probe position.

The robot 226 attempts to reach the position by controlling, for example, two dials while constraining the force on the tissue. Pressure sensors or other sensors or measurements (eg, motor current) may be used as feedback for satisfying these constraints. If no excessive force is applied during the motion, the robot 226 will reach an optimal position to view the target 402 . If the force limit is reached, the viewpoint of the imaging device will be suboptimal, but will be the most probable viewpoint taking into account the constraints.

Robotic control of imaging equipment can have many implementations for control of ultrasound probes. For example, a system for controlling a dial on a handle of a TEE probe and robotically controlling the position of a steerable probe tip, among other robotic systems and methods, may be employed.

To reduce impact on clinical workflow, the ultrasound probe(s) can be robotically controlled to maintain catheter position within the ultrasound data set. In cases where ultrasound quality is degraded or lost due to lack of coupling or pressure between the ultrasound transducer head and the surface (detected via image processing or manual observation), the robot can (by, for example, increase pressure on the surface or releasing the gel) detects and compensates for the coupling or pressure, and then proceeds to perform imaging. Robotic control can simultaneously maintain the imaging volume, can optimize the physical position of the transducer for image resolution, and can work within the physical constraints of clinical applications to maintain proper contact with tissue for imaging while minimizing force on the patient. In the case of a robotically controlled TEE probe, the control of the robot may include control of the two dials of the TEE probe for changing the position of the curved probe tip.

The present principles apply to any robotic control of an imaging probe (eg, ultrasound) using input from a shape sensing instrument. This applies to guide wires, catheters (both manual and robotic), etc., and can be extended to other devices and applications, eg, endoscopes, bronchoscopes, and other such applications.

Referring to FIG. 5 , a method for physically tracking an image of a device is shown in accordance with an illustrative embodiment. In block 502, a shape sensing enabled device or instrument is positioned within an interior region to be imaged. Shape sensing enabled devices may include catheters or other instruments. In block 504, an interior region of the subject is imaged using a probe and/or transducer of an intraoperative imaging system to generate a region-specific image. The probe can include an ultrasound probe, although other imaging equipment can be used. The images may comprise two-dimensional or three-dimensional images.

In block 508, the coordinate system of the shape sensing enabled device is registered with the coordinate system of the intraoperative imaging system. This can be performed using a number of different methods. In block 512, the probe is robotically positioned relative to the shape sensing enabled device such that the shape sensing enabled device is positioned within the image. The robot can be programmed to center the shape sensing enabled device in the image, although other geometric arrangements or relationships can be employed.

In block 518, the probe is robotically repositioned based on the movement of the shape sensing enabled device. As the shape sensing enabled device moves, the probe tracks the movement to ensure that the shape sensing enabled device and/or the surrounding area of interest are still in the displayed field of view. In block 520, images from the intraoperative imaging system are optimized to provide a field of view that includes the shape sensing enabled device or other preferred view. Optimizing may include maximizing the size of the device in the image, minimizing the size of the device in the image, centering the device in the image, and the like.

In block 522, positioning or repositioning the probe may include manipulating the probe between a starting position and a target position according to an allowable range of motion between the starting position and the target position. In block 524, the range of motion may be stored in a data structure, such as a look-up table. In block 526, the probe may be steered to the closest position, stored in a lookup table corresponding to the closest target position.

In block 528, robot movement is constrained, eg, to provide acoustic or other coupling between the probe and the object, prevent injury to the patient, constrain the motion of the robot, and the like. In block 530, the robot may be controlled such that the robot moves the probe between the plurality of shape sensing enabled devices or instruments to switch between images.

In reading the appended claims, it should be understood that:

a) the word "comprising" does not exclude the presence of elements or acts other than those listed in a given claim;

b) the word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements;

c) any reference signs in the claims do not limit the scope thereof;

d) several "units" may be represented by the same item or hardware or software implementing a structure or function; and

e) is not intended to require a specific order of actions unless specifically indicated.

Having described a preferred embodiment of robotic control of an imaging device with optical shape sensing (which is intended to be illustrative and not limiting), it should be noted that modifications and variations will occur to those skilled in the art in light of the above teaching. It is therefore to be understood that changes may be made in the particular embodiments of the present disclosure which are within the scope of the embodiments disclosed herein as outlined by the claims. Having thus described the details and features required by the patent laws, what is set forth in the appended claims and desired protection by Letters Patent is.

Claims (15)

1. a kind of system for tracking equipment image, comprising:

Imaging system (110) in art, have probe (146), and the probe is configurable to generate the image for region；

Shape senses enabled instrument (102), is configured with the shape and senses in enabled instrument relative to the region It can position at least partly, the shape senses enabled instrument with the seat with the co-registration of coordinate systems used of imaging system in the art Mark system；And

Robot (108) is configured as coordinating the probe and the shape senses the movement enabled between instrument, so that institute Stating the enabled instrument of shape sensing moves the probe relative to the movement in the region, and the shape is sensed enabled instrument Device maintains in described image, wherein the robot (108) is configured as allowing according between initial position and target position Motion range the probe is manipulated between the initial position and the target position.

2. system according to claim 1, wherein the probe (146) includes ultrasonic probe, and described image includes Two dimensional image or 3-D image.

3. system according to claim 1 further includes robot control system (140), the robot control system quilt It is configured to store range of motion information in a lookup table, the robot control system is configured as the probe being manipulated to It is stored in the proximal most position of target position described in the distance in the look-up table.

4. system according to claim 2, wherein the robot (108) is restrained to provide the transducing of the probe Device coupling and it is restrained to prevent the injury to patient.

5. system according to claim 1 further includes robot control system (300), the robot control system packet Nested control loop is included, the nesting control loop includes the first feedback loop and second feed back loop, wherein described first is anti- It is fed back to road and uses the shape sensing feedback for sensing enabled instrument from the shape, the second feed back loop is compiled using robot Code device information is as the motion feedback for being directed to the probe, so that the robot control system maintains the shape sensing enabled Spatial relationship between instrument and the probe, wherein first feedback loop is compared with the second feed back loop Higher level control loop.

6. system according to claim 1 further includes multiple enabled instruments of shapes sensing, and further includes for robot The switch mode of control so that the robot senses the mobile probe between enabled instrument in the multiple shape, with It is switched between each image.

7. system according to claim 1, wherein the image of imaging system, which is optimized to provide, in the art includes The shape senses the visual field of enabled instrument.

8. a kind of system for tracking equipment image, comprising:

Ultrasonic image-forming system (110), has probe (146), and the probe (146) is configurable to generate the figure for region Picture；

Shape senses enabled instrument (102), is configured with the shape and senses in enabled instrument relative to the region It can position at least partly；

Robot (108) is configured as coordinating the probe and the shape senses the movement enabled between instrument, wherein The robot (108) is configured as according to the motion range allowed between initial position and target position in the initial position The probe is manipulated between the target position；And

Robot control system (140) comprising nested control loop, it is described nesting control loop include the first feedback loop and Second feed back loop, wherein first feedback loop uses the shape for sensing enabled instrument from the shape to sense feedback, The second feed back loop is using robot encoder information as the motion feedback for being directed to the probe, wherein the machine People's control system maintains the shape to sense the spatial relationship between enabled instrument and the probe, so that shape sensing makes Energy instrument moves the probe relative to the movement in the region, the enabled instrument of shape sensing is maintained described In image.

9. system according to claim 8, wherein described image includes two dimensional image or 3-D image.

10. system according to claim 8, wherein the robot control system (140) is configured as motion range Information stores in a lookup table, and the robot control system, which is configured as the probe being manipulated to, is stored in the lookup The proximal most position of target position described in distance in table.

11. system according to claim 8, wherein the robot (108) is restrained to provide acoustical coupling and by about Beam is to prevent the injury to patient.

12. system according to claim 8 further includes multiple enabled instruments of shapes sensing, and further includes for machine The switch mode of people's control, so that the robot senses the mobile probe between enabled instrument in the multiple shape, with It is switched between each image.

13. system according to claim 8, wherein the image from the ultrasonic image-forming system is optimized to provide packet Include the visual field that the shape senses enabled instrument.

14. a kind of device for tracking equipment image, the memory including processor and storage computer-readable instruction, In, the operation of the computer-readable instruction enables the processor:

(502) are imaged using interior zone of the probe for imaging system in art to object, to generate for described interior The image in portion region, shape sense enabled instrument and are positioned relative to the interior zone of the object；

The coordinate system that the shape senses enabled instrument is registrated (508) with the coordinate system of imaging system in the art；

Enabled instrument is sensed relative to the shape and positions (512) described probe in a manner of robot, so that the shape senses Enabled instrument is positioned in described image；And

(518) described probe is relocated in a manner of robot according to the position that the shape senses the update of enabled instrument；And And

Wherein, it includes according to initial position and mesh that (512) are positioned in a manner of robot and relocate (518) in a manner of robot The motion range allowed between cursor position manipulates (524) described probe between the initial position and the target position.

15. device according to claim 14, wherein the operation of the computer-readable instruction also enables the processor:

The described image of optimization (520) imaging system in the art includes that the shape senses enabled instrument to provide Visual field.