Vista Robotics
Navigation Module Design Input
The navigation module utilizes the NDI optical camera (“Camera”) to track specific objects,
namely rigid bodies, in real-time. The Camera will also track movements of Surgical Tools.
Then, it shows the rigid body poses in the Vista Robotics app image viewer. This module
establishes a network connection with the NDI camera and reads the rigid bodies' pose data.
The rigid bodies are generally attached to surgical tools.
Below is the list of Vista Robotics’ surgical tools with unique rigid bodies. Each rigid body has a
custom ROM file which enables the Camera to track its position. Figure 6 shows the screw
inserter and a custom designed rigid body attached to the tool.
I. Discectomy and endplate preparation device
II. Screw inserter
III. Interbody cage inserter
IV. Pilot hole obturator
V. MIS awl
VI. MIS tube/retractor
VII. Needle
VIII. Hand probe
The CAD file of each surgical tool listed above, including the Vista Robotics discectomy and
endplate preparation device, is available and the transformation from rigid bodies to tool’s tip is
calculated. Figure 1 demonstrates the visualization requirement achieved in the Cygna-6D
software. Cygna-6D is a software that is created by the NDI team to track and visualize rigid
bodies.
Figure 1. The Vista Robotics discectomy and endplate preparation tool model is presented. The custom
rigid body with markers is attached to the tool.
Given the CAD file and the transformation, displaying the surgical tool poses in real-time is
feasible. For rigid bodies that are not attached to the surgical tools (specially the rigid body
attached to the patient’s iliac crest (DRB) and the rigid body on the robotic cart), the
visualization on the monitor is not required and it is sufficient to show the visibility status of them
to the Camera. For instance, the robotic cart’s movement is monitored by the rigid body
attached to it, and one rigid body is attached to the patient’s iliac crest which is called Dynamic
Reference Base (DRB) to monitor the movement of the patient.
The specific tasks to be done are listed below:
● Connect to the NDI camera: The C# application establishes a TCP connection with the
NDI camera which is available in the local network. Figure 2 shows the connection
control window in the developed C# application.
Figure 2. The connection control window is presented that enables the user to establish a network
connection with the NDI camera.
● Read and show the rigid bodies visibility status. At the left middle side of Figure 3, the
visible rigid bodies are shown in green and the blocked rigid bodies are shown in red.
The criteria for the visibility of rigid bodies are defined entirely in the implemented C#
application.
Figure 3. A screenshot of implemented C# application developed by the Vista team is presented. The
rigid bodies’ status are visible on the left side.
● Utilize the real-time pose data to visualize all the surgical tools listed above: The pose
data container composed of linear transformation parameters. The transformation
parameters are applicable in VTK image viewer to demonstrate the surgical pose in real-
time. Figure 1 is an example model that shows the desired output. In addition to the 3D
visualization, 2D projections of the surgical tool must be visualized in axial, sagittal, and
coronal views. Figure 4 shows the desired output in the Globus ExcelsiusGPS solution
where the surgical tool is visualized in red and projected to axial, sagittal, and coronal
planes. Figure 5 demonstrates 2D projections of the surgical tool (the Vista Robotics’
discectomy tool) in axial and sagittal views. The navigation of the surgical tools desirably
will be similar to the visualization in Fig. 5 when the green arrows will be removed.
Figure 4. The 3D visualization of the surgical tool in red is projected to 2D axial, sagittal, and coronal
views (the screenshot is taken from https://www.youtube.com/watch?v=aJzVry0bww0).
Figure 5. 3D STL file of the Vista Robotics’ discectomy tool is projected into axial and sagittal views. The
navigation of the surgical tools desirably will be similar to this visualization.
Remarks:
1. Except for the visualization of surgical tools, the module is already implemented in a
Visual Studio project and requires integration with the Vista UI app.
2. Establishing a connection with the NDI camera is already implemented in the Vista app;
however, the Emorphis team should verify if the submodule works properly.
Figure 6. The screw inserter is presented and a custom designed rigid body with four
markers attached to the tool.
3. The Emorphis team does not need to design and implement the UI for this module. The
navigation module UI is already designed and implemented in the Vista app (please see
Figure 7 as an example that we will work on to add more control). Emorphis team may
ask Vista team if any modifications are needed.
Figure 7. The navigation module UI is shown in the Vista Robotics app. However, the rigid body names
should be modified as in Remark 3.
The navigation and robotic control modules are tightly connected and must be implemented in
one UI tab.
The Robotic control module connects to the KUKA LBR Med 14 robot and enables the user to
communicate with the robot through the Vista app. The module checks for the “Brake test”
requirement and provides control to the user to register the NDI camera coordinates with the
robot coordinate system. Then, the module provides a UI to the user to move the robot to the
predetermined target and execute predefined movements such as elliptical and sweeping
moves. This module includes controls to configure the footswitch and manipulate the I/O
devices, such as the cutter and the steering mechanism of the discectomy and endplate
preparation device. The cutter and the steering mechanism have magnetic encoders that
monitor the tool position. The discectomy planning output must be compatible with the robotic
movement commands. The module records the events, user actions, exceptions, and errors in
log files.
The following items break down the robotic control module into submodules. The code for the
submodules is available in C# and Java external projects. Therefore, we require the Emorphis
team to integrate the available codes with the Vista C# application.
1. External connection to the robot:
Vista desktop is connected physically to the robot, and the desktop is configured to
control the robot externally. The robot and the Vista desktop are assigned static IP
addresses known to the client C# application on the desktop and the server Java
application on the robot. The local IP addresses of the robot and the desktop are
172.31.1.147 and 172.31.1.220, respectively, and should not be changed. The robot
controller's UDP connection on port 30005 can be established after executing a series of
handshaking between the robot and the desktop (see Figure 8). In the client C# app, the
user is provided with a GUI control to connect to the robot, but a control button must be
implemented on the Vista Robotics app.
Figure 8. External connection window to the robot controller in the client C# app is shown.
2. Brake test:
The KUKA LBR Med 14 robot requires testing the seven joints every 24 hours. This test
is called the Brake test. The client C# application is equipped to handle the brake test
performance. Client C# application asks the user to confirm before implementing this
test. A brake test (if required) must be done automatically after connecting to the robot,
and a warning message must be provided to the user to wait until the test is completed.
Figure 9 presents a similar message when the Brake test is required in the client C#
application after connecting successfully to the robot.
Figure 9. The “Robot Brake Test Required” message window is presented in the client C#
application.
3. Robotic and tracker coordinate registration:
The NDI camera navigation system tracks rigid bodies by translating their position in
space to a global coordinate system. The KUKA robot uses a similar coordinate system
with a different reference point and scaling factors. The client C# application provides a
user interface to execute the registration of these two coordinate systems. The
registration output transformation is saved, and the other submodules will employ it.
Figure 10 shows the registration UI in the client C# application. The submodule allows
users to add, remove, and re-order the registration points. Besides, there are controls to
cancel the registration.
Figure 10. The coordinate registration window in the client C# application is demonstrated.
Similar controls must be implemented in the Vista Robotics app.
4. Robotic movement control:
Once the output of the coordinates registration is available, the user is provided with
moving the robotic arm to a target position. The target position is a calculated trajectory
by the robotic surgery submodule (see Submodule 8). The client C# application provides
a user interface to move the robot to the target. Besides moving to the target, the UI
enables the user to move the robot elliptically and the sweeping motion during the
discectomy and endplate preparation procedure. The elliptical movement and the
sweeping motion must be employed on the selected discectomy planning points, and its
radius must be proportional to the discectomy planning's spheres. Then, the sweeping
motion is performed in several cycles along the annular inside the disc space. During the
discectomy procedure, the tip of the discectomy and endplate preparation device must
be displayed in the axial and sagittal views (see Figure 5).
The server Java application has a convenient function of moving the robot's seven joints
into desired angles. This submodule will consist of an interface to realize this movement.
5. Input/Output (I/O) and footswitch controls:
The input and output signals are configured in the server-side Java application. In
particular, the configured I/O allows the user to manipulate the cutter and steering
mechanism of the discectomy tool. The Java app also provides classes to program the
behavior of the footswitch connected to the robot controller. This submodule will equip
the user with an interface that reconfigures the aforementioned devices and monitors
their parameters.
6. Encoders control:
The cutter and the steering mechanism have magnetic encoders that monitor the tool
position. The readable positions, which are 0-65535, must be shown in real time during
the work with the robot. Also, the position of the steering mechanism plays an essential
role during the discectomy procedure. The I/O parameters control the cutter's nominal
speed. However, the actual speed must be calculated using its encoder. The actual
cutter speed gives rough feedback of the underlying tissue. If the actual speed reduces
to an assigned threshold, the task must be terminated, and the robot performs the
following planned procedure.
7. Compatibility with the surgical planning output:
The output of the surgical planning, including the discectomy planning, must be
compatible with the robotic movement commands. The UI will allow the user to move the
robot to the calculated trajectory. This submodule must verify with the user the selected
spheres and the steering mechanism's requirement in terms of angular degrees. The
output of the screw trajectory planning and interbody cage planning output must be
compatible with the robotic movement commands so that a proper trajectory is set for
surgeon to install each of the required screws and inserting the required interbody cage.
8. Robotic surgery:
The preoperative planning phase supplies the necessary information to carry out the
robotic surgery. This submodule consists of four parts which are: 1) Pilot hole trajectory;
2) automatic discectomy and endplate preparation; 3) screw trajectory; 4) interbody cage
trajectory. The UI allows the user to execute the four stages of robotic-assisted fusion
surgery.
9. Saving logs:
The module records the events, user actions, exceptions, and errors in log files similar to
the current logging mechanism in the client C# application.
