CA2144505A1 - Computer based medical procedure simulation system - Google Patents

Abstract

A virtual reality medical simulation system provides a highly realistic simulation of a variety medical procedures, particularly catheter based procedures such as balloon angioplasty. A catheter interface device tracks a catheter wire in translation and rotation and output a signal to a computer program that displaysmovement of a virtual catheter within a virtual arterial tree including an occluded region. The catheter tip is navigated through the arterial tree and into the occluded region, whereupon the computer program generates a tactile feedback force that is output to the catheter interface device to simulate increased resistance. Once the catheter is properly positioned within the occlusion, a virtual balloon catheter can be inflated to remove the occlusion. A variety of special effect enhancements are included in the simulation, including 3-D imaging, simulated radio-opaque dye infusion, bleeding and atrial pulsation in synchronism with an on-screen EKG
display. The simulation also includes case specific or randomly generated patient conditions and complications, as well as various user dependent error conditionssuch as puncture of the arterial wall and balloon rupture.

Description

2144~0~

COMP[lTER BASED MEDICAL PROCEDllRE SIMULATION SYSTEM
~CK(~ROTINIl OF TI~F INVF.I~l'ION
The present invention relates to computerbased simulation of medical ~1~ ' e.g., surgical procedures. A parlicular application of the present invention concerns a system for providing a realistic avirtual reality" simulation of a cardiac ~-h ~ procedure.
Continuing public debate concerning rising health care costs has pin-pointed continuing medical education (CME) and graduate medical education (GME) as major expenses to address in the health reform effort. More than $6 billion is spent on CME and GME annually. Much of this cost is attributed to expensive training, resources and material. The use of virtual reality (VR) im medical education and training can reduce the expenses associated with changes in medical technology, as well as provide enhanced physician training.
VR technology has been used in a wide variety of fields, including militarv, A~ h~ ..ti, chemistry and ' ' - The application of VR technology to education and training, however, is just beginning to be eA-ploited. VR
lC . ~ are becoming more applicable in medicine, in part because of the high costs of traditional training resources (lab animals, physican's time, etc.).
lAhe value of computer-assistc-d instruction has long been raA~ ~i7~
During the past decade, numerous medical centers around the country have designed and usc~d software for medical education. The software in use is primatily designed to run on IBM PC compatible and Macintosh computers.
These ill~LlU~,IiUllal programs also have many advantages over traditional lecture--214450~

oriented learning, e.g., they are interactive and allow for self-paced instruction.
Many of the programs also , ! ' which require the user to make patient ~ decisions. If incorrect decisions are made, text describing why the choice was incorrect is provided.
More recently, medical education programs have been created using personal computer graphics for the instruction of anatomy. This type of program provides increased P~ tjnn~l value because it gives students the u~ to quickly access anatomic images and related textual data. Rather than spending hours in the gross anatomy lab dissecting cadavers (and being exposed to toxic chemical i~ ~), these programs allow students to work in the library or at home--and spend as much time as needed to learn the material. The programs also provide textbook references -sc, students can ~' '~, read about the clinically relevant aspects.
Although these computer-assisted programs offer many advantages in medical education, there are some inherent limitations and drawbacks. For example, these programs are confined to two ~" (2-D) imaging. This limits the scope of depicting anatomical ~ I ' 'y with respect to showing the relation of superficial and deeper structures. These programs are similarly limited by lacking the ability to rotate in space so that anterior andposterior ' 1 can oe seen. Additionally, while highly valuable for anatomic inst~uction, these programs do not allow for practice of actual medicalprocedures under realistic conditions.
A system capable of providing a realistic real-time simulation of surgical procedures would help reduce the operative risk associated with new i ' ~, in medicine, allowing better j r of ski11s from training to the operating room. A i ' demand exists for enhancing the way that physicians learn new invasive rroc~l~c Appropriate education in new medical and surgical procedures is often outpaced by the desire of physicians to , a procedure into their practice.
As one example, until a few years ago, the treatment for coronary artery disease was open heart surgery, an expensive and very difficult surgical procedure.

21~45~

As an alternative to open heart surgery, ~. i ' coronary ~ r ~r ~PTCA) is now generaUy considered safe and effective to relieve the same conditions that heart surgery did just a few years ago. With the recent availability of this new procedure, the training of physiciams presents a ' ' ' problem. Currently, physiciam experience depends on the availability of a weekend course and practicing the procr~dure in an animal lab.
two days rarely provides enough practice time. ~.'' "~" animal anatomy is quite different tham human anatomy. A realistic virtual reality simulation couldoffer the physician the ability to practice hundreds of procr dures in a realistic r_l.v t, and with realistic anatomy, prior to patient contact.
~ ' _ ' surgical ~ have the potential for improving surgical morbidity and mortality. Studies have shown that for a wide lange of diagnostic and therapeutic procedures, doctors doing their first few to r~everal dozen cases are much more likely to make a greater number of errors. This I ' has been referrr, d to as the "learning cune. " Adequate proctoring of learners by ;- ~1 surgeons is , as there are few surgeons l~; - J
enough to proctor their colleagues. It is difficult for physicians, ~;- '~, those in rural areas, to travel to larger medical centers for training- The ~ro~
also places a burden on e~perts who, aiready busy with their clinical ' could become u.~ ' ' with proctoring requests.
Another potential ad~rantage of VR surgical training is in the ~ d~ud~Lu~-of surgical training. Such ' dr~iull would help insure, e.g., that all board-certified physicians are top-notch. Pro~riding training with computer ~ g would allow rnajor medical centers to directly interact with students amd physicians in a rural setting. By so doing, the ~dr~n~rioling and granting of privileges for traditional ~rl~ ' as weU as newer procedures like la~u~u,uh; surgery, could be objectively provided based on ' dr~ skills: t, regardless of where a surgeon is located. The chaUenge has been to create a surgical simulation that is r~ 1 realistic that ~,he motor skills and anatomic . ~ - ,. ,. . acquired in the simulation are readily transferred to ~ ' of the actual procedure.

~ 4 2~5Qs In addition to realistically simulating the involved anatomy, i.e., body parts, a realistic computer-baced surgieal simulator must also provide a realistic surgical 1...- A key in this regard is to provide the involved proxy medieal 'Sc) with a ~istic l~ok and feel. This can be ' ' through the use of computer peripheral deviees that mimie the aetual ~Ss) used by a surgeon to perform an aetual procedure. In this respeet, the eloser the proxy surgical instrument is to the aetual surgieal t, the more realistie the surgical simulation.
One eomputer interfaee device that has been used in surgieal ~
is the Immersion PROBE produced by Immersion ~nTn~tion of Palo Alto, California. The Immersion PROBE eomprises a pen-like stylus supported on a li~' ... ~ ' Iinkage. The-stylus is held between the fingers like a medieal instrument and is wielded d~t~ l. 'y. The Immersion PROBE reports the position and orientation of the stylus to a host computer via a standard serial port interface. The Immersion PROBE linkages provide six degrees of fre~dom.
Sensors provided at the linkage joints convey spatial COUA.-- ' (X~Y~Z) and orientation (roll, pitch, yaw) of the stylus to the host computer.
While the T PROBE is a useful general purpose 3-D
/,.~. interface tool, it does not elo ely resemble in feel or ~ an actual surgieal or medieal _ For example, it's pen-likie stylus and si~
degrees of freedom bear little - to the type of elongated flexible member used in a -~ - procedure. An additional r' ' ~ of the Immersion PROBE is that it has no provision for providing tactile feedback to the user. Tactile feedbaek is very important to e~eate a realistic feel.
Hon U.S. Patent No. 4,907,973 discloses a computer-based surgical simulation system including a peripheral device designed to simulate a surgieal ih.~ t, as well as the involved anatomy, i.e., body parts. In particular, the Hon patent discloses a system for computer simulation of a ~
operation. Therein, a eatheter-like device is negotiated through a moek arterialpath of a physical internal body part modeling device. Sensors on the eatheter traek the progress of the catheter through the mock arterial path. In one 21~05 t, a vessel . ~ simulator within the mock arterial path provides a fLl~ed resistance to the progress of the catheter.
While the Hon system has the potential for providing a more realistic feel of a catheter device as compared with the Immersion PROBE, it lacks flexibility for easily altering the involved anatomy and L ' 'llle simulation shown on the computer monitor appears to correspond to the structure of ehe physical model. Providing different anatomies, patients conditions and procedures would require additional physical models for each.
Another g of the Hon system is its lack of "special effects~ for realistic~lly simulating changes in the virtual C~lv- responsive to operator and Lr r ~or example, in the catheter insertion simulation of Hon, no provision is made for (l) a rea1istic simulation of bleeding, e.g., in the event an arterial wall is punctured, (2) radio-opaque dye infusion to render thearteries visible under a llu~lv~v~ or C3) catheter balloon inflation and consequent removal of an inclusion. Moreover, the Hon system is over-simplified and less than realistic, since no provision is made for tracking and displaying catheter rotation in the virtual ~ T L ' '~ in an actual ` -h~ procedure, rotation of the catheter wire is used, e.g., to direct the catheter in the proper direction.
Sr~l~T~Ry OF 1~1~ INVI~ON
In view of the foregoing, a principal object of the present invention is to provide for the ~ ' of a simulated medical procedure under highly realistic conditions.
It is a further object of the invention to provide a h`igh degree of flexibility, thereby allowing ~ ' of a variety of medical procedures under different patient conditions, imcluding actual patient conditions.
Yet another object of the present invention is to combine with a realistic simulation l'~n~'n~P tutorial aspects such as expert assistance and evaluation.
These and other objects are achieved by a computer based medical procedure simulation system in accvl with the present invention. In one such system, first memory means are provided for storing image data ICL,~ of -6- 214~SOS
a virtual internal body e..VllUIIIII~ . Second memory means are provided for storing image data , ~ a virtual medical instrument extending within the internal body ~ Display means are provided for displaying images ; ' ~ to the image data stored in the first and second memory means. A
first computer peripheral device tracks the movement of a physical member ~ the virtual medical t, and outputs a signal based thereon.
First input means are provided for receiving the signal from the peripheral device.
First calculating means are provided for calculating position data ~ the position of the virtual medical instrument within the virtual internal body ~.v t, based on the signal from the peripheral device. First updating means are provided for updating the image data, ~ the virtual medical instrument extending within the virtual interna~ body ~ t, based on said calculated position data.
An e~cemplary ~ lbC~ of the present invention is a cardiac simulator used to train ~ -g;~l~ to perform specific . ~h I' ;,~;.... procedures such as ~II;iù~l~ly and stent placement. The ~diulo~ l works with the system in the same way that he/she would perform am actual ~h 1~ i.e., by ' 1, a proxy catheter. The location and movement of the catheter is monitored by watching a simulated ~ r view, and the ~ can feel the interaction between the catheter and the ar~ery by the force required to move the catheter.
These and other objects, features and advantages of the present invention will be readily apparent and fully understood from the following detailed ~ ~rjr~ n of preferred ~ hJo' taken in connection with the appended drawings.
nU~F DF~R~Pl'lON OF 1~. DR~AWINGS
Figure I is a schematic view of an overaU medical procedure simulation system in aco~ with the present invention.
Figure 2 is a schematic view illustrating the operating principles of a catheter interface device used in the system shown in Fig. 1.

2l44~as Figures 3-17 are logic ffow diagrams for program used in the simulation system of Fig. 1.
Figures 18-29 are, L~ Vt~ screen displays produced in the simulation system of the present invention.
Figure 30 is a front-left perspective view of the catheter interface device used in the simulation system shown in Fig. 1.
Figure 31 is a rear-right perspective view of the catheter interface device shown in Fig. 30, with the cover removed.
Figure 32 is a front elevational view of the inkrnal ,; l, y of the catheter interface device, with the electrical . omitted for visual clarity.
Figure 33 is a rear elevational view of the internal . y of the - - - catheter interface device, with the electrical omitted for visual clarity.Figure 34 is a right-side elevational view of the internal . ~ y of the catheter interface device, with the electrical ~ omitted for visual clarity.
Figure 35 is a left-side elevational view of the internal y of the catheter interface device, with the electrical, omitted for visual clarity.
Figure 36 is a top-plan view of the internal ~x , y of the catheter interface device.
Figure 37 is a cross-sectional view taken along a line 37-37 in Fig. 36.
DF.TA~.Fl- Dli'.C~RTPllON OF T~F. ~ KFn li.l~OD~
An overall simulation system 1 in accordance with the present invention is illustrated in Fig. 1. The system comprises two primary parts: a ~- li I
interface devico 3 and a computer 5 storing a softwarc program which works in ~ ~ with the device. In the e~emplary l..I.~bL]d' ' described in detail herein, the system simulates a cardiac ~-h ~ - procedure, and interface device 3 is a catheter interface device. Catheter interface device 3 tracks the ' and rotational movement of a proxy catheter 7 provided in the form of a wire or like elongated flexible object. Catheter interface device 3 also provides selective tactile feedback resistance to the motion of the prw~y catheter.
The translation and rotation of the wire are converted into digital signals which are sent over line 9 to the simulation program. The virtual reality simulation is -8- -2 L4~o~
presented on a high resolution computer monitor 10. The feedback resistance is controlled by digital signals 11 sent by the program back to the catheter interface device over line 11.
In addition to the catheter interface device, the system comprises a switching ' e.g., an ON-OFF foot switch 13. When activated in one mode, the switch triggers the software to simulate a release of radio-opaque contrast material. This aUows for ~ ' of the coronary arteries, including the area in the artery that is obstructed, as in an actual procedure. In a second mode c ' when the virtual catheter enters an occluded portion of the virtual artery, the switch serves to simulate a syringe operated to selectively inflate and deflate a virtual baUoon catheter. Randomly generated or user selected patient data will determine how many successive inflations ana deflations of the virtual baUoon catheter are required to i~u~i~rully remove the occlusion.In place of a simple footswitch providing a dual function, separate devices (e.g., prwcy syringes) may be employed to actuate dye infusion and balloon inflation.
Other devices that may be ~ ' into the various types of . ~-~ h .
procedures may be employed as well.
For enhanced effect, the scre~n ' can be presented in 3-D. This is ~ - ' through the use of a " . system, e.g., CrystalEyes available from Stereo Graphics, Inc. of San Rafael, California. This product uses eyeglasses 15 having active liquid crystal shutters which open and close sixty times per second. The shutters are ~J ~ ' ' to monitor 10 via an infrared emitter 17 mounted on the front of the monitor, and a receiver 19 mounted on the glasses.
Although not used in an actual: ~- r- ~ procedure, 3-D imagery provides a more r' ' of the cardiac anatomy for these types of ' A surgeon's ability to see organs and tissues in 3-D, irl ld~uv~pic , and similar r' ~I procedures, will give physicians an enhanced ability to view and manipulate those organs and tissues.
Digit~l, compact-disc quality ' ' audio can be used to provide an u~ ~h~nal narration in the inventive system, as well as to provide soothing IJ~C~ ' music as is often provided during actual surgery. For this purpose, ~ 9 ~1~450~
the iUustrated system includes a CD-ROM drive 21, an audio mixer 23 (for mixing ' music from the CD-ROM drive and a narrative "voice-over" stored on the computer harddrive), and a pair of amplified speakers 25.
The operating principles of the catheter interface device tracking are ' "~, illustrated in Fig. 2. To obtain translation tracking, a friction wheel 27 presses against the catheter wire 7. This causes the wheel to rotate when the wire is pushed or pulled.
An optical encoder 29 is attached to the friction wheel for measuring the rotation. Rotation is obtained by passing the catheter wire through a semi-tight grommet 31. When the wire is rotated, a wheel 33 attached to grommet 31 rotates. An optical sensor 35 mounted adjacent to the wheel measures the rotation of wheel 33 and outputs a . " g signal. At the same time, grommet 31 aUows the catheter wire to slide i ' 11y Ih~ ,h.
A feedback resistance force is achieved by having wire 7 slide through a wire pressing 37 actuated by a solenoid 39. When the solenoid is activated, it pushes a plunger 41 against one side of a sleeve 43, pressing the catheter wire 7 and thereby making it more difficult to move the same.
An exemplary catheter based simulation is the simulation of PTCA. As mentioned above, PTCA is a technique for relieving partial blockage of arteries i~ the heart. In the actual procedure, a wire-like guiding catheter is threaded through an artery in the leg, up to the blocked area in the heart. The surgeon can see where the wire is going by injecting radio-opaque contrast material into theartery, thereby rendering the artery visible on a n~ (X-ray) display. The physician directs the catheter's placement by twisting and pushing as it is guided through the artery. Once the end of the catheter is in the proper position, the surgeon c~m thread a baUoon catheter onto the guiding catheter. Once the ballooncatheter is properly placed, the surgeon can inflate the balloon to open the occluded portion of the artery so that regular blood flow is restored. ~Iore recently, a self-guiding baUoon catheter have been used to perform the procedure, thus . " ~ the IJ-, ' y step of placing a separate guiding catheter.

O- 21~4~os The exemplary simulation system shown and described in detail herein proceeds very much like an actual PTCA procedure, p~i 7.~ one using a self-guiding balloon catheter. ~ " a number of simulation ' are provided, as described in detail herein below. It will be 3~ ' that the principles of the invention could be applied in order to create a double catheter insertion simulating more closely the more W.l ~, ' technique of first inseTtinga guiding catheter and then inserting a separate balloon catheter thereover.
The software monitors the translation and rotation of catheter wire, as the ~-liol~g;i,l , it. Using this r ", the program tracks the progress of the catheter through a simulated arterial tree forming part of a 3-,l:. ", l internal body ell~ The simulation program uses geometry -data which can be created for each specific procedure (and virtual patient). The location and orientation of catheter 7 is converted into an image by the program, which image is displayed on computer monitor 10. The image has the same n, ~v,~ appearance as a ~-l;vl~ ,l sees during an actual procedure (but with the optional of 3-D imaging) (see the .r~ ;v~ screen displays of Figs. æ-263. In addition, other views which are not possible during a procedure can be provided by the simulation program to enhance the value of the simulation (see Figs. 27-29). Depending on the position and orienhtion of the virtual catheter within the virtual arterial tree, the simulation prograun may calculate and output to catheter interface device 3 a resishnce force signal so that the i ~ can e~cperience the fe~i of the catheter insertion, just as helshe would during the actual procedure. In particular, in the preferred ...bc " t, a hctile fe~dback force is generated when the virtual catheter entersan occluded portion of the virtual artery.
The system can add additional realism by displaying at the outset patient history and laboratory data screens (see Figs. 18-20). The system can also Vl~V~ variations in complications cvll- r " l" to different patient conditions, the ~.. of which can be randomly generated by the computer, or selected by the user.

~` -11- 21~k~
In order to provide additional realism, the system can simulate procedural error conditions. For example, if the catheter wire is moved too abruptly, this triggers the simulation of a puncture in the arterial wall, and associated bleeding (see Fig. 25). If the virtual balloon catheter is inflated to an e~cessive amount, a realistic simulation of balloon rupture occurs.
As seen in Figs. 21-29, a simulated el~L-, v (EKG) waveform can also be displayed on the monitor 10. Provision is made for changing the waveform from "abnormal" to "normal~ during the course of the simulation.
Provision can also be made for the random occurrence of ' ' such as atrial ~' Additional realism is provided by simulating pulsation of the arterial tree (and tne catheter therein) in ~ ' with the EKG.
- Additional abilities that can be provided by the simulation system include, among others, interactive control of the point of view, recording of the procedure, and monitoring by an expert assistant. Changing the point of view is important.
For example, viewing the arterial tree from different p~L~ allows the K;~I or student to better learn the 3~ ;h ~ Of the arterial branches. This is difficult to master in practice since only a 2~
projection of the 3~ ' ' geometry is available in the n - -- ....~ "~. view usedduring an actual ~ procedure. The progress of the ~ " ', can be re~orded and played back for review and evaluation. A virtual "expert assistant"can be included to monitor the actions of the ~ 1 during the simulation and can provide r ~ such as warnings and c ~
Many different catheter-based procedures can be simulated with the inventive system. The specific pro~edure is created by the data used in the interaction aspects of the simulation program. Thus, the same catheter interfacedevice can be used with many different procedures while the simulation program is ~ I for each procedure. This enables the ' simulation system to be used to create training h.~ ' for many different clinical procedures and patient conditions. The patient conditions can be simulated or actual. With respect to actual conditions, those conditions could be taken fwm - 12- 21~05 lC~rl vc past patients for use in general training, or could be taken from a current patient for use in a pre-operative surgical rehearsal.
The logic flow of the exemplary PTCA simulation program (and ' '-VC screen displays thereof) are now described in detail, with reference to Figs. 3-29.
Figure 3 depicts thrce primary modules of the main program: elror checking 45, case l 47 amd calibration 49. In error checking module 45, features and function of the system are diagnosed for presence and ~ ,. As previously mentioned, a ~, ' 3-D viewing system can be utilizcd by the system to enhance the simulation. To determine if tne trar,smitter is prcsent, a call to the devir~e is made at step 49. If the transmitter is prcsent, the video is adjusted for three /' ' perception at step 51. If the transmitter is not present, the ~ A ' ' - viewing function is disabled at step 53 and the program pror eeds without the visual ' The system may ~ r utilize other types of visual perception ~ ' devices, such as ~--' ' displays, and projection devices (including ~ .r projection devices).
As previously mentioned, the system utilizcs a catheter interface device 3 that provides for direct human interaction with the computcr ' The presence and ~ ' ~, of catheter interface device 7 is ~' ' at step 55.
If the device is not detected or if it fails a diagnostic routine, the error is reporhed via the display and the simulator program will exit to the host operating system(step 57).
Given that ~ h.; l;r.~ll interface device 3 passes the previous error cher~king routines, the program next proceeds to the detection of footswitch 13 used for dye infusion and balloon inflation (step 59). If the system fails to detect the footswitch, an error is visually rcporh~d and the program exits to the host operating system (step 61).
To, ,..~hl~ the program polling rate, simulation timing functions, and threshold arrays, the program l ' a routine to assess the speed of the host computer system (step 63). All ,~ ll.ull;~ull variables are ûutput to global dah -13- 214~iD5 uffers to be utilized by time dependent ' U~L;A._D, array ~' threshold and polling count ~ uuLl~ within the simulatiûn program. Ne~t, the program proceeds t~ a case ~ - module 47.
In case l module 47, a patient case and its relative data are retrieved (step 6S) via a random selection subroutine 67, from an on-line database 69. This stage may be replaced or IL~ ' ~ by a routine allowing the user to select a specific patient case from the case files, or to input data from a selected patient case not in the stored files. The latter would be ~; '~, useful fûr conducting a ~lW~ldLiv~ rehearsal. The case specific data which is stûred in memory in the preferrwd ...1,~' can include EKG datd (normal and abnûrmal), occlusion location data and dl~giO,ul~ y success datd, i.e., the requirwd number of successive balloon inflation cycles required to remove the occlusion.
(The ~ .y success data is artificial since in an actual procedure the number ûf baDoon inflations required to remûve an occlusion typically is not known in advance.) The case ~' variables are set in step 71.
Patient r '- is delivered via a l at step 12.
This may include a patient history screen as shown in Fig. 18, an ~uL~iugld~l~ as shown in Fig. 19, and an indication of h~AA~lJ as seen in ~ig. 20. ~
addition to displaying patient ;..r....~ n the screen, c4..~i.d;.~, or r '- is conveyed by a digitally recorded voice providwd through the ...~i~. audio system. Tf footswitch 13 is activated at step 75, the wiD proceed ("fast forward~) Lo the next topic (step 77). So long as the program poDing does not detect an activated footswitch, the program wiD
proceed in a linear fashion through each topic, until the last topic is reached (step 79).
~ efore the simulation actually begins, the catheter interface device is calibrated within calibration module 49 (step ~l). To calibrate catheter interface device 3, the catheter wire 7 is ~ U..;~ with the program by the user. The user is instructed by the program to pull the guide wire out while a status bar on the screen display of monitor 10 decreases to indicate the approach of the catheter starting point. This calibration is required for the program to ascertain the point -14- 21445~
of origin for screen 1~, and translation tracking. Rotation degree is also set to a starting position at this time.
Referring now to Fig. 4, upon completion of the calibration routine, the program will proceed to a catheter placement simulation module 83. At this stage, the screen displays a simulated n~ ray) view (see Fig. 22), wherein the patient's rib-cage 84 is clearly seen. The arterial tree remains invisible until a subsequent dye infusion routine is e~ecuted. Once the dye infusion routine is executed, the arterial tree 86 including an occluded region 88, and the virtual catheter 90, become clearly visible (see Figs. 23-29). The 1,~ ' image can be created by digituing an actual r~ image and, if desired for clarity, by modifying the digitued image data to remove unwanted structures and "noise~.
In the present ' " t, this 1,~ ' remains inanimate throughout the simulation (as it generally would during an actual procedure). As also seen in Figs. 21-29, the screen presents along its boltom border a moving EKG waveform 92, based on the stored patient data.
In catbeter placement simulation module 83, a timer subroutine 85 simulates real time by counting units that are ~ - ' by the polling rate time adjustment step 63. Referring to Fig. 5 showing the timer ' the timer unit is defined as an integer with a value of 1. If the current sum timer value is greater than 1 at step 85a, then the value is increased by 1 unit at step 85b. The value is set to 1 upon initiation of the timer, at step 85c.
Referring back to Fig. 4, tne program proceeds to an EKG data ... subroutine 87. Referring to Fig. 6, if an occlusion (blockage) is indicated in the case specific arterial dataset (blockage=1) at step 87a then the program will check to see that the abnormal EKG data is stored in tbe program memory buffer (step 87b). If it is not yet stored, the case ~ data is retrieved at step 87c from the EKG database 87d. The EKG data base 87d can provide sinus rhythm patterns as well as atrial fibrillation data, and other case ~' cardiac events. The buffer space is given a local definition, i.e.
CASE ABNORMAL at step 87e, and the EKG data is placed in a com_n buffer - 15 - 21~0S
(step 87f). Alternately if the correct buffer nalne is found at step 87b, then tbe program wiU proceed out of the subroutine.
Conversely, if it is deterrnined that a blockage does not exist at step 87a, then tbe program wiU check to see that the EKG data is set to CASE_NORMAL
at step 87g. If it is, the program will proceed out of the ! ' ~ '- Otherwise, the CASE NORMAL data is retrieved at step 87h from the EKG database 87d.
The buffer name is set to CASE_NORMAL at step 87i, and the data in the EKG
data buffer is u.. ~ at step 87f. The program provides the onscreen EKG
simulation based on this data in the EKG Buffer, which is derived from the stored case data.
Referling '~ back to Fig. 4, the onscreen EKG display is produced and updated within subroutine 89. Referling to Fig. 7, the update EKG
subroutine will check to see that the EKG data is translated into a usable array at step 89a. If an array is not detected, the EKG data is retrieved at step 89b from the EKG data buffer 89c and sorted into an array. The array is processed at step89d by translating the value from the EKG buffer array 89c, to a floating-point number between 0 and 1. A value equal to 75% to 80% of the highest value in the translated array is set as an audible beep threshold value at step 89e. The adjusted array is next defined as the spline ;- r - array at step 89f. If it is 1~ ' ' J at step 89a that the spline d~ F ~ array has already been set, the subroutine wiU bypass the array definition process.
Each time the program polls update EKG screen subroutine 89, the spline ~ r ' array is advanced one place holder (step 89g). A range from the array is then used to deform the EKG screen spline at step 89h. The real patientdata may be plotted and the EKG screen decay can be simulated. If it is determined at step 89i that the apex of the spline is at or above the deflned beep threshold value, then an audio beep is generated by the sound system 89j, at step 89k.
The simulation can include, as a simulated, ' randomly generated periods of EKG il~ ily, e.g., atrial fihrill This is illustrated in the screen display of Fig. 21, wherein the surgeon is instructed to stand-by until normal sinus rhythm resumes. At this point in an actual procedure, a drug would be ' to the patient. Such a step could be included in the present simulation, e.g., using footswitch-13 as a trigger ~ ly, the simulation can be simplified by simply notifying the surgeon that the drug has been . Another feature that can be provided is a graphic time clock 94 (as shown in Fig. 21) notifying the surgeon of the time remaining before the drug takes effect (and sinus rhythm is resumed).
Referring again to the main program of Fig. 4, at subroutine 91 an arterial pulse is realistically simulated by s~ u..~..~; arterial and catheter movement with the EKG pulse data. Referring to Fig. 8, subroutine 91 will retrieve a spline .1. f..~ .\ array at step 91a, from a spline fi f ~ buffer 91b. The spline l- f -~ ';---- array is the data used to represent the virtual arterial tree that will appear upon dye infusion. The . 5~ of the arterial tree can be adapted from . 'ly available 3-D graphics packages (such as from ~lewPoint Data Labs of Orem, Utah) or can be ' '~ , generated based upon physical models or actual anatomies. Offset data points are . ~ ' ' from the array data at step 91c. The geometry is offset (redrawn) and the screen is updated at step 91d. The program then proceeds to step 93 of the main program (Fig. 4), where the status of footswitch 13 is checked. At this stage, foots~vitch 13 is used to initiate the infusion of contrasting dye (subroutine 95). The default position of the switch is OFF and the program will simulate rapid dissipation of the dye into the bl~ (as would occur in an actual procedure) when the footswitch is released. At step 93, a check is made to see whether the footswitch has been depressed: If not, dye infusion subroutine 95 is by-passed.
When the footswitch is depressed, the program proceeds to the dye infusion subroutine 95. Referring to Fig. 9, dye infusion is activated at step 95a, whereupon a ramp ~ random flow of values is streamed into an infusion value definition (see 95b) used to determine the shade or c-. ~ of the dye.
By altedng the value of the dye - at vadous points, the shade vadations simulate the mixing of blood and dye. By ramping the value stream, simulated dye that is infused behind dye infused earlier will appear less diffused, - 17- 214~50S
thereby looking to be of higher . By ' the sbade ~ value stream as it is ramped upwardly, it is possible to create a stril~ngly realistic simulation of one fluid (in this case dye) diffusing into another fluid (in this case blood). The constraints on the ramped random value stream 95b c;m be adjusted to provide the ~lr of different fluid viscosities and rates of diffusion. ~ ly, a par~icle system or procedural material map may be used.
The path of the dye is mapped (at step 95c) in accordance Witll the artery data set 95d. The flow of the dye is mapped IJlU~;I~..;~.I~ from the top of the artery down through the branches. When tbe footswitch is released, the mapped contrasting dye is reverse mapped to simulate diffusion in a downward direction,as if blood were flowing from above and ~washing~ the dye from the site.
If it has been determined at step 95e that a puncture in the arterial wall has been mapped on the arterial geometry, and footswitch 13 is in the ON position, the puncture origin 95f is mapped at step 95g, to simulate bleeding from the puncture into the i~Ul~ . body cavity. Such bleeding is shown at 96 in the screen display of Fig. 25. If no origin is detected, the program will proceed. As w~l~ be described, a puncture caused to be mapped onto the geometry when the user moves the catheter wire 7 of the catheter interface device 3 too abruptly.
The next step in the main program is a catheter movement sampling subroutine 97. Catheter interface device 7 includes an electronic interface having a software driver for ~ tbe signals produced by the catheter wire tracking sensors. As shown in Fig. 10, the data that is provided by the driver is sampledat step 97a, at a rate that has been d ' by the program at step 63.
Tr-~g' ' ' translation 97b and rotation 97c of the wire through the catheter interface device are processed to determine the wire location relative to the calibration point position established in step 81. The ~axis (wll- r ' 6 to the catheter insertion depth) and the rohtion data are placed in an array buffer in step 97d.
The program then proceeds to subroutine 99 for ~ " the location and orientation of the virtual catheter. As shown in Fig. 11, this is computed at -18- 2~4~5 step 99a from the Za~is and rotation value array stored in array buffer 97d, a geometry offset value 99b, and the constraints provided by the artery data set 99b.
By taking account of a geometry offset value (which is a product of the arterialgeometry data and the EKG data), a simulated flutter or vibration of the catheter in ~ with the arterial pulse is achieved. The catheter location and orientation data are placed in a data buffer at step 99d for use in a subsequentcatheter update subroutine.
With the arterial tree il ' by simulated ~ u~u~,ic dye, the catheter is navigated through the arteries toward the occlusion region. Figs. 23-26 show the ~IUI, ' of virtual catheter 90 through arterial tree 86 and into occluded region 88. As can be seen in these screen displays, virtual catheter 90has an angled or bent end portion 98. This allows for steering the catheter downdifferent pathways, e.g., branches in the virhlal arterial tree, based upon the rohtiorial orienhtion of the catheter wire.
To determine whether the catheter has entered occluded region 88, within subroutine 101 it is determined if the spline origin (IC~ the catheter end) is within the blockage range stored as part of the case specific dah. Referring to Fig. 12, the blockage range data lOla, and catheter location definition lOlb arecompared at step lOlc. A local value of 1 is output at step lOld if the .
determines that the catheter location is within the blockage range. 1~ ~ly, if the . indicates that the catheter is not within the occlusion region, a value of 0 is passed at step lOle.
If subroutine 101 returns a value of 1, then the program will output a signal to the catheter interface device to impart force feedback (a.k.a. haptic feedback, resistive feedback) to the catheter wire (at step 103). The force feedback signal can be provided at a single preset level, or could be provided in gradations dependent on the location of the catheter and the cross-sectional ~" of the occluded region. Then, the program proceeds to a balloon inflation simulation module 105. If subroutine 101 returns a value of 0, the program will proceed to subroutine 107 for checking whether the catheter has punctured the arterial wall.

19- 214~5~5 Puncture simulation in the illustrated program is based on a detected velocity of catheter wire 7 within the catheter interface device 3. This provides a realistic simulation in that during an actual procedure an abrupt movement of the catheter can often result in an arterial puncture. Likewise, in the ' if the catheter wire is moved abruptly by the user, the velocity of the catheter exceeds a preset level and thereby triggers the simulation of a puncture.
Referring to Fig. 13, in puncture detecting subroutine 107, the prograrn will first check at step 107a for the presence of a previously defined puncture origin 107b. If a puncture origin 107b does exist, then the program checks at step 107c whether an audio response (e.g., verbal of the error) through the ' sound system 107d is completed. During the audio response, the program continues to loop, resulting in the simulation of bleeding from the puncture origin (see 96 in Fig. 25), due to execution of dye infusion subroutine95. At this time, a flashing screen message may also appear to alert the uær of the procedural mistake. Following completion of the audio response, the simulation proceeds to i step 107e, whereupon the screen display can be fro~en to preserve the ~ r result for evaluation.
If a puncture origin is not detected at step 107a, the preænce of a previously logged velocity variable is determined at step 107f. If none is detected, the current time 107g is placed in a local variable definition (buffer A) at step 107h amd the program proceeds. If a previously recorded value is detected im step 107f, then that variable is moved to a second buffer (buffer B), in step 107i. The current time 107g i5 placed in the primary buffer (A) at step 107j. The time between events is then calculated at step 107k. If the calculated elapæd time is~IP~Prm~ to be bdow a preset value at step 107e, this indicates a velocity in excess of the velocity constraint. Under this condition, the pumcture origin 107b is set to the current catheter location (step 107m). The program will then activate the audio responæ to the puncture event (step 107n), and program control retums to the main program. If the time between events calculated in step 107k is found~m step 107e) to be within the defined . the program will proceeds to the next stage without a puncture origin having been defined.

~ -20- 214~
An altemative method of detecting a puncture would be to utilize a collision detecting , t, whereby a puncture would be simulated upon an impact of the virtual catheter tip against a wall of the virb~al artery.
In the next main program step ( ' 111), the catheter position and orientation are updated. '~he catheter is l~J.. ' by a spline generated from data points. Referring to Fig. 15, to determine the length of the spline, the starting point (i.e. catheter origin llla), and the current location of the catheter lllb, are retrieved and processed in step lllc. In step Illd, the spline length definition computed in step 11 lc is combined with geometry offset data 11 le. The curvature of the spline (i.e., spline ~ f~ ) iS derived from the geometry offset data 11 ld, which is a product of the artery geometry and the EKG pulse If a puncture origin is present (11 lf), the path of the spline will not be . ' by the geometry map at that point, thus allowing the catheter to pass through the puncture, e.g., as seen at 96 in the screen display of Fig. 25.The rotation of the catheter wire is reflected by the position of the bent or angled wire end portion of the virtual catheter. The catheter is pointed or navigated on the path chosen by the user by rotating the wire (and bent end portion thereof). If it is ' ' in step lllg that the end of the catheter is located at a branch in the arterial data set 11 lh (a~ seen at 100 in the screen display of Fig.
24), the program will determine the direction of travel or trajectory at step Illi based upon retrieved rotation data lllj. Finally, the spline is deformed and redrawn in step 11 Ik, and program control proceeds.
Referring bach to Fig. 4, when the catheter is determined to be within the occlusion range at step 101, and after rul~ '~lI ~h is activated at step 103, the program proceeds to balloon inflation module 105. Therein, the program will first magnify the occlusion region and change the viewing ~live (step 113), as seen in the screen displays of Figs. 27-29, in order to enhance ~ .. of the arterial structure and the effect of the procedure thereon. Contrasting dye is l1y activated, the visual - r ' are adjusted to the new ~.~ iv~, and the polling cycle is redirected to loop within the balloon inflation module.The required program: . are replicated within this stage, i.e., catheter ~ -21- 21~S05 update 111, timer 85, EKG data 87, arterial pulse update 91, and footswitch status check 93. Within the balloon inflation module, footswitch 13 becomes operative to activate balloon inflation rather than dye infusion. If step 93 determines that the footswitch is activated, the program proceeds to a ballovn inflation subroutine 115.
Referring to Fig. 14 showing balloon inflation subroutine llS, if the footswitch is depressed so that a value of 1 is detected at step llSa, then the prvgram proceeds to step 1 lSb wherein it is l~f ' ' whether an inflation statusbuffvr 1 ISc has been previously; ' ' ' If the buffer has been; ' ' ' i, the value within the buffer is increased by one unit, at step 115d. The value frvm the inflation status buffer is then used to deform (inflate) the balloon tip 102 of the virtual catheter (step 115e) and then tv; v~ deform the occlusion gevmetry outwardly (step llSf). These effects are seen clearly in the screen display of Fig. 2v.
If it is determined at step 1 lSb that an inflation status buffer has not yet been previously . ' ' ' l, then one is created at step I ISg. The value is set to one unit at step 1 lSh, and the value from inflation status buffer is then used to deform (inflate) the balloon tip of the catheter (step llSe), and then to deform the occlusion geometry outwardly (step llSf).
If the footswitch value is not equal to 1 at the time step I lSa is polled, thenthe program wiU determine at step I ISi if an inflation status buffer 1 lSj has been previously . ' ' ' If it has not, the program proceeds to loop within the mainballoon inflation module lOS. If it has, the buffer is reduced by I unit at stepllSk, thus serving to; lly deflate the baUoon catheter. If the buffer is reduced to a value of 0 (step llSI), effectively deflating the balloon (as seen in Fig. 29), the buffer is released from memory at step I lSm. If the condition exists that the buffer has been reduced by 1 unit (step 1 lSk) but has not been reduced to a value of 0, then the .I~A ' value from the inflation status buffer is then used to deform (in this case deflate) the balloon tip of the catheter, and then to ~Ll~ -r '' ,,~.~1 deform the occlusion geometry, at steps llSe and llSf .

~ -22- 2144~05 On exiting balloon inflation subroutine 115, program control pro~eeds to a balloon rupture subroutine 117. A threshold value for inflation is set upon the case specific arterial dataset retrieval. This threshold should be equal to tne value of units required to fully deform the blocked region of the artery so that it is of healthy ' It is possible also to introduce the threshold at a lower value to provide for a risk ~ ' ' element to the program. Referring to Fig. 16, if the value retrieved from the inflation status buffer 117a is determined to exceed the threshold value at step 117b, then the balloon volume is set to 0 units (step 117c), abruptly denating the balloon (as seen in Fig. 29) and thereby simulating balloon rupture. This action can be buffered over time to deflate at a visually correct rate. Various events such as ' g and vascular damage may be added to this scenario to enhance the program's realism. When a rupture has occurred, the rupture audio response 117d is activated at step 117e, and theprogram then proceeds to a final summation of the simulation step 117f.
The desired outcome (objective) of the simulation is clearance of tne occlusion and restoration of normal sinus rhythm. If balloon rupture is avoided in subroutine 117, pro~ess control proceeds to subroutine 119 for d~
when the occlusion has been successfully cleared. Referring to Fig. 17, tne program will detect whether the balloon has been allowed to inflate ~urr~ ay to cause complete ~' ' of the occlusion range cross-sections to the proper (corrected) diameter. If so, an occlusion value of 0 will be detected at step ll9a.
If the cross-sections have not been fuay corrected, then the program will continue to lo~p.
In am actual procedure, removal of a blockage typically requires repeated balloon inflation cycles. Similarly, in the simulation, repeated inflation cycles are required to ~u~r~flly clear the occlusion. The required number of successive inflation cycles is determined by a success value definition which is part of the retrieved case specific data. For each polling cycle wherein it is ~' ' that the blockage has been fuay deformed by the baaoon to the corrected cross-section, a count value is ~ t~i at step 119c and a ~ is made at step Il9d to see if the success value definition has been reached. If so, the program ,3 21 ~ 4 S Q S
proceeds to a successful concluding summation at step 119e. The concluding summation may, e.g., include an of the successful result, and provide a display of the elapsed time of the procedure. If the success value hasnot been reached, the program continues through the polling loop. T ' ''~"
because of the inflation time limitation imposed by the balloon rupture subroutine 117, a user cannot achieve a successful clearance of the occlusion by simply g tbe footswitch in the depressed (ON) position. Rather, it is necessary for the user to su~,~ ly in~late the balloon and allow it to deflate (via the ~ ~ function of step ll5k) at least several times, as in an actual procedure.
In the preferred ' " t, the software is ~l~" ' using the industry standard language C+ + and OpenGL. The use of these t~o languages has several ad~ -ability to port the software to different hardware platforms.OpenGL has been licensed by many different workshtion including Windows NT, ATT, DEC, and DOS.
OpenGL is an open, ~' ' standard for describing 3-,l;", l with objects.
ability of software to adapt to upcoming hardware OpenGL handles most graphics routines, and its use is scaled to the parlicular hardware that is available on the computer.
, ' li~y with Nintendo/Silicon Graphics, Inc. hardware as well as set-top boxes intended for interactive television. The availability of ~ platforms for VR will open u~.; for use in third world countlies, rural settings amd individual homes and offices.
In order to maintain real-time frame rates (30 fps), it is important to limit picture detail to the essential portions of the screen. The limited resource is the graphics rendering speed (the number of polygons per second), and the software must know where to best allocate this resource to determine what features being displayed should have the most detail. In the exemplary cardiac . ,ll. l. .;,-l,.."

2~4~505 simulation described herein, the real time graphics are limited to the arterial tree and the catheter therein; the remaining 1.~.. ' of the simulated ' .,~,;c view is inanimate (as it would be in an actual procedure). With this t, real time rates are readily attainable.
Specific atbention will now be given to the preferred system hardware for the d~ . _~sc~ d cardiac . ' simulation system. There are a number of different hardware platforms that can be utilized for VR ~ Current include:
~ i~h ~nrl systems. VR can be run on systems such as the SiLicon Graphics, Inc. RealityEngine. These are costly and logistically the most difficult to t~ansport due to their large size. The Silicon Graphics Indigo 2 Extreme ~Jlh~ ti~ has the graphics and computer power necessary for many VR .~ and is less costly.
p~ systems. Alt~.ough the current generation of personal computers could be used for some limibed VR ~ , they lack the d hardware found in graphics w- which facilitates the production of real-time complex 3-D ~ ' of objects. Personal compubers are optimized for 2-D graphics, and as a result, they are ideal for playback of digitized linear sequences which can be obtained from 3-D graphics sysbems (VR).Personal computers are also well suited for the display of 2-D ~ of 3-Dl' The a~ y application of the present invention was developed on a Silicon Graphics Indigo 2 Extreme ~.. A complete hardware ; - for this application is set forth below:
Silicon Graphics Indigo 2 Extreme graphic hardware ~IIPS
R4000/100 Mhz, o4MB RAM, IGB hard drive, 19" SGI monitor, CD ROM drive, DAT tape drive.
W .~ equipment (e.g., CrystalEyes eyeglasses and emitter + cable).
audio equipment (one pair of sterwo power speakers, Mackie 1202 (or compatible) mixer or another pair of power speakers.

-25- 21~4511S
audio CD for prvviding continuous l,..~ . ' music (narrative "voice-over" can be recorded onto hard drive).
" (two stereo mini-male to RCA male cables, six RCA
female to l/4" male phone jack, power strip).
catheter interface device (as described in detail ~ b~lv..).
foot pedal switch peripheral, e.g., as is available from Immersion ~mlmrpt j~n In the preferred ' - ' t, the ~ h ~ - simulation system of the present invention employs a virtual catheter interface produced for use in applicants' system by Immersion t'~ til~n This device is now described in detail with references to Figs. 30-37.
Catheter interface device 3 tracks the movement of wire 7 in a 3-D space, with the movement of the wire ~ ~ to two degrees of freedom. This provides a rcalistic simulation of an actual catheter insertion pro~edure, wherein once the catheter is inserted into a patient, it is limited to two degre~s of fre~dom.
More L - ' ~.y, within a patient's body, the catheter wire is ~ ' to translation (in-out motion) and rotation.
The catneter wire may be formed of a variety of known materials, e.g., metal or nylon. Generally, the gauge should not be bigger than 050 (0.050"
diameter), and the insertion length should be at least 14~. The catheter wire can be an actual catheter wire, such as is ~ lly available, e.g., from Cordis Corporation of Miami Lakes, Florida. An actual catheter should be modified such that any tool at the end (such as any cutting edges or baUoon) is removed leaving only the handle amd the shaft.
Catheter wire 7 csm include an end mounted handle or "grip" portion 121.
The "grip" portion can be any w..~. ' device used to manipulate a catheter.
~1 .,.1~, the catheter shaft could be gripped directly. While the particular described herein comprises a wire, it should be understood that other catheters and catheter-like mcmbers could be used as well, including L~i~
elongated fle~ible tubular catheters.

~ 26 2~505 An electronic interface 123 serves to electrically couple the force feedback actuator and motion sensors contained in catheter interface device 7 to the computer 5. An electronic interface that is ~.ui ' '~, adapted for the present invention is ~ "y available from Immersion ~ n This is basically the same electronic interface designed for use with Immersion's PROBE. Unlike the PROBE setup, a serial interface is configured to be bi~'- l, so as to allow for ~ of a tactile feedback signal from the computer to the force feedback actuator. The electronic interface has six channels Wll~ to the six degrees of freedom of the Immersion PROBE. However, with the present invention, the electronic interface requires use of only two channels for the tracking of catheter wire 7, since the catheter wire is c ...~ -FA to two degrees of freedom. A third channel is used for 1,, - ~- - .;; of the tactile feedback signal.
The electronic interface includes a software driver. The update speed of the device is between 500Hz and 2kHz--this speed cannot be exceeded with a single serial port. If necessary to achieve a cleaner "feel," the device can be connected with two serial ports running ' '~" whereby a 2X speed ih~ u.~ llL is obtained by doubling the data h capability. Other methods of improving interface speeds between the cahheter interface device and h'le computer include standard and high specd ethernet use of SCSI
or direct memory access parts (DMA), or a variety of more rapid standard interfaces. In addihon, dedicated hardware to facilitate ~ , can be used, such as boards for rapid data , The operahve of catheter interface device 3 are mounted on top of the housing of elechronic interface 123 within a separate metal housing 125 having a removable cover 127. A rubber-like tubular member 129 is used to assure free passage of wire 7 hhrough the housing cover 127. Housing 125 could instead be modeled after the external bodypart(s) located within the vicinity of the cahheter insertion location, e.g., a leg or arm, in order to render the simulation more j n to the user.
The operahve of catheter interface device 3 comprise a block assembly 131 forming a through-passage 133 (see Flg. 37) and mounting locations - 27 - ~ Q ~
for a first input transducer (sensor) 135 that senses the i ' ' movement of wire 7 and produces a Wll~ elect,ical signal for input to computer 5, an output transducer (actuator) 137 that imparts a resistance force to wire 7 responsive to a tactile feedback signal re~eived from computer 5, and a second ænsor -- a rotational transducer -- 139 for t,-acking the rotation of wire 7.
Block assembly 131 comprises a plurality of blocks formed of aluminum or other ~ ' material, such as plastic. The blocks may be cast, molded, and/or mac'nined as a monoblock members, with the r " ~ sens~M and actuator attached thereto and/or illCul~ ' therein.
A variety of; ' readily available in the . ;~1 market are suitable for use in the present invention. For example, the input;
(sensors) can include encoded wheel i , , optical encoders, etc. The output transducer (actuator) could be a stepper motor, servo motor, magnetic particle brake, friction bra,ie, pneumatic actuator, etc. Hybrid or bi-directional i ' that pair input and output i ' together could also be used in place of separate sensor and actuator devices. A simple bi-directional trmsducer that could be used is a permanent magnet electric motor/generator.
It should a,'so be noted that the present invention can uti,ize both absolute and relative sensors. An absolute sensor is one in which the angle of the sensoris known in absolute terms, such as with an analog, Relative sensors only provide relative angle r '-, and thus require an initial calibration step, as is provided in the present simulation system.
In the preferred . '-' t, actuator 137 is a solenoid capable of producing a force output of at least S ounces for imparting tactile feedback in the form of passive resistance (frictional drag) on wire 7. As best seen in Figs. 35and 37, a solenoid plunger 141 has attached to its end a lower pressing plate 143.
Lower pressing plate 143 engages wire 7 against an upper pressing plate 145 by applying a force in a direction substantia,ly ,~"~"li. ul~ to t'ne translation direction of wire 7. This produces a frictiona,' force a,ong the translation direction of wire 7. Such solenoids, controllable to a thousand or more selectable resistance force values, are available commercia,ly from, e.g., Guardian Electric of ~ -28- 21~;0~
Wocdstock, Illinois. It will be a~ that other actuator devices may be employed in the present invention. For e~ample, the actuator could be a pneumatic or hydraulic device which applies a force to the wire. It will be .~",. ' by those skilled in the art that the choice of ~
d~L.~ Lic, pneumatic or hydraulic actuation will depend largely on the required response time, and the cost and co A1 ~, of the device.
EI~LI., .... ~ are generally preferred as they typiQlly have a fast response time, are low cost, and are simpler and smaller than hydraulic and pneumatic devices.
One or both of upper and lower press plates 143, 145 Qn comprise a resilient friction (brake) pad 147 (shown attached to upper press plate 145), for providing a smooth braking action to wire 7. Materials ~ , for the resilient brake pad will be apparent to those of ordinary skill in the art.
As best seen in Figs. 36 and 37, the translation sensor , a friction wheel 149, which wheel is mounted on a rotatable shaft 151. The friction wheel engages wire 7 with a normal force such that translation of wire 7 Quses rotation of shaft 151. As best seen in Fig. 35, shaft 151 is coupled to a disk 153 made of clear plastic material and having a number of dark radial bands 155 formed on its ~, ' ~ ll~. As best seen in Fig. 36, a ~ .. pair 157a,b including a light source and a detector is positioned on opposite sides of disk 153 in alignment with bands 155. As disk 153 rotates about its central axis, bands 155 , cause light to impinge and not impinge upon the detector, thereby creating an electronic signal which is passed to electronic interface device 123 over a bus wire 157. In the illustrated preferred ~ ~ " of catheter interface device 3, i ' movement of the v~ire is tracked with a resolution of at least 0.002". One suitable type of sensor is am optical enccder wheel type input transducer (Model Sl) marketed by U.S. Digital of Vancouver, Ur"~h;l.l;Lon. The same basic type of sensor can be used for rotation sensor 139.
In the preferred ' ' t, the rotation sensor 139 includes a disk 159 rotatably coupled with wire 7. Disk 159 is preferably made from a clear plastic material and has a number of dark radial bands 160 forrned on its d ~

2~44~

A l ' ' pair 161 including a light source and a det~ctor is positioned on opposite sides of disk 159 in alignment with bands 160. As disk 159 rotates about its central axis, the bands ' ~, aUow light emanating from the light source to impinge and not impinge upon the detector, thus producing an electronic signal which is also pass~d to electronic interface device 123 over bus wire 157. A
second 1 ' ' could be provided so as to aUow ~' of the rotation direction, as is weU known to those skilled in the art.
The; l" ~ between wire 7 and the central aperture of disk 159 is preferably ~,., ,,.,,~1' h ~I by the formation of a friction seal. This can be provided by a resilient (e.g., rubber or plastic) grommet 163 mounted within the hub of disk 159. Grommet 163 receives wire 7 with a semi-tight fit causing disk 159 to rotate coaxially with the wire, while at the same time allowing relatively free wire translation. Preferably, the mass of the rotating transdu~er parts is kept very smaU
so that it only takes a smaU amount of friction to ensure coaxial rotation of wire 7 and disk 159 without slippage. Because the level of friction is so small, it does not ' "~, impede ~ motion (i.e., in-out motion) of wire 7.
AILIlld~ ly~ wire 7 could include a lengthwise e~tending rib, slot or flat, which is engaged with a . ' ~ key-way provided on disk 159. While more costly and difficult to ~ ~i, the use of a non-circular wire and ~ key-way would ensure a slip-free connection between the wire and the disk, without , ' ~ "- ,~ l friction.
The principles of the present invention can also be applied to create an .J... (IV) line insertion simulator. In the IV insertion simulator, a hardware device and software work together to reproduce the vistlal and physicalexperien_e of inserting an IV ne~dle into an artery. The simulator would work byhaving the user grasp a proxy needle connected to a position monitoring device.
Preferably, a custom IV hardware interface device would be designed for this purpose. Alt~ ~ld~ .ly~ the Immersion PROBE could be us~d for I ,( g a general simulation (without tactile feedback). The insertion would be carried out by interacting with a simulation of the activity on a computer generated display.
The display would show the needle as the user moves it along the part of the body 2 1 ~ 5 into which the needle is to be inserted. As the needle is pushed into the skin, the simulation software would detect the contact and display an ~, ~ view of the process. For instance, if the needle were properly inserted into an artery, then a small amount of bleeding could be displayed along with a change in the shape of the skin where the needle has passed under. In addition, a force mimicking that ~ by the user during an actual IV insertion could be generated by the IV interface device, im response to commands from the simulation program, similar to the: cardiac ~ ' simulator described in detail herein.
The concepts of the present invention are applicable to provide a realistic simulation of a wide range of other medical ~ ' including ~uLi~_ul~ly invasive surgical pro~edures wherein a device is inserted into an internal bcdy _w~, or directly into body tissues. The procedures which can be simulated using the present inventive techniques span every specialty of medicine, including but not limited to cardiology, invasive radiology, ~ y, neurology, h.`~ ;y and, of course, general surgery. Procedures that can be simulated with the inventive system include, but are not limited to:
placement and/or use of therapeutic ' ' devices, e.g., stent placement and coil placement for aneurism treatment.
i.lt~ (IV) line and central line placement, e.g., jugular, femoral.
u~_tiu" into the body of materials or devices to enhance the ability to discem internal body palts, e.g., ~ ,. . 1~ intra-coronary ultrasound devices, and fiber-optic scopes.
placement of devices to obtain tissue and fluid samples from the body, e.g., lumbar puncture, and liver and kidney biopsy.
needleplacement~e.g~epidural~rh~ ' J ,ul3-.y of therapeutic agents at the cite of injury/disease (e.g., use of a catheter for precise placement of ~ agents at the cite of an v~ lal tumor).

-214~05 placement and of devices to remove fluids or tissues, e.g., urinary catheters, and pleural fluid removal.
devices for trauma t"l, t, e.g., chest tube placement.
devices for airway ~ t, e.g., fiber-optic l.
ligid ~ 6~ and intubation ~
devices for assessing disease and pathology, e.g., fiber-optic scope manipulation (e.g., ~.y~OSCOpC5, bl~ ~lCS~,u~y~ flexible rY~ CUIU..J tC~Y~ and e,.du~u~).
devices for minimally invasive surgery, e-g-, IdP~IUt~Ur^~
. .
The present invention has been described in terms of preferred and exemplary; ' ' thereof. Other~ andvariations within tne scope and spirit of the appended claims will, given the benefit of this disclo~u~ occur to persons of ordinary skill in the art.

Claims (7)

1. A computer based medical procedure simulation system, comprising:
first memory means for storing image data representative of a virtual internal body environment, second memory means for storing image data representing a virtual medical instrument extending within said internal body environment;
display means for displaying images corresponding to said image data stored in said first and second memory means;
a first computer peripheral device that tracks the movement of a physical member representing the virtual medical instrument, and outputs a signal based thereon;
first input means for receiving the signal from said peripheral device;
first calculating means for calculating position data representing the position of the virtual medical instrument within said virtual internal body environment, based on the signal from the peripheral device; and first updating means for updating the image data representing the virtual medical instrument extending within said virtual internal body environment, based on said calculated position data.

2. A surgical simulation system according to claim 1, wherein said virtual medical instrument comprises a virtual catheter, said physical member comprises an elongated relatively flexible member representing said virtual catheter, and said peripheral device tracks the translational movement of the elongated relatively flexible member.

3. A surgical simulation system according to claim 2, wherein:
the end of said virtual catheter is angled;
said peripheral device tracks the rotational movement of the elongated relatively flexible member and outputs a signal based thereon;
said first calculating means calculates orientation data representing an orientation of the angled end of the virtual catheter, based on the signal from the first peripheral device; and said updating means updates the image data representing the virtual catheter based on the calculated orientation data.

4. A surgical simulation system according to claim 1, further comprising:
determining means for determining the location of the virtual medical instrument within the virtual internal body environment; and output means for outputting a tactile feedback response signal to said first computer peripheral device based on said location of the virtual medical instrument;
wherein, said first computer peripheral device comprises a mechanism for imparting to said physical member a tactile feedback force responsive to said tactile feedback response signal.

5. A surgical simulation system according to claim 4, wherein said tactile feedback force is a force resisting movement of said physical member.

6. A surgical simulation according to claim 5, wherein said virtual medical instrument comprises a virtual catheter and said physical member comprises an elongated relatively flexible member representing said virtual catheter.

7. A surgical simulation system according to claim 1, further comprising:
second calculating means for calculating the velocity of the virtual medical instrument within the virtual internal body environment, based upon the signal from the peripheral device; and second updating means for modifying the image data representative of the virtual internal body environment, based on the calculated velocity data. 8. A surgical simulation system according to claim 7, wherein the second updating means modifies the image data representative of the virtual internal body environment when the calculated velocity of the virtual medical instrument exceeds a preset velocity.
9. A surgical simulation system according to claim 8, wherein the second updating means modifies the image data representative of the virtual internal body environment to simulate a puncture of internal passageway, and associated bleeding.
10. A surgical simulation system according to claim 1, further comprising third memory means for storing image data representing a simulated real-time wave form of a signal representing a physiological parameter, and wherein said display means displays said simulated real-time wave form together with said images corresponding to said image data stored in said first and second memory means.
11. A surgical simulation system according to claim 10, wherein said simulated real-time waveform is an electrocardiogram (EKG) waveform.
12. A surgical simulation system according to claim 11, further comprising second updating means for modifying the image data representative of the virtual internal body environment, based on the image data representing the simulated real-time waveform.
13. A surgical simulation system according to claim 12, wherein the virtual internal body environment comprises an arterial tree, and the image datamodification based on the simulated real-time waveform simulates a pulsation of the arterial tree in synchronism with the EKG waveform.
14. A surgical simulation system according to claim 1, further comprising:
a second computer peripheral device comprising a switch switchable between ON and OFF positions, and outputting a corresponding signal;
a second input means for receiving the signal from said second computer peripheral device; and second updating means for modifying the image data representative of the virtual internal body environment, based on the signal from said second computer peripheral device.
15. A surgical simulation system according to claim 14, wherein the virtual internal body environment comprises an arterial tree, and said image data modification based on the signal from the second computer peripheral device simulates the release and flow of fluoroscopic dye through the arterial tree when the switch is in the ON position.
6. A surgical simulation system according to claim 1, further a second computer peripheral device comprising a switch switchable between ON and OFF positions, and outputting a corresponding signal;
a second input means for receiving the signal from said second computer peripheral device; and second updating means for modifying the image data representing the virtual medical instrument extending within said virtual internal body environment based on the signal from said second computer peripheral device.
17. A surgical simulation system according to claim 16, wherein the virtual internal body environment comprises an arterial tree having an occlusiontherein, and said image data modification based on the signal from the second computer peripheral device simulates the inflation of a balloon catheter within said occlusion.
18. A surgical simulation system according to claim 17, wherein an inflation amount of the simulated balloon catheter increases with time while theswitch is held in the ON position.
19. A surgical simulation system according to claim 18, further third updating means for modifying the image data representative of the virtual arterial tree, based on the inflation amount of the simulated balloon catheter.
20. A surgical simulation system according to claim 19, wherein the image data modification based on the inflation amount of the simulated balloon catheter simulates the clearing of the occlusion.
21. A surgical simulation system according to claim 18, wherein the image data modification based on the signal from the second computer peripheral device simulates the rupture of the simulated balloon catheter in the event the switch is held in the ON position for a time period exceeding a preset time limit.