CN107067856A - A kind of medical simulation training system and method - Google Patents

Abstract

The invention discloses a medical simulation training system and method. The system includes: an object model building module, which collects three-dimensional model data of a specific part of a human body, and constructs a corresponding three-dimensional virtual model object based on the collected three-dimensional model data; a model database, Store the object data of the 3D virtual model; the sensor tracking module collects the operation data of the user on the 3D solid model; the data processing module processes the collected data to obtain the shape data of the 3D solid model and the user's operation data on the 3D solid model and outputs it; the image The generation module generates a 3D virtual image including simulation operation result information according to the shape data of the 3D solid model and the user's operation data; the display module displays and outputs the 3D virtual image for the user to view the simulation operation result. The system and method of this embodiment have a strong sense of immersion, simulate real medical scenes to the greatest extent, and reduce the cost of clinical teaching and training such as surgery and intravenous injection.

Description

A medical simulation training system and method

technical field

The invention relates to the technical field of virtual reality, in particular to a medical simulation training system and method.

Background technique

In the field of medicine, traditional medical simulation training, for example, the practical teaching of surgical operations is very simple, basically not up to the level of clinical application, some medical colleges and hospitals use simulated organ rubber for teaching, through the demonstration of surgical videos, using real organs Specimen explanation, cadaver anatomy teaching, and expert operation observation and other forms are aimed at medical assistance such as surgical training, surgical rehearsal, clinical diagnosis, remote surgery, and rehabilitation treatment.

There are many disadvantages in this method: 1. Simulating organs for teaching, the rubber material itself is easy to age, and needs to be replaced after being used many times. The price of rubber organs is relatively high, and frequent replacement of teaching aids greatly increases the teaching cost. 2. In the teaching of cadaver anatomy, students are taught through cadaver anatomy in the laboratory. This teaching method of human anatomy is often very expensive, and there is a large gap between the corpse and the living body, which cannot enable students to complete intuitive and easy learning and hands-on operation. . 3. The form of expert operation observation, to understand and learn various surgical operations through multiple senses such as audio-visual and tactile senses. For young doctors or interns, it is impossible for young doctors or interns to operate in person and compare them with expert operation methods in time to find their own mistakes and Insufficient, no doubt make the training of young medical staff waste a lot of money and time, can not get good training effect, also seriously affect the practice efficiency.

In summary, the existing medical simulation training technical solutions have high training costs, poor training effects and low efficiency.

Contents of the invention

The present invention provides a medical simulation training system and method to solve the technical problems of high training cost, poor training effect and low efficiency in the existing medical simulation training technical solutions in the prior art.

According to one aspect of the present invention, a kind of medical simulation training system is provided. The system includes:

The object model building module is used to collect 3D solid model data of specific parts of the human body, and constructs a corresponding 3D virtual model object based on the collected 3D solid model data, and sends the constructed 3D virtual model object to the model database;

A model database for storing 3D virtual model object data;

The sensor tracking module is used to collect real-time position data and pressure data of the device held by the user through the sensor set on the device held by the user, and track and collect the position data and pressure data of the three-dimensional model through the sensor in the three-dimensional model that the device acts on;

The data processing module processes the position data and pressure data collected by the sensor tracking module, obtains the shape data of the 3D solid model and the user's operation data on the 3D solid model, and combines the shape data of the 3D solid model and the user's operation data on the 3D solid model The operation data is output to the image generation module;

The image generation module generates a three-dimensional virtual image including simulation operation result information according to the shape data of the three-dimensional solid model and the user's operation data on the three-dimensional solid model, and sends the generated three-dimensional virtual image to the display module;

The display module is used for displaying and outputting a three-dimensional virtual image for users to view simulation operation results.

According to another aspect of the present invention, a kind of medical simulation training method is provided, the method comprises:

Collecting 3D stereoscopic model data of specific parts of the human body, and constructing corresponding 3D virtual model objects based on the collected 3D stereoscopic model data;

Store the constructed 3D virtual model object;

Collect real-time position data and pressure data of the device held by the user through the sensor set on the device held by the user, and track and collect the position data and pressure data of the three-dimensional solid model through the sensor in the three-dimensional solid model that the device acts on;

Process the collected position data and pressure data to obtain the shape data of the three-dimensional solid model and the user's operation data on the three-dimensional solid model;

generating a 3D virtual image including simulation operation result information according to the shape data of the 3D solid model and the user's operation data on the 3D solid model;

Display the output 3D virtual image for the user to view the simulation operation results.

The beneficial effects of the embodiments of the present invention are: the medical simulation training system and method of the embodiments of the present invention use virtual reality technology to collect 3D stereoscopic model data of specific parts of the human body and construct virtual model objects, and receive user operations on the 3D stereoscopic model Data and generates corresponding virtual reality images according to the operation data, and presents the generated virtual reality images on the virtual reality display device, which is convenient for users to watch the operation results. Therefore, by installing sensors on the equipment and operating objects operated in the real space, the user's operation data is collected in real time, and corresponding 3D virtual reality images are produced to provide users with simulated tactile perception and simulated visual perception, improving the user's sense of immersion , highly interactive. The operation data can be directly collected and updated in real time, and the operation results can be truly fed back to improve the training effect. It meets the needs of medical teaching and operation training, and is simple, efficient, easy to operate, convenient for clinical promotion and popularization, and has high application value.

Description of drawings

Fig. 1 is a structural block diagram of a kind of medical simulation training system of an embodiment of the present invention;

Fig. 2 is a structural block diagram of a kind of intravenous injection simulation training system of an embodiment of the present invention;

Fig. 3 is a structural block diagram of a kind of surgical operation simulation training system of an embodiment of the present invention;

Fig. 4 is a flowchart of a medical simulation training method according to an embodiment of the present invention.

detailed description

Virtual Reality (Virtual Reality) technology, also known as Spirit Realm technology, is a new technology that has developed rapidly in recent years. Virtual reality is to use computer simulation to generate a three-dimensional virtual world, providing users with simulations of vision, hearing, touch and other senses, so that users can observe things in the three-dimensional space in a timely and unlimited manner as if they were actually there. This technology uses a series of computer-related science and technology such as computer graphics, human-computer interaction technology, and 3D modeling to simulate real space and sense human behavior in a virtual environment (such as body movement, head rotation, etc.) in a timely manner. ), and provide timely feedback on its behavior, so as to simulate the real scene to the greatest extent.

The design idea of the present invention is to apply virtual reality technology to medical simulation training, generate a three-dimensional image of a virtual space through virtual reality technology, and collect in real time the user's operation data on a three-dimensional model of a specific part of the human body in the real world. The operation data calculates the operation result data, and superimposes the operation result data on the virtual model object, so as to generate a virtual image including the operation result data, and display the output to the user for the user to view. In this way, the simulation of the user's senses such as vision and touch is provided, so that the user can observe the model objects in the three-dimensional space in a timely and unlimited manner as if they were in the scene, which improves the effect of medical training, and does not need to purchase expensive training equipment, saving costs.

Embodiment one

Fig. 1 is a structural block diagram of a kind of medical simulation training system of an embodiment of the present invention, referring to Fig. 1, the medical simulation training system 10 of the present embodiment comprises:

The object model building module 101 is used to collect 3D solid model data of specific parts of the human body, and constructs a corresponding 3D virtual model object based on the collected 3D solid model data, and sends the constructed 3D virtual model object to the model database 104;

Model database 104, used for storing three-dimensional virtual model object data;

The sensor tracking module 103 is used to collect real-time position data and pressure data of the device held by the user through the sensor set on the device held by the user, and track and collect the position data and pressure data of the three-dimensional solid model through the sensor in the three-dimensional solid model that the device acts on;

The data processing module 102 processes the collected data, obtains the shape data of the three-dimensional solid model and the user's operation data on the three-dimensional solid model, and outputs the shape data of the three-dimensional solid model and the user's operation data on the three-dimensional solid model to the image generation module 105;

The image generation module 105 generates a three-dimensional virtual image including simulation operation result information according to the shape data of the three-dimensional solid model and the user's operation data on the three-dimensional solid model, and sends the generated three-dimensional virtual image to the display module 106;

The display module 106 is configured to display and output a three-dimensional virtual image for the user to view the simulation operation result.

As shown in Figure 1, it can be seen that the medical simulation training system based on virtual reality in this embodiment collects 3D stereoscopic model data of specific parts of the human body, and constructs corresponding 3D virtual model objects based on the collected 3D stereoscopic model data. The 3D virtual model object is sent to the model database, and then the user can select the 3D virtual model object in the model database. Using virtual reality technology, when the user operates the scalpel and other instruments in the real world to operate the 3D solid model, every step The operation data can be collected by the sensors on the scalpel and other instruments and the sensors on the three-dimensional model, and the collected data are transmitted to the data processing module, which calculates and recognizes each step of the user's operation data (such as , operating force data, operating direction data, and data of the operated parts, etc.), and then sent to the image generating unit, which generates the corresponding three-dimensional operation result image according to the operating data, and then through the display on the three-dimensional display, the user can see The operation result of each operation step on the virtual model object improves the effect of medical training and reduces the training cost, which is convenient for large-scale promotion and application.

In one embodiment of the present invention, the image generation module 105 is specifically configured to identify the 3D model corresponding to the shape data in the preset model database according to the shape data of the 3D model, and The operation data, call the 3D image processing library, and generate the 3D image including the simulation operation result information.

In one embodiment of the present invention, the object model construction module 101 is used to collect the three-dimensional model data of the human arm and internal organs, and construct the corresponding three-dimensional virtual model of the human arm based on the collected three-dimensional model data of the human arm and internal organs objects, and the corresponding three-dimensional virtual model objects of internal organs.

In one embodiment of the present invention, the equipment held by the user includes: a virtual reality scalpel or a virtual reality syringe, and a sensor tracking module 103, which is used to collect real-time position data of the equipment held by the user through the position sensor, and collect the three-dimensional effect of the equipment through the position sensor. Real-time operating position data of the three-dimensional model; and, collecting real-time pressure data of the device held by the user through the pressure sensor, and collecting real-time pressure data of the three-dimensional model of the device acting on the device through the pressure sensor.

In one embodiment of the present invention, the display module 106 is provided with a camera for tracking the user's eye movement, or an acceleration sensor is provided in the display module for real-time collection of user head movement data to track the user's head movement, And determine the user's point of view in real time according to eyeball movement or head movement,

When receiving the three-dimensional virtual image sent by the image generating module 105, the display and output position of the three-dimensional virtual image is adjusted according to the user's viewpoint determined in real time. In one embodiment of the present invention, the display module 106 is virtual reality glasses or a virtual reality helmet.

In one embodiment of the present invention, the image generating module 105 is also used to generate a 3D virtual image according to the optical characteristics of the glasses in the virtual reality glasses or the virtual reality helmet before sending the 3D virtual image to the virtual reality glasses or the virtual reality helmet. Perform distortion correction.

The structure of the virtual reality-based medical simulation training system of the present invention will be specifically described below in conjunction with two specific application scenarios. The two specific application scenarios are: surgical simulation training and nursing intravenous injection simulation training.

Embodiment two

Provided in the present embodiment is based on virtual reality nursing intravenous injection simulation training, specifically, needs to use the following tools during the application of virtual reality based intravenous injection simulation training system:

Intravenous injection simulation training arm three-dimensional model, that is, the object operated by the user;

Syringes, i.e. devices held by the user;

Computer, computer is used for model data storage and data processing, image generation, data transmission, etc.;

A VR display device (such as VR glasses) is used to display a virtual reality three-dimensional image.

The realization principle of the virtual reality-based intravenous injection simulation training system is to arrange sensors in the simulated blood vessels (or intramuscular injection area) in the three-dimensional model of the arm, collect the dynamic data of the blood vessels in the hand, and centrally store the collected data. In this embodiment, the dynamic data of hand blood vessel trends can be classified according to different age groups, the collected data can be stored centrally, a complete arm physiological metabolism model database can be constructed, and sensors can be set in the intramuscular injection area and important blood vessels in the model. Once all the sensors are touched Touch, transmit the collected data to the VR glasses, and then realize the object model of the virtual injection operation through programming. The operator can simulate the intravenous injection of the arm model in the virtual reality space by wearing 3D glasses. In addition, the skin tissue The thickness of the fat layer and the thickness of the blood vessels can also be freely switched in the model database, and the real feedback of the intravenous (muscle) injection effect is given to the VR glasses through the sense of touch connected to the sensor. The technical solutions of the embodiments of the present invention have a strong sense of immersion, strong interactivity, and are simple, efficient, and easy to operate.

Fig. 2 is a structural block diagram of a kind of intravenous injection simulation training system of an embodiment of the present invention, referring to Fig. 2, the intravenous injection simulation training system 20 of the present embodiment comprises:

The object model building unit 201 uses virtual reality technology to collect three-dimensional arm model data, sets a virtual three-dimensional arm model that meets the situation, and stores it in the computer run by the object model building unit 201 for simulation operation. In addition, in the actual application process, the 3D model can also be provided to the 3D application program interface for the design of the intravenous (muscular) injection scheme.

The data processing unit 202 is used to process the data transmitted by the data transmission unit 204, and obtain the information therein: for example: user viewpoint information Viewport, operation object position information Position, motion information Motion, and pass the identified information to the image generation unit Step 203 is to generate a three-dimensional virtual image, and generate a virtual scene according to a corresponding injection operation.

Wherein, the viewpoint information Viewport can be collected in the following manner: a camera is installed in the VR head-mounted device to track the user's eyeball movement, and the user's viewpoint is determined according to the eyeball movement.

When the user wears the VR head-mounted device and is in the virtual scene of intravenous injection simulation training, the sensor on the human arm model and the sensor on the syringe are used to recognize the user's actions, and the action instructions are transmitted to the data processing unit of the computer.

The image generation unit 203 receives the viewpoint information, position information, motion information, etc. sent by the data processing unit 202, and processes the information, extracts the shape (object outline) information of the three-dimensional solid model operated by the user, and then The object model data corresponding to the object shape information is identified in the model database, and a 3D image processing library (for example, OpenGL or Direct3D) is called to construct a 3D model and generate and render a 3D image.

OpenGL (Open Graphics Library) refers to a professional graphics program interface that defines a cross-programming language and cross-platform programming interface specification. It is used for three-dimensional images (two-dimensional ones are also available), and is a powerful and convenient bottom layer. graphics library.

Direct 3D is a 3D graphics API based on Microsoft's common object mode COM (Common Object Mode), which can bypass the Graphics Display Interface (GDI) and directly perform the underlying operations of various hardware that support the API, improve the running speed, and is suitable for multimedia and entertainment , real-time 3D animation and other extensive and practical 3D graphics calculations.

The data transmission unit 204 receives the information sent by the object sensor tracking unit 206 in real time, and transmits it to the data processing unit 202 for processing, and tracks the syringe (note, the syringe here is the device in the device access unit, that is, the syringe needs to be connected into the computer and communicate with the data transmission unit, so as to transmit the sensor data in the syringe to the data transmission unit, so that the data transmission unit sends the information to the data processing unit) and transmits the operation information to the data processing unit for processing to generate 3D virtual image.

The data processing unit can be set in a computer, and the computer provides a device access interface to facilitate data transmission between the injector and the object sensor tracking unit 206 in the three-dimensional model.

The object sensor tracking unit 206 is set in the three-dimensional arm model and the syringe. In actual operation, since the three-dimensional arm model and the syringe distribute multiple sensors, once the syringe is inserted into the blood vessel on the three-dimensional model of the arm, the data processing unit will be based on the sensors in the syringe. And the information provided by the three-dimensional model sensor of the arm determines whether the user's current operation is correct. If the user's current operation is incorrect, after the current operation data is transmitted to the VR display device, the VR display device immediately presents a three-dimensional virtual image of blood vessel rupture, bleeding, and redness. , to achieve the most realistic injection simulation effect. In addition, the simulation effect data of various shapes of arms can be stored in the model database of the computer for backup during the user's practice.

The sensors here include: a position sensor and a pressure sensor. In practical applications, the real-time position data of the device held by the user is collected through the position sensor, and the real-time operating position data of the three-dimensional model of the device's action is collected through the position sensor; The real-time pressure data of the device held by the user, and the real-time pressure data of the three-dimensional model of the device's action are collected through the pressure sensor.

The display unit 207 is used to receive the 3D image data generated by the image generation unit 203 and display it in a display device, such as VR glasses, and the user can view the operation simulation effect on the virtual reality interactive operation platform by wearing VR glasses; the VR glasses here can be Replaced by a VR headset, there is no limit to this.

Optionally, the virtual reality-based intravenous injection simulation training system of this embodiment also includes: a verification unit 205, the verification unit records the number of user operation errors according to the user's operation data, and then performs user operations according to the settings in the model database. Score and rate the user's operations based on the specified weight assignment and quantitative evaluation information for subsequent viewing and reference to improve the success rate of operations.

The specific working process of the intravenous injection simulation training system based on virtual reality technology in the embodiment of the present invention includes:

Start the intravenous injection simulation training system based on virtual reality technology, enter the initialization work, and the user can select the operation object (that is, a specific human arm model).

Step 201: The user picks up the syringe to operate, the object sensor tracking unit 206 transmits the position information and action information for real-time sensing, and the data processing unit 202 performs gesture recognition and other work, and notifies the image generation unit 203 to generate a three-dimensional image according to the recognized corresponding operation. image.

Step 202: During the injection operation, the object sensor tracking unit 207 interacts with the data transmission unit 204 to track the injection operation to obtain injection operation information, and transmits the information to the data processing unit 202 for processing. In addition, the collected data can also periodically send an update request to the object sensor tracking unit 207, so as to obtain the latest data from the object sensor tracking unit 207, so as to collect user action instructions and accurately collect user operation information.

Step 203: The data processing unit 202 processes the data transmitted from the data transmission unit 204, and after the user performs an injection operation on the three-dimensional model of the arm in the real space, the image generation unit 203 starts the virtual environment subroutine in the computer and calls the three-dimensional model of the arm. The data of the virtual model enables the three-dimensional virtual model to provide timely feedback to the operator's action instructions, thereby presenting the results of the simulated injection operation in front of the user.

Step 204: After the simulated injection operation is completed, the simulated injection operation is evaluated according to the total number of recorded operations and the number of operation errors, and a reference score is issued.

Compared with the existing venipuncture models, this embodiment has the advantages of high reusability, low cost, high degree of venipuncture simulation, and intuitive venipuncture simulation results. Proficiency and other issues. It reduces teaching costs and medical risks, saves training resources, improves teaching quality, and is simple, efficient, easy to operate, convenient for clinical promotion and popularization, and has high application value and significance.

Embodiment three

This embodiment provides a surgical operation simulation training system based on virtual reality. Specifically, the following tools are required for the application of the virtual reality-based surgical simulation training system:

(1) Virtual reality VR scalpel (optionally also includes surgical gloves), where the VR scalpel contains sensors, which can be used for data collection and motion sensing to perform gesture recognition and identify the user's surgical actions. The sensors here include: a position sensor and a pressure sensor. In practical applications, the real-time position data of the device held by the user is collected through the position sensor, and the real-time operating position data of the three-dimensional model of the device's action is collected through the position sensor; The real-time pressure data of the device held by the user, and the real-time pressure data of the three-dimensional model of the device's action are collected through the pressure sensor.

(2) Surgical objects (i.e. three-dimensional models of specific organs of the human body), the system can pre-set the three-dimensional model data of the human body for the user to select the three-dimensional model of human organs as the surgical object, and the main positions of the model organs and the distribution of major blood vessels Multiple sensors. In this way, during the user's operation, once the scalpel touches a blood vessel, the sensor will send sensing information to the data processing unit for further data analysis and processing.

(3) Computer, which is used to realize model data storage and data processing, image generation, data transmission and other functions;

(4) VR display device (such as VR glasses or VR helmet), used to display virtual reality three-dimensional images.

The present invention has three advantages in applying virtual technology to surgical simulation training: first, it can truly present the dynamic process of physiological metabolism, adjust the data rate of organ metabolism in different age groups, and highly simulate the surgical scene. Second, provide real-time feedback on the progress of the operation, use the model-sensing technology to sense the operator's operation results, and give real feedback on the operation process, laying good practical skills for the operator and improving the success rate of the operation. Third, reduce the cost of clinical operation teaching.

Fig. 3 is a structural block diagram of a kind of surgical operation simulation training system of an embodiment of the present invention, referring to Fig. 3, this surgical operation simulation training system 30 comprises:

The viewpoint determination unit 301, the viewpoint determination unit here includes two subunits, respectively the eyeball movement tracking determination viewpoint subunit and the head movement tracking determination viewpoint subunit, in the virtual reality display device, for example, a camera is set in the VR helmet, using The camera performs eye tracking and determines the user's current viewpoint according to the real-time movement data of the user's eyeballs; or, a camera and a specific sensor (for example, an acceleration sensor) are set in the VR helmet to track the head movement position to determine the viewpoint. It should be noted that the way of determining the viewpoint in this embodiment can be realized by using existing technology, so the more detailed implementation of how to determine the viewpoint will not be repeated in this embodiment, and you can refer to the relevant description in the prior art .

The surgical object model 302, the surgical object model here is correspondingly generated by the image processing unit in the computer according to the three-dimensional model of the human organ selected by the user. For example, if the user selects a lung model in the real space to perform surgery, then the image generation unit in the virtual reality-based surgical simulation training system of this embodiment selects the model data of the corresponding lung from the surgery model database , construct a three-dimensional virtual model, and present it in a virtual reality display device.

The hand motion collection unit 303, where the hand motion collection unit uses the sensor on the scalpel or surgical glove to complete the collection of the user's hand motion, for example, the current position of the user's hand, the direction information of the user's operation, and the strength The position of the surgical object model operated by the user can be determined according to the information and the information collected by the sensor in the three-dimensional model of the specific part of the human body. Continuing from the previous example, the user is currently performing surgery on the lungs, and the sensor on the three-dimensional lung model in real space collects whether the scalpel held by the user has touched the blood vessels on the lungs. If there is contact, the sensor can Immediately collect this data and send this operation data to the data processing unit, and then the data processing unit processes the data and sends it to the image generation unit. The image generation unit generates a corresponding virtual reality three-dimensional image according to the user's operation on the blood vessel, presenting in front of the user's eyes.

The operation model database 304 stores organ model data of various specific parts of the human body in the operation model database for users to choose.

The data processing unit 305, the data processing unit performs data analysis and processing by using the data transmitted by multiple sensors distributed in the organ (eg, lung) model and the sensors distributed on the scalpel. Once the scalpel touches the corresponding position of the sensor, it is determined whether the current surgical operation is correct according to the information provided by the sensor, and the data sensed by the sensor is transmitted to the data processing unit for data analysis: The operation data of touching the blood vessel is obtained and sent to the image generation unit, and the image generation unit generates a three-dimensional image of the ruptured blood vessel and sends it to the display unit for presentation.

The image generation unit 306 is configured to generate a three-dimensional virtual organ model, and generate and render a corresponding three-dimensional virtual image according to the data sent by the data processing unit 305 .

In addition, due to the optical characteristics of the lenses in the VR head-mounted device (both are concave lenses, image distortion will occur when displayed), the image generation unit 306 in this embodiment distorts the generated image, that is, the generated three-dimensional virtual image is sent to Correct the distortion of the image before feeding it to the display unit to achieve the desired display effect.

The virtual reality display unit 307 is configured to display the three-dimensional virtual image generated by the image generation unit 306, and the user can see the simulated operation effect of the real operation and the VR operation scene by wearing the virtual reality display unit 307.

The specific working process of the surgical simulation training system based on virtual reality technology in the embodiment of the present invention includes:

Step 301: the user puts on the VR headset and picks up the scalpel.

Step 302: The image processing unit in the computer generates a corresponding surgical virtual model image based on the surgical object in the real space and transmits it to the VR display device.

Step 303: The user performs a surgical operation and performs surgical action recognition.

Step 304: The VR headset determines the position of the user's head or tracks the user's eyeballs to determine the viewpoint.

Step 305: During the surgical operation, the sensors on the scalpel and the three-dimensional human body model sense the surgical cutting point of the surgical object held by the user with the scalpel, and give feedback.

Step 306: Generate a 3D image according to the feedback information and transmit it to the VR headset for display.

So far, the surgical operation simulation training system based on virtual reality technology in this embodiment provides visual and tactile simulation perception through virtual reality technology, and simulates real surgical operations with strong interactivity. It reduces teaching costs and medical risks, saves medical training resources, improves teaching quality, is simple, efficient, easy to operate, and is convenient for popularization.

Embodiment Four

Fig. 4 is a flowchart of a medical simulation training method according to an embodiment of the present invention. Referring to Fig. 4, the medical simulation training method based on virtual reality of the present embodiment includes the following steps:

Step S401, collecting 3D solid model data of a specific part of the human body, constructing a corresponding 3D virtual model object based on the collected 3D solid model data, and storing the constructed 3D virtual model object;

Step S402, collect real-time position data and pressure data of the device held by the user through the sensor set on the device held by the user, and track and collect the position data and pressure data of the three-dimensional solid model through the sensor in the three-dimensional solid model that the device acts on;

Step S403, processing the collected position data and pressure data to obtain the shape data of the three-dimensional solid model and the user's operation data on the three-dimensional solid model;

Step S404, generating a 3D virtual image including simulation operation result information according to the shape data of the 3D solid model and the user's operation data on the 3D solid model;

Step S405, displaying and outputting a three-dimensional virtual image for the user to check the simulation operation result.

In one embodiment of the present invention, step S404, generating a 3D virtual image including simulation operation result information according to the shape data of the 3D solid model and the user's operation data includes:

According to the shape data of the 3D solid model, identify the 3D solid model corresponding to the shape data in the pre-set model database, and call the 3D image processing library according to the user's operation data on the 3D solid model, and generate the information including the simulation operation result information 3D image.

In one embodiment of the present invention, step S401, collecting 3D stereoscopic model data of a specific part of the human body includes:

Collect 3D model data of human arms and internal organs,

The instruments held by the user include: a virtual reality scalpel or a virtual reality syringe,

Tracking and collecting the user's operation data on the three-dimensional model through the sensor set on the device held by the user and the sensor in the three-dimensional model that the device acts on includes:

Collect the real-time position data of the device held by the user through the position sensor, and collect the real-time operation position data of the three-dimensional model of the device's action through the position sensor; and collect the real-time pressure data of the device held by the user through the pressure sensor, and collect the device's action through the pressure sensor The real-time pressure data of the 3D solid model.

In one embodiment of the present invention, the method shown in FIG. 4 further includes: tracking the user's eye movement, or collecting the user's head movement data in real time to track the user's head movement, and determining the user's head movement in real time according to the eye movement or head movement. Viewpoint: when receiving the 3D virtual image, adjust the display and output position of the 3D virtual image according to the user's viewpoint determined in real time.

Displaying and outputting the three-dimensional virtual image includes displaying and outputting the three-dimensional virtual image through virtual reality glasses or a virtual reality helmet.

In step S404 in one embodiment of the present invention, generating a 3D virtual image including simulation operation result information according to the shape data of the 3D stereoscopic model and the user's operation data also includes: according to the optical characteristics of the lenses in the virtual reality glasses or virtual reality helmets , performing distortion correction on the three-dimensional virtual image before sending the three-dimensional virtual image to the virtual reality glasses or the virtual reality helmet.

To sum up, the medical simulation training method of this embodiment uses virtual reality technology to collect 3D solid model data of specific parts of the human body and construct a virtual model object, receives user operation data on the 3D solid model and generates corresponding The virtual reality image is to present the generated virtual reality image on the virtual reality display device, so that the user can watch the operation result conveniently. Therefore, by installing sensors on the equipment and operating objects operated in the real space, the user's operation data is collected in real time, and corresponding 3D virtual reality images are produced to provide users with simulated tactile perception and simulated visual perception, improving the user's sense of immersion , highly interactive. The operation data can be directly collected and updated in real time, and the operation results can be truly fed back to improve the training effect. It meets the needs of medical teaching and operation training, and is simple, efficient, easy to operate, convenient for clinical promotion and popularization, and has high application value.

The above descriptions are only specific implementations of the present invention. Under the above teaching of the present invention, those skilled in the art can make other improvements or modifications on the basis of the above embodiments. Those skilled in the art should understand that the above specific description is only to better explain the object of the present invention, and the protection scope of the present invention shall be determined by the protection scope of the claims.

Claims (10)

1. A medical simulation training system, characterized in that the system comprises: The object model building module is used to collect 3D solid model data of specific parts of the human body, and constructs a corresponding 3D virtual model object based on the collected 3D solid model data, and sends the constructed 3D virtual model object to the model database; A model database for storing 3D virtual model object data; The sensor tracking module is used to collect real-time position data and pressure data of the device held by the user through the sensor set on the device held by the user, and track and collect the position data and pressure data of the three-dimensional model through the sensor in the three-dimensional model that the device acts on; The data processing module processes the position data and pressure data collected by the sensor tracking module, obtains the shape data of the 3D solid model and the user's operation data on the 3D solid model, and combines the shape data of the 3D solid model and the user's operation data on the 3D solid model The operation data is output to the image generation module; The image generation module generates a three-dimensional virtual image including simulation operation result information according to the shape data of the three-dimensional solid model and the user's operation data on the three-dimensional solid model, and sends the generated three-dimensional virtual image to the display module; The display module is used for displaying and outputting a three-dimensional virtual image for users to view simulation operation results. 2. The system according to claim 1, wherein the image generating module is specifically configured to, according to the shape data of the three-dimensional solid model, identify the image corresponding to the shape data in the preset model database. 3D solid model, and according to the user's operation data on the 3D solid model, call the 3D image processing library to generate a 3D image including the simulation operation result information. 3. The system according to claim 1, wherein the object model building module is specifically used to collect the three-dimensional model data of human arms and internal organs, and construct the three-dimensional model data based on the collected human arms and internal organs 3D virtual model objects corresponding to human arms, and 3D virtual model objects corresponding to internal organs. 4. The system according to claim 1, wherein the instruments held by the user include: a virtual reality scalpel or a virtual reality syringe, The sensor tracking module is specifically used to collect real-time position data of the device held by the user through the position sensor, and collect real-time position data of the three-dimensional model of the device through the position sensor; and collect real-time pressure data of the device held by the user through the pressure sensor , and collect real-time pressure data of the three-dimensional model of the device's action through the pressure sensor. 5. The system according to claim 1, wherein the display module is provided with a camera for tracking the user's eye movement, or an acceleration sensor is provided for the display module to collect the user's head in real time. Motion data tracks the user's head movement, and determines the user's point of view in real time according to the eye movement or head movement, When receiving the 3D virtual image sent by the image generating module, the display and output position of the 3D virtual image is adjusted according to the user's viewpoint determined in real time. 6. The system according to claim 5, wherein the display module is virtual reality glasses or a virtual reality helmet. 7. The system according to claim 6, wherein the image generating module is further configured to send a three-dimensional virtual image to the virtual reality glasses or the virtual Before the real helmet, the distortion correction process is performed on the three-dimensional virtual image. 8. A medical simulation training method, characterized in that the method comprises: Collecting 3D solid model data of a specific part of the human body, constructing a corresponding 3D virtual model object based on the collected 3D solid model data, and storing the constructed 3D virtual model object; Collect real-time position data and pressure data of the device held by the user through the sensor set on the device held by the user, and track and collect the position data and pressure data of the three-dimensional solid model through the sensor in the three-dimensional solid model that the device acts on; Process the collected position data and pressure data to obtain the shape data of the three-dimensional solid model and the user's operation data on the three-dimensional solid model; generating a 3D virtual image including simulation operation result information according to the shape data of the 3D solid model and the user's operation data on the 3D solid model; Display the output 3D virtual image for the user to view the simulation operation results. 9. The method according to claim 8, wherein said generating a 3D virtual image comprising simulated operation result information according to the shape data of the 3D solid model and the user's operation data comprises: According to the shape data of the 3D solid model, identify the 3D solid model corresponding to the shape data in the pre-set model database, and call the 3D image processing library according to the user's operation data on the 3D solid model, and generate the information including the simulation operation result information 3D image. 10. The method according to claim 8, wherein the three-dimensional model data of the specific parts of the collected human body comprises: Collect 3D model data of human arms and internal organs, The instruments held by the user include: a virtual reality scalpel or a virtual reality syringe, The tracking and collection of the user's operation data on the three-dimensional model through the sensor set on the device held by the user and the sensor in the three-dimensional model of the device includes: Collect the real-time position data of the device held by the user through the position sensor, and collect the real-time position data of the three-dimensional model of the device's action through the position sensor; and collect the real-time pressure data of the device held by the user through the pressure sensor, and collect the real-time position data of the device's action through the pressure sensor Real-time pressure data for 3D solid models.