CN115273583A - A multi-person interactive orthopaedic clinical teaching method based on mixed reality - Google Patents

Abstract

The application relates to a multi-person interaction orthopedics clinical teaching method based on mixed reality, the teaching method comprises a plurality of orthopedics clinical teaching modes, a corresponding virtual teaching scene can be automatically generated by selecting the orthopedics clinical teaching mode, and the virtual teaching scene and image information of a user are generated and combined to form an image which is displayed on a three-dimensional display device; the user uses the hand motion track generation instruction to control the operation of the virtual prop in the virtual teaching scene through the worn interactive device, so that the effect of simulating teaching is achieved. The teaching method adopts a mixed reality technology to realize multi-user interactive teaching among teachers, students and a mixed reality environment, through the teaching method, the teachers can teach in a virtual orthopedics clinical teaching scene, the students can watch teaching contents and simulation clinical operation contents more intuitively and stereoscopically, immersion is more sufficient, meanwhile, the teaching places of the teachers are not limited, and the convenience of online education is greatly improved.

Description

A multi-person interactive orthopedic clinical teaching method based on mixed reality

technical field

The present application relates to a clinical teaching method of orthopedics, in particular to a multi-person interactive clinical teaching method of orthopedics based on mixed reality.

Background technique

Orthopedics is a highly professional and practical discipline, which not only contains more abstract concepts and a variety of theoretical knowledge, but also involves complex anatomical skills of bones, muscles and blood vessels. An orthopedic specialist not only needs to have experience in the diagnosis, treatment, and prevention of common orthopedic diseases, but also needs to master the skills of orthopedic surgery. Orthopedic diseases are complex and subdivided. How to establish effective orthopedic teaching methods is a subject worth studying. Orthopedic clinical teaching is an important part of orthopedic teaching.

Traditional orthopedic clinical teaching, especially in clinical practice teaching, mostly adopts the 1:1 apprenticeship model, including teachers and students directly teaching in the operating room and acquiring clinical skills under the supervision of experts. However, this model At least the following questions must be met:

1. Due to time and place constraints, teachers and students need to teach in the operating room at the same time.

2. The cost is high, and real teaching scenes and teaching props must be provided.

3. The efficiency is not high, limited by the teaching scene and the guarantee of teaching effect, the number of students participating in the training at one time is limited.

4. Human anatomy faces ethical and moral issues.

In view of the above problems, the traditional orthopedic clinical teaching methods have been greatly limited.

Contents of the invention

One of the ideas of this application is to solve the problem that the traditional orthopedic clinical teaching method is limited by time and location. This application uses network technology to transmit the content of the teacher's clinical teaching to the three-dimensional display device worn by the students in real time, so that the students can watch it.

The second idea of this application is to solve the limitation that teachers must enter real teaching scenes (such as operating room) and use real teaching props for clinical teaching in traditional orthopedic clinical teaching; at the same time, because the teaching scenes and teaching props used are real , leading to high teaching costs; the teaching props include operating tables, orthopedic surgical instruments, auxiliary consumables, and orthopedic anatomical models.

This application provides virtual orthopedic clinical teaching scenes and teaching props for teachers to simulate clinical teaching. Teachers use virtual teaching props (such as virtual orthopedic anatomical models) to perform simulated clinical teaching in virtual teaching scenes. The virtual orthopedic anatomical model is constructed through computer equipment according to the teaching objectives, and one or more virtual orthopedic anatomical models are constructed at the same time.

This application uses interactive devices (such as data gloves) to track and identify the teacher's hand movements, and then controls the virtual teaching props to simulate the teaching effect through hand movements.

Further, on the basis of ideas 1 and 2, idea 3 of this application is to ensure the efficiency and effect of virtual teaching. This application utilizes mixed reality technology to integrate teachers and virtual scenes and display them on students' three-dimensional display devices. Students can watch real-time teaching images by wearing the three-dimensional display devices to achieve the goal of participating in teaching in real scenes. Effect. Each student wears a three-dimensional display device to realize a one-to-many orthopedic clinical teaching mode.

Combining the above ideas, in the first aspect, the present application provides a multi-person interactive orthopedic clinical teaching method based on mixed reality. The teaching method includes the following steps:

The user logs in to the client, and the client records the user information;

The teacher user selects the orthopedic clinical teaching mode on the user terminal;

The client imports the corresponding virtual teaching scene according to the orthopedic clinical teaching mode;

The 3D camera device 4 obtains the image information of the teacher user and uploads it to the computer device;

The computer device displays the image information on the virtual teaching scene to form a merged image, and displays the merged image on the three-dimensional display device worn by the user;

The user recognizes and captures the hand movement trajectory information through the interactive device worn by the user, and the hand movement trajectory information generates control instructions, and the control instructions control the operation of the virtual props in the virtual teaching scene;

The user simulates the orthopedic clinical teaching content in the virtual teaching scene through the operation of virtual props, and the teaching content is displayed in the form of a real-time 3D image on the 3D display worn by the user for viewing and learning;

After the teaching content is over, the user terminal generates and saves a video of orthopedic clinical teaching content.

In a specific embodiment, a user logs in an account through the software terminal and records personal information, and the user includes a teacher user and a student user.

In a specific embodiment, the orthopedics clinical teaching mode includes: classroom teaching, clinical simulation, and video courseware, and the orthopedics clinical teaching mode is selected and operated by a teacher user.

In a specific embodiment, the teaching scene includes a virtual orthopedic operating room and a virtual classroom;

In a further embodiment, the virtual teaching scene has essential elements of virtual teaching props, and the virtual teaching props include a first virtual teaching prop and a second virtual teaching prop;

Wherein, the first virtual teaching prop is applied to the virtual orthopedic operating room teaching scene, and is an essential element for constructing a virtual orthopedic operating room teaching scene;

The second virtual teaching prop is applied to the virtual classroom teaching scene and is an essential element for constructing the virtual classroom teaching scene.

In a further embodiment, the first virtual teaching props include a virtual operating bed, a virtual orthopedic anatomical model, virtual orthopedic surgical instruments and equipment consumables.

In a further embodiment, the virtual orthopedic anatomical model includes: at least one of a virtual skull, a virtual trunk bone, a virtual limb bone, and a virtual human joint.

Wherein, the virtual trunk bones include virtual vertebrae, virtual ribs, and virtual sternum; the virtual limb bones include at least one of virtual upper limb bones and virtual lower limb bones.

In a further embodiment, the virtual orthopedic surgical instruments include: virtual surgical forceps, virtual scalpels, virtual surgical scissors, and virtual puncture needles.

In a further embodiment, the virtual equipment consumables include: virtual implant screws, virtual implant bone plates, virtual implant joints, and virtual electric bone drills.

In a further embodiment, the second virtual teaching props include: a virtual classroom, a virtual podium, a virtual desk, a virtual white, and a blackboard.

In a further embodiment, during the orthopedic clinical teaching process, the teacher user selects a teaching scene with the corresponding virtual teaching props to perform simulated orthopedic clinical teaching according to the orthopedic clinical teaching objectives.

In a further embodiment, during the orthopedic clinical teaching process, the teacher user edits the virtual teaching scene through the client software, and the editing includes adding, deleting and replacing virtual teaching props.

In a further embodiment, the new virtual teaching props are constructed by means of data programming, and imported into the user terminal for editing the virtual teaching scene.

In a further embodiment, when the orthopedic clinical teaching scene is a virtual orthopedic operating room, the virtual teaching prop imported into it is the first virtual teaching prop; when the orthopedic clinical teaching scene is a virtual classroom, the virtual teaching prop imported into it is the second virtual teaching prop. Virtual teaching props.

In a further embodiment, the teacher user and the student user identify and acquire hand movement trajectory data information through wearing interactive devices 3 (such as data gloves), and the hand movement trajectory data information generates control instructions, and the control instructions are related to The virtual teaching props establish a one-to-one mapping relationship, and control the operation of the virtual teaching props, so as to realize the simulation operation used to show the actual clinical teaching effect during orthopedic clinical teaching.

In a further embodiment, the teaching video includes video information including:

Display the teacher user image information, virtual teaching scene information and virtual orthopedic clinical teaching operations performed.

In a further embodiment, the orthopedic clinical teaching images are played in the teaching mode of video courseware.

In a further embodiment, during the orthopedic clinical teaching process, interactive operations are performed between teacher users and student users as required, and the interactive operations include:

The teacher's client terminal imports questions or question banks, and answers questions from the student's client terminal;

Questions from the student client, and answers to questions from the teacher client.

In a further embodiment, the three-dimensional display device includes a sound module for playing audio information generated during the teaching process.

In a second aspect, the present application provides a device for realizing the teaching method described in the first aspect, the device includes a user terminal, a computer device, an interactive device, a 3D camera device, and a three-dimensional display device.

Compared with the prior art, the present application has at least the following beneficial effects:

This application relates to a multi-person interactive orthopedic clinical teaching method based on mixed reality. The teaching method applies mixed reality technology to orthopedic clinical teaching, and realizes the interaction between teachers and students through the interaction between teachers, students and the mixed reality environment. Multi-person interactive teaching between students and students.

Through the teaching method, the teacher conducts simulated classroom teaching and simulated orthopedic clinical operation teaching in the virtual orthopedic clinical teaching scene, and students can watch the teaching content and simulated clinical operation content more intuitively and three-dimensionally. It improves the participation of students, is not easy to be distracted, and achieves the purpose of improving training effectiveness. At the same time, the teaching location of teachers is not limited, which greatly improves the convenience of online education.

The teaching method solves the traditional orthopedic clinical teaching mode which is limited by time and place, low cost, low efficiency and human anatomy faces ethical and moral problems.

Description of drawings

The application will be described in further detail below in conjunction with the accompanying drawings and preferred embodiments, but those skilled in the art will appreciate that these drawings are only drawn for the purpose of explaining preferred embodiments, and therefore should not be used as a reference to this application. Restrictions on scope of application. In addition, unless otherwise specified, the drawings are only schematically intended to conceptually represent the composition or configuration of depicted objects and may contain exaggerated representations, and the drawings are not necessarily drawn to scale.

FIG. 1 is a schematic diagram of a mixed reality device provided by an embodiment of the present application.

Fig. 2 is a flow chart of the orthopedics clinical teaching method provided by the embodiment of the present application.

Detailed ways

In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, not to limit the present application.

Embodiment 1: hand interaction process

Hand interaction, that is, controlling the operation of virtual props through hand movements, is an important part of realizing the teaching method described in this application, and this embodiment will describe this process in detail.

The hand interaction device 3 used in the embodiment of the present application is a data glove. The data glove constructs a virtual hand model in the virtual teaching scene according to the structure of the human hand, and overlaps with the hand structure of the user wearing the data glove and displays it in the virtual teaching scene.

Data information acquisition: the data glove is embedded with 16 optical sensors, which are used to measure the positioning and shape data information for the virtual hand model. The data information includes the flexion and extension angle, translation amount and Angle information between fingers, etc.

Data connection and reading: the data glove is connected to the computer device 2 through a data line, and the computer reads the data information transmitted by the data glove and generates control instructions, which are used to control the virtual teaching props in the virtual teaching scene .

Calibration of data gloves: Due to the difference in the size and structure of different users’ hands, in order to ensure the maximum sensitivity of data gloves and the accuracy of input data information, data gloves have a calibration function, which is calibrated through the relevant software of the connected user terminal 1. There are automatic calibration and manual calibration.

Control of the virtual hand model: the computer device 2 collects hand information through a program, establishes a virtual hand model, and puts it into the virtual teaching scene;

The virtual hand model coincides with the user's hand structure;

The motion trajectory of the user's hand generates data information, and the computer device 2 assigns the collected data information to the virtual hand model, so that the virtual hand model can display corresponding continuous and smooth hand movements on the 3D display device, and the data information is determined by the data information. The sensor on the glove transmits to the computer device 2;

The operation of the data glove on the virtual teaching props is divided into selection, operation and release. When the data glove touches the virtual teaching props, the purpose of selection is achieved; the operation of the virtual teaching props is realized by adjusting the movements of the user's hands; through the virtual hand model When the "release" function button on the 3D display device is touched, the virtual hand model is separated from the virtual teaching prop, and at this time, the next teaching prop can be selected or the simulation teaching operation can be ended.

Embodiment 2 clinical simulation teaching mode

The teaching method of the embodiment of the present application is realized by the device shown in FIG. 1. FIG. 2 is a flow chart of implementing a multi-person interactive orthopedic clinical teaching method based on mixed reality. As shown in FIG. 2, the orthopedic clinical teaching method includes the following steps:

S1: The user logs in to client 1, and client 1 records user information.

A user needs to register an account before logging into the user terminal 1 , which includes a teacher user terminal 1 and a student user terminal 1 , and the users register and operate on their respective user terminal 1 .

S2: The teacher user selects the orthopedic clinical teaching mode on the user terminal 1.

There are three modes of orthopedic clinical teaching, including: classroom teaching, clinical simulation and video courseware. The selection of the teaching mode is performed by the teacher user on the teacher user terminal 1, and this embodiment is described in detail with the teaching mode of clinical simulation.

S3: Client 1 imports the corresponding virtual teaching scene according to the orthopedic clinical teaching mode.

According to the clinical simulation teaching mode selected in the S2 step, a teaching scene is automatically generated, and the teaching scene (3D) has the first virtual teaching props for constructing a virtual orthopedic operating room teaching scene;

Wherein, the first virtual teaching props include a virtual operating bed, a virtual orthopedic anatomical model, virtual orthopedic surgical instruments and equipment consumables;

The virtual orthopedic anatomical model includes: at least one of a virtual skull, a virtual trunk bone, a virtual limb bone, and a virtual human joint;

The virtual trunk bones include virtual vertebrae, virtual ribs, and virtual sternum; the virtual limb bones include at least one of virtual upper limb bones and virtual lower limb bones;

The virtual orthopedic surgical instruments include: virtual surgical forceps, virtual scalpels, virtual surgical scissors, and virtual puncture needles;

The virtual equipment consumables include: virtual implant screws, virtual implant bone plates, virtual implant joints, and virtual electric bone drills.

The teaching scene is pre-set at the teacher's client terminal 1, and the teacher uses the client to edit the teaching scene, and the editing of the teaching scene includes:

Adjust the color, size and position display of the teaching scene;

Addition, deletion and replacement of virtual teaching props.

According to needs, the user constructs a new first virtual teaching prop by programming data and imports it into the computer device 2, and matches the teaching scene of the virtual orthopedic operating room for editing of the teaching scene of the virtual orthopedic operating room.

S4: The 3D camera device 4 acquires the image information of the teacher user, and uploads it to the computer device 2 .

S5: The computer device 2 displays the image information on the virtual teaching scene to form a merged image, and displays the merged image on the three-dimensional display device 5 worn by the user. First, the human body image of the teacher user is displayed in the virtual orthopedic operating room for teaching Scenes.

S6: The user simulates the orthopedic clinical teaching content in the virtual teaching scene through the operation of the virtual props, and the teaching content is displayed in the form of a real-time 3D image on the worn 3D display for the user to watch and learn.

As in the hand interaction process described in Embodiment 1, the teacher user identifies and obtains the hand movement track data information of the teacher through the wearing interactive device 3 (such as a data glove), and the hand movement track data information generates control instructions , the control instruction and the first virtual teaching prop are mapped one by one, and the operation of the first virtual teaching prop is controlled, so as to realize the simulation for showing the actual clinical operation effect during orthopedic clinical teaching operate.

The student user performs the simulation operation of the clinical operation in the orthopedics clinical teaching process by wearing the interactive device 3 .

During the orthopedic clinical teaching process, the teacher user and the student user perform interactive operations as required, and the interactive operations include:

The teacher's client terminal 1 imports questions (or question banks), and answers the questions of the student's client terminal 1;

Questions from the student client terminal 1 and answers to questions from the teacher client terminal 1.

S7: The teaching content ends, and the client 1 generates and saves a video of orthopedic clinical teaching content.

After the orthopedic clinical teaching task is completed, an orthopedic clinical teaching image is generated, and the teaching image includes video information including:

The human body image of the teacher, the virtual orthopedic operating room teaching scene and the performed virtual orthopedic clinical teaching operation are displayed.

The teacher user generates the teaching image and saves it to the user terminal 1, selects the teaching mode of the video courseware through the teacher user terminal 1, and the students can watch the teaching image by wearing the three-dimensional display device 5.

Embodiment 3 Teacher's teaching mode

Compared with Example 1, the difference of Example 2 lies in the different teaching modes selected in the orthopedics clinical teaching process, resulting in different generated teaching scenarios and achieving different teaching purposes. The teaching mode of the classroom teaching selected in this embodiment matches the teaching scene of the virtual classroom. Through this teaching mode, the teaching of theoretical knowledge in orthopedic clinical teaching is mainly carried out. As shown in Figure 2, the specific method steps are as follows:

S1: The user logs in to client 1, and client 1 records user information.

A user needs to register an account before logging into the user terminal 1 , which includes a teacher user terminal 1 and a student user terminal 1 , and the users register and operate on their respective user terminal 1 .

S2: The teacher user selects the orthopedic clinical teaching mode on the user terminal 1.

The teacher user selects the classroom teaching mode on the teacher user terminal 1 .

S3: Client 1 imports the corresponding virtual teaching scene according to the orthopedic clinical teaching mode.

According to the classroom teaching mode selected in the S2 step, the teaching scene is automatically generated, and the teaching scene (3D) has the second virtual teaching props for constructing the virtual classroom teaching scene;

Wherein, the second virtual teaching props include a virtual classroom, a virtual podium, a virtual desk, and a virtual white (black) board.

The teaching scene is pre-set at the teacher's client terminal 1, and the teacher uses the client to edit the teaching scene, and the editing of the teaching scene includes:

Adjust the color, size and position display of the teaching scene;

Addition, deletion and replacement of virtual teaching props.

According to needs, the user constructs a new second virtual teaching prop by programming the data and imports it into the computer device 2, and matches the teaching scene of the virtual classroom for editing the teaching scene of the virtual classroom.

S4: The 3D camera device 4 acquires the image information of the teacher user, and uploads it to the computer device 2 .

S5: The computer device 2 displays the image information on the virtual teaching scene to form a combined image, and displays the combined image on the three-dimensional display device 5 worn by the user.

S6: The user simulates the orthopedic clinical teaching content in the virtual teaching scene through the operation of the virtual props, and the teaching content is displayed in the form of a real-time 3D image on the worn 3D display for the user to watch and learn.

As in the hand interaction process described in Embodiment 1, the teacher user identifies and obtains the hand movement trajectory data information of the teacher through the wearing interactive device 3 (such as a data glove), and the hand movement trajectory data information generates a control instruction, The control instruction establishes a one-to-one action mapping relationship with the second virtual teaching props, and controls the operation of the second virtual teaching props (such as the operation of virtual chalk writing on a virtual blackboard), so as to achieve clinical teaching in orthopedics. Instructional operations for demonstrating the effects of actual classroom instruction.

The teacher user imports courseware or teaching video through the teacher user terminal 1, and displays it on the three-dimensional display device 5 worn by the students for students to watch.

During the orthopedic clinical teaching process, the teacher user and the student user perform interactive operations as required, and the interactive operations include:

The teacher's client terminal 1 imports questions (or question banks), and answers the questions of the student's client terminal 1;

Questions from the student client terminal 1 and answers to questions from the teacher client terminal 1.

S7: The teaching content ends, and the client 1 generates and saves a video of the orthopedic clinical teaching content.

After the orthopedic clinical teaching task is completed, an orthopedic clinical teaching image is generated, and the teaching image includes video information including:

The human body image of the teacher, the virtual classroom teaching scene and the performed virtual orthopedic clinical teaching operation are displayed.

The teacher user generates the teaching image and saves it to the user terminal 1, selects the teaching mode of the video courseware through the teacher user terminal 1, and the students can watch the teaching image by wearing the three-dimensional display device 5.

To sum up, this application provides a multi-person interactive orthopedic clinical teaching method based on mixed reality, which applies mixed reality technology to orthopedic teaching. Interaction between teachers and students, between students and students to achieve multi-person interactive teaching.

Through the teaching method, the teacher conducts simulated classroom teaching and simulated orthopedic clinical operation teaching in the mixed reality scene, and students can watch the teaching content and simulation content more intuitively and three-dimensionally, with a more immersive sense and further improve the participation of students , it is not easy to be distracted, so as to achieve the purpose of improving the training effect. At the same time, the teaching place of the teacher is not restricted, which greatly improves the convenience of online education.

The teaching method solves the traditional orthopedic clinical teaching mode limited by time and place, high cost, low efficiency and ethical and moral problems faced by human anatomy.

The present application has been introduced in detail above, and specific examples are used in this paper to illustrate the principles and implementation modes of the present application. The descriptions of the above embodiments are only used to help understand the present application and its core ideas. It should be pointed out that those skilled in the art can make some improvements and modifications to the application without departing from the principles of the application, and these improvements and modifications also fall within the protection scope of the claims of the application.

Claims (11)

1. a multi-person interactive orthopedics clinical teaching method based on mixed reality, is characterized in that, described teaching method comprises the steps: The user logs in to the client, and the client records the user information; The teacher user selects the orthopedic clinical teaching mode on the user terminal; The client imports the corresponding virtual teaching scene according to the orthopedic clinical teaching mode; The 3D camera equipment obtains the image information of the teacher user and uploads it to the computer equipment; The computer device displays the image information on the virtual teaching scene to form a merged image, and displays the merged image on the three-dimensional display device worn by the user; The user recognizes and captures the hand movement trajectory information through the interactive device worn by the user, and the hand movement trajectory information generates control instructions, and the control instructions control the operation of the virtual props in the virtual teaching scene; The user simulates the orthopedic clinical teaching content in the virtual teaching scene through the operation of virtual props, and the teaching content is displayed in the form of a real-time 3D image on the 3D display worn by the user for viewing and learning; After the teaching content is over, the user terminal generates and saves a video of orthopedic clinical teaching content. 2. The teaching method according to claim 1, wherein the orthopedic clinical teaching mode includes: classroom teaching, clinical simulation, and video courseware, and the orthopedic clinical teaching mode is selected and operated by a teacher user. 3. The teaching method according to claim 1, wherein the teaching scene comprises a virtual orthopedic operating room and a virtual classroom. 4. The teaching method according to claim 1 or 3, wherein the virtual teaching scene has essential elements of virtual teaching props, and the virtual teaching props include a first virtual teaching prop and a second virtual teaching prop; Wherein, the first virtual teaching prop is applied to the virtual orthopedic operating room teaching scene, and is an essential element for constructing a virtual orthopedic operating room teaching scene; The second virtual teaching prop is applied to the virtual classroom teaching scene and is an essential element for constructing the virtual classroom teaching scene. 5. The teaching method according to claim 4, wherein the first virtual teaching props include a virtual operating bed, a virtual orthopedic anatomical model, virtual orthopedic surgical instruments and equipment consumables. 6 . The teaching method according to claim 4 , wherein the second virtual teaching props include: a virtual classroom, a virtual podium, a virtual desk, a virtual white, and a blackboard. 7. The teaching method according to claim 1 or 4, characterized in that, in the orthopedic clinical teaching process, the teacher user edits the virtual teaching scene through the client software, and the editing includes the addition of virtual teaching props , delete, replace. 8. The teaching method according to claim 1, characterized in that, the teacher user and the student user identify and obtain hand movement trajectory data information through wearing interactive devices, and the hand movement trajectory data information generates control instructions, and the The control instruction establishes a one-to-one mapping relationship with the virtual teaching props, and controls the operation of the virtual teaching props, thereby realizing the simulation operation for displaying the actual clinical teaching effect during orthopedic clinical teaching. 9. The teaching method according to claim 1, wherein the video information contained in the three-dimensional image of the teaching content includes: Display the teacher user image information, virtual teaching scene information and virtual orthopedic clinical teaching operations performed. 10. The teaching method according to claim 1, characterized in that, during the orthopedic clinical teaching process, teacher users and student users perform interactive operations as required, and the interactive operations include: The teacher's client terminal imports questions or question banks, and answers questions from the student's client terminal; Questions from the student client, and answers to questions from the teacher client. 11. A virtual reality device for realizing the teaching method according to any one of claims 1 to 10, characterized in that the device comprises a user terminal, a computer device, an interactive device, a 3D camera device, and a three-dimensional display device.

Images