CN113359994A - Teaching content configuration and interaction scheme implementation method suitable for AR education application - Google Patents
Teaching content configuration and interaction scheme implementation method suitable for AR education application Download PDFInfo
- Publication number
- CN113359994A CN113359994A CN202110702895.3A CN202110702895A CN113359994A CN 113359994 A CN113359994 A CN 113359994A CN 202110702895 A CN202110702895 A CN 202110702895A CN 113359994 A CN113359994 A CN 113359994A
- Authority
- CN
- China
- Prior art keywords
- scene
- interaction
- teaching
- data
- client
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000003993 interaction Effects 0.000 title claims abstract description 127
- 238000000034 method Methods 0.000 title claims abstract description 58
- 230000002452 interceptive effect Effects 0.000 claims abstract description 81
- 238000013461 design Methods 0.000 claims abstract description 77
- 238000013515 script Methods 0.000 claims description 36
- 230000005540 biological transmission Effects 0.000 claims description 26
- 238000011161 development Methods 0.000 claims description 20
- 230000008569 process Effects 0.000 claims description 20
- 230000006978 adaptation Effects 0.000 claims description 9
- 238000003860 storage Methods 0.000 claims description 8
- 238000001514 detection method Methods 0.000 claims description 7
- 230000008859 change Effects 0.000 claims description 6
- 230000003044 adaptive effect Effects 0.000 claims description 5
- 239000000470 constituent Substances 0.000 claims description 4
- 230000006870 function Effects 0.000 claims description 4
- 238000010276 construction Methods 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims description 3
- 230000001960 triggered effect Effects 0.000 claims description 3
- 238000012546 transfer Methods 0.000 claims 1
- 230000003190 augmentative effect Effects 0.000 abstract description 38
- 238000009877 rendering Methods 0.000 abstract description 6
- 238000005516 engineering process Methods 0.000 description 14
- 238000012545 processing Methods 0.000 description 12
- 230000000694 effects Effects 0.000 description 11
- 238000010586 diagram Methods 0.000 description 7
- CURLTUGMZLYLDI-UHFFFAOYSA-N Carbon dioxide Chemical group O=C=O CURLTUGMZLYLDI-UHFFFAOYSA-N 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000013500 data storage Methods 0.000 description 3
- 229910002092 carbon dioxide Inorganic materials 0.000 description 2
- 239000001569 carbon dioxide Substances 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000012938 design process Methods 0.000 description 1
- 238000007654 immersion Methods 0.000 description 1
- 230000008595 infiltration Effects 0.000 description 1
- 238000001764 infiltration Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000000638 stimulation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/20—Education
- G06Q50/205—Education administration or guidance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B5/00—Electrically-operated educational appliances
- G09B5/08—Electrically-operated educational appliances providing for individual presentation of information to a plurality of student stations
- G09B5/14—Electrically-operated educational appliances providing for individual presentation of information to a plurality of student stations with provision for individual teacher-student communication
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2210/00—Indexing scheme for image generation or computer graphics
- G06T2210/61—Scene description
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Educational Technology (AREA)
- General Engineering & Computer Science (AREA)
- Educational Administration (AREA)
- Tourism & Hospitality (AREA)
- Strategic Management (AREA)
- Marketing (AREA)
- Health & Medical Sciences (AREA)
- Economics (AREA)
- General Health & Medical Sciences (AREA)
- Human Resources & Organizations (AREA)
- Human Computer Interaction (AREA)
- Primary Health Care (AREA)
- General Business, Economics & Management (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- Software Systems (AREA)
- Electrically Operated Instructional Devices (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
本发明涉及一种适用于AR教育应用的教学内容配置与交互方案实施的方法。首先，系统开发者预先在客户端进行交互方案的设计，随后由教师在服务端通过Web页面进行教学内容的配置与规范，最后将教学数据发送至客户端，由客户端采用相应的交互方案进行实施，渲染生成的虚拟场景交由学生使用。本发明采用了客户端/服务端的架构与增强现实教育应用的理念，通过服务端网页配置教学内容并在客户端实施交互方案，进而呈现出一套可供学生学习的增强现实场景，最终根据教师的意愿对学生所学习的增强现实课程进行自主设计与管理。
The present invention relates to a method for teaching content configuration and interaction scheme implementation suitable for AR education applications. First, the system developer designs the interaction scheme on the client side in advance, then the teacher configures and standardizes the teaching content on the server side through the Web page, and finally sends the teaching data to the client side, and the client side adopts the corresponding interaction scheme to carry out Implementation, rendering the generated virtual scene for students to use. The invention adopts the client/server architecture and the concept of augmented reality educational application, configures the teaching content through the server web page and implements the interactive scheme on the client, and then presents a set of augmented reality scenes for students to learn, and finally according to the teacher Willingness to independently design and manage the augmented reality courses that students learn.
Description
Technical Field
The invention relates to a method for configuring teaching contents and implementing an interaction scheme, which is suitable for AR education application.
Background
The existing augmented reality education related patents:
1. patent numbers: CN201911221301.6, patent name: a portable augmented reality medical image observation auxiliary device based on a smart phone.
This patent provides a portable augmented reality medical image observation auxiliary assembly based on smart mobile phone, through the built-in data processing system of equipment, presents the image source that the smart mobile phone screen demonstrates and object and reality object in the real environment simultaneously in people's eye, reaches augmented reality's interactive effect. The patent focuses on medical image processing and presentation, but the information data and the AR effect are all realized by auxiliary equipment, which undoubtedly increases the weight and operation burden of the equipment. The scheme designed by the invention is an architecture mode with separated front and back ends, and the storage of most resources and the processing of data are realized at the server end, so that the volume of the application program of the client end is reduced, the operation expense of hardware equipment of the client end is reduced, and the popularization of the application program is facilitated.
2. Patent numbers: CN201911049722.5, patent name: an intelligent classroom based on augmented reality technology.
This patent provides a wisdom classroom based on augmented reality technique, realizes the 3D dynamic effect on the mobile device through technologies such as augmented reality, image recognition and big data, provides better study experience for the student through forms such as animation, little class, simulation model's interactive operation. This patent is focused on the interactive realization effect of AR, and the interactive scheme of the teaching of customization also loses the flexibility to a certain extent, can't satisfy the demand that the teacher adjusted to having the scheme at any time promptly. The AR interaction effect is designed in a universal mode, a teacher can directly modify the interaction scheme in time through a Web page of a server side, and application re-development is not needed through a special teaching content producer. This can greatly improve the maneuverability of teaching adjustment, and simultaneously embody the diversity of AR interaction effect. In addition, the customization cost of the teaching contents is reduced, and the manufacturing period of the teaching contents is shortened.
Augmented Reality (AR) technology is a technology that combines digital information of a computer with a real environment, and by superimposing virtual objects with a real scene, an experiencer is completely immersed in a virtual-real combined situation. The immersion experience, the combination of virtuality and reality and the infiltration learning of the technology enable more and more people to apply the technology to the field of education. The education experts breed a new form of teaching method by combining AR technology with education, thereby improving the teaching quality.
The prior art solution has the following two problems with respect to how to configure the storage, processing and presentation of teaching contents in the augmented reality education application system:
(1) in order to enhance the effect of audio-visual perception sensory stimulation of interactive experience, most of the existing augmented reality education application systems are developed and manufactured in a closed manner according to the specific teaching needs of teachers, and the potential requirements of the teachers on re-innovation of teaching contents are difficult to be met from the aspect of visitation. The reason for this is that the teacher's design opinions on the teaching contents can only be presented before the application development is completed. After the application is customized and put into teaching use, when problems occur and the teaching content needs to be adjusted, the teacher can only provide new requirements for readjustment and deployment of the system developer. The process greatly increases the development time and the use cost of the augmented reality education application system, and limits the flexible implementation of the teaching strategy from another angle.
(2) Most of the existing augmented reality education application systems adopt an off-line mode, namely, data support of a cloud terminal is not needed, and the configuration and interaction of teaching contents cannot be modified in time at a client. The storage and processing of a lot of teaching contents are distributed in the client, so that on one hand, the client program is relatively bloated; on the other hand, the task execution efficiency of the mobile terminal with limited hardware resources is reduced due to the parallel data processing and rendering, and the use experience of the user is influenced. In addition, the development situation of education digitization promotes the close combination of new technologies such as artificial intelligence and big data and the like and the education field, the technologies rely on the hardware support of distributed computing and high-performance data processing, and from the viewpoint of technical iteration and evolution, the inherent defects of the original teaching content configuration and the implementation of the interactive scheme further need the data processing capability of the cloud end to share the operation cost of the client end.
Disclosure of Invention
The invention aims to provide a method for configuring teaching contents and implementing an interaction scheme, which is suitable for AR education application, and is used for presenting the work of constructing a virtual learning scene in a mode of filling a Web page, helping a teacher to construct the virtual learning scene and facilitating the subsequent adjustment of the scene.
In order to achieve the purpose, the technical scheme of the invention is as follows: a method for configuring teaching contents and implementing an interactive scheme suitable for AR education application includes designing an interactive scheme at a client in advance by a system developer, configuring and standardizing the teaching contents at a server by a teacher through a Web page, sending teaching data to the client, implementing the teaching contents by the client through a corresponding interactive scheme, and rendering a generated virtual scene for students to use.
In an embodiment of the present invention, the configuration and specification of the teaching content include the location where the teaching resources including pictures, videos, texts and models are stored, and the processing manner of these data includes the provision of data, the specification of data and the transmission of data.
In an embodiment of the invention, Unity3d is used as a development engine for designing the interactive scheme and implementing the interactive scheme.
In an embodiment of the invention, the design of the interaction scheme is that a system developer designs an interaction scheme for each scene learned by students in a client in advance, and the design comprises layout design and interaction design of virtual elements in the scene, wherein the layout design of the virtual elements refers to planning of positions and sizes of the elements and composition contents of a chartlet in the virtual scene, and the interaction design of the virtual elements represents how the composition elements in the scene interact with the students; designing an interaction scheme in different scenes by a system developer according to different teaching requirements; and after the system developer sets up the interaction scheme of the teaching system client, a corresponding data interface is provided for the server.
In an embodiment of the present invention, implementation of an interactive scheme, that is, deployment of an interactive scheme refers to deployment and presentation of teaching contents on each designed interactive scheme of a client after configuration and specification of the teaching contents; the deployment process comprises three parts of resource transmission, resource loading and resource presentation.
In one embodiment of the invention, the client side is provided with an education application system, after the development of an interaction scheme design stage and a teaching content configuration stage of the education application system is finished, a system developer provides a front-end Web interface at the client side, a teacher fills and stores course contents according to the guidance of the interface, and the client side and the server side establish data connection through Http; the students use the client of the education application system to carry out virtual learning; in addition, the teacher can also modify the data of the existing teaching content through the Web page, so as to update the interactive content in the virtual scene of the client.
In one embodiment of the invention, the design of the interaction scheme is that each learning Scene is used as a basic design unit, and two parts of Scene element layout and Scene element interaction are designed in a Scene; the scene element layout comprises three parts, namely element selection, element state setting and UI hierarchy setting; scene element interaction is to carry out interactive design and script mounting on the selected elements; scene element layout and scene element interaction of the learning scenes jointly form a scene template, and the mutual switching of the learning scenes corresponds to the selection of different scene templates; the concrete implementation is as follows:
(1) scene element layout
Firstly, a system developer selects constituent elements in a learning scene, and selects a plurality of expression forms including models, videos, buttons and pictures; then, setting parameters of the states of the elements in the scene, and after the setting of the element positions is completed, setting parameters related to the element states including rotation angles, display sizes and rigid body attributes; finally, UI hierarchy setting needs to be carried out on the elements with the set state parameters, the higher the hierarchy is, the higher the UI display priority is, and the shielding priority order among the elements in the picture is embodied, namely, the elements with low priority are shielded by the elements with high priority; after the layout design of scene elements is completed, a complete element composition structure is built in a learning scene;
(2) scene element interaction
The scene element interaction is to use the script of the Unity3d to carry out interactive design on elements in a learning scene, and mount the written script on the corresponding interactive elements, so as to construct a complete scene interaction design scheme; the student interacts with elements in the learning scene in a preset operation mode, wherein the operation mode comprises rotation, zooming and double-click operations; after different interaction modes are selected for the interaction script, the interaction script is mounted to the corresponding scene element, so that the element is triggered by a user and then presents a preset interaction state; a system developer designs program interfaces of various interaction modes in advance at a client, so that the subsequent calling of the interfaces is facilitated to realize the corresponding mounting of scripts and elements;
(3) scene template and learning scene switching
After the element layout and the element interaction design in a learning scene are finished, the construction of a scene template is finished; the scene template is a set of integral scene presentation scheme formed by the layout of all elements in a learning scene and the interaction among the elements;
after all element layouts and element interactions in one scene template are set, a system developer designs and customizes collocation and interaction schemes of different elements in different scenes in advance in a scene template mode according to course requirements of a teacher; a teacher configures specific teaching contents for a scene template designed by a system developer on a Web page of a server, and different scene templates are selected at the server to be equal to different virtual learning scenes established at a client;
after all element layouts and element interactions in a scene template are set, a system developer also needs to define a switching relation between learning scenes; scene switching is equivalent to selecting different scene templates for teachers and system developers; for students, it appears as a change in the virtual learning scenario; a learning scenario may select a template from a plurality of selectable scenario templates as an element layout and interaction scheme adopted by the current scenario; the teacher sets the corresponding relation between the learning scene and the scene template on the Web page of the server according to the teaching requirement of the teacher and configures the content of the corresponding scene template, and the students can click corresponding buttons in the scene according to the learning requirement to realize the change of the learning scene.
In an embodiment of the present invention, implementation of an interaction scheme, that is, deployment of the interaction scheme, that is, a client establishes transmission with a server through Http, and after downloading a resource to a local end, the deployment process of the interaction scheme is implemented through two stages, namely, resource loading and resource presentation, which are specifically as follows:
(1) resource transmission phase
The client side needs to acquire and download data of teaching contents configured by a teacher at the server side from the server side; firstly, a client sends a POST request to a server to acquire a URL (Uniform resource locator) address corresponding to data for downloading teaching contents; then, the client side carries out resource transmission under an Http transmission protocol by calling an interface encapsulated in the UntiyWebRequest class, model resources are downloaded into streamAssetsBefiles of the client side in a binary stream transmission mode, and preparation is made for loading and presenting subsequent resources in a learning scene;
(2) resource loading phase
Loading the teaching elements downloaded in the teaching content of the client, namely setting the state parameters of the elements in the scene template; loading two-dimensional resources including videos, characters and music can complete state parameter setting of elements only by calling corresponding element loading program interfaces and utilizing a two-dimensional component self-adaption function integrated by the Unity3 d; the three-dimensional model self-adaptation is to call a Load method for resource loading of the model stored in the streaming assets folder, Instantiate the loading model by an instantiation method, and automatically adjust the model to be in a proper size and position by a collision detection method;
(3) resource presentation
The resource presenting part comprises two steps of mounting of the interaction script and presenting of the scene elements; the client reads the interactive information and calls an interface for mounting the interactive script, wherein the interactive information refers to a part of the resource transmission teaching content; the mounting script interface can realize the accurate mounting of the scene elements and the interactive mode only by inputting the corresponding relation parameters of the interactive script and the scene elements; then, the scene elements of the mounted interactive script are displayed at the expected positions of the scene template after the element layout setting is finished; and finally, presenting data configured by the teacher at the server in a scene template designed by the interactive scheme, and constructing a virtual learning scene for students for the students to experience.
In an embodiment of the present invention, the configuration and specification of the teaching content is that a teacher logs in a web page, teaching data is provided according to three steps of scene selection, data provision and data specification, then a server performs automatic specification processing on the data provided by the teacher and stores the data in a database, and when a data request of a client occurs, corresponding data is sent, which is specifically as follows:
(1) selection of scenes
A system developer presets a virtual scene capable of interacting according to the requirement of a teacher at the beginning of teaching application development, namely, the system developer is equivalent to a frame for providing teaching contents for the teacher; the teacher can provide corresponding teaching data according to different scenes in the subsequent teaching content configuration process, so that different interactive scenes with the teaching data are displayed at the client;
(2) provision of data
A teacher selects a scene template, and then provides a specific data server program including teaching contents in various forms including characters, pictures, videos, models and sounds according to the requirements of data required to be provided by different templates to establish a mapping relation between a real path and a virtual path convenient for database storage, and when the specified contents need to be called, the real path is directly provided for a background virtual path, so that the teaching contents can be read;
for three-dimensional model data required to be provided by a teacher, a model instance needs to be newly established, and course information, model information and interaction information are filled in; when the three-dimensional model is selected, if the example model selected by the teacher is the model existing in the database, the client generates an interactive model according to the model information contained in the example when reading the example information provided by the teacher; if the teacher needs to upload a new model as an example model, the teacher needs to designate the example model as other models, and the example is created; after the model is created, uploading corresponding information including a model file, a model preview and information indicating the length, the width and the height of the model on an editing page of a corresponding example; after a teacher designs a model instance through a page, a server stores model files uploaded by the teacher to a corresponding folder and stores model information to a database;
(3) specification of data
The format of data required to be provided by a teacher needs to be standardized, including the standardization of the format of a received picture, the format of a received video and the format of a received model file; aiming at uploading text teaching contents, the specified constraint conditions are that blank contents cannot be uploaded and the number of characters cannot exceed the preset number;
(4) transmission of data
Opening data services provided by a server program to a webpage and a client in a program interface mode; when the server receives the data request, the access operation to the database is automatically carried out, and after the records are read, the records are automatically packaged into json strings and sent to the data requester.
Compared with the prior art, the invention has the following beneficial effects:
1. the invention considers the balance problem of the universality and the diversity of teaching content configuration and interaction schemes when a system developer constructs an augmented reality education application program. The teacher can configure teaching contents according to own will, and the autonomy of the teacher to the customized augmented reality course is greatly improved. In addition, after the interactive scheme is deployed, on one hand, teachers can still edit teaching contents at any time, and on the other hand, models and interactive modes in existing augmented reality courses can be customized again by proposing new requirements to system developers. The novel solution not only breaks the limitation of customization of most of existing augmented reality education application systems on the market, but also keeps the diversity of teaching content interaction, also shortens the development period of teaching application programs, provides a new solution for teachers to participate in the design and adjustment of augmented reality courses, and further improves the teaching effect of the augmented reality education auxiliary system.
2. The method for implementing the teaching content configuration and the interaction scheme is beneficial to light deployment of the client. The teacher can directly configure the teaching contents through the webpage of the server side, and can participate in customizing and editing the learning scene without mastering the Unity3d development technology, so that the convenience degree of the teacher in designing and configuring the teaching contents is greatly improved. In addition, the data storage and analysis operation of the teaching contents are mainly completed by the server side, the client side and the server transmit data through an Http protocol, and the design mode of 'thin client side and fat server side' is convenient for the popularization of the augmented reality education application system at the present of network popularization.
Drawings
FIG. 1 is a flowchart of a method performed by an embodiment of a teaching content configuration and interaction scheme for AR education applications according to the present invention;
FIG. 2 is a diagram of an educational application system with a role relationship;
FIG. 3 is a logic diagram of an interaction scheme design;
FIG. 4 is a schematic view of a scene cut;
FIG. 5 is a schematic diagram of the design and configuration of teaching content;
FIG. 6 is a flow chart of data provision;
FIG. 7 is a flow chart of three-dimensional model upload and storage;
fig. 8 is a diagram illustrating server-side data transmission;
FIG. 9 is an interaction scenario deployment flow diagram;
FIG. 10 is a flow chart of resource transmission;
FIG. 11 is a first flowchart of three-dimensional model adaptation;
FIG. 12 is a second flowchart of three-dimensional model adaptation;
FIG. 13 is a schematic diagram of model adaptation, (a) before model adaptation, (b) after model adaptation;
FIG. 14 is a schematic diagram of script mounting and resource rendering.
Detailed Description
The technical scheme of the invention is specifically explained below with reference to the accompanying drawings.
The invention relates to a method for configuring teaching contents and implementing an interactive scheme, which is suitable for AR education application.
The following is a specific implementation of the present invention.
The technical scheme of the invention is as shown in figure 1, an education application system is constructed by adopting a C/S (client and server) architecture, and by using the system, teachers can customize augmented reality education application programs without learning professional software. The method and the system display the work of constructing the virtual learning scene in a mode of filling in a Web page, help teachers build the virtual learning scene and facilitate subsequent adjustment of the scene.
The execution flow of the invention is shown in figure 1, firstly, a system developer designs an interactive scheme at a client in advance, then a teacher configures and standardizes teaching contents at a server through a Web page, finally teaching data is sent to the client, the system adopts a corresponding interactive scheme for implementation, and a virtual scene generated by rendering is handed to students for use. Based on the above process, the general technical scheme of the invention is divided into two parts, namely the configuration of the teaching content and the design and implementation of the interaction scheme.
(a) The configuration of the teaching content, namely the position where teaching resources such as pictures, videos, characters and models are stored, and the data processing mode, namely the processing comprises the data providing, the data specification and the data sending.
(b) The design and implementation of the interactive scheme of the invention select Unity3d as a development engine, which comprises a front stage and a rear stage, which are respectively the design of the interactive scheme and the deployment of the interactive scheme.
i. The design of the interactive scheme is that a system developer designs the interactive scheme for each scene learned by students in a client in advance, the interactive scheme comprises layout design and interactive design of virtual elements in the scene, and the layout design refers to planning of the positions, sizes, chartlets and other composition contents of the elements in the virtual scene. The interaction design represents how the constituent elements in the scene are designed to interact with the student, such as rotation, zooming, clicking and other interaction operations. And (3) designing an interaction scheme in different scenes by a developer according to different teaching requirements. And after the system developer sets up the interaction scheme of the teaching system client, a corresponding data interface is provided for the server.
And ii, the deployment of the interactive scheme refers to the deployment and the presentation of the teaching contents on each designed scheme of the client after the configuration and the specification of the teaching contents. The deployment process comprises three parts of resource transmission, resource loading and resource presentation.
1. Relationship of educational application systems to roles
As shown in fig. 2, in the early stage of development of the education application system, the teacher submits the requirement of the content of the course to the system developer, and the developer performs design and development of the server and the client of the education application system according to the requirement of the teacher. After the interactive scheme design stage and the teaching content configuration stage of the application system are both developed, a system developer provides a front-end Web interface at a client, a teacher fills and stores course content according to the guidance of the interface, and the client and a server establish data connection through Http. The students use the clients of the education application system for virtual learning. In addition, the teacher can also modify the data of the existing teaching content through the Web page, so as to update the interactive content in the virtual scene of the client.
2. Design of interaction scheme
As shown in fig. 3, the system developer designs an interaction scheme for an educational application by using each learning Scene as a basic design unit, and designs two parts, namely a Scene element layout and a Scene element interaction, in a Scene. The scene element layout comprises three parts, namely element selection, element state setting and UI hierarchy setting; the scene element interaction is to carry out interactive design and script mounting on the selected elements. The scene element layout and scene element interaction of the learning scene jointly form a scene template, and the mutual switching of the learning scenes corresponds to the selection of different scene templates. Each link in the interactive design process is discussed in detail below.
(1) Scene element layout
Firstly, a system developer selects constituent elements in a learning scene, and can select various expression forms such as models, videos, buttons and pictures. Then, the state of these elements in the scene is subjected to parameter setting, and the relative coordinates of the element in fig. 3 are the coordinate positions of the element relative to the parent element, for the position setting of the dependent element. And the world coordinate is the coordinate position of the element in the whole learning scene and is used for positioning the position of the parent element. After the setting of the element position is completed, parameters related to the state of other elements such as the rotation angle, the display size, and the rigid body attribute need to be set. Finally, UI hierarchy setting needs to be performed on the elements with the set state parameters, the higher the hierarchy is, the higher the UI display priority is, and the occlusion priority order among the elements in the picture is embodied, that is, the elements with low priority are occluded by the elements with high priority.
After the layout design of the scene elements is completed, a complete element composition structure is built in the learning scene, and the design of the interaction script and the mounting of the element script are carried out subsequently.
(2) Scene element interaction
The scene element interaction is to use the script of Unity3d to carry out interactive design on elements in the learning scene, and mount the written script on the corresponding interactive elements, so as to construct a complete scene interaction design scheme. The interactive mode of the invention refers to that students interact with elements in a learning scene through a specific operation mode, such as rotating, zooming, double-clicking and the like. After different interaction modes are selected for the interaction script, the interaction script is mounted to the corresponding scene element, so that the element is triggered by a user and then presents a preset interaction state. Considering the problem of element and interaction universality, some interaction modes with universality can be selected, such as double-click operation of a button, development and gradual effect of a picture, rotation and scaling effect of a model and the like. A system developer designs program interfaces of various interaction modes in advance at a client, and the subsequent calling of the interfaces is facilitated to realize the corresponding mounting of scripts and elements.
(3) Scene template and learning scene switching
After the element layout and the element interaction design in a learning scene are finished, the construction of a scene template is finished. The scene template refers to a set of overall scene presentation schemes formed by the layout of all elements in a learning scene and the interaction among the elements.
The template customization and calling relation is shown in fig. 2, so that a teacher can conveniently and quickly arrange a learning scene when building teaching contents, and a system developer designs and customizes collocation and interaction schemes of different elements in different scenes in advance in a scene template mode according to the course requirements of the teacher. A teacher configures specific teaching contents for a scene template designed by a system developer on a Web page of a server, and different scene templates are selected at the server, which is equal to the establishment of different virtual learning scenes at a client, so that the interestingness of course knowledge is greatly improved, and the initiative of students participating in learning is mobilized.
As shown in fig. 4, after all element layouts and element interactions in a scene template are set, the system developer needs to define the switching relationship between learning scenes. Scene switching is equivalent to selecting different scene templates for teachers and system developers; it appears to the student as a change in the virtual learning scenario. A learning scenario may select a template from a plurality of selectable scenario templates as an element layout and interaction scheme to be adopted by the current scenario. The teacher sets the corresponding relation between the learning scene and the scene template on the Web page of the server according to the teaching requirement of the teacher and configures the content of the corresponding scene template, and the students can click corresponding buttons in the scene according to the learning requirement to realize the change of the learning scene and realize more comprehensive and deep knowledge.
3. Configuration of instructional content
The design and configuration of the teaching content are shown in fig. 5, wherein the configuration of the teaching content is located at the server side, and the configuration process is from two to eight. After the teacher logs in the webpage, the configuration of the teaching content can be carried out. Firstly, a system developer designs a learning scene, then a teacher logs in a webpage, and provides teaching data according to three steps of scene selection, data provision and data specification. And then the server side automatically standardizes the data provided by the teacher and stores the data in the database, and when a data request of the client side occurs, the server side sends corresponding data. And finally, the client requests the scene data and then requests the teaching data, thereby laying a foundation for the deployment of the next interaction scheme.
(1) Selection of scenes
The system developer presets interactive virtual scenes according to the requirements of teachers at the beginning of application development, and the step is equivalent to providing a frame of teaching contents for the teachers. The teacher can provide corresponding teaching data according to different scenes in the subsequent teaching content configuration process, so that different interactive scenes with the teaching data are displayed on the client.
(2) Provision of data
As shown in fig. 6, the teaching content provided by the teacher at the server needs to match the design of the interaction scheme of the learning scenario at the client, so that a scenario corresponds to a table in the database and also corresponds to a web page that the teacher can operate. The teacher data providing includes both the operation of adding, deleting, modifying and checking the existing teaching content and the operation of uploading the new teaching content data. The teacher selects the scene template first, and then provides specific data including teaching contents in various forms such as characters, pictures, videos, models and sounds according to the requirements of the data required to be provided by different templates. When providing the data of the teaching contents using the web page, the teacher fills the teaching contents according to the data structure of the learning scene preset by the system developer. During data storage, the text content can be directly stored in the data table, unstructured data such as pictures, videos and models are stored in corresponding records of the database in the form of virtual paths in the server, and files uploaded by teachers are stored in the paths of the real folders. The server program is responsible for establishing a mapping relation between the real path and the virtual path convenient for database storage, and when the designated content needs to be called, the designated content is directly provided for the background virtual path, so that the teaching content can be read.
For the three-dimensional model class data that the teacher needs to provide, as shown in fig. 7. A new model instance is required to be created, and course information, model information and interaction information are filled in. When the three-dimensional model is selected, if the example model selected by the teacher is the model existing in the database, the client generates an interactive model according to the model information contained in the example when reading the example information provided by the teacher; and if the teacher needs to upload a new model as an example model, the teacher needs to designate the example model as other models, and the example is created. And after the model file is created, uploading corresponding information such as the model file, the model preview and the information for indicating the length, the width, the height and the like of the model on an editing page of the corresponding example. If difficulty is encountered in uploading the new model, a help page can be opened for reference. After the teacher designs the model instance through the page, the server stores the model file uploaded by the teacher to the corresponding folder and stores the model information to the database.
(3) Specification of data
Because the storage space of the server is limited, files such as pictures, videos and models provided by teachers are not recommended to exceed 100MB in size, and the server-side program can limit the size of the files. For the convenience of reading of the client, the format required to be provided by the teacher needs to be specified, the format of the received picture is png \ jpg \ gif and the like, the format of the received video is mp4 only, and the format of the received model file is fbx only. For uploading text teaching contents, the specified constraint conditions are that blank contents cannot be uploaded and the number of characters cannot exceed 300. After the teacher uploads the data of the teaching content, before the data is stored in the database, the server program can mark information such as date and name on the data record to be generated, so that the subsequent data editing operation of the teacher is facilitated. After the specification of the web page of the server and the specification of the server are carried out, the server stores the information into a database in a corresponding data structure for the calling of the client.
(4) Transmission of data
As shown in fig. 8. And for the data service provided by the server program, the data service is opened to the webpage and the client in the form of program interface. When the server receives a data request (generally in the form of a json string), the access operation to the database is automatically carried out, and after the records are read, the records are automatically packaged into the format of the json string and sent to a data requester.
4. Deployment of interaction schemes
After the configuration of the teaching content is completed, each element in the interactive scheme has a data source as a support. As shown in fig. 9, the teaching content data transmitted by the server includes teaching elements and interactive information, the teaching elements are scene element portions, such as models, videos, and texts, required to be filled in the teaching template in the interactive scheme design, and the interactive information is an interactive mode corresponding to the elements. The client establishes transmission with the server through Http, and after downloading the resources to the local end, the deployment process of the interactive scheme is realized through two stages of resource loading and resource presentation.
(1) Resource transmission phase
The client needs to acquire and download the data of the teaching content configured by the teacher at the server from the server. As shown in fig. 10, the resource transmission flowchart firstly sends a POST request to the server, and acquires a URL address corresponding to data for downloading teaching content. And then, the client transmits the resources under the Http transmission protocol by calling an interface encapsulated in the UntiyWebRequest class, and downloads the model resources into the streamAssetsFolders of the client in a binary stream transmission mode to prepare for the loading and presentation of the subsequent resources in the learning scene.
(2) Resource loading phase
In the stage, the teaching elements downloaded in the teaching content of the client are loaded, namely, the state parameters of the elements in the scene template are set. The loading of two-dimensional resources such as videos, characters, music and the like can be completed by only calling corresponding element loading program interfaces and utilizing the two-dimensional component self-adaption function integrated by the Unity3 d. However, Unity3d cannot adaptively set the size and position of the three-dimensional model uploaded by the teacher from the server, and adaptive means that the size and position of the three-dimensional model can be displayed in the learning scene with the parameters of the desired design. Fig. 11 shows a flow chart of the three-dimensional model self-adaptation, in which a Load method is called for resource loading of a model stored in a StreamingAssets folder, a loading model is instantiated by an instantate method, and the model is automatically adjusted to be in an appropriate size and position by a collision detection method.
As shown in fig. 12, the adaptive process is implemented by placing the model in a collision detection adaptive Box (Box) and assigning rigid body and collision body attributes to the model. And detecting whether the size of the model exceeds the expectation through the collision between the edge of the model and the Box, and if so, continuously adjusting the size of the model until the collision is not detected any more.
The self-adaptive process is implemented by setting the position of the model as the central position of the collision body, and setting the size of the collision body according to a certain ratio through the length, the width and the height of the model acquired from the server. Then, a collision detection process is started, and for an oversized model, if the oversized model collides with Box, the size of the original model is reduced by 0.1 Unity scale unit (hereinafter referred to as "unit"), and if the oversized model still is in a collision condition, the size is reduced by 0.1 unit. And continuously detecting the collision detection function until no collision occurs, and finishing the self-adaptive process. For an undersized model, no collision with Box occurs during initialization, so the model needs to be enlarged until a collision is detected. For the size of the critical collision object, the size is reduced by 0.1 unit, and the adaptive process can be ended and presented in the interactive scene.
As shown in fig. 13, fig. 13(a) shows the size and position of the carbon dioxide model before adaptation, the gray cube is the collision detection Box, and after the model adaptation process is performed, as shown in fig. 13(b), the size and position of the carbon dioxide model are in the expected setting state after adaptation.
(3) Resource presentation
The resource presenting part comprises two steps of mounting of the interaction script and presenting of the scene element. Script mounting and resource rendering As shown in FIG. 14, a client of an educational application reads interactive information and invokes an interface on which to mount an interactive script, the interactive information being a part of the resource transmission tutorial in FIG. 9. The mounting script interface can realize the accurate mounting of the scene elements and the interactive mode only by inputting the corresponding relation parameters of the interactive script and the scene elements. Subsequently, the scene element mounting the interaction script will appear at the expected position of the scene template after the element layout setting is completed. And finally, presenting data configured by the teacher at the server in a scene template designed by the interactive scheme, and constructing a virtual learning scene for students for the students to experience. The students can perform rich interactive experience in virtual learning scenes presented by the teacher in the scene template designed by the Web page of the server, so that the diversity of learning modes of the students is improved, and an efficient and convenient mode is provided for the teacher in designing and teaching processes.
The invention provides a new method for configuring teaching contents and implementing an interactive scheme, which is suitable for augmented reality education application from the perspective of teaching crowds. The method has the following advantages:
the invention considers the balance problem of the universality and the diversity of teaching content configuration and interaction schemes when a system developer constructs an augmented reality education application program. The teacher can configure teaching contents according to own will, and the autonomy of the teacher to the customized augmented reality course is greatly improved. In addition, after the interactive scheme is deployed, on one hand, teachers can still edit teaching contents at any time, and on the other hand, models and interactive modes in existing augmented reality courses can be customized again by proposing new requirements to system developers. The novel solution not only breaks the limitation of customization of most of existing augmented reality education application systems on the market, but also keeps the diversity of teaching content interaction, also shortens the development period of teaching application programs, provides a new solution for teachers to participate in the design and adjustment of augmented reality courses, and further improves the teaching effect of the augmented reality education auxiliary system.
The method for implementing the teaching content configuration and the interaction scheme is beneficial to light deployment of the client. The teacher can directly configure the teaching contents through the webpage of the server side, and can participate in customizing and editing the learning scene without mastering the Unity3d development technology, so that the convenience degree of the teacher in designing and configuring the teaching contents is greatly improved. In addition, the data storage and analysis operation of the teaching contents are mainly completed by the server side, the client side and the server transmit data through an Http protocol, and the design mode of 'thin client side and fat server side' is convenient for the popularization of the augmented reality education application system at the present of network popularization.
The method can be used for theoretical research of the augmented reality teaching technology, and provides a new solution for exploring generalization and systematization of the augmented reality teaching technology in the engineering application process. In addition, the augmented reality teaching application used by the novel design method is a typical network application, is divided into a client and a server, can help a system developer to rapidly develop a whole set of augmented reality education application system, provides more augmented reality course design choices for teachers, also obtains richer course learning experience for students, and fully plays the role of the augmented reality technology in the education field; the existing teaching contents are edited through the webpage of the server, so that the later maintenance and the updating of the developed teaching system are facilitated, and a novel technical scheme is provided for the generalization development of the augmented reality education application.
The above are preferred embodiments of the present invention, and all changes made according to the technical scheme of the present invention that produce functional effects do not exceed the scope of the technical scheme of the present invention belong to the protection scope of the present invention.
Claims (9)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110702895.3A CN113359994B (en) | 2021-06-24 | 2021-06-24 | Method for teaching content configuration and interaction scheme implementation suitable for AR educational applications |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110702895.3A CN113359994B (en) | 2021-06-24 | 2021-06-24 | Method for teaching content configuration and interaction scheme implementation suitable for AR educational applications |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113359994A true CN113359994A (en) | 2021-09-07 |
| CN113359994B CN113359994B (en) | 2022-09-13 |
Family
ID=77536195
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110702895.3A Active CN113359994B (en) | 2021-06-24 | 2021-06-24 | Method for teaching content configuration and interaction scheme implementation suitable for AR educational applications |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113359994B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114237389A (en) * | 2021-12-06 | 2022-03-25 | 华中师范大学 | Holographic imaging-based in-situ induction forming method in enhanced teaching environment |
| CN116091738A (en) * | 2023-04-07 | 2023-05-09 | 湖南快乐阳光互动娱乐传媒有限公司 | Virtual AR generation method, system, electronic equipment and storage medium |
| CN116360593A (en) * | 2023-03-10 | 2023-06-30 | 深圳职业技术学院 | Teaching method, system and medium of immersion teaching plan system based on XR technology |
| CN116563462A (en) * | 2023-05-09 | 2023-08-08 | 南京强者科技有限公司 | Three-dimensional simulation system and method for equipment training |
| TWI828527B (en) * | 2023-01-19 | 2024-01-01 | 王一互動科技有限公司 | Method for automatic guide and electronic apparatus |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101650749A (en) * | 2009-09-02 | 2010-02-17 | 上海理工大学 | Dynamic dragging and constructing module system with 3D virtual element for on-line exhibition |
| CN103606310A (en) * | 2013-10-29 | 2014-02-26 | 中兴通讯股份有限公司 | Teaching method and system |
| CN104732826A (en) * | 2015-03-27 | 2015-06-24 | 宋晓宇 | Augmented reality (AR) interactive teaching material and teaching resource cloud platform |
| CN110503582A (en) * | 2019-07-16 | 2019-11-26 | 王霞 | Cloud system is educated based on the group interaction of mixed reality and multidimensional reality technology |
| US20210027645A1 (en) * | 2019-07-26 | 2021-01-28 | Jody Sarah Nicol | Collaborative Educational E-Learning Multi and Single Device, Supplemental Pedagogical Data Management UX/UI System Technology Platform Using Immersive Interactive Mixed Reality |
| CN112558759A (en) * | 2020-11-30 | 2021-03-26 | 苏州端云创新科技有限公司 | Education-based VR (virtual reality) interaction method, interaction development platform and storage medium |
-
2021
- 2021-06-24 CN CN202110702895.3A patent/CN113359994B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101650749A (en) * | 2009-09-02 | 2010-02-17 | 上海理工大学 | Dynamic dragging and constructing module system with 3D virtual element for on-line exhibition |
| CN103606310A (en) * | 2013-10-29 | 2014-02-26 | 中兴通讯股份有限公司 | Teaching method and system |
| CN104732826A (en) * | 2015-03-27 | 2015-06-24 | 宋晓宇 | Augmented reality (AR) interactive teaching material and teaching resource cloud platform |
| CN110503582A (en) * | 2019-07-16 | 2019-11-26 | 王霞 | Cloud system is educated based on the group interaction of mixed reality and multidimensional reality technology |
| US20210027645A1 (en) * | 2019-07-26 | 2021-01-28 | Jody Sarah Nicol | Collaborative Educational E-Learning Multi and Single Device, Supplemental Pedagogical Data Management UX/UI System Technology Platform Using Immersive Interactive Mixed Reality |
| CN112558759A (en) * | 2020-11-30 | 2021-03-26 | 苏州端云创新科技有限公司 | Education-based VR (virtual reality) interaction method, interaction development platform and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 汤显峰、沈丽燕、董榕、杨玉辉、张宇燕: "基于云渲染的VR/AR智慧教室的设计与应用", 《现代教育技术》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114237389A (en) * | 2021-12-06 | 2022-03-25 | 华中师范大学 | Holographic imaging-based in-situ induction forming method in enhanced teaching environment |
| TWI828527B (en) * | 2023-01-19 | 2024-01-01 | 王一互動科技有限公司 | Method for automatic guide and electronic apparatus |
| CN116360593A (en) * | 2023-03-10 | 2023-06-30 | 深圳职业技术学院 | Teaching method, system and medium of immersion teaching plan system based on XR technology |
| CN116360593B (en) * | 2023-03-10 | 2023-11-24 | 深圳职业技术学院 | Teaching methods, systems and media of immersive lesson plan system based on XR technology |
| CN116091738A (en) * | 2023-04-07 | 2023-05-09 | 湖南快乐阳光互动娱乐传媒有限公司 | Virtual AR generation method, system, electronic equipment and storage medium |
| CN116563462A (en) * | 2023-05-09 | 2023-08-08 | 南京强者科技有限公司 | Three-dimensional simulation system and method for equipment training |
| CN116563462B (en) * | 2023-05-09 | 2023-12-15 | 南京强者科技有限公司 | Three-dimensional simulation system and method for equipment training |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113359994B (en) | 2022-09-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113359994B (en) | Method for teaching content configuration and interaction scheme implementation suitable for AR educational applications | |
| US6466239B2 (en) | Method and apparatus for editing data used in creating a three-dimensional virtual reality environment | |
| CN109166374B (en) | Teaching system based on virtual reality technology | |
| US12266273B2 (en) | Augmented reality platform for collaborative classrooms | |
| US6654785B1 (en) | System for providing a synchronized display of information slides on a plurality of computer workstations over a computer network | |
| US6552721B1 (en) | Graphic data generating apparatus, graphic data generation method, and medium of the same | |
| CN107632824B (en) | A kind of generation method of augmented reality module, generating means and generate system | |
| US6401237B1 (en) | Method and apparatus for editing data used in creating a three-dimensional virtual reality environment | |
| CN103258338A (en) | Method and system for driving simulated virtual environments with real data | |
| CN107294838A (en) | Animation producing method, device, system and the terminal of social networking application | |
| Adão et al. | A rapid prototyping tool to produce 360 video-based immersive experiences enhanced with virtual/multimedia elements | |
| CN110201396B (en) | Method and device for generating game editor and plug-in and data processing | |
| WO2022170230A1 (en) | Shared mixed reality and platform-agnostic format | |
| CN102811352A (en) | Moving image generation method and moving image generation device | |
| CN110502228A (en) | A method of VR content creation and generation VR application program based on cloud platform | |
| JP2022500795A (en) | Avatar animation | |
| US6377255B1 (en) | Pattern data generator, pattern data generating method and its medium | |
| CN116437153A (en) | Previewing method and device of virtual model, electronic equipment and storage medium | |
| JP2019532385A (en) | System for configuring or modifying a virtual reality sequence, configuration method, and system for reading the sequence | |
| Walczak et al. | Dynamic creation of interactive mixed reality presentations | |
| JPH0981782A (en) | Method and system for three-dimensional virtual reality space, medium and method for recording information, medium and method for transmitting information, information processing method, client terminal, and common-use server terminal | |
| JPH10222698A (en) | Communication device and communication method for three-dimensional virtual space | |
| WO2024230324A1 (en) | Child application processing method and apparatus, computer device, and storage medium | |
| EP1602251A2 (en) | System and method for facilitating ip telephony applications | |
| US20050033822A1 (en) | Method and apparatus for information distribution and retrieval |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |