TWI835289B - Virtual and real interaction method, computing system used for virtual world, and virtual reality system - Google Patents

Abstract

A virtual and real interaction method, a computing system used for a virtual world, and a virtual reality system are provided. In the method, a first object model is generated according to a first sensing data, a second object model is generated according to a second sensing data, the behaviors of the first object model and the second object model in a virtual scene are determined according to the first sensing data and the second sensing data, a first image stream is generated according to the behavior of the first object in the virtual scene, and a second image stream is generated according to the behavior of the second object in the virtual scene. The first image stream is provided to be displayed by a remote display apparatus. The second image stream is provided to be displayed by a local display apparatus. Accordingly, the interaction experience could be improved.

Description

Virtual-real interaction method, computing system for virtual world and virtual reality system

The present invention relates to a simulation experience technology, and in particular, to a virtual-real interaction method, a computing system for a virtual world, and a virtual reality system.

Today, technologies such as Virtual Reality (VR), Augmented Reality (AR), Mixed Reality (MR), and Extended Reality (XR) are used to simulate feelings, perceptions, and/or environments. technology is welcomed. The above-mentioned technology can be applied in a variety of fields (eg, gaming, military training, medical care, remote work, etc.). In the past virtual worlds, built reality content or pre-recorded venues were usually used, making it impossible for users to obtain real-time two-way interaction during the experience.

In view of this, embodiments of the present invention provide a virtual-real interaction method, a computing system for a virtual world, and a virtual reality system that can fuse image streams corresponding to objects in different locations and allow two objects to interact in the same virtual scene. in interaction.

The virtual-real interaction method in the embodiment of the present invention includes (but is not limited to) the following steps: generate a first object model based on the first sensing data; generate a second object model based on the second sensing data; generate a second object model based on the first sensing data and the second sensing data. The sensing data determines the behavior of the first object model and the second object model in the virtual scene; the first image stream is generated based on the behavior of the first object model in the virtual scene; and the second image stream is generated based on the behavior of the second object model in the virtual scene. Two video streaming. The first sensing data is obtained by sensing the first physical object. The second sensing data is obtained by sensing the second physical object. The first image stream is used for display by the remote display device. The second image stream is used for display on the local display device.

The computing system used in the virtual world according to the embodiment of the present invention includes (but is not limited to) one or more memories and one or more processors. Memory is used to store one or more programs. The processor is coupled to the memory. The processor is configured to load the program code to execute: generate a first object model based on the first sensing data; generate a second object model based on the second sensing data; determine based on the first sensing data and the second sensing data. The behavior of the first object model and the second object model in the virtual scene; the first image stream is generated based on the behavior of the first object model in the virtual scene; the second image stream is generated based on the behavior of the second object model in the virtual scene . The first sensing data is obtained by sensing the first physical object. The second sensing data is obtained by sensing the second physical object. The first image stream is used for display by the remote display device. The second image stream is used for display on the local display device.

The virtual reality system according to the embodiment of the present invention includes (but is not limited to) two first space sensing devices, one or more computing devices and a local display device. The space sensing device is used to sense the first physical object to obtain first sensing data. The computing device is configured to: generate a first object model based on the first sensing data; generate a second object model based on the second sensing data; determine the first object model and the second object model based on the first sensing data and the second sensing data. The behavior of the two object models in the virtual scene; the first image stream is generated based on the behavior of the first object model in the virtual scene; the second image stream is generated based on the behavior of the second object model in the virtual scene. The second sensing data is obtained by sensing the second physical object through two second space sensing devices. The first image stream is used for display by the remote display device. The local display device is used to display the second image stream.

Based on the above, according to the virtual-real interaction method, the computing system for the virtual world, and the virtual reality system of the embodiments of the present invention, different objects are sensed respectively to generate corresponding object models, and the behaviors of the two object models in the same virtual scene are determined. , and generate image streams for display on different display devices. In this way, the movement of objects can be sensed in real time, and the two objects can interact reasonably and smoothly in the virtual scene, thus improving the experience of the virtual world.

In order to make the above features and advantages of the present invention more clearly understood, embodiments are given below and described in detail with reference to the accompanying drawings.

FIG. 1 is a component block diagram of a virtual reality system 1 according to an embodiment of the present invention. Please refer to Figure 1. The virtual reality system 1 includes (but is not limited to) one or more space sensing devices 11, one or more wearable devices 12, one or more mobile devices 13, local The display device 14 , one or more space sensing devices 21 , one or more wearable devices 22 , one or more mobile devices 23 , the remote display device 24 and the server 30 .

In one embodiment, one or more space sensing devices 11 , one or more wearable devices 12 , one or more mobile devices 13 and the local display device 14 are located at the first location/environment/ Space/field (hereinafter collectively referred to as the first location), and one or more space sensing devices 21, one or more wearable devices 22, one or more mobile devices 23 and remote display The device 24 is located in a second location/environment/space/field (hereinafter collectively referred to as the second location). In this embodiment, it is assumed that the local user is located at the first location and the remote user is located at the second location. However, embodiments of the present invention do not limit the distance between two locations and the objects located thereon (that is, not limited to people, but may also be sports-related objects such as balls, toys, controllers, hitting tools, etc.).

It should be noted that "local" and "remote" are named from the perspective of local users, so they may have different definitions or naming methods for remote users or other users.

It should also be noted that in Figure 1, the local user is the student and the remote user is the coach. However, in some application scenarios, since the coach does not necessarily need to present his or her own holographic image and can teach students only through voice, the dotted line in Figure 1 indicates that the remote user (coach) can selectively use or not use it. Space sensing device 21 and wearable device 22.

FIG. 2A is a component block diagram of the space sensing device 11 according to an embodiment of the present invention. Referring to FIG. 2A , the space sensing device 11 includes (but is not limited to) an image sensing module 111 , a motion tracking module 112 , a communication module 113 , a distance sensor 114 , a memory 115 and a processor 116 .

The image sensing module 111 may be a camera, an image scanner, a video camera, a depth camera, a stereo camera, or other devices used to capture images. In one embodiment, the image sensing module 111 may include an image sensor (such as a Charge Coupled Device (CCD), a Complementary Metal-Oxide-Semiconductor (CMOS), etc.), Optical lenses, image control circuits and other components. It should be noted that the lens specifications (for example, imaging aperture, magnification, focal length, imaging viewing angle, image sensor size, etc.) of the image sensing module 111 and its quantity can be adjusted according to actual needs. For example, the image sensing module 111 includes a 180-degree lens, thereby providing a capture range with a larger field of view. In one embodiment, the image sensing module 111 is used to capture image and/or depth information as sensing data.

The motion tracking module 112 may be an accelerometer, a gyroscope, a magnetometer, an electronic compass, an inertial sensing unit, or a sensor with three or more axes. In one embodiment, the motion tracking module 112 is used to obtain motion-related information such as speed, acceleration, angular velocity, inclination, displacement, etc., and use it as sensing data.

The communication module 113 may support mobile communications such as fourth generation (4G) or other generations, Wi-Fi, Bluetooth, infrared, radio frequency identification (Radio Frequency Identification, RFID), Ethernet, optical fiber network It can be a communication transceiver, serial communication interface (such as RS-232), Universal Serial Bus (USB), Thunderbolt or other communication transmission interface. In the embodiment of the present invention, the communication module 113 is used to transmit or receive data with other electronic devices (for example, a wearable device 12 or a mobile device 13).

The distance sensor 114 may be a radar, a Time of Flight (ToF) camera, a LiDAR scanner, a depth sensor, an infrared rangefinder, an ultrasonic sensor, or other ranging-related sensors. In one embodiment, the distance sensor 114 can detect the azimuth angle of the object to be detected. That is, the azimuth angle of the object to be measured relative to the distance sensor 114 . In another embodiment, the distance sensor 114 can detect the distance of the object to be detected. That is, the distance of the object to be measured relative to the distance sensor 114 . In some embodiments, the detection results (eg, azimuth angle and/or distance) of the one or more distance sensors 114 can be used as sensing data.

The memory 115 can be any type of fixed or removable random access memory (Radom Access Memory, RAM), read only memory (Read Only Memory, ROM), flash memory (flash memory), traditional hard disk (Hard Disk Drive, HDD), solid-state drive (Solid-State Drive, SSD) or similar components. In one embodiment, the memory 115 is used to store program codes, software modules, configurations, data (such as sensing data, object models, image streams, etc.) or files, and their implementation will be described in detail later. example.

The processor 116 is coupled to the image sensing module 111, the motion tracking module 112, the communication module 113, the distance sensor 114 and the memory 115. The processor 116 may be a central processing unit (CPU), a graphics processing unit (GPU), or other programmable general-purpose or special-purpose microprocessor (Microprocessor), digital signal processing Digital Signal Processor (DSP), programmable controller, Field Programmable Gate Array (FPGA), Application-Specific Integrated Circuit (ASIC), neural network accelerator or other similar elements or combinations of the above elements. In one embodiment, the processor 116 is used to execute all or part of the operations of the space sensing device 11, and can load and execute each program code, software module, file and data stored in the memory 115. In some embodiments, the functions of the processor 116 may be implemented through software or a chip.

The implementation and components of the space sensing device 21 can be referred to the description of the space sensing device 11 and will not be described again here.

FIG. 2B is a component block diagram of the wearable device 12 according to an embodiment of the present invention. Please refer to 2B. The wearable device 12 may be a smart bracelet, a smart watch, a handheld controller, a smart waist ring, a smart anklet, a smart headgear, a head-mounted display or other sensing devices worn by human body parts. The wearable device 12 includes (but is not limited to) a motion tracking module 122 , a communication module 123 , a memory 125 and a processor 126 . For the introduction of the motion tracking module 122, the communication module 123, the memory 125 and the processor 126, please refer to the description of the motion tracking module 112, the communication module 113, the memory 115 and the processor 116 respectively, and will not be described again here.

The implementation aspects and components of the wearable device 22 may be referred to the description of the wearable device 12 and will not be described again here.

FIG. 2C is a component block diagram of the mobile device 13 according to an embodiment of the present invention. Referring to FIG. 2C , the mobile device 13 may be a mobile phone, a tablet computer or a laptop computer. The mobile device 13 includes (but is not limited to) a communication module 133 , a memory 135 and a processor 136 . For the introduction of the communication module 133, the memory 135 and the processor 136, please refer to the description of the communication module 113, the memory 115 and the processor 116 respectively, and will not be described again here.

The implementation aspects and components of the mobile device 23 may be referred to the description of the mobile device 13 and will not be described again here.

FIG. 2D is a component block diagram of the local display device 14 according to an embodiment of the present invention. Referring to FIG. 2D , the local display device 14 may be a head-mounted display or smart glasses. The local display device 14 includes (but is not limited to) an image sensing module 141, a motion tracking module 142, a communication module 143, a display 144, a memory 145 and a processor 146.

The display 144 may be a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic light-emitting diode (OLED), a quantum dot display (Quantum dot display), or other type display. In one embodiment, the display 144 is used to display images.

The introduction of the image sensing module 141, motion tracking module 142, communication module 143, memory 145 and processor 146 can be referred to the image sensing module 111, motion tracking module 112, communication module 113, memory respectively. The description of 115 and processor 116 will not be repeated here.

In some embodiments, the local display device 14 may not include the motion tracking module 142 and/or the image sensing module 141 and may be a television or a screen.

The implementation aspects and components of the remote display device 24 may be referred to the description of the local display device 14 and will not be described again here.

FIG. 2E is a component block diagram of the server 30 according to an embodiment of the present invention. Referring to FIG. 2E , the server 30 includes (but is not limited to) a communication module 33 , a memory 35 and a processor 36 . For the introduction of the communication module 33, the memory 35 and the processor 36, please refer to the description of the communication module 113, the memory 115 and the processor 116 respectively, and will not be repeated here.

Figure 3A is a schematic diagram of a set of two space sensing devices according to an embodiment of the present invention. Figure 3B is a schematic diagram of one of the space sensing devices based on Figure 3A. Figure 3C is a schematic diagram of another space sensing device based on Figure 3A. A schematic diagram of a mobile device 13 on which the device is placed. As shown in FIG. 3A , two space sensing devices 11 can be combined together to facilitate portability. As shown in FIG. 3C , the space sensing device 11 has a platform for placing the mobile device 13 or other objects. In one embodiment, the space sensing device 11 further includes a wireless charging module (not shown) and is used to provide power to the mobile device 13 or other electronic devices. The space sensing device 11 may be provided with a notch/window/opening 119 for the image sensing module 111 and/or the distance sensor 114 to transmit and receive signals.

Referring to FIGS. 2A to 2E , in one embodiment, the computing system 2 includes one or more memories 115, 125, 135, 145, 35 and one or more processors 116, 126, 136, 146, 36. One or more processors 116, 126, 136, 146, 36 load the program code stored in the memory 115, 125, 135, 145, 35 to execute/implement the virtual-real interaction method of the embodiment of the present invention introduced later. In some embodiments, multiple devices may be integrated into one device.

In the following, the method described in the embodiment of the present invention will be described with reference to various devices, components and modules in the virtual reality system 1 and/or the computing system 2 . Each process of this method can be adjusted according to the implementation situation, and is not limited thereto. For convenience of explanation, the processor 36 of the server 30 (refer to FIG. 2E) will be used as the execution subject of the method proposed in the embodiment of the present invention. However, all or part of the operations executed by any processor 116, 126, 136, 146, 36 may be executed or implemented by another processor 116, 126, 136, 146, 36, and the embodiments of the present invention are not limited thereto. The execution subject of the method proposed in the embodiment of the invention. In addition, data transfer between devices can be realized through communication modules 113, 123, 133, 143 or 33 respectively.

Figure 4 is a flow chart of a virtual-real interaction method according to an embodiment of the present invention. Referring to FIG. 4 , the processor 36 generates a first object model based on the first sensing data (step S410 ). Specifically, referring to FIG. 1 and FIGS. 2A to 2D , the first sensing data is the image sense of the space sensing device 11 , the wearable device 12 and/or the local display device 14 located at the first location (ie, the local user). Sensing data of the measurement modules 111, 141, motion tracking modules 112, 122, 142 and/or the distance sensor 114. For example, information such as image, depth, distance, speed, rotation, position, orientation, etc.

The first sensing data is obtained by sensing the first physical object located at the first location. For example, the distance between the space sensing device 11 and the first physical object, the moving speed and displacement of the first physical object, or the depth information of the first physical object.

In one embodiment, the first physical object is a first character. The processor 36 may use holographic technology to generate a three-dimensional first object model. Holographic images use the principles of interference and refraction to record the amplitude and/or phase information in the light waves reflected or transmitted by the object, so that the recorded images give people a three-dimensional visual experience. The image sensing module 111 can emit light signals through lasers and receive echo signals through sensing elements. For example, the processor 116 of the spatial sensing device 11 can generate sensing data related to holographic photography based on the echo signals. For example, the amplitude and/or phase of the aforementioned light wave. The processor 36 can generate the first object model of the first physical object based on the sensing data related to the holographic photography.

In another embodiment, the processor 36 may generate the first object model using three-dimensional imaging technologies such as time-of-flight, point-line scanning, structured light projection, optical deflection, and stereoscopic vision.

In one embodiment, the first physical object is a first article. The first item is not a character. Processor 36 can identify the first item. For example, the processor 36 may be based on a neural network algorithm (eg, YOLO (You only look once), Region Based Convolutional Neural Networks (R-CNN), or Fast R-CNN ( Fast CNN)) or feature matching-based algorithms (e.g., Histogram of Oriented Gradient (HOG), Scale-Invariant Feature Transform (SIFT), Harr, or accelerated robust features ( Speeded Up Robust Features (SURF) feature comparison) to achieve item identification. The processor 36 can determine whether the object in the image captured by the image sensing modules 111 and 141 is the default first object. According to different application requirements, the first object may be a ball, a ball frame, a sports tool, a batting tool and other sports-related objects.

In addition to using an algorithm to identify the first item, in another embodiment, the type of the first item may be obtained based on an input device (not shown, such as a keyboard, a mouse, a gesture recognition module or a voice input module). changed according to the input command. For example, if the input command is a voice command of "kick ball", then the type of the first item is related to football. For another example, if the input command is "throw the ball" pointed by the gesture command, then the type of the first item is related to basketball or baseball.

After identifying the first object, the processor 36 may obtain the pre-stored first object model based on the identification result of the first object. That is, the first object model is a three-dimensional model that is pre-created, obtained or stored. For example, the processor 36 can download the first object model through the Internet through the communication module 133 and pre-store it for subsequent use. For another example, the first object is scanned with a three-dimensional scanner, and a first object model is created based on the scan and stored for subsequent use. In this way, in addition to saving software and hardware resources for reconstructing the first object model from the holographic image, the movement of other irrelevant objects can also be eliminated.

For example, FIG. 5A is a schematic diagram of a first object at a first location according to an embodiment of the present invention. Please refer to FIG. 1 and FIG. 5A at the same time. The first object may include a character 51 , a football 52 and a football frame 53 . The processor 36 can generate an object model of the character 51 based on the holographic image, and load the pre-stored object models of the football 52 and the football frame 53 based on the recognition results.

In one embodiment, the image capture module 114 or 141 includes a 180-degree lens. The processor 36 can combine the images captured by two or more image capture modules 114 or 141 with overlapping fields of view into a 360-degree panoramic image. For example, the processor 36 splices two or more images through plane projection conversion (Homography), image warping (Warping), and image blending (blending) technologies.

Please continue to refer to FIG. 4 , the processor 36 generates a second object model based on the second sensing data (step S420 ). Specifically, referring to FIG. 1 , the second sensing data is the image sensing module of the space sensing device 21 , the wearable device 22 and/or the remote display device 24 located at the second location (ie, the remote user). , the sensing data of the motion tracking module and/or the distance sensor (not shown in the figure, but please refer to the same module in Figures 2A to 2E). For example, information such as image, depth, distance, speed, force intensity, rotation, position, orientation, etc.

The second sensing data is obtained by sensing the second physical object located at the second location. For example, the distance between the space sensing device 11 and the second physical object, the moving speed and displacement of the second physical object, or the depth information of the second physical object.

In one embodiment, the second physical object is a second character. The processor 36 may use holographic imaging technology to generate a three-dimensional second object model. The second object model of the second character can be generated with reference to the aforementioned description of the first object model of the first character, and will not be described again here. In addition, in other embodiments, the processor 36 may also use other three-dimensional imaging technologies such as time-of-flight, point-line scanning, structured light projection, optical deflection, stereoscopic vision, etc. to generate the second object model.

In one embodiment, the second physical object may also be a second object, that is, a ball, a frame, a sports tool, a batting tool and other sports-related objects. The second object model of the second object may be generated with reference to the aforementioned description of the first object model of the first character, and will not be described again here. It should be noted that since the processor 36 has generated the first object model of the first item, in some cases it is not necessarily necessary to generate the second object model of the second item.

For example, FIG. 5B is a schematic diagram of a second object at a second location according to an embodiment of the present invention. Please refer to FIG. 1 and FIG. 5B at the same time. The second object may only include the character 55 . The processor 36 can generate an object model of the character 55 based on the holographic image. That is to say, the object models (ie, holographic images) of the characters 51 and 55 can share the pre-stored object models of the football 52 and the football frame 53 in the virtual scene.

Referring to FIG. 4 , the processor 36 determines the behavior of the first object model and the second object model in the virtual scene based on the first sensing data and the second sensing data (step S430 ). Specifically, a virtual scene (or virtual world) is a virtual space generated through spatial scanning or a virtual space simulated by a computing device. The processor 36 can be based on the image sensing modules 111, 141, motion tracking modules 112, 122, 142 and/or the distance sensor 114 of the space sensing device 11, the wearable device 12 and/or the local display device 14. The sensing data determines the motion information of the first physical object in the real space of the first location. Similarly, the processor 36 determines the second step based on the sensing data of the image sensing module, motion tracking module and/or distance sensor of the space sensing device 21 , the wearable device 22 and/or the remote display device 24 . Movement information of physical objects in the real space of the second location. Action information is, for example, speed, direction and/or displacement.

The processor 36 can respectively determine the behavior of the first physical object and the second physical object in the virtual scene based on the motion information of the two objects in their respective real spaces. The processor 36 can simulate the behavior of the first physical object and the second physical object in their respective real spaces in the virtual scene. For example, if the first entity object kicks a ball, then the first object model kicks the ball. For another example, if the second entity object runs, the first object model moves quickly.

The processor 36 generates a first image stream based on the behavior of the first object model in the virtual scene (step S440), and the processor 36 generates a second image stream based on the behavior of the second object model in the virtual scene (step S450). Specifically, in order to allow physical objects located in different locations to interact, the processor 36 generates a first object model and a second object model in a virtual scene. For example, the virtual scene includes a first object model that generates a holographic image of the character 51 . The object model and the second object model of the character 55 are loaded from the server 30 to the pre-stored first object model of the football 52 and the football frame 53 . The processor 36 can respectively project or convert the positions of the first physical object and the second physical object in the real space into the virtual scene, and determine the positions of the first object model and the second object model in the virtual scene accordingly. In addition, the processor 36 simulates the behaviors of the first object model and the second object model in the virtual scene according to the behavior determined in step S430. For example, if the behavior of the first entity object in the real space is to kick a ball, the first object model simulates the action of kicking the ball in the virtual scene.

In one embodiment, the processor 36 may determine the interaction between the first object model and the second object model based on the first sensing data and the second sensing data. For example, the processor 36 can determine whether there are collisions, touches, overlaps and other interactive situations between the two object models based on the positions of the first and second object models in the virtual scene.

The processor 36 can determine the behavior of the first object model and the second object model according to the interaction situation. For example, if the second object model is stationary in the virtual scene, and the first object model moves and collides with the second object model, the second object model will simulate the movement reaction caused by the collision. It should be noted that, depending on the physical characteristics and behaviors of different physical objects and/or application scenarios, the interactions of the first and second object models may be different. Users can design interactive content according to actual needs, which are not limited by the embodiments of the present invention.

The processor 36 captures images of the virtual scene and the first object model (if present in the field of view) from a virtual perspective, and generates images of one or more frames in the first image stream accordingly. For example, a tracking camera system is a camera system or a fixed-angle camera system used to track character movement in a virtual camera system. In addition, the processor 36 captures images of the virtual scene and the second object model (if present in the field of view) from a virtual perspective, and generates images of one or more frames in the second image stream accordingly.

The first image stream generated by the first object model can be displayed by the remote display device 24 . For example, the server 30 transmits the first image stream to the remote display device 24 . The display of the remote display device 24 (not shown in the figure, please refer to the display 144 of the local display device 14) can display the first image stream. For example, the character 55 at the second location can see from its remote display device 24 that the virtual scene includes the holographic image generated from the character 51 at the first location, including by identifying the football 52 and the football at the first location. After block 53, the pre-stored images of the virtual football and the football frame are loaded from the server.

On the other hand, the second image stream generated by the second object model can be displayed by the local display device 14 . For example, the server 30 transmits the second image stream to the local display device 14 through the mobile device 13, and the display 144 of the local display device 14 can display the second image stream. For example, the character 51 in the first location can see from its local display device 14 that the virtual scene includes a holographic image generated from the character 51 in the second location, and includes the aforementioned pre-stored images of the virtual football and football frame.

In an application scenario, in order to have the same standard for calculating the field ranges of the first location (eg, local) and the second location (eg, remote), the spatial sensing devices 11 and 11 used in the first location and the second location are 21 should be placed in the same position, direction and distance.

Figure 6 is a flow chart of spatial correction according to an embodiment of the present invention. Referring to FIG. 6 , the processor 36 may determine the first space based on the third sensing data of the first sensing device (eg, the space sensing device 11 ) (step S610 ). Specifically, the third sensing data is obtained by sensing the space located at the first location. For example, the relative distance between two space sensing devices 11 or the space sensing device 11 and an obstacle (eg, wall, table or chair) and the direction of the space sensing device 11 . The processor 36 can determine the first space based on the distance and direction between the two space sensing devices 11 or the space sensing device 11 and the obstacle. For example, the processor 36 determines the sensing ranges of the two space sensing devices 11 and uses the union of the sensing ranges as the first space.

The processor 36 may compare the space specifications of the first space and the virtual scene (step S620). Specifically, the processor 36 may define spatial specifications according to the type of virtual scene and/or application situation. For example, football practice requires a space of 5*10 meters. For another example, rhythmic dance requires a space of 2*3 meters. The processor 36 can determine the difference in length and orientation between the first space and the space specification, and generate a comparison result accordingly.

The processor 36 may generate a first space adjustment prompt based on the comparison result between the first space and the space specification (step S630). If the comparison result between the first space and the space specification is the same or the difference is less than the corresponding threshold, there is no need to adjust the position of the first sensing device. That is, the space sensing device 11 remains in place. The user interface of the mobile device 13 or the local display device 14 may present a visual prompt that the space is aligned or play an auditory prompt (ie, the first spatial adjustment prompt) through a speaker (not shown).

If the comparison result between the first space and the space specification is different or the difference is greater than the corresponding threshold, the processor 36 needs to adjust the position of the first sensing device. That is, the position or orientation of the spatial sensing device 11 is changed. The first spatial adjustment prompt is used to adjust the position or orientation of the first sensing device. The user interface of the mobile device 13 or the local display device 14 may present visual prompts of the movement distance and/or steering angle or play an auditory prompt (ie, the first spatial adjustment prompt) through a speaker (not shown).

7A to 7F are schematic diagrams of the placement location of a space sensing device according to an embodiment of the present invention. Please refer to FIG. 7A first. The sensing ranges F of the two space sensing devices 11 form a first space S. As shown in FIG. The first space S can be formed only when the two sensing ranges F overlap to a certain extent (for example, 50%, 75% or 80%) or the distance between their origins is within a minimum safe distance. The processor 36 can determine whether the first space S can be formed through the orientation of the sensing range F. Although the positions of the space sensing devices 11 in Figures 7B, 7D and 7F are approximately the same as those in Figures 7A, 7C and 7E, the orientations of the two space sensing devices 11 in Figures 7B, 7D and 7F are There is no relative, and the overlapping range of the sensing range F cannot form the first space S. Therefore, the user can be reminded to change the position and/or orientation of the space sensing device 11 through the first space adjustment prompt.

In addition to the alignment between the plurality of spatial sensing devices 11 at their locations, the spaces at the first location and the second location also need to be corrected.

Figure 8 is a flow chart of spatial correction according to an embodiment of the present invention. Referring to FIG. 8 , the processor 36 may determine the second space according to the fourth sensing data of the second sensing device (eg, the space sensing device 21 ) (step S810 ). Specifically, the fourth sensing data is obtained by sensing the space located at the second location. For example, the relative distance between two space sensing devices 21 or the space sensing device 21 and an obstacle (eg, wall, table or chair) and the direction of the space sensing device 21 . The processor 36 can determine the second space based on the distance and direction between the two space sensing devices 21 or the space sensing device 21 and the obstacle. For example, the processor 36 determines the sensing ranges of the two space sensing devices 21 and uses the union of the sensing ranges as the second space.

The processor 36 may compare the first space and the second space (step S820). The processor 36 can determine the difference in length and orientation between the first space and the second space, and generate a comparison result accordingly.

The processor 36 may generate a second space adjustment prompt based on the comparison result between the first space and the second space (step S830). If the comparison result between the first space and the second space is the same or the difference is less than the corresponding threshold, there is no need to adjust the position of the first or second sensing device. That is, the space sensing device 11 or 21 remains in place. The user interface of the mobile device 13 , 23 , the local display device 14 or the remote display device 24 may present a visual cue that the space is aligned or play an auditory cue (ie, a second spatial adjustment cue) through a speaker (not shown).

If the comparison result between the first space and the second space is different or the difference is greater than the corresponding threshold, the processor 36 determines that the position of the first or second sensing device needs to be adjusted. That is, the position or orientation of the spatial sensing device 11 or 21 is changed. The second spatial adjustment prompt is used to adjust the position or orientation of the first or second sensing device. The user interface of the mobile device 13, 23, the local display device 14 or the remote display device 24 may present visual cues of the movement distance and/or steering angle or play auditory cues (i.e., second space adjustment) through a speaker (not shown). hint).

In one embodiment, the processor 36 uses the smaller of the first space and the second space as a reference, and generates a space adjustment prompt for the larger of the first space and the second space. That is, the position and orientation of the smaller one in the first space and the second space remain unchanged, but the position and/or orientation of the larger one in the first space and the second space are adjusted.

For example, FIG. 9 is a schematic diagram of device movement suggestions according to an embodiment of the present invention. Please refer to FIG. 9 , because the space S1 formed by the two space sensing devices 21a and 21b at the remote end (for example, the second location) is larger than the space formed by the space sensing devices 11a and 11b at the local location (for example, the first location). S2, therefore, the user interface of the mobile device 23 can present a prompt for position and/or orientation adjustment. For example, the space sensing device 21a at the top of the drawing is suggested to be close to the space sensing device 21b at the bottom of the drawing.

On the contrary, in other embodiments, the processor 36 uses the larger of the first space and the second space as a reference, and generates a space adjustment prompt for the smaller of the first space and the second space. That is, the position and orientation of the larger one in the first space and the second space remain unchanged, but the position and/or orientation of the smaller one in the first space and the second space are adjusted.

10A and 10B are schematic diagrams of a user interface for position correction according to an embodiment of the present invention. Please refer to FIG. 10A and FIG. 10B . Since the two spatial sensing devices 11 or 21 can detect the current orientation and relative distance, the processor 36 can determine whether the current distance complies with the definition of the field mode (ie, spatial specifications) (eg, length). X and Y), and can present user interface prompts through the speaker of the mobile device 13 or 23 or on the display screen, allowing the user to adjust/correct the position and/or orientation of the two space sensing devices 11 or 21 accordingly, until Adjusting to meet the space specifications will allow the user interface to switch to the next step. The spatial sensing device on the right side of the figure in Figure 10A should be turned 90 degrees counterclockwise to achieve the minimum safe distance A between the sensors/modules shown in Figure 10B.

If the placement of the space sensing devices 11 or 21 complies with the above-mentioned alignment and/or correction, the spaces formed by the space sensing devices 11 or 21 used at the first location and the second location will be approximately the same, which will help Subsequently, the positions of the first physical object and the second physical object at the first location and the second location are calculated respectively.

In one embodiment, the first sensing data includes first position information of the first physical object in the space where it is located. The location information may include the relative distance or absolute position/coordinates of the first physical object and the space sensing device 11 . For example, the fixed-point two-space sensing device 11 generates the first position information of the space where the first physical object is located based on the distance measurement data of the distance sensor 114 and/or the image sensing module 111. The processor 136 calculates and generates the first position information of the space where the first physical object is located. A position information is assigned to the two-axis vector data of the coordinate system. For example, assign the location to the closest coordinate in the coordinate system. When the image sensing module 111 captures moving portraits/objects, the moving distance data and fixed-point distance data measured by the image sensing module 111 (which may also refer to the ranging data of the distance sensor 114) can be communicated through The module 113 transmits back to the mobile device 13 . The processor 136 of the mobile device 13 can calculate the vector positions corresponding to the moving distance data and the fixed-point distance data on the two-axis coordinate system.

For example, FIG. 11 is a schematic diagram of the coordinate system CS according to an embodiment of the present invention. Referring to FIG. 11 , it is assumed that the origin of the coordinate system CS of a certain space sensing device 11 a is located at the left center, and the origin of the coordinate system CS of another space sensing device 11 b is located at the right center. Taking the space sensing device 11a on the left as an example, the coordinates of the triangular pattern are (4,-2), and the coordinates of the square pattern are (2,1). Taking the space sensing device 11b on the right as an example, the coordinates of the triangular pattern are (3,-2), and the coordinates of the square pattern are (5,1). The processor 36 may integrate the coordinate systems of the two spatial sensing devices 11 into a single coordinate system CS. For example, take the coordinate system CS of the left space sensing device 11 as a reference.

The processor 36 may convert the first position information into second position information in the plane coordinate system of the virtual scene. The processor 36 may convert the first position information into the second position information according to the proportional relationship between the coordinate system of the first position information and the plane coordinate system of the virtual scene. For example, the coordinates (2,1) in Figure 11 are converted to (4,-2).

The behavior of the first object model is related to the second location information. In other words, the position of the first object model is obtained through coordinate system conversion. When the position of the first object model changes, its behavior also changes. This allows remote and local users to experience a more accurate and realistic distance experience when they see each other's images from their respective display devices.

In order to facilitate readers to understand the spirit of the embodiments of the present invention, application scenarios are described below.

FIG. 12 is a schematic diagram of an operation process according to an embodiment of the present invention, and FIG. 13 is a schematic diagram of a user interface and operation process according to an embodiment of the present invention. Please refer to FIGS. 12 and 13 . FIG. 12 shows the operation flow of the user interface of the mobile device 13 , and FIG. 13 shows an example of the user interface. In step S121, if the "Start Training" option is selected on the home page, the process proceeds to step S125 of exercise selection. If the "viewing record" option is selected, the cloud connection in step S122 is performed.

In step S122, the mobile device 13 is connected to the server 30. In step S123, different sports events can be browsed through scrolling, and a specific sport event can be selected accordingly. After selecting a sports event, you can scroll through the different records of this event (preview images can be displayed to assist selection). After selecting the record, the mobile device 13 can play the recorded image (step S124).

In step S125, a preset sports program can be selected or a new program can be created. After selecting a sports event, the detailed items of this event can be selected (step S126).

FIG. 14 is a schematic diagram of a user interface and operation process according to an embodiment of the present invention. Referring to FIG. 12 and FIG. 14 , after selecting the detailed item, in step S127 , the operation prompt of the space sensing device 11 or 21 can be performed, and the connection pairing between other devices can be performed. For example, the spatial sensing device 11 or 21 performs spatial scanning in the respective spaces (ie, with reference to the embodiments described in FIGS. 6 to 11 ), and simultaneously connects the respective local/remote display device 14 or 24 and the wearable device. 12 or 22.

In step S128, a selection of teaching items for multiple devices is provided. For example, the setting teaching of the wearable device 12 or the usage teaching of the mobile device 13 . After selecting a teaching item, the teaching details of this item can be provided (step S129).

FIG. 15 is a schematic diagram of a user interface and operation process according to an embodiment of the present invention. Please refer to Figure 12 and Figure 15. In step 1210, if there is only one local user in the venue, you only need to click "User Join", and the image sensing modules 111 of the two space sensing devices 11 can scan in real time. and create "3D reconstructed portrait".

In one embodiment, the processor 36 generates the second object model in the virtual scene according to the detection result of the control operation. The control operation may be a user operation received through an input device of the mobile device (not shown, such as a keyboard, a touch panel, or a mouse). For example, pressing a physical button, clicking a virtual button, or a toggle switch. In response to no control operation being detected, the processor 36 disables/does not generate the second object model in the virtual scene. At this time, there is no second object model in the first image stream. In response to detecting the control operation, the processor 36 allows the second object model to be generated in the virtual scene. At this time, there is a second object model in the first image stream.

For example, if there is more than one person in the venue, according to the person wearing the bracelet (i.e., the wearable device 12), the location information of the bracelet in the venue is determined after pressing the bracelet, and the space sensing device 11 is notified of the pair of people wearing the bracelet. Real-time scanning and creation of "3D reconstructed portraits" of people.

In step S1211, the mobile device 13 establishes a connection with the wearable device 12. The mobile device 13 can determine the position of the wearable device 12 based on the sensing data generated by the wearable device 12, and accordingly display the corresponding object model in the virtual scene, allowing the display 144 of the local display device 14 to display the object model. Finally, other steps (for example, steps S121, S124, S126 or S1210) may be returned as required.

16A to 16D are schematic diagrams of a two-player football situation according to an embodiment of the present invention. Please refer to Figures 16A to 16D. For double shooting, space sensors 11 or 21 are provided at both the first location and the second location. When the local user who owns the physical goal and physical football (for example, located at the first location) presses the Host button of the bracelet to become the attacker and kick the ball, the remote user (for example, located at the second location) becomes the goalkeeper to block. . At this time, the local user can see the virtual position/posture of the remote user's holographic image (as shown in Figure 16A). When there is no physical football (as shown in FIG. 16B, FIG. 16C, and FIG. 16D) and no physical goal (as shown in FIG. 16B, FIG. 16D), the pre-existing object models of the football and the goal (hereinafter referred to as "virtual football") are loaded from the server 30 ), local/remote users can still be attackers and defenders respectively. Then, when the local user presses the Host button of the bracelet to kick a ball, the camera module in the window of the space sensing device 11 captures the movement posture of the local user, and the server 30 can calculate based on the user's posture. The virtual football movement path is transmitted to the remote display device 24 of the remote user and displayed on the screen for viewing.

For single shot or long pass training, there is no far-end user in Figures 16A to 16D. The local user who owns a physical goal and a physical football presses the Host button of the bracelet to kick the ball. When there is no physical goal or physical football, the user of this device can also press the Host button of the bracelet to kick the ball. The remote user can see the local user's kicking status through the display of the remote display device 24 and press the bracelet to provide voice guidance to the local user. Alternatively, if the remote user also has a space sensing device 21, he or she can press the Host key to switch to a virtual movement posture generated by the remote user. The local user can see the remote user from the display 144 of the local display device 14. The user's demonstration actions in the virtual scene.

For single-player dribbling training, the local user with a physical football presses the Host button of the bracelet to dribble. If there is no physical football, the virtual football is displayed by the local display device 14 . In addition, local users can also press the host button of the bracelet to play virtual football.

For two-person dribbling training, when the local user who owns a physical football presses the Host button of the bracelet to become the attacker and perform dribbling, the image sensing module 111 in the window of the space sensing device 11 captures the movement of the local user. posture and the physical football, the remote user can see the interaction between the local user's holographic image and the virtual football in the virtual scene from the remote display device 24 .

Assume that the remote user becomes the defender and intercepts. If the interception is successful, it ends. If the limbs of this machine and the remote user overlap, the ball will disappear and the game will be suspended. After separation, the ball will appear in place.

On the other hand, if there is no physical football, the local display device 14 displays the virtual football. Then, when the local user presses the Host button of the bracelet to perform dribbling, the image sensing module 111 in the window of the space sensing device 11 captures the movement posture of the local user. The mobile device 13 can calculate the moving path of the virtual football based on the user's posture, so that the remote user can also see the interaction between the local user's holographic image and the virtual football in the virtual scene from the remote display device 24 .

17A to 17D are schematic diagrams of a single football situation according to an embodiment of the present invention. Please refer to Figure 17A to Figure 17D. For the competition mode, a physical football is required on the local or both locations. Press the Host button of the bracelet to start time calculation for the remote user. For the number of picking/dumping games, the image sensing module 111 in the space sensing device 11 captures football images for the cloud or the computing center in the space sensor to calculate the number of times. If the ball falls on the floor, the calculation of this round ends (Figure 17A, Figure 17B). For the obstacle course dribbling time competition: the server calculates that the football correctly bypasses the virtual pyramid, and the sensor stops timing after completing the route and passing it (Figure 3, Figure 4). If it is necessary to use a virtual pyramid, the corresponding object model can be generated in the virtual scene.

On the other hand, the remote user can see the local user's kicking status through the display of the display device 24 and press the bracelet to provide voice guidance to the local user. Alternatively, when there is also a space sensing device 21 at the second location, the Host key can be pressed to switch to the remote user generating a virtual movement posture, and accordingly display the remote user's movement posture through the display 144 of the local display device 14 . action poses.

Figure 18 is a schematic diagram of the position layout of a spatial sensing device for fitness training according to an embodiment of the present invention. Please refer to Figure 18. For fitness training, since fitness movements are basically based on the front posture and the postures are mostly symmetrical, one space sensing device 11 is placed in front of the user, and the other space sensing device 11 is placed in front of the user. side of the user, and accordingly constructs the action corresponding to the sensing data into the virtual image (i.e., image streaming).

On the teaching end, the instructor's fitness postures and movements can be recorded and transmitted to the learners in real time for viewing so that they can follow the movements. Alternatively, the learner's fitness postures and movements can be recorded and transmitted to the instructor for review in real time. For joint training use, multiple users can view each other's fitness postures and movements.

Figure 19 is a schematic diagram of a multi-sensing scenario according to an embodiment of the present invention. Referring to Figure 17, the wearable device 12 (for example, a bracelet) can record and set correct action information (for example, displacement, distance, direction, force, etc.), and the image obtained by the space sensing device 11 can be provided to the local display device The display 144 of 14 is used to prompt the user whether each action is in place.

It should be noted that the content of the aforementioned application scenarios is only for example illustration, and users can change the training content according to actual needs.

To sum up, in the virtual-real interaction method, the computing system for the virtual world and the virtual reality system of the embodiments of the present invention, the image sensing module is used to capture the actions of characters and interactive objects (for example, balls, ball frames). images and generate location data. After the obtained character image data is sent to the server, a three-dimensional reconstructed portrait can be reconstructed through holographic imaging technology. Only the image of the interactive object is captured, and based on the selection of the selected sports event and/or the AI model stored in the memory to identify the characteristics of the object image, the pre-stored three-dimensional object image is loaded from the server database to save money. Rebuild three resources to maintain the image of the item.

In this way, users can have different operating methods and experiences in different sports. In addition to the ability for multiple users to have fun together, there is also a teaching interaction method between coaches and students, allowing users to have the same experience of outdoor sports at home.

Although the present invention has been disclosed above through embodiments, they are not intended to limit the present invention. Anyone with ordinary knowledge in the technical field may make some modifications and modifications without departing from the spirit and scope of the present invention. Therefore, The protection scope of the present invention shall be determined by the appended patent application scope.

1:Virtual reality system 11, 11a, 11b, 21, 21a, 21b: space sensing device 12, 22: Wearable devices 13, 23:Mobile device 14:Local display device 24:Remote display device 30:Server 2:Computing system 111, 141: Image sensing module 112, 122, 142: motion tracking module 113, 123, 133, 143, 33: Communication module 114: Distance sensor 115, 125, 135, 145, 35: memory 116, 126, 136, 146, 36: Processor 144:Display S410~S450, S610~S630, S810~S830, S121~S1211: steps 51, 55: Characters 52:Football 53: Football Frame S, S1, S2: space S: sensing range X, Y: length A: Minimum safety distance

FIG. 1 is a component block diagram of a virtual reality system according to an embodiment of the present invention. FIG. 2A is a component block diagram of a space sensing device according to an embodiment of the present invention. FIG. 2B is a component block diagram of a wearable device according to an embodiment of the present invention. FIG. 2C is a block diagram of a mobile device according to an embodiment of the present invention. FIG. 2D is a component block diagram of a local display device according to an embodiment of the present invention. FIG. 2E is a component block diagram of a server according to an embodiment of the present invention. FIG. 3A is a schematic diagram of a set of two space sensing devices according to an embodiment of the present invention. FIG. 3B is a schematic diagram of one of the space sensing devices according to FIG. 3A. FIG. 3C is a schematic diagram of a mobile device placed on another space sensing device according to FIG. 3A . Figure 4 is a flow chart of a virtual-real interaction method according to an embodiment of the present invention. FIG. 5A is a schematic diagram of the first object at the first location according to an embodiment of the present invention. FIG. 5B is a schematic diagram of the second object at the second location according to an embodiment of the present invention. Figure 6 is a flow chart of spatial correction according to an embodiment of the present invention. 7A to 7F are schematic diagrams of the placement location of a space sensing device according to an embodiment of the present invention. Figure 8 is a flow chart of spatial correction according to an embodiment of the present invention. FIG. 9 is a schematic diagram of device movement suggestions according to an embodiment of the present invention. 10A and 10B are schematic diagrams of a user interface for position correction according to an embodiment of the present invention. FIG. 11 is a schematic diagram of a coordinate system according to an embodiment of the present invention. FIG. 12 is a schematic diagram of an operation flow according to an embodiment of the present invention. FIG. 13 is a schematic diagram of a user interface and operation process according to an embodiment of the present invention. FIG. 14 is a schematic diagram of a user interface and operation process according to an embodiment of the present invention. FIG. 15 is a schematic diagram of a user interface and operation process according to an embodiment of the present invention. 16A to 16D are schematic diagrams of a two-player football situation according to an embodiment of the present invention. 17A to 17D are schematic diagrams of a single football situation according to an embodiment of the present invention. Figure 18 is a schematic diagram of the position layout of a spatial sensing device for fitness training according to an embodiment of the present invention. Figure 19 is a schematic diagram of a multi-sensing scenario according to an embodiment of the present invention.

1:Virtual reality system

11, 21: Space sensing device

12, 22: Wearable devices

13, 23:Mobile device

14:Local display device

24:Remote display device

30:Server

Claims (16)

A virtual-real interaction method includes: generating a first object model based on a first sensing data, wherein the first sensing data is obtained by sensing a first physical object; generating a first object model based on a second sensing data. Two object models, wherein the second sensing data is obtained by sensing a second physical object; determining where the first object model and the second object model are based on the first sensing data and the second sensing data. Behavior in a virtual scene, wherein the step of determining the behavior of the first object model and the second object model in the virtual scene includes: determining the first object based on the first sensing data and the second sensing data An interaction situation between the model and the second object model; and determining the behavior of the first object model and the second object model in the virtual scene based on the interaction situation; based on the behavior of the first object model in the virtual scene The behavior generates a first image stream, wherein the first image stream is used for display by a remote display device; and a second image stream is generated according to the behavior of the second object model in the virtual scene, wherein the first image stream is used for display by a remote display device; The two image streams are used for display on a local display device. The virtual-real interaction method as claimed in claim 1, wherein the first physical object is a first character, the second physical object is a second character, and the step of generating the first object model and the second object model includes : Using a holographic image (Holographic) technology to generate a three-dimensional model of the first object model and the second object model. The virtual-real interaction method as described in claim 1, wherein the first physical object is a first object, and the step of generating the first object model includes: identifying the first object; and obtaining the first object model based on the identification result of the first object. The pre-stored first object model. The virtual-real interaction method as described in claim 1, wherein the first sensing data includes a first position information of the first physical object in the space where it is located, and determines the position of the first object model and the second object model in the space. The step of behaving in the virtual scene includes: converting the first position information into a second position information in a plane coordinate system of the virtual scene, wherein the behavior of the first object model is related to the second position information. The virtual-real interaction method as described in claim 1 further includes: generating the second object model in the virtual scene according to the detection result of a control operation, wherein in response to the non-detection of the control operation, disabling the virtual object model is disabled. The second object model is generated in the scene; and in response to detecting the control operation, the second object model is allowed to be generated in the virtual scene. The virtual-real interaction method as claimed in claim 1, further comprising: determining a first space based on a third sensing data of a first sensing device, wherein the third sensing data includes information related to another first sensing device. the relative distance between and the The orientation of the first sensing device, and the first sensing device is used to sense the first physical object; compare the first space and a spatial specification of the virtual scene; and based on the first space and the spatial specification The comparison result generates a first spatial adjustment prompt, wherein the first spatial adjustment prompt is used to adjust the position or orientation of the first sensing device. The virtual-real interaction method as described in claim 6, further comprising: determining a second space based on a fourth sensing data of a second sensing device, wherein the fourth sensing data includes a second space with another second sensing device. the relative distance between them and the orientation of the second sensing device, and the second sensing device is used to sense the second physical object; compare the first space and the second space; and based on the first space and The comparison result of the second space generates a second space adjustment prompt, wherein the second space adjustment prompt is used to adjust the position or orientation of the first sensing device or the second sensing device. A computing system for a virtual world, including: at least one memory to store at least one program code; and at least one processor coupled to the at least one memory and configured to load the at least one program code to Execution: Generate a first object model based on a first sensing data, wherein the first sensing data is obtained by sensing a first physical object; generate a second object model based on a second sensing data, wherein The second The sensing data is obtained by sensing a second physical object; the behavior of the first object model and the second object model in a virtual scene is determined based on the first sensing data and the second sensing data, wherein The at least one processor is further configured to: determine an interaction situation between the first object model and the second object model based on the first sensing data and the second sensing data; and based on the interaction situation Determine the behavior of the first object model and the second object model in the virtual scene; generate a first image stream based on the behavior of the first object model in the virtual scene, wherein the first image stream is used to provide A remote display device displays; and a second image stream is generated according to the behavior of the second object model in the virtual scene, wherein the second image stream is used for display by a local display device. The computing system for a virtual world as claimed in claim 8, wherein the first physical object is a first character, the second physical object is a second character, and the at least one processor is further configured to execute : Using a holographic imaging technology to generate the three-dimensional first object model and the second object model. The computing system for a virtual world as claimed in claim 8, wherein the first physical object is a first object, and the at least one processor is further configured to perform: identifying the first object; and according to the first object The identification result of an object obtains the pre-stored first object model. The computing system for a virtual world as described in claim 8, wherein the first sensing data includes a first position information of the first physical object in the space where it is located, and the at least one processor is further configured to Execution: convert the first position information into a second position information in a plane coordinate system of the virtual scene, wherein the behavior of the first object model is related to the second position information. The computing system for a virtual world as described in claim 8, wherein the at least one processor is further configured to perform: generating the second object model in the virtual scene according to a detection result of a control operation, wherein the response When the control operation is not detected, generation of the second object model in the virtual scene is disabled; and in response to detection of the control operation, generation of the second object model in the virtual scene is allowed. The computing system for a virtual world as claimed in claim 8, wherein the at least one processor is further configured to perform: determining a first space based on a third sensing data of a first sensing device, wherein the The third sensing data includes the relative distance to another first sensing device and the orientation of the first sensing device, and the first sensing device is used to sense the first physical object; compare the first space and a spatial specification of the virtual scene; and generating a first spatial tone based on the comparison result of the first space and the spatial specification. A whole prompt, wherein the first spatial adjustment prompt is used to adjust the position or orientation of the first sensing device. The computing system for a virtual world as claimed in claim 13, wherein the at least one processor is further configured to perform: determining a second space according to a fourth sensing data of a second sensing device, wherein the The fourth sensing data includes the relative distance to another second sensing device and the orientation of the second sensing device, and the second sensing device is used to sense the second physical object; compare the first space and the second space; and generate a second space adjustment prompt based on the comparison result of the first space and the second space, wherein the second space adjustment prompt is used to adjust the first sensing device or the second sensing The location or orientation of the device. A virtual reality system, including: two first space sensing devices, used to sense a first physical object to obtain a first sensing data; at least one computing device configured to: according to the first The sensing data generates a first object model; a second object model is generated based on a second sensing data, wherein the second sensing data is obtained by sensing a second physical object through two second space sensing devices. ; Determine the behavior of the first object model and the second object model in a virtual scene based on the first sensing data and the second sensing data, wherein the at least one operation The computing device is further configured to: determine an interaction situation between the first object model and the second object model based on the first sensing data and the second sensing data; and determine the first interaction situation based on the interaction situation. The behavior of the object model and the second object model in the virtual scene; generating a first image stream based on the behavior of the first object model in the virtual scene, wherein the first image stream is used for a remote display device display; and generating a second image stream based on the behavior of the second object model in the virtual scene; and a local display device for displaying the second image stream. The virtual reality system as claimed in claim 15, further comprising: at least one wearable device for being worn by a first character and generating a first sub-data accordingly, wherein the two first space sensing devices A second sub-data is generated, and the first sensing data includes the first sub-data and the second sub-data.

Images