CN115981544A - Interaction method, device, electronic device and storage medium based on extended reality - Google Patents

Abstract

Provided is an interaction method, device, electronic device, and storage medium based on extended reality. By displaying the virtual reality space, a virtual keyboard object is displayed in the virtual reality space in response to a user's first preset operation. The virtual keyboard object includes multiple The first virtual keys arranged around the preset blank area, and the selected first virtual keys are determined in response to the user's second preset operation. In the present disclosure, the virtual keyboard object includes a plurality of first virtual keys arranged around the preset blank area, so that the first virtual keys are located in each direction of the preset blank area, so that an interactive mode suitable for extended reality can be used to realize the user's The interaction with the virtual keyboard object improves the information input efficiency in the extended reality scene.

Description

Interaction method, device, electronic device and storage medium based on extended reality

technical field

The present disclosure relates to the field of computer technology, and in particular to an interaction method, device, electronic device and storage medium based on extended reality.

Background technique

Extended reality technology (Extended Reality, referred to as XR) can combine reality and virtuality through computers to provide users with a virtual reality space that can interact with humans and computers. In the virtual reality space, users can conduct social interaction, entertainment, study, work, remote office, create UGC (User Generated Content, User Generated Content), etc. through virtual reality devices such as head mount displays (HMD). . However, the virtual keyboard layout provided by the relevant extended reality application adopts the previous keyboard layout, which is not suitable for the interaction habits of the extended reality, and reduces the efficiency of information input in the extended reality scene.

Contents of the invention

This Summary is provided to introduce a simplified form of concepts that are described in detail later in the Detailed Description. This summary of the invention is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to be used to limit the scope of the claimed technical solution.

In a first aspect, according to one or more embodiments of the present disclosure, an interaction method based on extended reality is provided, including:

display virtual reality space;

In response to a user's first preset operation, displaying a virtual keyboard object in the virtual reality space, the virtual keyboard object including a plurality of first virtual keys arranged around a preset blank area;

In response to the user's second preset operation, the selected first virtual key is determined.

In a second aspect, according to one or more embodiments of the present disclosure, an interaction device based on extended reality is provided, including:

a space display unit for displaying a virtual reality space;

A keyboard display unit, configured to display a virtual keyboard object in the virtual reality space in response to a user's first preset operation, where the virtual keyboard object includes a plurality of first virtual keys arranged around a preset blank area;

The key determining unit is configured to determine the selected first virtual key in response to the user's second preset operation.

In a third aspect, according to one or more embodiments of the present disclosure, an electronic device is provided, including: at least one memory and at least one processor; wherein, the memory is used to store program codes, and the processor is used to call The program code stored in the memory enables the electronic device to execute the extended reality-based interaction method provided according to one or more embodiments of the present disclosure.

In a fourth aspect, according to one or more embodiments of the present disclosure, a non-transitory computer storage medium is provided, the non-transitory computer storage medium stores program code, and when the program code is executed by a computer device, the The computer device executes the extended reality-based interaction method provided according to one or more embodiments of the present disclosure.

According to one or more embodiments of the present disclosure, by making the virtual keyboard object include a plurality of first virtual keys arranged around the preset blank area, the first virtual keys are located in each direction of the preset blank area, so that An interaction mode suitable for the extended reality is used to realize the interaction between the user and the virtual keyboard object, and the efficiency of information input in the extended reality scene is improved.

Description of drawings

The above and other features, advantages and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic and that elements and elements are not necessarily drawn to scale.

Fig. 1 is a flowchart of an interaction method based on extended reality provided according to an embodiment of the present disclosure;

FIG. 2 is a schematic diagram of an extended reality device according to an embodiment of the present disclosure;

FIG. 3 is an optional schematic diagram of a virtual field of view of an extended reality device provided according to an embodiment of the present disclosure;

4 is a schematic diagram of a virtual keyboard object provided according to an embodiment of the present disclosure;

5 is a schematic diagram of a first virtual keyboard object and a second virtual keyboard object provided according to another embodiment of the present disclosure;

Fig. 6 is a schematic structural diagram of an interaction device based on extended reality provided according to an embodiment of the present disclosure;

Fig. 7 is a schematic structural diagram of an electronic device provided according to an embodiment of the present disclosure.

Detailed ways

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the drawings, it should be understood that the disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein; A more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for exemplary purposes only, and are not intended to limit the protection scope of the present disclosure.

It should be understood that the steps described in the embodiments of the present disclosure may be executed in different orders, and/or executed in parallel. Furthermore, embodiments may include additional steps and/or omit performing illustrated steps. The scope of the present disclosure is not limited in this regard.

As used herein, the term "comprise" and its variations are open-ended, ie "including but not limited to". The term "based on" is "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one further embodiment"; the term "some embodiments" means "at least some embodiments." The term "responsive to" and related terms means that one signal or event is affected to some extent, but not necessarily completely or directly, by another signal or event. If event x occurs "in response to" event y, then x may respond to y directly or indirectly. For example, the occurrence of y may eventually lead to the occurrence of x, but there may be other intermediate events and/or conditions. In other cases, y may not necessarily result in the occurrence of x, and x may occur even if y has not occurred. Furthermore, the term "responsive to" can also mean "responsive at least in part to".

The term "determining" broadly covers a wide variety of actions, which may include obtaining, calculating, computing, processing, deriving, investigating, looking up (e.g., looking up in a table, database, or other data structure), ascertaining, and the like, Also includes receiving (eg, receiving information), accessing (eg, accessing data in memory) and the like, as well as parsing, selecting, selecting, establishing, and the like, and the like. Relevant definitions of other terms will be given in the description below. Relevant definitions of other terms will be given in the description below.

It should be noted that concepts such as "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the sequence of functions performed by these devices, modules or units or interdependence.

It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative and not restrictive, and those skilled in the art should understand that unless the context clearly indicates otherwise, it should be understood as "one or more" multiple".

For the purposes of this disclosure, the phrase "A and/or B" means (A), (B), or (A and B).

The names of messages or information exchanged between multiple devices in the embodiments of the present disclosure are used for illustrative purposes only, and are not used to limit the scope of these messages or information.

The extended reality-based interaction method provided by one or more embodiments of the present disclosure adopts an extended reality (Extended Reality, XR for short) technology. Extended reality technology can combine reality and virtuality through computers to provide users with a virtual reality space that can interact with humans and computers. In the virtual reality space, users can conduct social interaction, entertainment, study, work, telecommuting, create UGC (User Generated Content, User Generated Content), etc. .

Referring to Figure 2, users can enter the virtual reality space through extended reality devices such as head-mounted VR glasses, and control their own virtual characters (Avatar) in the virtual reality space to socially interact, entertain and learn with virtual characters controlled by other users. , remote office, etc.

In the virtual reality space, the user can implement related interactive operations through a controller, and the controller can be a handle, for example, the user can perform related operation control by operating a button of the handle. Of course, in other embodiments, the target object in the extended reality device may also be controlled by using gestures or voice or a multi-modal control method instead of using a controller.

The extended reality devices described in the embodiments of the present disclosure may include but not limited to the following types:

The computer-side extended reality (PCVR) device uses the PC side to perform related calculations and data output of the extended reality function, and the external computer-side extended reality device uses the data output from the PC side to realize the effect of the extended reality.

The mobile extended reality device supports setting up mobile terminals (such as smart phones) in various ways (such as a head-mounted display with a special card slot), and through a wired or wireless connection with the mobile terminal, the mobile terminal performs extended reality Function-related calculations, and output data to mobile extended reality devices, such as watching extended reality videos through APPs on mobile terminals.

The all-in-one extended reality device has a processor for performing calculations related to virtual functions, so it has independent extended reality input and output functions, does not need to be connected to a PC or mobile terminal, and has a high degree of freedom of use.

Of course, the form realized by the extended reality device is not limited to this, and can be further miniaturized or enlarged as required.

The extended reality device is equipped with a posture detection sensor (such as a nine-axis sensor), which is used to detect the posture change of the extended reality device in real time. If the user wears the extended reality device, when the posture of the user's head changes, the head will be moved The real-time attitude of the user is transmitted to the processor, so as to calculate the gaze point of the user's line of sight in the virtual environment, and calculate the image in the 3D model of the virtual environment according to the gaze point in the user's gaze range (that is, the virtual field of view), and display it on the display screen Display, making people feel as if they are in the immersive experience of watching in the real environment.

Fig. 3 shows an optional schematic diagram of the virtual field of view of the extended reality device provided by an embodiment of the present disclosure. The horizontal field of view and the vertical field of view are used to describe the distribution range of the virtual field of view in the virtual environment, and the vertical The distribution range of the direction is represented by the vertical field of view BOC, and the distribution range of the horizontal direction is represented by the horizontal field of view AOB. The human eye can always perceive the image in the virtual field of view in the virtual environment through the lens. It can be understood that the field of view The larger , the larger the size of the virtual field of view, and the larger the area of the virtual environment that the user can perceive. Wherein, the angle of view indicates the distribution range of the viewing angle when the environment is perceived through the lens. For example, the field of view of the extended reality device indicates the distribution range of the viewing angle of the human eye when the virtual environment is perceived through the lens of the extended reality device; for another example, for a mobile terminal equipped with a camera, the field of view of the camera The angle is the distribution range of the angle of view that the camera has when it perceives the real environment and shoots.

Extended reality devices, such as HMDs, integrate several cameras (such as depth cameras, RGB cameras, etc.), and the purpose of the cameras is not limited to providing a through view. Camera images and an integrated inertial measurement unit (IMU) provide data that can be processed by computer vision methods to automatically analyze and understand the environment. Also, HMDs are designed to support not only passive computer vision analysis, but also active computer vision analysis. Passive computer vision methods analyze image information captured from the environment. These methods can be monoscopic (images from a single camera) or stereoscopic (images from two cameras). They include but are not limited to feature tracking, object recognition, and depth estimation. Active computer vision methods add information to the environment by projecting patterns that are visible to the camera but not necessarily to the human visual system. Such technologies include time-of-flight (ToF) cameras, laser scanning or structured light to simplify the stereo matching problem. Active computer vision is used to achieve scene depth reconstruction.

Referring to FIG. 1 , FIG. 1 shows a flow chart of an XR-based interaction method 100 provided by an embodiment of the present disclosure. The method 100 includes step S120-step S160.

Step S120: Displaying the virtual reality space.

Step S140: In response to the user's first preset operation, display a virtual keyboard object in the virtual reality space, where the virtual keyboard object includes a plurality of first virtual keys arranged around a preset blank area.

Step S160: Determine the selected first virtual key in response to the user's second preset operation.

Wherein, the virtual reality space may be a simulated environment of the real world, a semi-simulation and semi-fictional virtual scene, or a purely fictional virtual scene. The virtual scene may be any one of a two-dimensional virtual scene, a 2.5-dimensional virtual scene, or a three-dimensional virtual scene, and the embodiment of the present application does not limit the dimensions of the virtual scene. For example, the virtual scene may include sky, land, ocean, etc. The land may include environmental elements such as deserts and cities, and the user may control virtual objects to move in the virtual scene.

The first preset operation and the second preset operation include but are not limited to somatosensory control operations, gesture control operations, eye movement operations, touch operations, voice control instructions, or operations on external control devices. For example, the user can arouse the virtual keyboard object in the virtual reality space by triggering the preset button on the controller of the extended reality (for example, the handle of the VR device), and select a certain first key from the virtual keyboard object through such as the handle ray. Virtual Key.

In some embodiments, the first virtual keys may be arranged in a ring, and the center of the ring is the preset blank area. Referring to FIG. 4 , in the virtual reality space, the substantially circular virtual keyboard object 20 includes eight first virtual keys 30 . Each first virtual key corresponds to 3-4 letters.

An input box 40 is also displayed in the virtual reality space. The input box 40 displays the character input by the user by triggering the first virtual key, and a plurality of candidate entries determined according to the character.

In some embodiments, the second preset operation includes an operation for controlling the virtual ray to select the first virtual key; the method 100 further includes: after displaying the virtual keyboard object, setting the initial direction of the virtual ray to Default blank space for orientation. For example, referring to FIG. 4 , the initial direction of the virtual ray may be located at the center of the ring. In this embodiment, by arranging the first virtual keys around the preset blank area, the first virtual keys are located in each direction of the preset blank area, and the initial direction of the virtual ray is set to face the preset blank area, so that the user can select the first virtual button located in each direction by moving the virtual ray around the preset blank area, thereby reducing the distance between the initial position of the virtual ray and the first virtual button to be selected as much as possible. The distance improves the efficiency of information input in extended reality scenarios.

In some embodiments, the second preset operation includes an operation of changing the pose of the controller; step S160 includes: responding to the operation of changing the pose of the controller, determining the direction of motion of the controller; based on the The direction of movement determines the selected first virtual key. In this embodiment, by arranging the first virtual keys around the preset blank area, the first virtual keys are located in each direction of the preset blank area, so that the user can drive in a certain direction (such as shaking, flipping, By moving the controller, the first virtual button located in this direction can be selected without moving the virtual ray (such as the handle ray) to the position of the first virtual button, thereby improving the information input efficiency in the extended reality scene.

In some embodiments, step S160 includes: in response to the second preset operation, rotating the virtual keyboard object or rotating the key pointing object used to indicate the currently selected key, and turning the key pointing object to the key pointing object after the rotation stops. The indicated first virtual key is used as the selected first virtual key. In this embodiment, by arranging the first virtual keys around the preset blank area, the first virtual keys are located in each direction of the preset blank area, so that the ring-shaped virtual button can be rotated around the preset blank area. The keyboard or the rotation button indicates the object, and then the first virtual button is selected through the rotation interaction mode, which improves the efficiency of information input in the extended reality scene.

According to one or more embodiments of the present disclosure, by making the virtual keyboard object include a plurality of first virtual keys arranged around the preset blank area, the first virtual keys are located in each direction of the preset blank area, so as to be compatible with the extended In order to adapt to the real space-based interaction characteristics, an interaction mode suitable for extended reality can be adopted to realize the interaction between the user and the virtual keyboard object, which improves the efficiency of information input in the extended reality scene.

In some embodiments, the characters corresponding to the first virtual keys are the same as the characters corresponding to the keys in the Jiugongge input method; the number of the first virtual keys is eight. In this embodiment, the mapping between the first virtual key and the character inherits the Jiugongge input method, which reduces the user's learning cost and improves the information input efficiency in the extended reality scene.

In some embodiments, the virtual keyboard object includes a first virtual keyboard object and a second virtual keyboard object arranged side by side; the first virtual keyboard object includes a plurality of first virtual keyboard objects arranged around a first preset blank area button, the second virtual keyboard object includes a plurality of first virtual buttons arranged around a second preset blank area; the first virtual keyboard object is used to be controlled by the first handheld controller, and the second virtual The keyboard object is used to be controlled by the second handheld controller.

In this embodiment, by setting dual virtual keyboard objects suitable for two-handed operation, the user can control different virtual keyboard objects by manipulating two handles, so that the information input interaction is more suitable for the extended reality scene, and the extended reality scene is improved. Information input efficiency under.

Exemplarily, with reference to FIG. 5 , in the virtual reality space, the substantially ring-shaped first virtual keyboard object 21 includes four first virtual keys 31 arranged around a preset blank area, and the four first virtual keys 31 are respectively located at corresponding The top, bottom, left, and right sides of the preset blank area. Similarly, the substantially ring-shaped first virtual keyboard object 22 includes four first virtual keys 32 arranged around a preset blank area, and the four first virtual keys 32 are respectively located at the top, bottom, left, and bottom of the corresponding preset blank area. Right. An input method switching interface 50 is also displayed in the virtual reality space, through which the user can select an input method.

In this example, the virtual keys in each virtual keyboard object are located on the upper, lower, left, and right sides of the corresponding preset blank area. In this way, when the second preset operation includes the operation of changing the pose of the controller, the user can only By driving the controller up, down, left or right (such as shaking, flipping, moving), the only first virtual button located in the corresponding direction can be selected, thereby further improving the efficiency and accuracy of virtual button selection , which greatly reduces the situation of misoperation caused by inaccurate driving directions, thereby improving the efficiency of information input in extended reality scenarios.

In some embodiments, the candidate entry displayed in the input box may be selected in response to the first preset button set on the controller being triggered; the confirmation may be confirmed in response to the second preset button set on the controller being triggered. The selected candidate entry; it is also possible to switch the candidate entry displayed in the input box (for example, to turn the page) in response to the triggering of the third preset button set on the controller. Exemplarily, the first preset button includes a joystick.

Correspondingly, referring to FIG. 6 , an information interaction device 600 is provided according to an embodiment of the present disclosure, including:

A space display unit 601, configured to display a virtual reality space;

A keyboard display unit 602, configured to display a virtual keyboard object in the virtual reality space in response to a user's first preset operation, where the virtual keyboard object includes a plurality of first virtual keys arranged around a preset blank area;

The key determining unit 603 is configured to determine the selected first virtual key in response to the user's second preset operation.

In some embodiments, the characters corresponding to the first virtual keys are the same as the characters corresponding to the keys in the Jiugongge input method; the number of the first virtual keys is eight.

In some embodiments, the virtual keyboard object includes a first virtual keyboard object and a second virtual keyboard object arranged side by side; the first virtual keyboard object includes a plurality of first virtual keyboard objects arranged around a first preset blank area button, the second virtual keyboard object includes a plurality of first virtual buttons arranged around a second preset blank area; the first virtual keyboard object is used to be controlled by the first handheld controller, and the second virtual The keyboard object is used to be controlled by the second handheld controller.

In some embodiments, the second preset operation includes an operation for controlling the virtual ray to select the first virtual key; the method further includes: after the virtual keyboard object is displayed, setting the initial direction of the virtual ray to Set to face the preset empty space.

In some embodiments, the second preset operation includes an operation of changing the pose of the controller; the determining the selected first virtual key in response to the second preset operation of the user includes: responding to the change The controller pose operation determines the motion direction of the controller; and determines the selected first virtual key based on the motion direction.

In some embodiments, the determining the selected first virtual key in response to the second preset operation of the user includes: in response to the second preset operation, rotating the virtual keyboard object or rotating for indicating The key pointing object of the currently selected key, the selected first virtual key is determined based on the first virtual key indicated by the key pointing object after the rotation stops.

In some embodiments, the first virtual keys are arranged in a ring.

As for the embodiment of the device, since it basically corresponds to the embodiment of the method, for relevant parts, please refer to part of the description of the embodiment of the method. The apparatus embodiments described above are illustrative only, where modules illustrated as separate modules may or may not be separate. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. It can be understood and implemented by those skilled in the art without creative effort.

Accordingly, according to one or more embodiments of the present disclosure, an electronic device is provided, including:

at least one memory and at least one processor;

Wherein, the memory is used to store program codes, and the processor is used to invoke the program codes stored in the memory to enable the electronic device to execute the extended reality-based interaction method provided according to one or more embodiments of the present disclosure.

Accordingly, according to one or more embodiments of the present disclosure, there is provided a non-transitory computer storage medium, the non-transitory computer storage medium stores program codes, and the program codes can be executed by a computer device to cause the computer device to execute An interaction method based on extended reality provided by one or more embodiments of the present disclosure.

Referring now to FIG. 7 , it shows a schematic structural diagram of an electronic device (such as a terminal device or a server) 800 suitable for implementing an embodiment of the present disclosure. The terminal equipment in the embodiment of the present disclosure may include but not limited to such as mobile phone, notebook computer, digital broadcast receiver, PDA (personal digital assistant), PAD (tablet computer), PMP (portable multimedia player), vehicle terminal (such as mobile terminals such as car navigation terminals) and fixed terminals such as digital TVs, desktop computers and the like. The electronic device shown in FIG. 7 is only an example, and should not limit the functions and application scope of the embodiments of the present disclosure.

As shown in FIG. 7 , an electronic device 800 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) Various appropriate actions and processes are executed by programs in the memory (RAM) 803 . In the RAM 803, various programs and data necessary for the operation of the electronic device 800 are also stored. The processing device 801, ROM 802, and RAM 803 are connected to each other through a bus 804. An input/output (I/O) interface 805 is also connected to the bus 804 .

Typically, the following devices can be connected to the I/O interface 805: input devices 806 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a liquid crystal display (LCD), speaker, vibration an output device 807 such as a computer; a storage device 808 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 809. The communication means 809 may allow the electronic device 800 to communicate with other devices wirelessly or by wire to exchange data. While FIG. 7 shows electronic device 800 having various means, it is to be understood that implementing or having all of the means shown is not a requirement. More or fewer means may alternatively be implemented or provided.

In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, where the computer program includes program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via communication means 809, or from storage means 808, or from ROM 802. When the computer program is executed by the processing device 801, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are executed.

It should be noted that the above-mentioned computer-readable medium in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. A computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave carrying computer-readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can transmit, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device . Program code embodied on a computer readable medium may be transmitted by any appropriate medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

In some embodiments, the client and the server can communicate using any currently known or future-developed network protocols such as HTTP (HyperText Transfer Protocol), and can communicate with digital data in any form or medium (eg, communication network) interconnections. Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), internetworks (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network of.

The above-mentioned computer-readable medium may be included in the above-mentioned electronic device, or may exist independently without being incorporated into the electronic device.

The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the electronic device, the electronic device is made to execute the above-mentioned method of the present disclosure.

Computer program code for carrying out the operations of the present disclosure can be written in one or more programming languages, or combinations thereof, including object-oriented programming languages—such as Java, Smalltalk, C++, and conventional Procedural Programming Language - such as "C" or a similar programming language. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as through an Internet service provider). Internet connection).

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, program segment, or portion of code that contains one or more logical functions for implementing specified executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by a dedicated hardware-based system that performs the specified functions or operations , or may be implemented by a combination of dedicated hardware and computer instructions.

The units involved in the embodiments described in the present disclosure may be implemented by software or by hardware. Wherein, the name of a unit does not constitute a limitation of the unit itself under certain circumstances.

The functions described herein above may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), System on Chips (SOCs), Complex Programmable Logical device (CPLD) and so on.

In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, portable computer discs, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.

According to one or more embodiments of the present disclosure, an interaction method based on extended reality is provided, including: displaying a virtual reality space; and displaying a virtual keyboard object in the virtual reality space in response to a user's first preset operation , the virtual keyboard object includes a plurality of first virtual keys arranged around a preset blank area; in response to a second preset operation of the user, the selected first virtual key is determined.

According to one or more embodiments of the present disclosure, the characters corresponding to the first virtual keys are the same as the characters corresponding to the keys in the Jiugongge input method; the number of the first virtual keys is eight.

According to one or more embodiments of the present disclosure, the virtual keyboard object includes a first virtual keyboard object and a second virtual keyboard object arranged side by side; the first virtual keyboard object includes a plurality of The first virtual key is distributed, and the second virtual keyboard object includes a plurality of first virtual keys arranged around the second preset blank area; the first virtual keyboard object is used to be controlled by the first hand-held controller, The second virtual keyboard object is for being controlled by a second handheld controller.

According to one or more embodiments of the present disclosure, the second preset operation includes an operation for controlling the virtual ray to select the first virtual key; the method further includes: after displaying the virtual keyboard object, setting the The initial direction of the virtual ray is set to face the preset blank area.

According to one or more embodiments of the present disclosure, the second preset operation includes an operation of changing the pose of the controller; the determining the selected first virtual key in response to the second preset operation of the user includes: In response to the operation of changing the pose of the controller, determine the motion direction of the controller; and determine the selected first virtual key based on the motion direction.

According to one or more embodiments of the present disclosure, the determining the selected first virtual key in response to the second preset operation of the user includes: rotating the virtual keyboard object in response to the second preset operation Alternatively, the key pointing object used to indicate the currently selected key is rotated, and the selected first virtual key is determined based on the first virtual key indicated by the key pointing object after the rotation stops.

According to one or more embodiments of the present disclosure, the first virtual keys are arranged in a ring.

According to one or more embodiments of the present disclosure, there is provided an information interaction device based on extended reality, including: a space display unit for displaying a virtual reality space; a keyboard display unit for responding to the user's first preset The operation is to display a virtual keyboard object in the virtual reality space, and the virtual keyboard object includes a plurality of first virtual keys arranged around a preset blank area; the button determination unit is configured to respond to a second preset operation of the user to determine the selected first virtual key.

According to one or more embodiments of the present disclosure, there is provided an electronic device, including: at least one memory and at least one processor; wherein, the memory is used to store program codes, and the processor is used to call the The stored program code enables the electronic device to execute the extended reality-based interaction method provided according to one or more embodiments of the present disclosure.

According to one or more embodiments of the present disclosure, there is provided a non-transitory computer storage medium, the non-transitory computer storage medium stores program code, and when the program code is executed by a computer device, the computer device The extended reality-based interaction method provided according to one or more embodiments of the present disclosure is executed.

The above description is only a preferred embodiment of the present disclosure and an illustration of the applied technical principle. Those skilled in the art should understand that the disclosure scope involved in this disclosure is not limited to the technical solution formed by the specific combination of the above-mentioned technical features, but also covers the technical solutions formed by the above-mentioned technical features or Other technical solutions formed by any combination of equivalent features. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

In addition, while operations are depicted in a particular order, this should not be understood as requiring that the operations be performed in the particular order shown or to be performed in sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while the above discussion contains several specific implementation details, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims (10)

1. An interactive method based on extended reality, comprising: display virtual reality space; In response to a user's first preset operation, displaying a virtual keyboard object in the virtual reality space, the virtual keyboard object including a plurality of first virtual keys arranged around a preset blank area; In response to the user's second preset operation, the selected first virtual key is determined. 2. The method of claim 1, wherein, The characters corresponding to each first virtual key are the same as the characters corresponding to each key in the Jiugongge input method; the number of the first virtual keys is 8. 3. The method of claim 1, wherein, The virtual keyboard object includes a first virtual keyboard object and a second virtual keyboard object arranged side by side; the first virtual keyboard object includes a plurality of first virtual keys arranged around a first preset blank area, and the second virtual keyboard object The virtual keyboard object includes a plurality of first virtual keys arranged around a second preset blank area; The first virtual keyboard object is used to be controlled by the first handheld controller, and the second virtual keyboard object is used to be controlled by the second handheld controller. 4. The method according to claim 1, wherein the second preset operation includes an operation for controlling the virtual ray to select the first virtual button; The method further includes: after displaying the virtual keyboard object, setting an initial direction of the virtual ray to face the preset blank area. 5. The method according to claim 1, wherein the second preset operation comprises an operation of changing the pose of the controller; The determining the selected first virtual key in response to the user's second preset operation includes: determining the motion direction of the controller in response to the operation of changing the controller pose; determining based on the motion direction The selected first virtual key. 6. The method according to claim 1, wherein the determining the selected first virtual key in response to the second preset operation of the user comprises: in response to the second preset operation, rotating the The virtual keyboard object or the key pointing object used to indicate the currently selected key is rotated, and the selected first virtual key is determined based on the first virtual key indicated by the key pointing object after the rotation stops. 7. The method according to claim 1, wherein the first virtual keys are arranged in a ring. 8. An information interaction device based on extended reality, comprising: a space display unit for displaying a virtual reality space; A keyboard display unit, configured to display a virtual keyboard object in the virtual reality space in response to a user's first preset operation, where the virtual keyboard object includes a plurality of first virtual keys arranged around a preset blank area; The key determining unit is configured to determine the selected first virtual key in response to the user's second preset operation. 9. An electronic device, characterized in that it comprises: at least one memory and at least one processor; Wherein, the memory is used for storing program codes, and the processor is used for invoking the program codes stored in the memory to make the electronic device execute the method according to any one of claims 1-7. 10. A non-transitory computer storage medium, characterized in that, The non-transitory computer storage medium stores program codes, and when the program codes are executed by the computer equipment, the computer equipment executes the method described in any one of claims 1-7.

Images