CN110442245A - Display methods, device, terminal device and storage medium based on physical keyboard - Google Patents

Abstract

The embodiment of the present application discloses a display method, device, terminal device and storage medium based on a physical keyboard. The display method based on a physical keyboard includes: acquiring first relative spatial position information between the terminal device and the physical keyboard, the The physical keyboard includes a plurality of keys; one or more keys with a control function in the physical keyboard are determined; according to the determined control function corresponding to each key, the virtual indication content corresponding to each key is obtained respectively; For the first relative spatial position information, obtain the second relative spatial position information between the terminal device and each button respectively; based on the second relative spatial position information corresponding to each button and the virtual indication content, A virtual indication matching each of the determined keys is generated, and the display position of the virtual indication corresponds to the matched key. The method can facilitate the user to control through the physical keyboard, and realize the control function required by the user.

Description

Display method, device, terminal device and storage medium based on physical keyboard

technical field

The present application relates to the field of display technology, and more particularly, to a display method, device, terminal device and storage medium based on a physical keyboard.

Background technique

Currently, physical keyboards are often used to enter information and commands. In the use of the physical keyboard, the user usually inputs according to the content printed on the keycap of the physical keyboard. However, when the user uses the keyboard to input information and instructions, there is a situation in which the key function is unclear or forgotten, which is inconvenient for the user to operate.

SUMMARY OF THE INVENTION

The embodiments of the present application propose a display method, device, terminal device and storage medium based on a physical keyboard, which can facilitate the user to control through the physical keyboard.

In a first aspect, an embodiment of the present application provides a display method based on a physical keyboard, the method comprising: acquiring first relative spatial position information between a terminal device and a physical keyboard, the physical keyboard including a plurality of keys; determining One or more keys with a control function in the physical keyboard; according to the determined control function corresponding to each key, the virtual indication content corresponding to each key is obtained respectively; according to the first relative spatial position information , respectively obtain the second relative spatial position information between the terminal device and each key; The virtual indication of the virtual indication, the display position of the virtual indication corresponds to the matched key.

In a second aspect, an embodiment of the present application provides a display device based on a physical keyboard, the device includes: a first position acquisition module, a key determination module, a content acquisition module, a second position acquisition module, and a content generation module, wherein, The first position acquisition module is used to acquire the first relative spatial position information between the terminal device and a physical keyboard, and the physical keyboard includes a plurality of keys; the key determination module is used to determine that the physical keyboard has a control function one or more keys; the content acquisition module is used to obtain virtual indication content corresponding to each key according to the determined control function corresponding to each key; the second position acquisition module is used to obtain the virtual indication content corresponding to each key; For the first relative spatial position information, the second relative spatial position information between the terminal device and each key is obtained respectively; the content generation module is configured to based on the second relative space corresponding to each key The position information and the virtual instruction content are used to generate virtual instructions respectively matched with the determined keys, and the display positions of the virtual instructions correspond to the matched keys.

In a third aspect, embodiments of the present application provide a terminal device, including: one or more processors; a memory; and one or more application programs, wherein the one or more application programs are stored in the memory and The one or more programs are configured to be executed by the one or more processors, and the one or more programs are configured to execute the display method based on the physical keyboard provided in the first aspect.

In a fourth aspect, an embodiment of the present application provides a storage medium, where program codes are stored in the computer-readable storage medium, and the program codes can be invoked by a processor to execute the display based on the physical keyboard provided in the first aspect above method.

In the solution provided by the present application, by obtaining the first relative spatial position information between the terminal device and the physical keyboard, the physical keyboard includes a plurality of keys, one or more keys with a control function in the physical keyboard are determined, and each key is determined according to The corresponding control function obtains the virtual indication content corresponding to each key, respectively obtains the second relative spatial position information between the terminal device and each key according to the first relative spatial position information, and based on the corresponding information of each key The second relative spatial position information and the virtual indication content generate virtual indications that are respectively matched with the determined buttons, and the display positions of the virtual indications correspond to the matched keys, so that the user can operate the keys with control functions according to the virtual indications, Realize the control function of virtual content, which is convenient for users to interact with virtual content.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the drawings that are used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can also be obtained from these drawings without creative effort.

FIG. 1 shows a schematic diagram of an application scenario applicable to this embodiment of the present application.

FIG. 2 shows a flowchart of a display method based on a physical keyboard according to an embodiment of the present application.

FIG. 3 shows a schematic diagram of a display effect provided according to an embodiment of the present application.

FIG. 4 shows a flowchart of a display method based on a physical keyboard according to another embodiment of the present application.

FIG. 5 shows a flowchart of a display method based on a physical keyboard according to yet another embodiment of the present application.

FIG. 6 shows a flowchart of a display method based on a physical keyboard according to yet another embodiment of the present application.

FIG. 7 shows a schematic diagram of a display effect provided according to another embodiment of the present application.

FIG. 8 shows a schematic diagram of another display effect provided according to still another embodiment of the present application.

FIG. 9 shows a flowchart of a display method based on a physical keyboard according to still another embodiment of the present application.

FIG. 10 shows a schematic diagram of a display effect provided according to another embodiment of the present application.

FIG. 11 shows a structural block diagram of a display device based on a physical keyboard provided according to an embodiment of the present application.

FIG. 12 is a block diagram of a terminal device for executing the display method based on a physical keyboard according to an embodiment of the present application.

FIG. 13 is a storage unit for storing or carrying a program code for implementing a display method based on a physical keyboard according to an embodiment of the present application according to an embodiment of the present application.

Detailed ways

In order to make those skilled in the art better understand the solutions of the present application, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application.

The following describes application scenarios of the virtual content display method provided by the embodiments of the present application.

Referring to FIG. 1, an augmented reality system 10 provided in one embodiment is shown. The augmented reality system 10 includes: a terminal device 100 and a physical keyboard 200 , wherein the terminal device 100 is connected to the physical keyboard 200 .

In some embodiments, the terminal device 100 may be a head-mounted display device, or a mobile device such as a mobile phone or a tablet. When the terminal device 100 is a head-mounted display device, the head-mounted display device may be an integrated head-mounted display device, or may be a head-mounted display device connected to an external electronic device. The terminal device 100 can also be a smart terminal such as a mobile phone connected to an external/accessible head-mounted display device, that is, the terminal device 100 can be used as a processing and storage device of the head-mounted display device, and can be inserted into or connected to the external head-mounted display device. , the virtual content is displayed on the head-mounted display device.

In some embodiments, markers 201 may be provided on the physical keyboard. Wherein, the marker 201 may be attached to or integrated on the physical keyboard 200, which is not limited herein. The number of markers 201 provided on the physical keyboard 200 may not be limited, and the number of markers 201 may be one or more. As an embodiment, the number of markers 201 is 4, and the 4 markers 201 can be respectively disposed in the areas where the four corners of the physical keyboard 200 are located, so as to improve the ability to locate the physical keyboard 200 and its keys through the markers 201 Accuracy.

In some embodiments, the terminal device 100 and the physical keyboard 200 can be connected by communication methods such as Bluetooth, WiFi (Wireless-Fidelity, wireless fidelity), ZigBee (Zifeng technology), etc., or can be connected by wired communication such as data lines. Not limited.

When the terminal device 100 and the physical keyboard 200 are used together, the marker 201 can be located within the visual range of the terminal device 100 , so that the terminal device 100 can capture an image containing the marker 201 to identify and track the marker 201 , obtain the spatial position information such as the position and posture of the marker 201 relative to the terminal device 100, and the identification results such as the identity information of the marker 201, and then obtain the spatial position information such as the position and posture of the physical keyboard 200 relative to the terminal device 100, so as to realize the physical Location tracking of the keyboard 200 and the various keys it contains. The terminal device 100 can display corresponding virtual content according to the relative position and posture information with the physical keyboard 200 .

In some embodiments, the marker 201 is a pattern with a topology structure, and the topology structure refers to the connectivity between sub-markers and feature points in the marker. In some embodiments, the marker 201 may also be a light-spot mark, and the terminal device can obtain spatial position information such as relative position and attitude by tracking the light spot. In a specific embodiment, a light spot and an inertial measurement unit (IMU) can be set on the physical keyboard 200, and the terminal device can collect an image of the light spot on the physical keyboard 200 through an image sensor, and use the inertial measurement unit The measurement data is acquired, and the relative spatial position information between the physical keyboard 200 and the terminal device 100 can be determined according to the light spot image and the measurement data, so as to realize the positioning and tracking of the physical keyboard 200 . The light spots set on the physical keyboard 200 may be visible light spots or infrared light spots, and the number of light spots may be one or a light spot sequence composed of multiple light spots.

In some implementations, a plurality of keys 202 may be included on the physical keyboard 200 . The physical keyboard can generate operation data according to the user's operation on the keys 202, and send the operation data to the terminal device 100. After receiving the operation data, the terminal device 100 can perform corresponding operations, such as controlling the displayed virtual content, etc. .

In some embodiments, the physical keyboard 200 may further include a touchpad, and the touchpad may be disposed on the surface of the physical keyboard 200 so that the user can operate and control through the touchpad.

In some embodiments, the physical keyboard 200 can also be connected to other peripheral devices, and other peripheral devices can generate operation data and send it to the terminal device through the physical keyboard 200, so as to realize operation and control through other peripheral devices. Other peripheral devices may include a mouse, a handle, etc., which are not limited here. The physical keyboard 200 and other peripherals can be connected in a wired manner, for example, other peripherals are connected to the peripheral interface of the physical keyboard 200 through a connecting line; the physical keyboard 200 and other peripherals can also be connected wirelessly, for example, by Bluetooth, ZigBee and other wireless communication methods are connected.

In some embodiments, the physical keyboard 200 may be a foldable portable keyboard, and the user can fold the physical keyboard 200 to reduce the occupied area of the physical keyboard 200, thereby facilitating the user to carry.

It should be noted that the physical keyboard 200 is not limited to the form of a physical keyboard. In some embodiments, the physical keyboard 200 may also be a virtual keyboard displayed on a touch screen of a mobile terminal (eg, a tablet computer, a mobile phone, etc.).

A specific display method based on a physical keyboard will be introduced below.

Referring to FIG. 2, an embodiment of the present application provides a display method based on a physical keyboard, which can be applied to a terminal device in the above-mentioned augmented reality system. The method may include:

Step S110: Obtain first relative spatial position information between the terminal device and a physical keyboard, where the physical keyboard includes a plurality of keys.

In some embodiments, the physical keyboard may be used in scenarios where a control function is implemented, such as scenarios where the physical keyboard is used to play electronic games, used to control video images, and used to input content.

When the physical keyboard is used to realize the control function, the terminal device and the physical keyboard can be in the same on-site environment, and the markers on the physical keyboard are within the field of view of the image acquisition device of the terminal device, so that the terminal device can track the markers by identifying the markers. to the physical keyboard.

In some implementations, the terminal device may acquire first relative spatial position information between the terminal device and the physical keyboard, so that the terminal device generates and displays related virtual content according to the first relative spatial position information. Specifically, the terminal device can identify the marker on the physical keyboard, so as to obtain the first relative spatial position information between the terminal device and the physical keyboard according to the recognition result of the identified marker.

The identification result of the terminal device identifying the marker includes at least the position information, posture information, etc. of the marker relative to the terminal device, as well as the identity information of the marker. Each marker can have different identity information, so that the terminal device can physically The first relative spatial position information between the terminal device and the physical keyboard is obtained from the position set on the keyboard and the above identification result. The first relative spatial position information between the terminal device and the physical keyboard may include relative position information and attitude information between the terminal device and the physical keyboard, and the attitude information may be the orientation and rotation angle of the physical keyboard relative to the terminal device.

Certainly, the specific manner of acquiring the first relative spatial position information between the terminal device and the physical keyboard may not be limited in this embodiment of the present application.

Step S120: Determine one or more keys having a control function in the physical keyboard.

In some embodiments, one or more keys of the physical keyboard may have control functions, that is, these keys may be configured with corresponding control functions, and the correspondence between keys with control functions and their specific control functions may be stored in the terminal device. For example, in a game scenario, at least part of the keys on the physical keyboard can be configured as skill keys, item keys, game parameter adjustment keys, etc.; for another example, in a video or music playback scenario, at least part of the keys on the physical keyboard Can be configured as playback control, pause control, volume control, playback progress control and other functions; also for example, in the scenario of text content input, at least some of the keys on the physical keyboard can be configured as shortcut functions such as copy and paste. The role of shortcut keys.

In some embodiments, when the physical keyboard is used to implement the control function, the terminal device may determine one or more keys with the control function among all the keys included in the physical keyboard. It depends on the actual situation and needs. Specifically, the terminal device may acquire one or more keys with control functions on the physical keyboard according to the locally stored correspondence between keys and control functions. The corresponding relationship may be stored in the terminal device in the form of a configuration file of the physical keyboard, and the type of the configuration file may be a table or the like, which is not limited herein. The buttons with control functions and the corresponding relationship between the buttons and the control functions may be configured by the user according to requirements and preferences, or may be preset by the manufacturer, which is not limited herein.

For example, in the scene of music playback control, some keys of the physical keyboard have playback control functions, and the corresponding relationship between the keys and the playback control functions may be the control function of the button F5 corresponding to the pause playback, the button F6 corresponding to the control function of the next song, and the corresponding relationship between the buttons and the playback control function. F7 corresponds to the control function of the previous song, and the button F8 corresponds to the function of double-speed playback. According to the corresponding relationship, the terminal device can determine that the buttons with the playback control function on the physical keyboard are F5, F6, F7 and F8. The above music playback control The corresponding relationship between the buttons and the control functions in the scene is only an example, and is not intended to be limiting.

Step S130: According to the determined control function corresponding to each key, obtain the virtual indication content corresponding to each key respectively.

The terminal device can obtain the virtual indication content corresponding to each key in the one or more keys with control function, the virtual indication content is used for the terminal device to generate a virtual indication corresponding to the key with control function, and the virtual indication can be used to indicate the key of the key. control function. The virtual indication content may include 3D model data for generating the virtual indication, and the 3D model data may include the color, model vertex coordinates, model outline data and the like for constructing a model corresponding to the 3D model. As an embodiment, the virtual indication content may be icon content, and the icon content is used to represent a specific control function; as another embodiment, the virtual indication content may also be text content, and the text content may be descriptive that summarizes the control function Text, such as a name, etc.; as another embodiment, the virtual indication content may also be a combination of icons and text, and the specific virtual indication content may not be limited.

In some embodiments, when the terminal device acquires the virtual indication content corresponding to each key with a control function, it can determine the virtual indication content corresponding to the control function of each key according to the determined control function corresponding to each key . The virtual indication content corresponding to different control functions may be different, the corresponding relationship between the virtual indication content and the control function may be stored in the terminal device, and the terminal device may determine the control corresponding to each button according to the corresponding relationship between the virtual indication content and the control function. The virtual instruction content corresponding to the function, and the virtual instruction content corresponding to the control function of each key is used as the virtual instruction content corresponding to each key.

Step S140: Acquire second relative spatial position information between the terminal device and each key, respectively, according to the first relative spatial position information.

The terminal device can determine the second relative spatial position information between the terminal device and each key with a control function according to the obtained first relative spatial position information between the terminal device and the physical keyboard, so that the virtual indication and the key can be generated subsequently. correspond.

In some embodiments, the terminal device may determine the position of each key in all keys on the physical keyboard (that is, the relative position of the key relative to the physical keyboard as a whole) and the first relative spatial position information between the terminal device and the physical keyboard. The second relative spatial position information between the terminal device and each key with a control function. As a specific implementation manner, the terminal device may determine, according to the positional relationship between each key on the physical keyboard and the marker, and the first relative spatial position information between the terminal device and the physical keyboard, The second relative spatial position information between the keys of the function.

In some embodiments, when the physical keyboard is a virtual keyboard displayed on the touch screen of the mobile terminal, since the positions of the keys in the virtual keyboards of different types or formats are displayed on the touch screen are different, the terminal device can The type or format of the virtual keyboard displayed by the terminal determines the position of each key of the virtual keyboard on the touch screen. For example, the virtual keyboard can be a 9-key type or a 26-key type, etc. and the first relative spatial position information to determine the second relative spatial position information between the terminal device and each key.

In this embodiment of the present application, the order in which the terminal device acquires the virtual indication content corresponding to each button and the sequence in which the terminal device acquires the second relative spatial position information between each button may not be limited. For example, step S140 may be performed in Step S130 is performed before step S130, and step S140 may also be performed after step S130.

Step S150: Based on the second relative spatial position information corresponding to each key and the virtual indication content, generate virtual indications respectively matching the determined keys, and the display positions of the virtual indications correspond to the matched keys.

After the terminal device obtains the second relative spatial position information between the terminal device and each button with control function, and the virtual indication content corresponding to each button with control function, it can generate a corresponding information about each button with control function. The virtual indications matched with the keys are displayed, and the generated virtual indications are displayed, so that the user can see the virtual indications matched with each key having a control function.

In some embodiments, the terminal device can obtain the rendering position of the virtual indication content in the virtual space according to the second relative spatial position information with each key having a control function, and then generate and render a three-dimensional image according to the rendering position. Virtual instructions, and display rendered virtual instructions. The position of the displayed virtual indication corresponds to the button corresponding to the virtual indication. The position of the virtual indication refers to the position where the terminal device superimposes and displays the virtual indication in the real environment by means of augmented reality. The virtual indication to the location in the real environment.

In some embodiments, the terminal device may obtain information about the second relative spatial position between each key with a control function and the relative positional relationship between each key with a control function and the corresponding virtual indication Each virtual indication needs to superimpose the spatial position coordinates of the displayed position, and convert each obtained spatial position coordinate into the spatial coordinates in the virtual space, so as to obtain the rendering coordinates of each virtual indication in the virtual space, that is, get The rendering position of each virtual indication. Wherein, the rendering position can be used as the rendering coordinates of the virtual indication, so as to realize the rendering of the virtual indication at the rendering position, and the rendering coordinates refer to the virtual indication in the virtual space with a preset origin (for example, the origin of the world coordinates, the origin of the virtual camera, etc.) 3D space coordinates. As an embodiment, the relative positional relationship between the position where each virtual indication needs to be superimposed and displayed and each key with control function may be that the position where the virtual indication needs to be superimposed and displayed is at the position of the key with control function.

After the terminal device obtains the rendering coordinates of each virtual indication in the virtual space, the terminal device can construct each virtual indication according to the content of each virtual indication, and render each virtual indication according to the rendering coordinates of each virtual indication, wherein rendering the virtual indication The instruction can obtain the vertex coordinates and color values of each vertex in the virtual instruction.

In some embodiments, after the terminal device generates each virtual indication, it can display all the virtual indications. Specifically, after the terminal device constructs and renders a three-dimensional virtual indication, it can convert the virtual indication into a virtual picture, and obtain display data of the virtual indication, and the display data can include the RGB value of each pixel in the display picture and the corresponding pixel point. Coordinates, etc., the terminal device can generate a display screen according to the display data, and project the display screen onto the display lens through the display screen or the projection module, thereby displaying a virtual instruction. The user can see the display lens through the display lens of the head-mounted display device. The virtual indications are superimposed and displayed on the keys with control functions on the physical keyboard in the real world, to achieve an augmented reality display effect, and it is convenient for users to know the control functions of each key according to the virtual indication corresponding to each key.

In some embodiments, when the terminal device also displays a virtual screen, the user can use the physical keyboard to interact with the virtual screen, so as to facilitate the user to operate the keys with the control function according to the virtual instructions, thereby realizing the control of the virtual screen. control.

For example, as shown in FIG. 3 , the key A in the physical keyboard 200 has the function of zooming in the content, and the key D has the function of reducing the content, then the terminal device can generate a virtual zoom-in function icon 301 corresponding to the key A and corresponding to the key D The virtual zoom-in function icon 302 is displayed, and the virtual zoom-in function icon 301 and the virtual zoom-out function icon 302 are displayed. The user can view the virtual zoom-in function at the position of the button A through the display lens of the head-mounted display device. The icon 301 and the virtual zoom-out function icon 302 located at the position of the button B, thus it is understood that the button A has the function of zooming in the content, and the button D has the function of reducing the content, which is convenient for the user to use the physical keyboard 200 to control the zoom-in content and reduce the content control.

In the display method based on the physical keyboard provided by the embodiment of the present application, the terminal device can generate virtual instructions matching the keys with control functions according to the specific control functions corresponding to the keys with control functions, so that the user can The function keys are operated to realize the control function of the virtual content, which is convenient for the user to interact with the virtual content.

Referring to FIG. 4, another embodiment of the present application provides a display method based on a physical keyboard, which can be applied to a terminal device in the above-mentioned display system. The method may include:

Step S210: Obtain first relative spatial position information between the terminal device and a physical keyboard, where the physical keyboard includes a plurality of keys.

In this embodiment of the present application, for step S210, reference may be made to the content of the foregoing embodiment, and details are not described herein again.

Step S220: Determine the current usage scenario.

In the embodiments of the present application, since different usage scenarios require different use of the keys of the physical keyboard and the control functions of the keys, for different usage scenarios, the keys with control functions corresponding to the usage scenarios and each of the keys with control functions may be configured button control function.

In some embodiments, the number of control functions required to be controlled by using a physical keyboard may be different for different usage scenarios, so different numbers of keys with control functions may be set in different usage scenarios. For example, in game scenarios of different games, Each game requires a different number of actions from the user, so the number of keys on a physical keyboard with controls varies. Further, according to the number of keys with a control function required by each usage scenario, the number of keys on the physical keyboard can be configured as the keys with a control function corresponding to each usage scenario.

In some embodiments, one or more identical keys may also exist between keys with control functions corresponding to different usage scenarios. In addition, in some different specific usage scenarios, these same buttons can also have the same functions. For example, in music playback scenarios and video playback scenarios, the buttons with control functions can all be set to F5, F6, F7 and F8 , and F5 is the control function of pausing playback in the music playback scene and video playback scene, F6 is the control function of the next program in the music playback scene and video playback scene, and F7 is in the music playback scene and the video playback scene. For the control function of the previous program, F8 is the control function of double-speed playback in the music playback scene and the video playback scene. The above buttons and control functions are only examples.

Of course, the keys with control functions corresponding to the usage scenarios in the application embodiments and the control functions corresponding to each key can be configured according to the actual usage scenarios, user requirements, etc., which are not limited herein.

The terminal device can acquire the current usage scenario, so as to determine the keys with control functions in the physical keyboard and the control function of each key according to the current usage scenario. As an implementation manner, the terminal device may determine the current usage scenario according to the currently running application. For example, when the application currently running on the terminal device is a video playback program, the current usage scenario may be determined as a video playback scenario. As another implementation manner, the terminal device may determine the target usage scenario as the current usage scenario according to the user's selection operation on the target usage scenario among the multiple usage scenarios. The manner in which the terminal device obtains the current usage scenario may not be limited.

Step S230: Determine one or more keys with control functions in the physical keyboard according to the usage scenario, and determine the control function corresponding to each key.

In some embodiments, the corresponding relationship between each usage scenario and a key with a control function and the corresponding relationship between each key in each usage scenario and a specific control function may be pre-configured. The terminal device can determine the corresponding control key in the physical keyboard corresponding to the current use scenario according to the corresponding relationship between the use scenario and the key with the control function, and determine the corresponding relationship between each key in each use scenario and the specific control function. The control function corresponding to each key in the current usage scene.

As an implementation manner, the terminal device may store a relationship correspondence table among scene identifiers, buttons with control functions, and control functions of each button, as shown in Table 1,

Table 1

As another implementation manner, a key configuration file for each usage scenario may be stored in the terminal device, and the configuration file may include information used to describe keys with control functions corresponding to the usage scenarios and information used to describe the corresponding keys of each key Information about control functions. The terminal device can read the configuration file corresponding to the usage scenario according to the current usage scenario, so as to obtain the keys on the physical keyboard corresponding to the current usage scenario and the control function corresponding to each key according to the configuration file.

Certainly, the implementation manner of specifically determining the key corresponding to the current usage scenario and the control function of each key may not be regarded as a limitation.

Step S240: According to the determined control function corresponding to each key, obtain the virtual indication content corresponding to each key respectively.

Step S250: Acquire second relative spatial position information between the terminal device and each key, respectively, according to the first relative spatial position information.

Step S260: Based on the second relative spatial position information corresponding to each key and the virtual indication content, generate virtual indications respectively matching the determined keys, and the display positions of the virtual indications correspond to the matched keys.

In this embodiment of the present application, for steps S240 to S260, reference may be made to the content of the foregoing embodiments, and details are not described herein again.

In the display method based on a physical keyboard provided by the embodiment of the present application, keys with control functions are determined according to the current usage scenario, and the control function corresponding to each key is determined, and then virtual instructions matching these keys are generated, so that the user can Virtual instructions, operate the keys with control functions, realize the control functions required by the current usage scene, and facilitate the user to control through the physical keyboard.

Referring to FIG. 5, another embodiment of the present application provides a display method based on a physical keyboard, which can be applied to a terminal device in the above-mentioned display system. The method may include:

Step S310: Obtain a selection instruction for selecting a key, and select one or more keys of the physical keyboard based on the selection instruction.

In some embodiments, when the physical keyboard is used to perform the control function, the keys on the physical keyboard with the control function can be set by the user. The terminal device may display a virtual key selection screen for selecting keys, and then select one or more keys of the physical keyboard according to a selection instruction input by the user, and configure the selected keys as keys with a control function. The input of the selection instruction can be input through a physical keyboard, or can be input according to other interactive devices, such as a mouse and so on.

In the above manner, users can set buttons with control functions to perform control operations according to personal needs and preferences, so as to meet the needs and preferences of different users. For example, in the same game scene, user 1 likes to use W, A, S, and D for direction control, and user 2 likes to use 1, 2, 3, and 5 for direction control, so that different users' preferences can be met.

Step S320: Acquire a setting instruction, and set the control function corresponding to each key based on the setting instruction.

In some embodiments, after selecting one or more keys with a control function in the physical keyboard, the terminal device can receive a setting instruction for setting the control function of each key, and according to the setting instruction, assign each key to corresponding The control function is set to the control function that matches the setting instruction.

After setting the control function corresponding to each key with control function, the terminal device can store the corresponding relationship between each key with control function and control function, so that when the physical keyboard is used to realize these control functions , the terminal device can determine the keys with control functions and the control functions corresponding to each key according to the corresponding relationship.

It should be noted that, in step S310 and step S320, each time a button with a control function can be selected, and then the control function of the button can be set, or all buttons with a control function can be selected, and then the corresponding button of each button can be set respectively. The control function is not limited here.

Step S330: Obtain first relative spatial position information between the terminal device and a physical keyboard, where the physical keyboard includes a plurality of keys.

Step S340: Determine one or more keys having a control function in the physical keyboard.

Step S350: According to the determined control function corresponding to each key, obtain the virtual indication content corresponding to each key respectively.

Step S360: Acquire second relative spatial position information between the terminal device and each key, respectively, according to the first relative spatial position information.

Step S370: Based on the second relative spatial position information corresponding to each key and the virtual indication content, generate virtual indications respectively matching the determined keys, and the display positions of the virtual indications correspond to the matched keys.

In this embodiment of the present application, for steps S330 to S370, reference may be made to the contents of the foregoing embodiments, and details are not described herein again.

In the display method based on a physical keyboard provided by the embodiment of the present application, by determining the control functions corresponding to the keys with control functions, and then generating virtual instructions matching these keys, the user can perform operations on the keys with control functions according to the virtual instructions. operation to realize the control function of the virtual content, so as to facilitate the user to interact with the virtual content. In addition, the buttons with control functions and the specific control functions corresponding to the buttons can be set by the user, so the needs and preferences of the users are also met.

Referring to FIG. 6, another embodiment of the present application provides a display method based on a physical keyboard, which can be applied to a terminal device in the above-mentioned display system. The method may include:

Step S400: Obtain first relative spatial position information between the terminal device and a physical keyboard, where the physical keyboard includes a plurality of keys.

Step S410: Determine one or more keys having a control function in the physical keyboard.

Step S420: According to the determined control function corresponding to each key, obtain the virtual indication content corresponding to each key respectively.

Step S430: Acquire second relative spatial position information between the terminal device and each key, respectively, according to the first relative spatial position information.

Step S440: Based on the second relative spatial position information corresponding to each key and the virtual indication content, generate virtual indications respectively matching the determined keys, and the display positions of the virtual indications correspond to the matched keys.

In this embodiment of the present application, for steps S410 to S440, reference may be made to the contents of the foregoing embodiments, and details are not described herein again.

Step S450: Acquire first content data of the virtual screen to be displayed.

In this embodiment of the present application, the terminal device may also display the virtual picture to be displayed. The virtual picture to be displayed may be a virtual picture corresponding to an operating system or an opened application. For example, the virtual screen to be displayed may be a virtual game screen; another example, the virtual screen to be displayed may be a virtual video screen, another example; the virtual screen to be displayed may be a virtual document screen, and the above virtual screen is only an example.

In some embodiments, the terminal device may acquire first content data of the virtual image to be displayed, the first content data is used by the terminal device to generate the virtual image, and the first content data may include three-dimensional model data used to generate the virtual image , for the specific content of the first content data, reference may be made to the description of the virtual indication content in the foregoing embodiment, which will not be repeated here.

Step S460: Obtain the first relative positional relationship between the first setting area and the physical keyboard, where the first setting area is an overlay area where the virtual image needs to be superimposed in the real scene, and the first setting area is outside the area where the physical keyboard is located .

In some embodiments, the terminal device may acquire the first relative positional relationship between the superimposed position where the virtual image needs to be superimposed in the real scene and the physical keyboard, so as to generate a virtual image for display. The virtual image can be displayed in one area, so the terminal device can obtain the first relative relationship between the superimposed area (the first set area) where the virtual image needs to be superimposed in the real scene and the physical keyboard.

In some embodiments, the first setting area may be located outside the area where the physical keyboard is located, so that the virtual image will not overlap with the area where the physical keyboard is located after being displayed, so that the user can view the content on the virtual image and the physical keyboard at the same time. The screen can be suspended and displayed in front of the terminal device, etc. As a specific implementation manner, the first setting area may be outside the area where the physical keyboard is located and correspond to one edge area of the physical keyboard, where the edge area refers to the area around the physical keyboard, such as the area where the physical keyboard is located outside and adjacent to the first side edge region.

In some embodiments, the first set area may also form a preset angle with the area where the physical keyboard is located, and the specific size of the preset angle may not be limited, for example, the preset angle may be 45°˜70°, For another example, it may be 65° to 90° or the like. Therefore, after the virtual image is displayed, it is convenient for the user to view the virtual image, and the display effect of the virtual image is improved.

In the above implementation manner, the displayed virtual picture can be used as a computer in combination with the physical keyboard, that is, the virtual picture simulates the display screen of the computer, the physical keyboard simulates the keyboard of the computer, and the user only needs to carry the physical keyboard and the terminal device. Realize the use of computer.

Step S470: Generate a virtual image and display the virtual image according to the first relative spatial position information, the first relative position relationship and the first content data.

In this embodiment of the present application, the terminal device may acquire the rendering position of the virtual image according to the first relative spatial position information and the first relative position relationship, and then display the virtual image after rendering the virtual image according to the rendering position and the first content data. . The terminal device generates a virtual picture and displays the virtual picture, and can refer to the description content of generating and displaying the virtual instruction in the foregoing embodiment.

It should be noted that the sequence of the process of generating and displaying the virtual indication by the terminal device and the process of generating and displaying the virtual picture by the terminal device may not be limited.

Step S480 : when one or more keys are triggered, acquire the display adjustment function corresponding to the triggered key.

In some embodiments, a control function corresponding to a key having a control function in the physical keyboard may be a display adjustment function. The display adjustment function may be display adjustment for the virtual image, for example, control functions such as scale enlargement, scale reduction, and position movement for the virtual image, which are not limited herein.

In some embodiments, the physical keyboard can detect a triggered key, generate operation data according to the triggered key, and send the operation data to the terminal device. The terminal device can receive the operation data generated by the physical keyboard, and determine the triggered keys in the physical keyboard. If the terminal device determines, according to the operation data sent by the physical keyboard, that a key is triggered among the keys with control functions, it can determine the display adjustment function corresponding to the triggered key, so as to perform corresponding display adjustment on the virtual screen.

Step S490: Perform display adjustment corresponding to the display adjustment function on the virtual screen.

In some embodiments, after determining the display adjustment function corresponding to the triggered key, the terminal device may perform display adjustment corresponding to the display adjustment function on the virtual screen. For example, when the display adjustment function corresponding to the triggered key is to scale the virtual screen, the virtual screen can be scaled according to the scale that needs to be scaled when the button is triggered once.

For example, in a scenario where a physical keyboard can be used for document editing, please refer to FIG. 7 and FIG. 8 at the same time, the virtual screen displayed by the terminal device can be a virtual document screen 303. In this scenario, the key A in the physical keyboard 200 has the right pair The document screen 303 has the function of scaling up, and the button D has the function of scaling down the document screen 303. The display position of the zoom function icon corresponding to the button A corresponds to the button A, and the display position of the zoom function icon corresponding to the button D is the same as that of the button D. Corresponding to the button D, when the button D is triggered, the scale of the document screen 303 is reduced.

In the display method of a virtual keyboard provided by the embodiment of the present application, the terminal device can generate a virtual instruction matching the key with a control function according to the specific control function corresponding to the key with a control function, and can also generate a virtual screen to be displayed. The screen is displayed, and the user can operate the keys with the function of adjusting the display of the virtual screen according to the virtual instructions, so as to realize the display adjustment of the virtual screen, and it is convenient for the user to interact with the virtual screen through the physical keyboard.

Referring to FIG. 9 , another embodiment of the present application provides a display method based on a physical keyboard, which can be applied to a terminal device in the above-mentioned display system. The method may include:

Step S500: Obtain first relative spatial position information between the terminal device and a physical keyboard, where the physical keyboard includes a plurality of keys.

Step S510: Determine one or more keys having a control function in the physical keyboard.

Step S520: According to the determined control function corresponding to each key, obtain the virtual indication content corresponding to each key respectively.

Step S530: Acquire second relative spatial position information between the terminal device and each key, respectively, according to the first relative spatial position information.

Step S540: Based on the second relative spatial position information corresponding to each key and the virtual indication content, generate virtual indications respectively matching the determined keys, and the display positions of the virtual indications correspond to the matched keys.

In this embodiment of the present application, for steps S510 to S540, reference may be made to the contents of the foregoing embodiments, and details are not described herein again.

Step S550: Acquire second content data of the virtual input box to be displayed.

In this embodiment of the present application, the terminal device may also display a virtual input box, and the virtual input box may be a text input box used to display text content such as input characters and candidate words in a text input scenario. For example, the virtual input box may be used to display candidate words corresponding to the input method; for another example, the virtual input box may be used to display the content of the text input area in the chat interface. The above virtual input box is only an example.

In some embodiments, the terminal device can obtain the second content data of the virtual input box to be displayed, the second content data is used by the terminal device to generate the virtual input box, and the specific content of the second content data and the first content data can refer to the aforementioned The description of the virtual indication content in the embodiment will not be repeated here.

Step S560: Obtain the second relative positional relationship between the second setting area and the physical keyboard, the second setting area is the superimposition area where the virtual input box needs to be superimposed in the real scene, and the second setting area is located at the edge of the physical keyboard The region where the location is located.

In the embodiment of the present application, different from the first setting area in the foregoing embodiments, the second setting area is located in the area where the edge of the physical keyboard is located, where the edge of the physical keyboard is the location of the periphery of the physical keyboard. For example, the second setting area may be an area that fits the upper edge of the physical keyboard opposite to the terminal device, and the upper edge of the physical keyboard may refer to the long edge close to the Esc (escape) key.

Step S570: Generate a virtual input box and display the virtual input box according to the first relative spatial position information, the second relative position relationship and the second content data.

In this embodiment of the present application, for step S570, reference may be made to the description of step S470 in the foregoing embodiments, and details are not repeated here.

For example, as shown in FIG. 10, the terminal device can generate a virtual input box 304 and display the virtual input box 304. The user can observe the virtual input box 304 and the upper edge of the physical keyboard 200 through the display lens of the head-mounted display device. The position fit gives the user the visual effect that the virtual input box 304 is adsorbed on the physical keyboard 200, so that the user can see the physical keyboard 200 and the virtual input box 304 at the same time when using the physical keyboard 200 for text input, which is convenient for the user to input text.

It should be noted that the sequence of the process of generating and displaying the virtual instruction by the terminal device and the process of generating and displaying the virtual input box by the terminal device may not be limited.

In some embodiments, a control function corresponding to a key with a control function in the physical keyboard may be a display adjustment function, and the terminal device may perform corresponding display adjustment on the virtual input box according to a triggered key in the key with a control function.

Step S580: Acquire the operation data of the keys used for inputting text in the physical keyboard.

In some embodiments, the physical keyboard may include keys for inputting text, and the specific keys for inputting text may not be limited. The terminal device can receive the operation data of the key for inputting text generated by the physical keyboard detecting the triggered key.

In some embodiments, the keys used for inputting text can be the same as the part of the keys with control functions. When the same part of keys needs to be used to input text, the mode of this part of the keys can be switched, so that this part of the keys can be switched. Switch from the mode for controlling functions to the mode for entering text.

Step S590: Generate virtual text information for display in the virtual input box according to the operation data, and display the virtual text information.

In this embodiment of the present application, the terminal device may acquire the text content input by the user according to the operation data of the keys used for inputting text, and generate virtual text information for displaying in the virtual input box according to the text content, and convert the virtual text information into the virtual text information. to display.

In some implementations, the implementation of the terminal device generating a virtual input box and the virtual input box in the embodiment of the present application may also be combined with the implementation of the terminal device generating a virtual screen and displaying the virtual screen in the foregoing embodiments, that is, The terminal device displays both the virtual screen in the foregoing embodiment and the virtual input box. For example, in the scenario where the terminal device displays the virtual document content in the foregoing embodiment, the terminal device can also display a virtual input box, and the virtual input box can be used to display the content (for example, candidate words) corresponding to the input method, so that the user can input When the text content is used, the keys on the physical keyboard and the corresponding content of the input method can be seen at the same time, which is convenient for the user to input text into the virtual document content.

In some embodiments, the terminal device may also generate virtual decoration content for superimposing and display on the physical keyboard, and display the virtual decoration content, so as to achieve a decoration effect on the physical keyboard and improve the visual effect of the physical keyboard. In a specific implementation, the terminal device can generate flickering virtual content with different colors, and the display position of the flickering virtual content can correspond to the surrounding positions of the physical keyboard, and display the flickering virtual content to realize the virtual Fluorescent content overlays are displayed around the physical keyboard. Different virtual decoration contents can be displayed for different usage conditions of the physical keyboard. For example, flickering virtual contents with different colors can be displayed during startup, and a halo effect can be superimposed on each button. When the button is pressed, The color of the halo effect, etc. may be changed, but not limited thereto. As another specific implementation, the terminal device can generate virtual keycap content corresponding to the keycap content of each key (text, patterns, etc. printed on the keycap), and the display position of the virtual keycap content corresponds to the key, The content of the virtual keycap is displayed, so that the content of the virtual keycap is superimposed and displayed on the keys. Of course, the above two specific embodiments can also be combined, so that fluorescent content is superimposed and displayed around the keys, and the virtual keycap content is superimposed and displayed on the keys, so as to achieve a decorative effect on the physical keyboard and improve the user's visual experience.

In the display method based on the physical keyboard provided by the embodiment of the present application, the terminal device can generate a virtual indication matching the key with the control function according to the specific control function corresponding to the key with the control function, so as to facilitate the user to understand the control function corresponding to the key. In addition, the terminal device can also generate a virtual input box to be displayed and display the virtual input box, and the position where the virtual input box is superimposed in the real scene is located in the area where the physical keyboard is located and is close to the area where the edge of the physical keyboard is located, which is convenient for users When the physical keyboard is used for text input, the physical keyboard and the virtual input box can be seen at the same time, which facilitates the user to input text.

Please refer to FIG. 11 , which shows a structural block diagram of a display device 400 based on a physical keyboard provided by the present application. The physical keyboard-based display device 400 is applied to the terminal device in the above-mentioned display system. The physical keyboard-based display device 400 includes: a first position acquisition module 410 , a key determination module 420 , a content acquisition module 430 , a second position acquisition module 440 and a content generation module 450 . Wherein, the first position acquisition module 410 is used to acquire the first relative spatial position information between the terminal device and the physical keyboard, and the physical keyboard includes a plurality of keys; the key determination module 420 is used to determine one or more keys with a control function in the physical keyboard The content acquisition module 430 is used for respectively acquiring virtual indication content corresponding to each button according to the determined control function corresponding to each button; the second position acquisition module 440 is used for respectively Obtain the second relative spatial position information between the terminal device and each key; the content generation module 450 is configured to generate virtual corresponding to the determined keys respectively based on the second relative spatial position information corresponding to each key and the virtual indication content. Indication, the display position of the virtual indication corresponds to the matching key.

In some embodiments, the key determination module 420 may include a scene determination unit and a key acquisition unit. The scene determination unit is used to determine the current usage scenario; the key acquisition unit is used to determine one or more keys with control functions in the physical keyboard according to the usage scenario, and determine the control function corresponding to each key.

In some embodiments, the physical keyboard-based display device 400 may further include an instruction acquisition module and a function setting module. The instruction acquisition module is used to acquire a selection instruction for selecting keys before acquiring the first relative spatial position information between the terminal device and the physical keyboard, and select one or more keys of the physical keyboard based on the selection instruction; the function setting module uses For obtaining the setting instruction, the control function corresponding to each key is set based on the setting instruction.

In some embodiments, the physical keyboard-based display device 400 may further include a first data acquisition module, a first position relationship acquisition module, and a picture generation module. The first data acquisition module is used for; the first position relationship acquisition module is used for; the picture generation module is used for generating a virtual picture and displaying the virtual picture according to the first relative spatial position information, the first relative position relationship and the first content data .

In this embodiment, the physical keyboard-based display device 400 may further include: a trigger detection module and a picture adjustment module. The trigger detection module is used to obtain the display adjustment function corresponding to the triggered key when a key in one or more keys is triggered; the image adjustment module is used to perform display adjustment corresponding to the display adjustment function on the virtual screen.

In some embodiments, the physical keyboard-based display device 400 may further include: a second data acquisition module, a second position relationship acquisition module, and an input box generation module. The second data acquisition module is used to acquire the second content data of the virtual input box to be displayed; the second positional relationship acquisition module is used to acquire the second relative positional relationship between the second setting area and the physical keyboard, the second setting The area is a superimposed area where the virtual input box needs to be superimposed in the real scene, and the second setting area is located in the area where the physical keyboard is located and is close to the area where the edge of the physical keyboard is located; the input box generation module is used for the first relative spatial position information, The second relative positional relationship and the second content data generate a virtual input box and display the virtual input box.

In this embodiment, the physical keyboard-based display device 400 may further include: an operation data acquisition module and an input control module. The operation data acquisition module is used for acquiring operation data of keys used for inputting text in the physical keyboard; the input control module is used for generating virtual text information for display in the virtual input box according to the operation data, and displaying the virtual text information.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, for the specific working process of the above-described devices and modules, reference may be made to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

In several embodiments provided in this application, the coupling between the modules may be electrical, mechanical or other forms of coupling.

In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist physically alone, or two or more modules may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware, and can also be implemented in the form of software function modules.

To sum up, in the solution provided by this application, by acquiring the first relative spatial position information between the terminal device and the physical keyboard, the physical keyboard includes a plurality of keys, and one or more keys with a control function in the physical keyboard are determined. For the control function corresponding to each key, obtain the virtual indication content corresponding to each key, respectively obtain the second relative spatial position information between the terminal device and each key according to the first relative spatial position information, based on each key The second relative spatial position information and the virtual indication content corresponding to the keys generate virtual indications that respectively match the determined keys, and the display positions of the virtual indications correspond to the matched keys, so that the user can control the keys with control functions according to the virtual indications. The operation is performed to realize the control function of the virtual content, so as to facilitate the user to interact with the virtual content.

Please refer to FIG. 12 , which shows a structural block diagram of a terminal device provided by an embodiment of the present application. The terminal device 100 may be a terminal device capable of running an application program, such as a smart phone, a tablet computer, a head-mounted display device, or the like. The terminal device 100 in this application may include one or more of the following components: a processor 110, a memory 120, and an image acquisition device 130. The memory 120 may store one or more computer programs, and the computer programs may be configured to be composed of one or more The plurality of processors 110 are executed for executing the methods described in the foregoing method embodiments.

The processor 110 may include one or more processing cores. The processor 110 uses various interfaces and lines to connect various parts of the entire terminal device 100, and executes by running or executing the instructions, programs, code sets or instruction sets stored in the memory 120, and calling the data stored in the memory 120. Various functions of the terminal device 100 and processing data. Optionally, the processor 110 may adopt at least one of digital signal processing (Digital Signal Processing, DSP), field-programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). implemented in hardware. The processor 110 may integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU), a modem, and the like. Among them, the CPU mainly handles the operating system, user interface and application programs, etc.; the GPU is used for rendering and drawing of the display content; the modem is used to handle wireless communication. It can be understood that, the above-mentioned modem may also not be integrated into the processor 110, and is implemented by a communication chip alone.

The memory 120 may include random access memory (Random Access Memory, RAM), or may include read-only memory (Read-Only Memory). Memory 120 may be used to store instructions, programs, codes, sets of codes, or sets of instructions. The memory 120 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.) , instructions for implementing the following method embodiments, and the like. The storage data area may also store data and the like created by the terminal 100 in use.

In this embodiment of the present application, the image acquisition device 130 is used to acquire images of markers. The image acquisition device 130 may be an infrared camera or a color camera, and the specific camera type is not limited in this embodiment of the present application.

In one embodiment, the terminal device may further include one or more of the following components: a display module, an optical module, a communication module, and a power supply. The display module may include a display control unit, the display control unit is configured to receive a display image of the virtual content rendered by the processor, display and project the display image on the optical module, so that the user can view the virtual content through the optical module . The display module may be a display screen or a projection device, etc., for displaying images. The optical module may adopt an off-axis optical system or a waveguide optical system, and the display image displayed by the display module can be projected to the user's eyes after passing through the optical module. The user can see the display image projected by the display module through the optical module. In some embodiments, the user can also observe the real environment through the optical module, and feel the visual effect after the virtual content and the real environment are superimposed. The communication module can be modules such as Bluetooth, WiFi, ZigBee, etc. The terminal device can communicate and connect with the interactive device through the communication module to exchange information and instructions. The power supply can supply power to the entire terminal equipment to ensure the normal operation of each component of the terminal equipment.

Please refer to FIG. 13 , which shows a structural block diagram of a computer-readable storage medium provided by an embodiment of the present application. The computer-readable medium 800 stores program codes, and the program codes can be invoked by the processor to execute the methods described in the above method embodiments.

The computer readable storage medium 800 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 800 includes a non-transitory computer-readable storage medium. Computer readable storage medium 800 has storage space for program code 810 to perform any of the method steps in the above-described methods. These program codes can be read from or written to one or more computer program products. Program code 810 may be compressed, for example, in a suitable form.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand: it can still be Modifications are made to the technical solutions described in the foregoing embodiments, or some technical features thereof are equivalently replaced; and these modifications or replacements do not drive the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims (10)

1. a display method based on physical keyboard, is characterized in that, described method comprises: acquiring first relative spatial position information between the terminal device and a physical keyboard, where the physical keyboard includes a plurality of keys; determining one or more keys with control functions in the physical keyboard; According to the determined control function corresponding to each key, obtain the virtual indication content corresponding to each key respectively; According to the first relative spatial position information, respectively obtain second relative spatial position information between the terminal device and each key; Based on the second relative spatial position information corresponding to each key and the virtual indication content, a virtual indication matching each of the determined keys is generated, and the display position of the virtual indication corresponds to the matched key. 2. The method according to claim 1, wherein the determining of one or more keys having a control function in the physical keyboard comprises: Determine the current usage scenario; One or more keys having control functions in the physical keyboard are determined according to the usage scenario, and a control function corresponding to each key is determined. 3 . The method according to claim 1 , wherein before acquiring the first relative spatial position information between the terminal device and the physical keyboard, the method further comprises: 4 . obtaining a selection instruction for selecting a key, and selecting one or more keys of a physical keyboard based on the selection instruction; A setting instruction is acquired, and the control function corresponding to each key is set based on the setting instruction. 4. The method according to any one of claims 1-3, wherein the method further comprises: acquiring first content data of the virtual screen to be displayed; Obtain the first relative positional relationship between the first setting area and the physical keyboard, the first setting area is the superposition area where the virtual image needs to be superimposed in the real scene, and the first setting area is in outside the area where the physical keyboard is located; The virtual screen is generated and displayed according to the first relative spatial position information, the first relative position relationship and the first content data. 5. The method according to claim 4, wherein the method further comprises: When a key is triggered in the one or more keys, obtain the display adjustment function corresponding to the triggered key; A display adjustment corresponding to the display adjustment function is performed on the virtual screen. 6. The method according to any one of claims 1-3, wherein the method further comprises: obtaining the second content data of the virtual input box to be displayed; Obtain a second relative positional relationship between a second setting area and the physical keyboard, where the second setting area is an overlay area where the virtual input box needs to be superimposed in the real scene, and the second setting area is located in the area where the edge of the physical keyboard is located; According to the first relative spatial position information, the second relative position relationship and the second content data, the virtual input box is generated and displayed. 7. The method according to claim 6, wherein the method further comprises: Obtain the operation data of the keys used for inputting text in the physical keyboard; The virtual text information for displaying in the virtual input box is generated according to the operation data, and the virtual text information is displayed. 8. A display device based on a physical keyboard, wherein the device comprises: a first position acquisition module, a key determination module, a content acquisition module, a second position acquisition module and a content generation module, wherein, The first position acquisition module is used to acquire first relative spatial position information between the terminal device and a physical keyboard, where the physical keyboard includes a plurality of keys; The key determination module is used to determine one or more keys with a control function in the physical keyboard; The content obtaining module is configured to obtain virtual indication content corresponding to each key according to the determined control function corresponding to each key; The second position obtaining module is configured to obtain the second relative space position information between the terminal device and each key according to the first relative space position information; The content generation module is configured to generate virtual indications that are respectively matched with the determined keys based on the second relative spatial position information corresponding to each key and the virtual indication content, and the display positions of the virtual indications correspond to the matched keys. . 9. A terminal device, comprising: one or more processors; memory; One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs are configured to perform such as The method of any one of claims 1-7. 10. A computer-readable storage medium, wherein a program code is stored in the computer-readable storage medium, and the program code can be invoked by a processor to execute any one of claims 1-7 Methods.