CN110688018A - Control method, device, terminal device and storage medium for virtual screen - Google Patents

Abstract

The present application discloses a control method, device, terminal device and storage medium for a virtual screen, and relates to the technical field of display. The method for controlling a virtual image is applied to a terminal device, and the method includes: acquiring a target image containing a target object; acquiring outline information of the target object according to the target image; determining a first virtual image to be switched in the currently displayed virtual image content; acquiring second virtual content according to the outline information, and switching and displaying the first virtual content as the second virtual content. The method can realize the simple and fast switching of virtual content by using the outline information of the object.

Description

Control method, device, terminal device and storage medium for virtual screen

technical field

The present application relates to the field of computer technology, and more particularly, to a method, device, terminal device and storage medium for controlling a virtual screen.

Background technique

In recent years, with the advancement of science and technology, technologies such as augmented reality (AR, Augmented Reality) and virtual reality (VR, Virtual Reality) have gradually become the focus of research at home and abroad. Taking augmented reality as an example, augmented reality is a technology that increases users' perception of the real world through information provided by a computer system. Perception of a real-world environment or data representing a real-world environment. In AR/VR technology, the control of virtual images is cumbersome, especially in the aspect of content switching of virtual images, so how to quickly and effectively switch the contents of virtual images is an urgent problem to be solved.

SUMMARY OF THE INVENTION

The embodiments of the present application provide a method, device, terminal device and storage medium for controlling a virtual screen, which can quickly and effectively switch and display a first virtual content to a second virtual content by using the outline information of an object.

In a first aspect, an embodiment of the present application provides a method for controlling a virtual image, which is applied to a terminal device. The method includes: acquiring a target image containing a target object; acquiring outline information of the target object according to the target image; determining a currently displayed virtual image The first virtual content to be switched in the screen; the second virtual content is acquired according to the outline information, and the first virtual content is switched and displayed as the second virtual content.

In a second aspect, an embodiment of the present application provides a virtual screen control device, which is applied to a terminal device. The device includes: an image acquisition module, an information identification module, a content determination module, and a content switching module, wherein the image acquisition module is used to acquire Contains a target image of the target object; the information identification module is used to obtain the outline information of the target object according to the target image; the content determination module is used to determine the first virtual content to be switched in the currently displayed virtual picture; the content switching module is used to according to the outline The information acquires the second virtual content, and switches and displays the first virtual content as the second virtual content.

In a third aspect, embodiments of the present application provide a terminal device, including: one or more processors; a memory; and one or more application programs, wherein the one or more application programs are stored in the memory and configured to be One or more processors execute, and one or more application programs are configured to execute the method for controlling a virtual screen provided by the first aspect.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where a program code is stored in the computer-readable storage medium, and the program code can be invoked by a processor to execute the control method for the virtual screen provided in the first aspect above .

In the solution provided by the embodiment of the present application, the terminal device may acquire a target image containing the target object, so as to acquire the outline information of the target object according to the target image, and then the terminal device may determine the first virtual content to be switched in the currently displayed virtual screen, Finally, the terminal device can obtain the second virtual content according to the outline information, and switch and display the first virtual content as the second virtual content, that is, the present application can use the outline information of the object to quickly switch the virtual content, and the virtual content can be switched. Operation is not only simple but also more intelligent.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the drawings that are used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can also be obtained from these drawings without creative effort.

FIG. 1 shows a schematic diagram of an application environment suitable for an embodiment of the present application;

FIG. 2 shows a flowchart of a method for controlling a virtual screen provided by an embodiment of the present application;

3a and 3b show a schematic diagram of a gesture in a method for controlling a virtual screen provided by an embodiment of the present application;

FIG. 4 shows a schematic diagram of another gesture in a method for controlling a virtual screen provided by an embodiment of the present application;

FIG. 5 shows a flowchart of a method for controlling a virtual screen provided by another embodiment of the present application;

6 shows a flowchart of step S240 in a method for controlling a virtual screen provided by another embodiment of the present application;

FIG. 7 shows a flowchart of step S242 in the method for controlling a virtual screen provided by another embodiment of the present application;

FIG. 8 shows another flowchart of step S242 in a method for controlling a virtual screen provided by another embodiment of the present application;

9 shows a schematic diagram of a first virtual content in a method for controlling a virtual screen provided by another embodiment of the present application;

10 shows a schematic diagram of virtual content switching in a method for controlling a virtual screen provided by another embodiment of the present application;

FIG. 11 shows yet another flowchart of step S242 in the method for controlling a virtual screen provided by another embodiment of the present application;

FIG. 12 shows yet another flowchart of step S242 in the method for controlling a virtual screen provided by another embodiment of the present application;

FIG. 13 shows yet another flowchart of step S242 in the method for controlling a virtual screen provided by another embodiment of the present application;

FIG. 14 shows a flowchart of a method for controlling a virtual screen provided by another embodiment of the present application;

FIG. 15 shows a flowchart of step S330 in a method for controlling a virtual screen provided by another embodiment of the present application;

FIG. 16 shows a block diagram of an apparatus for controlling a virtual screen provided by an embodiment of the present application;

FIG. 17 is a block diagram of a terminal device for executing a virtual screen control method according to an embodiment of the present application according to an embodiment of the present application;

FIG. 18 is a storage unit for storing or carrying a program code for implementing the control method of a virtual screen according to an embodiment of the present application according to an embodiment of the present application.

Detailed ways

In order to make those skilled in the art better understand the solutions of the present application, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application.

In AR/VR technology, the adjustment or switching of virtual content is an essential operation, for example, switching different virtual interfaces according to the needs of users, or performing certain virtual objects in the virtual interface according to the needs of users. switch. Virtual content switching usually needs to display an independent switching window, and then the user selects the target virtual content for switching. Obviously, the user's operation is complicated when switching virtual content, which also makes the switching of virtual content very cumbersome.

After research, the inventor proposes the control method, device, terminal device and storage medium of the virtual screen in the embodiments of the present application, which can reduce the complexity of switching virtual content.

An application scenario of the virtual picture control method provided by the embodiment of the present application is introduced below.

Referring to FIG. 1 , a display system provided by an embodiment of the present application is shown, including a terminal device 100 and a handheld object 200 .

In some embodiments, the terminal device 100 may be a head-mounted display device, a mobile phone, a tablet computer, a personal computer, etc., wherein the head-mounted display device may be an integrated head-mounted display device, or may be connected to an external electronic device Head mounted display device. The terminal device 100 can also be a smart terminal such as a mobile phone connected to an external/accessible head-mounted display device, that is, the terminal device 100 can be used as a processing and storage device of the head-mounted display device, and can be inserted into or connected to the external head-mounted display device. , the virtual content 300 is displayed through the head-mounted display device.

The virtual content 300 may be a virtual interface, a window in the virtual interface, or some virtual objects in the virtual interface, and the virtual objects may be virtual characters or some virtual props adapted to the virtual characters For example, guns or ammunition in a gun game can be called virtual objects, that is, virtual objects can include game accessories.

In some embodiments, the user may interact with the virtual content 300 displayed by the terminal device 100 through the handheld object 200 . The specific shape, structure and size of the handheld object 200 are not limited, and can be in various shapes, such as a mobile phone, a book, a cup, a bottle, a stick, and the like. For example, referring to FIG. 1 again, when the object 200 held by the user is a stick, the weapon held by the avatar in the virtual content 300 is also a virtual stick 303 .

In other embodiments, the user may also interact with the virtual content 300 through hand gestures, and at this time, the terminal device 100 may receive the user's gesture instruction, and control the virtual content 300 accordingly according to the gesture instruction.

Referring to FIG. 2 , an embodiment of the present application provides a method for controlling a virtual screen, which is applied to a terminal device, and the method may include:

Step S110: Acquire a target image including the target object.

Since the current virtual content switching usually needs to display an independent switching window, and then the user selects the target virtual content for switching, the switching of the virtual content is very cumbersome. Therefore, in the embodiment of the present application, the terminal device can quickly switch the virtual content by using the outline information of the object, which simplifies the switching operation. Specifically, the terminal device may acquire a target image containing the target object to determine the contour information of the target object.

The target object may be a handheld object or a user's hand. When the target object is a handheld object, the user can control the virtual content displayed in the terminal device by operating the handheld object, and when the target object is the user's hand, the user can control the terminal device by changing different gestures of the hand. Displayed virtual content.

In some embodiments, the terminal device may acquire a scene image including the user's hand region in real time, so as to capture a target image including a target object from the scene image. As a method, when the target object is the user's hand, the terminal device can determine whether the acquired scene image contains the user's hand by analyzing whether the characteristic shape of the finger and the characteristic shape of the palm appear in the scene image. area. When information such as the characteristic shape of the finger and the characteristic shape of the palm appears in the scene image, the terminal device can determine that the scene image contains the user's hand, so that the information including the characteristic shape of the finger and the characteristic shape of the palm can be directly The above-mentioned target image containing the target object can be obtained by cutting out the partial image. In other embodiments, the terminal device can also directly locate the field of view of the image acquisition device to the hand region of the user, so that the target image containing the target object can be acquired in real time.

The above-mentioned scene image or the above-mentioned target image may be acquired by the terminal device using an image acquisition device configured by itself, or may be acquired by an external image acquisition device, that is, the external image acquisition device may acquire multiple scene images or targets. The image can then be transmitted to the terminal device in a wired or wireless manner for use by the terminal device. In the embodiment of the present application, no matter it is the image acquisition device configured by the terminal device itself or the external image acquisition device, it can be used to collect the image of the real object and the scene image of the target scene, and the two can be either an infrared camera or an external image acquisition device. It is a visible camera, and the specific type is not limited in the embodiments of the present application.

It should be noted that, regardless of whether the target object is a handheld object or a user's hand, the terminal device can track it, and the tracking method can be the same or different. Obviously, in order to achieve the purpose of tracking the target object, the terminal device obtains the target. The image must contain an image of the target object.

In some embodiments, the target image may include other objects besides the target object, and these objects may or may not be related to the target object. For example, when the target object is a stick, a marker may be set on the stick, and the marker is related to the target object. The terminal device can obtain spatial position information such as the relative position and posture of the stick by tracking the marker, and then can realize the positioning and tracking of the stick. An object unrelated to the target object may be called an interfering object, and the interfering object may be a person or thing irrelevant to the display of virtual content in a real scene.

Step S120: Obtain contour information of the target object according to the target image.

In one embodiment, after acquiring the target image, the terminal device may acquire contour information of the target object in the target image by identifying and analyzing the target image. The contour information of the target object may include contour key points of the contour of the target object. The terminal device can extract the contour key points of the shape of the target object in the target image, so as to determine the contour information of the target object according to the position distribution of each contour key point. The contour key points of the shape of the target object may be arranged at equal intervals, or may be arranged correspondingly according to different feature positions, which is not limited herein. Further, when the terminal device acquires multiple target images containing the target object, the terminal device can also analyze and identify the multiple target images, so as to accurately acquire the three-dimensional contour information of the target object.

In other embodiments, the outline information of the target object acquired by the terminal device may also directly be information such as lines and dimensions of the outline of the target object. The lines can be smooth curves or simplified polylines.

Step S130: Determine the first virtual content to be switched in the currently displayed virtual screen.

In the embodiment of the present application, before switching the virtual content according to the outline information of the target object, the terminal device may determine the first virtual content to be switched in the currently displayed virtual screen.

In some embodiments, the terminal device can determine the first virtual content to be switched from the display content corresponding to the default area in the virtual screen, that is, in the current application scenario, the terminal device can switch the content according to the outline information of the target object. Only the virtual content of the default area. For example, in a shooting game scene, the area of the weapon held by the avatar in the virtual game screen is the default area for switching, and the specific gun in this area is the first virtual content to be switched.

In other embodiments, the terminal device may also be the area to be switched determined according to the outline information of the target object, so that the first virtual content to be switched may be determined according to the display content corresponding to the area to be switched. In one way, when the terminal device obtains the contour information of the target object, the contour information can be matched with the preset contour information. If the contour information of the target object matches the preset contour information, the area to be switched in the currently displayed virtual image can be determined, so as to determine the first virtual content to be switched according to the display content of the area to be switched. For example, in the shooting game scene, see Figure 3a, when the target object is the user's hand, the contour information corresponding to the hand showing the "gun" gesture matches the preset equipment contour information, and the terminal device can determine that the area to be switched is In the handheld weapon area of the avatar in the virtual game screen, the first virtual content to be switched is the specific handheld weapon in the handheld weapon area. For another example, referring to FIG. 3b, when the hand shows the "phone" gesture, the corresponding outline information matches the preset application outline information, and the terminal device can determine that the area to be switched is the display area of the virtual game screen, and the first area to be switched is the display area of the virtual game screen. The virtual content is the entire virtual game screen, that is, the first virtual content to be switched is the game application.

In some embodiments, multiple types of preset contour information may be stored in the terminal device, each type of preset contour information may include multiple specific preset contour information, and the preset contour information may be performed according to actual needs set up. For example, the preset weapon outline information of the equipment type may include preset gun outline information, preset fist outline information, preset grenade outline information, and the like. If the types of the preset contour information are different, the corresponding to-be-switched regions are also different, that is, the types of the preset contour information and the to-be-switched regions are stored in the mapping table through a certain corresponding relationship. In other words, when the contour information of the target object matches the preset contour information of the target type, the terminal device can acquire the mapping table between the type of preset contour information and the area to be switched from the memory, and based on the acquired mapping table, and obtain the corresponding to-be-switched area according to the target type, so as to determine the first virtual content according to the display content of the to-be-switched area. For example, the mapping table can be set as: the type of the preset outline information is A, and the corresponding to-be-switched area is a; the type of the preset outline information is B, and the corresponding to-be-switched area is b.

In some embodiments, the preset outline information may be preset and stored in the terminal device, or may be obtained after analyzing the user's behavioral habits, such as the user's habit of using a certain target object to switch virtual content When , the contour information of the target object can be recorded. It should be noted that the preset contour information may be fixed, or may be continuously optimized according to the actual situation. For example, if the terminal device is used by user A in the first time period, the first preset profile information a can be obtained according to the operation habits of user A; in the second time period, the terminal device is used by user B, the B's operating habit acquires the second preset contour information b. At this time, the preset contour information may be the newly acquired second preset contour information b, or may be a combination of the first preset contour information a and the second preset contour information. Information b is the third preset contour information obtained comprehensively, and so on. For example, when the first preset contour information is a roughly elongated contour, and when the second preset contour information is a cuboid contour of a certain width, the third preset contour information may be a comprehensively obtained elongated contour of a certain width. It can be understood that, the specific method for setting the preset contour information is not limited in the embodiments of the present application.

Step S140: Acquire the second virtual content according to the outline information, and switch and display the first virtual content as the second virtual content.

Since the contour information of the target object may include the contour key points, lines and dimensions of the target object, when the terminal device obtains the contour information, it can determine the second virtual content matching the contour information according to the contour information, and then use the above-mentioned contour information to determine the second virtual content that matches the contour information. The first virtual content to be switched is switched and displayed as the second virtual content. Therefore, the rapid switching of virtual content can be realized only through the outline information of the target object, and the simplification of the switching operation is realized.

In some embodiments, the second virtual content that matches the contour information may be that the contour information of the shape of the target object matches the contour information of the second virtual content. For example, when the target object is a long stick, and the outline information of the long stick matches the outline information of the virtual long sword, it can be determined that the second virtual content matching the outline information of the long stick is the virtual long sword.

In other embodiments, the second virtual content matched with the outline information may be that the functional attributes represented by the outline information of the shape of the target object match the functional attributes of the second virtual content. For example, referring to Fig. 4, when the target object is the user's hand, the functional attribute represented by the corresponding outline information when the hand is compared with the "box" gesture can be a photo frame, which matches the functional attribute of the virtual photo album application. It is determined that the second virtual content matching the contour information of the hand is a virtual photo album application.

In some embodiments, the first virtual content and the second virtual content may have similar attributes and similar functions, but their key attributes and key functions may be different. As a way, the first virtual content is located in the currently displayed virtual picture, and it plays an important role in the virtual picture, that is, the state of the first virtual content in the terminal device is "used state" or "selected" state". The second virtual content may be located in the currently displayed virtual screen, or may not be in the currently displayed virtual screen, and the state of the second virtual content in the terminal device is an "unused state" or a "to-be-selected state", whichever one Either the second virtual content can replace the currently displayed first virtual content. For example, the first virtual content to be switched in the currently displayed virtual picture is the gun A in the game equipment, and the second virtual content obtained according to the outline information of the target object is the fist. Among them, the gun A is being used by the game character a, that is, the state of the gun A is the "used" state. The fist can also be located in the virtual screen, but it is not used by the game character a, or it is not displayed in the virtual screen, that is, the state of the fist is "unused", and the fist can only be used after receiving the corresponding call command. displayed on the virtual screen. Obviously, gun A and fist have similar properties and functions. For example, both fist and gun A are attack properties and can be used to attack the enemy, but the key attributes and key functions of the two are different, such as the shape of gun A and fist. Not the same, or the force value of gun A is higher than the force value of fist.

After obtaining the second virtual content, the terminal device may switch and display the first virtual content as the second virtual content. Specifically, the terminal device may acquire a target switching instruction, and the target switching instruction may be an instruction automatically generated when the second virtual content matching the outline information is acquired. When the terminal device obtains the target switching instruction, it can switch the first virtual content in the virtual screen to the second virtual content for display. As a method, the terminal device obtains the first identifier corresponding to the first virtual content and the second identifier corresponding to the second virtual content respectively according to the target switching instruction; the first identifier corresponding to the first virtual content is modified to correspond to the second virtual content The second identifier of the first virtual content can be switched to display the second virtual content.

In the virtual image control method provided by the embodiment of the present application, the terminal device obtains the outline information of the target object by analyzing the target image including the target object, and then by confirming the outline information of the target object, the terminal device can quickly obtain the first virtual content and second virtual content. The first virtual content is in the currently displayed virtual picture, and the second virtual content is used to replace the first virtual content. Through the confirmation of the profile information, the embodiment of the present application can accurately and effectively switch the first virtual content to the second virtual content, and the switching operation is simple and easy to implement, which more satisfies the user's operation requirements for the virtual content.

Referring to FIG. 5, another embodiment of the present application provides a method for controlling a virtual screen, which is applied to a terminal device. The method may include:

Step S210: Acquire a target image including the target object.

Step S220: Obtain contour information of the target object according to the target image.

Step S230: Determine the first virtual content to be switched in the currently displayed virtual screen.

Step S240: Acquire the second virtual content according to the outline information, and switch and display the first virtual content as the second virtual content.

In some embodiments, the terminal device can quickly switch the virtual content according to the specific shape of the target object. Specifically, as shown in FIG. 6 , the obtaining of the second virtual content according to the outline information may include steps S241 to S242 .

Step S241: Obtain the shape of the target object according to the contour information.

In some embodiments, when the contour information obtained by the terminal device is the contour key points of the shape of the target object, each of the contour key points can be sequentially connected with lines to obtain the contour line of the shape of the target object, so that the terminal device can The contour line determines the shape of the target object. The line may be a smooth curve or a polyline, which is not limited herein. For example, when the target object is a long stick, the shape formed by the corresponding contour line can be a cylinder shape; when the target object is a finger, the shape formed by the contour line corresponding to the "gun" gesture compared with the finger can be a shape composed of two The combined shape of a cuboid (similar to a pistol).

As one approach, it is envisaged to create a shape database in which a plurality of shapes are stored. The terminal device can match the contour line of the target object with the contour line of the shape in the shape database to obtain a target shape that successfully matches the contour line of the target object, so that the terminal device can determine the shape of the target object as the target shape. The target shape that successfully matches the contour of the target object may be a shape with a matching degree greater than a matching threshold, such as a matching degree greater than 90%, or a shape corresponding to the highest matching degree, which is not limited here.

Step S242: Acquire second virtual content that matches the shape.

After determining the shape of the target object, the terminal device may acquire second virtual content that matches the shape. The shape of the second virtual content that matches the shape of the second virtual content may match the shape of the target object. For example, when the target object is a stick, the second virtual content may be a virtual stick; The function attribute matches the function attribute represented by the shape of the target object. For example, when the target object is a finger, when the finger indicates the "telephone" gesture, the second virtual content can be a phone call application, and the function attributes of both are Phone call function.

In some embodiments, the terminal device may store multiple preset shapes, and the preset trajectories may be set according to actual needs. If the preset shapes are different, the corresponding first virtual content may be different, that is, the preset shapes and the second virtual content are stored in the mapping table through a certain corresponding relationship. When the shape of the target object matches the preset shape, the terminal device can obtain the mapping between the preset shape and the second virtual content from its memory. table, and based on the acquired mapping table, acquire the corresponding second virtual content according to the shape of the target object.

In some embodiments, when there are multiple virtual contents matching the shape, the terminal device may determine the second virtual content according to the user's usage habits. Specifically, as shown in FIG. 7 , obtaining the second virtual content that matches the shape may include steps S2421 to S2423.

Step S2421: Obtain candidate objects matching the shape.

When the terminal device determines the shape of the target object, it can acquire all candidate objects matching the shape according to the shape. Specifically, the terminal device may look up whether a candidate object matching the shape of the target object is stored in the memory. If no candidate object matching the shape of the target object is found stored in the terminal device, it indicates that there is no virtual content matching the shape of the target object, which further indicates that the terminal device cannot perform a virtual content switching operation on the first virtual content. If a candidate object matching the shape of the target object is stored in the terminal device, the terminal device can acquire the candidate object found by the terminal device.

In some embodiments, when it finds one candidate object, the terminal device can directly switch and display the candidate object as the second virtual content, that is, switch and display the currently displayed first virtual content as the candidate object . and the candidate objects are displayed in the virtual screen for the user to select,

In an implementation manner, when there are multiple candidate objects found by the terminal device, the terminal device may display the multiple candidate objects on a virtual screen for the user to select, or may automatically select the best candidate object. Among them, the best candidate object can be specifically determined according to a specific application scenario. For example, in a game scene, the best candidate object may be the candidate object corresponding to the highest attack power, or the candidate object corresponding to the best comprehensive attribute.

Step S2422: When there are multiple candidate objects, obtain the usage frequency of each candidate object.

In some embodiments, when there are multiple candidate objects acquired by the terminal device, the terminal device may determine the second virtual content according to the usage frequency of each candidate object. As a method, the terminal device can record the number of times each candidate object is in the "use" state in real time, and perform real-time statistics and sorting to obtain a ranking list of the use frequency of all candidate objects. Therefore, when the terminal device acquires multiple candidate objects, the usage frequency of each candidate object can be determined according to the usage frequency ranking list.

As another method, the terminal device can also record the duration of each candidate object in the "use" state, and perform real-time statistics and sorting to obtain the usage frequency ranking of all candidate objects, so that when the terminal device obtains multiple candidates When the object is selected, the usage frequency of each candidate object can be determined according to the usage frequency ranking list.

For example, there are three candidate objects matching the shape of the target object in the terminal device, namely gun 301, gun 302, and gun 303. The terminal device can obtain that the use frequency of gun 301 is 55%, and the use frequency of gun 302 is 12%. , the use frequency of gun 303 is 33%, and the use frequency of these three guns is sorted from high to low to obtain gun 301 , gun 303 and gun 302 .

In some embodiments, after obtaining the sorting result of the frequency of use of the candidate objects, the terminal device can display it, and the user can select the second virtual content from high to low for switching display according to the display result.

Step S2423: Obtain the candidate object corresponding to the highest frequency of use as the second virtual content.

In some embodiments, when acquiring the usage frequency of each candidate object, the terminal device may directly acquire the candidate object corresponding to the highest usage frequency as the second virtual content. Specifically, the terminal device may determine the candidate object corresponding to the highest usage frequency from the plurality of candidate objects matching the shape according to the above-mentioned updated usage frequency ranking list, as the second virtual content. For example, in the above example, the terminal device may correspond to the firearm 301 with the highest frequency of use as the second virtual content.

In other embodiments, when the target object is a hand, the terminal device may determine different second virtual content according to different hands. Specifically, before acquiring the second virtual content that matches the shape, the method for controlling the virtual content may further include:

The mode corresponding to the hand is determined according to the target image, and the mode includes any one of a left-hand mode, a right-hand mode, and a two-hand mode.

In this embodiment, as shown in FIG. 8 , the obtaining of the second virtual content that matches the shape may include steps S2424 to S2425.

Step S2424: Determine the gesture corresponding to the hand according to the shape.

Step S2425: Acquire second virtual content matching the pattern and gesture.

In some embodiments, the terminal device may determine whether the target image contains one hand or both hands of the user according to the number of fingers or palms in the target image. When the terminal device detects that the target image contains both hands of the user, it is determined that it is in the two-hand mode, and the terminal device can determine the second virtual content according to the gesture in the two-hand mode. When the terminal device detects that the target image contains a single hand of the user, it is determined that it is in the one-hand mode at this time, and the terminal device can determine the second virtual content according to the gesture in the one-hand mode. The second virtual content determined in the two-hand mode and the second virtual content determined by the gesture in the one-hand mode may be the same or different, which is not limited herein.

In some embodiments, when the terminal device detects that the target image contains a single hand of the user, the terminal device can determine the shot according to the specific posture of the hand in the target image, the orientation of the fingers, the positional arrangement between the fingers, and other information. Either the user's left hand or the user's right hand. When the terminal device detects that the target image contains the user's left hand, the terminal device may determine the second virtual content according to the gesture in the left-hand mode. When the terminal device detects that the target image contains the user's right hand, the terminal device may determine the second virtual content according to the gesture in the right-hand mode. The second virtual content determined in the left-hand mode and the second virtual content determined by the gesture in the right-hand mode may be the same or different, which is not limited herein.

For example, please refer to FIG. 9 and FIG. 10. In the game scene, when the first virtual content is the weapon fist (FIG. 9) used by the game character a, when the user shows the "gun" gesture with the right hand, the terminal device can follow the right hand. When the mode and the "gun" gesture are selected, the second virtual content is determined as the default gun (203 in Figure 10), and the weapon used by the current game character a is switched from the fist to the default gun for display. For another example, when the user shows the "gun" gesture with the left hand, the terminal device can follow the left-hand mode and the "gun" gesture to determine that the second virtual content is a bullet, and use the default gun as the weapon used by the current game character a. Bullet loading display. For another example, when the user makes a gesture of "gun" with both hands, the terminal device can follow the two-hand mode and the gesture of "gun", determine that the second virtual content is a machine gun (204 in FIG. 10 ), and assign the current game character aThe weapon used is switched from the default gun to a machine gun for display.

In still other embodiments, the terminal device may also determine the second virtual content according to the specific size of the target object. Specifically, as shown in FIG. 11 , obtaining the second virtual content that matches the shape may include steps S2426 to S2428 .

Step S2426: Obtain the shape and contour parameters of the target object according to the contour information.

In some embodiments, when the contour information obtained by the terminal device is the contour key points of the shape of the target object, each of the contour key points can be sequentially connected with lines to obtain the contour line of the shape of the target object, so that the terminal device can The contour line determines the contour parameters of the target object. The contour parameters may be parameters such as the length, width, and height of the target object, and may also be parameters such as the length of each side of the target object, the bending angle of the curved surface, etc., which are not limited here.

Step S2427: Obtain one or more candidate objects matching the shape.

Step S2428: Obtain a candidate object matching the contour parameters from one or more candidate objects as the second virtual content.

In some embodiments, when acquiring one or more candidate objects matching the shape of the target object, the terminal device may determine the second virtual content from the one or more candidate objects according to the contour parameter. In one way, when the terminal device acquires a candidate object that matches the shape of the target object, the terminal device can directly switch and display the candidate object as the second virtual content. As another way, the terminal device may also adjust the contour parameters of the candidate object according to the specific contour parameters to fit the contour parameters of the target object, and switch and display the adjusted candidate object as the second virtual content. For example, when the target object is a plastic rod and the matching candidate is only a virtual long sword, the terminal device can adjust the length of the virtual long sword to the length of the plastic rod, and switch the adjusted virtual long sword to display.

As a further way, when the terminal device acquires multiple candidate objects that match the shape of the target object, the terminal device may acquire, from the multiple candidate objects, the candidate object that matches the contour parameters of the target object as the second virtual object content. The candidate object matching the contour parameter of the target object may be the candidate object with the highest matching degree, or may be the candidate object whose matching degree is greater than the matching threshold, which is not limited herein.

In still other embodiments, when the candidate objects matching the shape belong to different object categories, the terminal device may first determine the object category to which the second virtual content belongs according to the specific size of the target object, and then determine the specific second virtual content . Specifically, as shown in FIG. 12 , obtaining the second virtual content that matches the shape may include steps S2429 to S2433 .

Step S2429: Obtain the shape and contour parameters of the target object according to the contour information.

Step S2430: Acquire one or more object categories matching the shape, and determine the parameter conditions corresponding to each object category.

In some embodiments, when the shape of the target object is relatively common, candidate objects matching the shape may have different object categories. Therefore, when the terminal device determines the shape of the target object, it can first determine one or more object categories that match the shape, and determine the parameter conditions corresponding to each object category, so that the appropriate object category can be subsequently determined according to the parameter conditions. . For example, when the target object is a stick, the shape of the stick is a slender bar, and the object category matching this shape can be a dagger, a stick, a sword, etc., wherein the length parameter condition of the dagger is 10cm～30cm, and the length parameter of the stick The condition is 31cm～45cm, and the length parameter condition of the sword is 46cm～70cm.

Step S2431: Match the contour parameters with the parameter conditions corresponding to each object category to obtain the object category corresponding to the contour parameters.

In some implementation manners, the terminal device may match the contour parameters of the target object with the parameter conditions corresponding to each of the foregoing object categories, so as to obtain the object category corresponding to the contour parameters. Specifically, it can be determined which object category the contour parameter of the target object is within the parameter condition range of the target object category, and when the contour parameter is within the parameter condition range of the target object category, it can be determined that the target object belongs to the target object category. For example, in the above example, when the length of the stick is 47cm, the corresponding object type is sword.

Step S2432: Obtain one or more candidate objects included in the corresponding object category.

Step S2433: Obtain candidate objects matching the contour parameters from one or more candidate objects as the second virtual content.

In some embodiments, when the terminal device determines the object category of the candidate object, it can acquire one or more candidate objects under the object category. When there is only one candidate object under the object category, the terminal device can directly switch and display the candidate object as the second virtual content, or can specifically adjust the outline parameters of the candidate object according to the outline parameters of the target object. When there are multiple candidate objects under the object category, the terminal device may obtain, from the multiple candidate objects, a candidate object that matches the contour parameters of the target object as the second virtual content. The candidate object matching the contour parameter of the target object may be the candidate object with the highest matching degree, or may be the candidate object whose matching degree is greater than the matching threshold, which is not limited herein.

In still other embodiments, the terminal device may further render the second virtual content according to the material information of the target object, so as to fit the visual effect of the target object. Specifically, as shown in FIG. 13 , the obtaining of the second virtual content that matches the shape may include steps S2434 to S2437 .

Step S2434: Acquire material information of the target object according to the target image.

In some embodiments, the material information may be material information corresponding to the largest area in the target object. As a way, the terminal device can identify and analyze the color, texture, pattern and combination of the surface of the target object to determine the feature area with the largest layout area, and then the terminal device can extract the image of a part of the area from the feature area as the target The material information of the object.

Step S2435: Acquire the second virtual content according to the outline information.

Step S2436: Acquire preset texture information matching the material information.

In some embodiments, the terminal device may store a plurality of preset texture information. After determining the material information of the target object, the terminal device may match the material information with the preset texture information to obtain the information from the multiple preset textures. The information obtains the preset texture information that matches the material information. The preset texture information that matches the material information may be preset texture information with the highest matching degree, or may be preset texture information whose matching degree is greater than a matching threshold.

Step S2437: Render the second virtual content according to the preset texture information, and switch and display the first virtual content as the rendered second virtual content.

After acquiring the preset texture information, the terminal device can render the second virtual content according to the preset texture information, and switch the first virtual content to be displayed as the rendered second virtual content, so that the switched virtual content can be connected to the target. The objects fit together, and at the same time, the switched virtual content has a personalized effect.

In the virtual screen control method provided by the embodiment of the present application, the terminal device obtains the contour information of the target object by analyzing the target image including the target object, and then confirms the shape and contour parameters of the target object through the contour information, so that the terminal device can quickly obtain the first A virtual content and a second virtual content. The first virtual content is in the currently displayed virtual picture, and the second virtual content is used to replace the first virtual content. Through the confirmation of the shape and contour parameters of the target object, the embodiment of the present application can accurately and effectively switch the first virtual content to the second virtual content, and the switching operation is simple and easy to implement, which better meets the user's operation requirements for the virtual content.

Referring to FIG. 14, another embodiment of the present application provides a method for controlling a virtual screen, which is applied to a terminal device. The method may include:

Step S310: Acquire a target image including the target object.

Step S320: Obtain contour information of the target object according to the target image.

Step S330: Determine the first virtual content to be switched in the currently displayed virtual screen.

Wherein, as shown in FIG. 15, step S330 may include steps S331 to S332.

Step S331: Obtain user identity information, and read user personal data according to the user identity information.

In one embodiment, the user's identity information may include a user's identity identifier and a login password, etc., and the terminal device may obtain the user's identity information from a smart card or chip, wherein the smart card may include a Subscriber Identity Module (SIM) card; Obtain user identity information in wireless network, etc. User identity information and user personal data are stored in the terminal device or the cloud through a one-to-one correspondence. After the user's identity information is obtained, the corresponding user personal data can be found according to the identity information. User personal data can include the user's personal data. basic data, operational data, and status data. Among them, basic data refers to the user's personal basic information, such as the user's gender, age, hobbies, etc.; operation data refers to some operations that the user is accustomed to when using the terminal device, such as the user's habits when using the terminal device to play games. The switching of WeChat applications is realized through the "telephone" hand speed; the status data refers to the status of the terminal device and the time corresponding to the status. Common statuses include online, offline, invisible, and do not disturb.

Step S332: Determine the first virtual content to be switched in the currently displayed virtual screen according to the user's personal data.

In some embodiments, after obtaining the corresponding user personal data through the user's identity information, the terminal device can analyze the user's personal data to obtain a personal data analysis result, and then can determine the current state according to the obtained personal data analysis result. The first virtual content to be switched corresponding to the outline information in the displayed virtual picture. For the same profile information, the first virtual content switched by different users may be the same or different, and each user has its own unique behavior. Different virtual content switching scenarios. In an embodiment, the built-in image acquisition device or the external image acquisition device of the terminal device can be used to acquire images including user behaviors at different times, and then feature extraction, identification, and analysis can be performed on the images at different times, and then The behavior habit of the user can be obtained, and according to the behavior habit, the first virtual content corresponding to the motion track in the currently displayed virtual picture can be determined.

Step S340: Acquire the second virtual content according to the outline information, and switch and display the first virtual content as the second virtual content.

In the virtual image control method provided by the embodiment of the present application, the terminal device obtains the contour information of the target object by analyzing the target image including the target object, and then the terminal device can quickly obtain the first virtual content and the first virtual content by confirming the contour information of the target object. The second virtual content, wherein the first virtual content is in the currently displayed virtual screen, the first virtual content can be obtained through the user's personal data, wherein the user's personal data includes the user's behavioral habit data, and the behavioral habit data can be With the help of the artificial intelligence algorithm, the embodiment of the present application obtains the first virtual content through the user's behavior habit data, which can make the control of the virtual screen more intelligent, which is not only simple to operate, but also more in line with the user's operation requirements for the virtual content.

Please refer to FIG. 16 , which shows a structural block diagram of an apparatus 400 for controlling a virtual screen provided by an embodiment of the present application. The apparatus may include: an image acquisition module 410 , an information identification module 420 , a content determination module 430 , and a content switching module 440, wherein, the image acquisition module 410 is used to acquire the target image containing the target object; the information recognition module 420 is used to acquire the outline information of the target object according to the target image; the content determination module 430 is used to determine the currently displayed virtual screen to be switched The content switching module 440 is configured to acquire the second virtual content according to the outline information, and switch and display the first virtual content as the second virtual content.

In some embodiments, the content switching module may include: a shape acquisition unit and a content acquisition unit. Wherein, the shape obtaining unit is used for obtaining the shape of the target object according to the outline information; the content obtaining unit is used for obtaining the second virtual content matching the shape.

In some embodiments, the content acquisition unit may be specifically configured to: acquire candidate objects matching the shape; when there are multiple candidate objects, acquire the usage frequency of each candidate object; acquire the candidate object corresponding to the highest usage frequency as the first candidate object 2. Virtual content.

In some embodiments, the target object is a hand, and the control device 400 of the virtual picture may further include: a mode determination module. The mode determination module is configured to determine the mode corresponding to the hand according to the target image, and the mode includes any one of a left-hand mode, a right-hand mode, and a two-hand mode.

In this implementation manner, the content acquisition unit may be specifically configured to: determine the gesture corresponding to the hand according to the shape; and acquire the second virtual content matching the mode and the gesture.

In some embodiments, the content switching module 440 may be specifically configured to: obtain the shape and contour parameters of the target object according to the contour information; obtain one or more candidate objects matching the shape; obtain and contour from the one or more candidate objects The candidate object matched by the parameter is used as the second virtual content.

In some embodiments, the content switching module 440 may also be specifically configured to: obtain the shape and contour parameters of the target object according to the contour information; obtain one or more object categories matching the shape, and determine the parameter conditions corresponding to each object category ; Match the contour parameters with the parameter conditions corresponding to each object category, and obtain the object category corresponding to the contour parameters; obtain one or more candidate objects contained in the corresponding object category; obtain and contour parameters from one or more candidate objects The matched candidate object is used as the second virtual content.

In some embodiments, the content switching module 440 may also be specifically configured to: obtain the material information of the target object according to the target image; obtain the second virtual content according to the outline information; obtain preset texture information matching the material information; The information renders the second virtual content, and switches and displays the first virtual content as the rendered second virtual content.

In the virtual screen control device provided by the embodiment of the present application, the contour information of the target object is obtained by analyzing the target image including the target object, and then the terminal device can quickly obtain the first virtual content and the first virtual content by confirming the contour information of the target object. 2. Virtual content. The first virtual content is in the currently displayed virtual picture, and the second virtual content is used to replace the first virtual content. Through the confirmation of the profile information, the embodiment of the present application can accurately and effectively switch the first virtual content to the second virtual content, and the switching operation is simple and easy to implement, which more satisfies the user's operation requirements for the virtual content.

Please refer to FIG. 17 , which shows a structural block diagram of a terminal device provided by an embodiment of the present application. The terminal device 100 may be a terminal device capable of running an application program, such as a head-mounted display device. The terminal device 100 in this application may include one or more of the following components: a processor 110 and a memory 120, wherein the memory 120 stores one or more application programs, and the one or more application programs are configured to be composed of one or more The processor 110 executes one or more programs configured to execute the methods described in the foregoing method embodiments.

The processor 110 may include one or more processing cores. The processor 110 uses various interfaces and lines to connect various parts of the entire terminal device 100, and executes by running or executing the instructions, programs, code sets or instruction sets stored in the memory 120, and calling the data stored in the memory 120. Various functions of the terminal device 100 and processing data. Optionally, the processor 110 may employ at least one of a digital signal processing (Digital Signal Processing, DSP), a Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA), and a Programmable Logic Array (Programmable Logic Array, PLA). implemented in hardware. The processor 110 may integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU), a modem, and the like. Among them, the CPU mainly handles the operating system, user interface and application programs, etc.; the GPU is used for rendering and drawing of the display content; the modem is used to handle wireless communication. It can be understood that, the above-mentioned modem may also not be integrated into the processor 110, and is implemented by a communication chip alone.

The memory 120 may include random access memory (Random Access Memory, RAM), or may include read-only memory (Read-Only Memory). Memory 120 may be used to store instructions, programs, codes, sets of codes, or sets of instructions. The memory 120 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.) , instructions for implementing the following method embodiments, and the like. The storage data area may also store data and the like created by the terminal device 100 in use.

In some embodiments, the terminal device 100 may further include an image sensor 130 for capturing images of real objects and capturing scene images of a target scene. The image sensor 130 may be an infrared camera or a visible light camera, and the specific type is not limited in this embodiment of the present application.

In one embodiment, the terminal device is a head-mounted display device, which, in addition to the above-mentioned processor, memory, and image acquisition device, may also include one or more of the following components: an image acquisition device, a display module, an optical module, Communication module and power supply.

The image acquisition device is used to collect the image of the real object and the scene image of the target scene. The image acquisition device may be an infrared camera or a visible light camera, and the specific type is not limited in the embodiments of the present application.

The display module may include a display control unit. The display control unit is configured to receive the display image of the virtual content rendered by the processor, and then display and project the display image to the optical module, so that the user can view the virtual content through the optical module. The display module may be a display screen or a projection device, etc., and may be used to display images.

The optical module may adopt an off-axis optical system or a waveguide optical system, and the display image displayed by the display module can be projected to the user's eyes after passing through the optical module. The user sees the display image projected by the display module through the optical module. In some embodiments, the user can also observe the real environment through the optical module, and feel the augmented reality effect after the virtual content and the real environment are superimposed.

The communication module may be a module such as Bluetooth, WiFi (Wireless-Fidelity, wireless fidelity), ZigBEE (Zifeng technology), etc. The head-mounted display device can establish a communication connection with the terminal device through the communication module. The head-mounted display device communicatively connected with the terminal device can interact with the terminal device for information and instructions. For example, the head-mounted display device may receive image data sent by the terminal device through the communication module, and generate virtual content of the virtual world for display according to the received image data.

The power supply can supply power to the entire head-mounted display device to ensure the normal operation of each component of the head-mounted display device.

Please refer to FIG. 18 , which shows a structural block diagram of a computer-readable storage medium provided by an embodiment of the present application. The computer-readable storage medium 900 stores program codes, and the program codes can be invoked by the processor to execute the methods described in the above method embodiments.

The computer-readable storage medium 900 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 900 includes a non-transitory computer-readable storage medium. Computer readable storage medium 900 has storage space for program code 910 to perform any of the method steps in the above-described methods. These program codes can be read from or written to one or more computer program products. Program code 910 may be compressed, for example, in a suitable form.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand: it can still be Modifications are made to the technical solutions described in the foregoing embodiments, or some technical features thereof are equivalently replaced; and these modifications or replacements do not drive the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims (10)

1. A method for controlling a virtual screen, characterized in that, applied to a terminal device, the method comprising: Get the target image containing the target object; Acquiring contour information of the target object according to the target image; determining the first virtual content to be switched in the currently displayed virtual picture; Acquire second virtual content according to the outline information, and switch and display the first virtual content as the second virtual content. 2. The method according to claim 1, wherein the acquiring the second virtual content according to the outline information comprises: Obtain the shape of the target object according to the outline information; Obtain second virtual content that matches the shape. 3. The method according to claim 2, wherein the acquiring the second virtual content matching the shape comprises: obtain candidate objects that match the shape; When there are multiple candidate objects, obtain the usage frequency of each candidate object; The candidate object corresponding to the highest frequency of use is obtained as the second virtual content. 4. The method according to claim 2, wherein the target object is a hand, and before acquiring the second virtual content matching the shape, the method further comprises: Determine the mode corresponding to the hand according to the target image, and the mode includes any one of a left-hand mode, a right-hand mode, and a two-hand mode; The acquiring second virtual content matching the shape includes: Determine the gesture corresponding to the hand according to the shape; Obtain second virtual content that matches the pattern and the gesture. 5. The method according to claim 1, wherein the acquiring the second virtual content according to the outline information comprises: Obtain the shape and contour parameters of the target object according to the contour information; obtaining one or more candidate objects matching the shape; A candidate object matching the contour parameter is obtained from the one or more candidate objects as the second virtual content. 6. The method according to claim 1, wherein the acquiring the second virtual content according to the profile information comprises: Obtain the shape and contour parameters of the target object according to the contour information; Acquire one or more object categories matching the shape, and determine the parameter conditions corresponding to each object category; Matching the contour parameters with the parameter conditions corresponding to each object category to obtain the object category corresponding to the contour parameters; obtaining one or more candidate objects contained in the corresponding object category; A candidate object matching the contour parameter is obtained from the one or more candidate objects as the second virtual content. 7. The method according to any one of claims 1-6, wherein the acquiring second virtual content according to the outline information, and switching and displaying the first virtual content as the second virtual content ,include: Acquiring material information of the target object according to the target image; acquiring second virtual content according to the profile information; obtaining preset texture information matching the material information; The second virtual content is rendered according to the preset texture information, and the first virtual content is switched and displayed as the rendered second virtual content. 8. A control device for a virtual screen, characterized in that, when applied to a terminal device, the device comprises: The image acquisition module is used to acquire the target image containing the target object; an information identification module, configured to obtain contour information of the target object according to the target image; a content determination module, configured to determine the first virtual content to be switched in the currently displayed virtual picture; A content switching module, configured to acquire second virtual content according to the outline information, and switch and display the first virtual content as the second virtual content. 9. A terminal device, comprising: one or more processors; memory; one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, the one or more application programs are configured to execute The method of any one of claims 1-7. 10. A computer-readable storage medium, wherein a program code is stored in the computer-readable storage medium, and the program code can be invoked by a processor to execute any one of claims 1-7 Methods.

Images