CN110737326A - Virtual object display method and device, terminal equipment and storage medium - Google Patents

Abstract

The embodiments of the present application disclose a method, device, terminal device and storage medium for displaying a virtual object, which relate to the technical field of display. The method for displaying a virtual object is applied to a terminal device, and the method includes: acquiring first spatial position information of the terminal device relative to a marker; acquiring second spatial position information of the marker relative to a target area of a real object; The first spatial position information and the second spatial position information determine the third spatial position information of the terminal device relative to the target area; based on the third spatial position information, the virtual object is displayed on the physical target area. The method can realize that the virtual object is displayed in the target area of the real object.

Description

Display method, device, terminal device and storage medium of virtual object

technical field

The present application relates to the field of display technology, and more particularly, to a method, apparatus, terminal device and storage medium for displaying virtual objects.

Background technique

In recent years, with the advancement of science and technology, technologies such as augmented reality (AR, Augmented Reality) have gradually become the focus of research at home and abroad. Augmented reality is a technology that increases users' perception of the real world through information provided by computer systems. The content objects such as virtual objects, scenes or system prompt information are superimposed on the real scene to enhance or modify the perception of the real-world environment or data representing the real-world environment. In the existing augmented reality display technology, the terminal device cannot combine virtual objects with real objects to display well.

SUMMARY OF THE INVENTION

Embodiments of the present application provide a method, device, terminal device and storage medium for displaying virtual objects, so as to better realize the combined display of virtual objects and real objects.

In a first aspect, an embodiment of the present application provides a method for displaying a virtual object, which is applied to a terminal device. The method includes: acquiring first spatial position information of the terminal device relative to a marker; acquiring the marker relative to a real object the second spatial location information of the target area; based on the first spatial location information and the second spatial location information, determine the third spatial location information of the terminal device relative to the target area; based on the third spatial location information The location information displays the virtual object in the target area of the object.

In a second aspect, an embodiment of the present application provides a device for displaying a virtual object, which is applied to a terminal device. The device includes: a first position acquisition module, a second position acquisition module, a third position acquisition module, and a display execution module, Wherein, the first position acquisition module is used to acquire the first spatial position information of the terminal device relative to the marker; the second position acquisition module is used to acquire the second spatial position of the marker relative to the physical target area information; the third position acquisition module is configured to determine the third spatial position information of the terminal device relative to the target area based on the first spatial position information and the second spatial position information; the display execution module for displaying the virtual object in the target area based on the third spatial position information.

In a third aspect, an embodiment of the present application provides a terminal device, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor enables the processor to implement the above-mentioned first step. On the one hand, a display method of a virtual object is provided.

In a fourth aspect, an embodiment of the present application provides a computer storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the method for displaying a virtual object provided in the first aspect.

The above-mentioned display method, device, terminal device and storage medium for virtual objects provided by the present application, by acquiring the first spatial position information of the terminal device relative to the marker, and then acquiring the second spatial position information of the marker relative to the target area of the real object, Then, based on the first spatial position information and the second spatial position information, the third spatial position information of the terminal device relative to the target area is determined, and finally based on the third spatial position information, the virtual object is displayed in the real target area, so as to realize the virtual object. It is displayed in the target area of the real object, and the combined display of the real object and the virtual object is completed.

These and other aspects of the present application will be more clearly understood in the description of the following embodiments.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the drawings that are used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can also be obtained from these drawings without creative effort.

FIG. 1 shows a schematic diagram of an application scenario of a method for displaying a virtual object provided by an embodiment of the present application;

FIG. 2 shows another schematic diagram of an application scenario of the method for displaying a virtual object provided by an embodiment of the present application;

3 shows a structural block diagram of a terminal device according to an embodiment of the present application;

FIG. 4 shows a flowchart of a method for displaying a virtual object provided by an embodiment of the present application;

FIG. 5 shows a schematic diagram of an effect of a method for displaying a virtual object provided by an embodiment of the present application;

FIG. 6 is a schematic diagram showing another effect of the method for displaying a virtual object provided by an embodiment of the present application;

FIG. 7 shows a flowchart of a method for displaying a virtual object provided by another embodiment of the present application;

FIG. 8 shows a structural block diagram of a display device for a virtual object provided by an embodiment of the present application;

FIG. 9 shows another structural block diagram of a display device for a virtual object provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

Referring to FIG. 1 , it is a schematic diagram showing an application scenario of the method for displaying a virtual object provided by an embodiment of the present application, where the application scenario includes a display system 10 . The display system 10 includes: a terminal device 100 , a marker 200 and a real object 300 .

In this embodiment of the present application, the marker 200 may be within the field of view of the terminal device 100 , and the marker 200 may be placed on the surface of the object 300 , or may be placed near the object 300 . For example, referring to FIG. 2 , the object 300 is a plane, and the marker 200 can be attached to an area of the plane.

In this embodiment of the present application, the terminal device 100 may be a head-mounted display device, or may be a mobile device such as a mobile phone or a tablet. When the terminal device 100 is a head-mounted display device, the head-mounted display device may be an integrated head-mounted display device. The terminal device 100 can also be a smart terminal such as a mobile phone connected to an external head-mounted display device, that is, the terminal device 100 can be used as a processing and storage device of the head-mounted display device, and the external The display device performs a display function for virtual objects.

In this embodiment of the present application, referring to FIG. 3 , the terminal device 100 may include: a memory 120 , a processor 110 , a display device 130 , and an image capture device 140 . The memory 120 , the display device 130 and the image acquisition device 140 are all connected to the processor 110 .

The image capturing device 140 is used for capturing an image of the object to be photographed and sending it to the processor 110 . The image acquisition device 140 may be an infrared camera, a color camera, or the like, and the specific type of the image acquisition device is not limited in the embodiments of the present application.

Processor 110 may comprise any suitable type of general or special purpose microprocessor, digital signal processor or microcontroller. The processor may be configured to receive data and/or signals from various components of the system via, for example, a network. The processor 110 may also process data and/or signals to determine one or more operating conditions in the system. For example, the processor 110 generates the image data of the virtual world according to the pre-stored image data, and sends it to the display device for display; it can also receive the image data sent by the smart terminal or the computer through a wired or wireless network, according to the received image data The data generates an image of the virtual world for display; it can also identify and locate according to the image collected by the image acquisition device, and determine the corresponding display content in the virtual world according to the positioning information, and send it to the display device for display. It can be understood that the processor 110 is not limited to be installed in the terminal device 100 .

The memory 120 may be used to store software programs and modules, and the processor 110 executes various functional applications and data processing by running the software programs and modules stored in the memory 130 . Memory 130 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory.

In this embodiment of the present application, when the terminal device 100 is a mobile terminal connected to an external head-mounted display device, the display device and the camera of the external head-mounted display device are connected to the mobile terminal. It can be understood that, the processing performed by the processor in the foregoing embodiments is performed by the processor of the mobile terminal, and the data stored in the memory in the foregoing embodiment is stored by the memory of the mobile terminal.

Of course, the above components included in the terminal device 100 are only examples, and do not represent a limitation on the components included in the terminal device 100 in the embodiments of the present application. The terminal device 100 may also include more or less components, for example, a terminal The device 100 may also include a communication module connected to the processor. The communication module is used for communication between the terminal device 100 and other devices.

In this embodiment of the present application, the marker 200 is placed within the field of view of the camera of the terminal device 100 , that is, the camera can collect an image of the marker 200 . The image of the marker 200 is stored in the terminal device for locating the position of the terminal device relative to the marker. The marker 200 may include at least one sub-marker, and the sub-marker may be a pattern having a certain shape. In one embodiment, each sub-marker may have one or more feature points, wherein the shape of the feature points is not limited, and may be a dot, a ring, or a triangle or other shapes. In the embodiment of the present application, the distribution rules of the sub-markers in different markers are different, therefore, each marker 200 may have different identity information, and the terminal device 100 can obtain the The identity information corresponding to the marker 200, the identity information may be information such as a code that can be used to uniquely identify the marker 200, but is not limited thereto.

As an embodiment, the outline of the marker 200 can be a rectangle. Of course, the shape of the marker 200 can also be other shapes, which are not limited here. The rectangular area and multiple sub-markers in the area constitute a marker 200. In this embodiment of the present application, the marker 200 may be a pattern that the terminal device can identify and track. It should be noted that the specific marker 200 is not limited in the embodiments of the present application, and it is only required that the marker can be identified and tracked by the terminal device.

When the user is using the terminal device, when the marker 200 is within the field of view of the terminal device, the terminal device can collect a marker image including the marker 200 . The processor of the terminal device obtains the marker image and related information, identifies the marker image, obtains the identity information of the marker 200, and obtains the position and rotation relationship between the marker 200 and the camera of the terminal device, and further Obtain the position and rotation relationship of the marker relative to the terminal device.

For the above-mentioned display system, an embodiment of the present application provides a method for displaying a virtual object. Specifically, please refer to the following embodiments. Referring to FIG. 4, a display method of a virtual object is shown. As shown in Figure 4, the method includes:

Step S110: Obtain first spatial position information of the terminal device relative to the marker.

In this embodiment of the present application, the terminal device may identify markers within the field of view of the image acquisition device to obtain first spatial position information of the terminal device relative to the markers. When identifying the marker, in addition to obtaining the first spatial position information of the terminal device relative to the marker, the identity information and posture information of the marker can also be obtained.

Wherein, different markers may have different identity information, and the corresponding relationship between the markers and their identity information is stored in the terminal device. In addition, the correspondence between the identity information of the marker and the relevant data can be stored in the terminal device, so that after the terminal device recognizes the identity information of the marker, the identity information of the marker can be used to read the corresponding data of the marker. Data, for example, read the model data of the virtual object corresponding to the marker.

As an implementation manner, acquiring the first spatial position information of the terminal device relative to the marker may include: collecting an image containing the marker; identifying the image containing the marker to obtain the first spatial position information of the terminal device relative to the marker .

It can be understood that, the terminal device can use the image acquisition device to perform image acquisition on the markers within the field of view, so as to obtain an image containing the markers. The terminal device identifies and tracks the marker according to the image containing the marker, so as to obtain spatial position information of the terminal device relative to the marker. The spatial position information represents the six-degree-of-freedom information of the marker, including position information and attitude information. The attitude information is the orientation, rotation angle, etc. of the marker relative to the terminal device.

When the position of the terminal device changes, the marker image obtained by the terminal device using the camera will also change, that is, marker images from different perspectives are acquired, and the content of the markers in these marker images from different perspectives will be different. . Therefore, according to the marker images acquired by the terminal device at different positions, the orientation, rotation angle, etc. of the recognized marker relative to the terminal device will be different.

For example, when the outline of the marker is a rectangular sticker including multiple sub-markers with different patterns, the terminal device is located in front of and above the first side of the rectangular sticker, and when the image capture device faces the first side of the rectangular sticker, the terminal device The acquired image of the marker is an image obtained from the perspective of the first side of the marker. According to the image, it can be identified that the first side of the marker faces the terminal device, and the angle between the facing direction and the facing direction of the marker.

In addition, when the marker corresponds to the stored information, the stored information corresponding to the marker can also be obtained by identifying the marker. For example, if marker A corresponds to virtual object 1 for display, after obtaining the identity information of marker A by identifying marker A, the terminal device obtains virtual object 1 based on the corresponding relationship between marker A and virtual object 1 .

The terminal device can arbitrarily select a specific number of feature points from the image containing the marker as the target feature point to determine the real position information and attitude information between the terminal device (camera) and the marker with the target feature point. Wherein, the terminal device can obtain the pixel coordinates of all target feature points. Then, according to the pixel coordinates of all the feature points and the pre-acquired physical coordinates of all the feature points, the position information and attitude information between the terminal device and the marker are acquired, wherein the physical coordinates are the pre-acquired feature points corresponding to the marker. The coordinates in the physical coordinate system, the physical coordinates of each feature point can be acquired in advance and stored in the terminal device.

The terminal device can also determine the identity information of the marker by identifying the feature points in the image containing the marker.

Of course, the specific manner of identifying the image containing the marker to obtain the position information, posture information and identity information of the terminal device relative to the marker is not limited in the embodiments of the present application.

Step S120: Acquire second spatial position information of the marker relative to the target area of the object.

After the first spatial position information of the terminal device relative to the marker is obtained, the second spatial position information of the marker relative to the target area of the object can be obtained, so as to facilitate the subsequent determination of the relative positional relationship between the terminal device and the target area. The target area is a certain area where the virtual object needs to be displayed on the real object. For example, as shown in FIG. 5 , when the physical object 300 is a plane, the marker 200 is attached to the plane 300 . At this time, the target area 400 is an area on the plane. When the virtual object is subsequently displayed, the virtual object is displayed on the plane. of this area.

In the embodiment of the present application, the second spatial position information of the marker relative to the target area of the real object may be pre-stored in the terminal device, and the second spatial position information can be read by the terminal device. It may also be that the terminal device acquires the second spatial position information of the marker relative to the real target area in real time.

Step S130: Based on the first spatial location information and the second spatial location information, determine third spatial location information of the terminal device relative to the target area.

It can be understood that, after obtaining the first spatial position information of the terminal device relative to the marker and the second spatial position information of the marker relative to the target area of the real object, it can be based on the first spatial position information and the second spatial position information. The marker is used as a reference to obtain third spatial position information of the terminal device relative to the target area.

Step S140: Display the virtual object in the target area of the real object based on the third spatial position information.

In the embodiment of the present application, after obtaining the third spatial position information of the terminal device relative to the target area, the display coordinates of the virtual object in the virtual space can be calculated according to the third spatial position information, and the virtual object is displayed according to the display coordinates, Make the virtual object display in the target area that needs to be displayed.

As a method, based on the third spatial position information, displaying the virtual object in the target area of the real object may include:

acquiring the zoom ratio between the virtual object and the target area; determining the display coordinates of the virtual object based on the zoom ratio and the third spatial position information; and displaying the virtual object in the target area based on the display coordinates.

In the embodiment of the present application, the above-mentioned virtual object may be corresponding to a marker, that is, corresponding to the identity information of the marker, and the virtual object may also be a virtual object pre-created in the terminal device. When the virtual object is displayed in combination with the real object, the size of the model of the virtual object to be displayed may be larger or smaller than the target area size of the real object. If the size of the virtual object is not scaled, the displayed virtual object will be too large. If it is too large or too small, the virtual object cannot be aligned with the target area of the real object, and the effect of displaying the virtual object in the target area of the real object cannot be achieved.

Therefore, after obtaining the third spatial position information of the terminal relative to the target area, the zoom ratio between the virtual object and the target area can be obtained, so as to scale the virtual object and display it in the real target area according to the display coordinates.

Further, the scaling ratio can be calculated according to the ratio of the virtual object to the target area of the real object, so as to realize the scaling of the virtual object. Specifically, the scaling ratio is the ratio of the size of the virtual object to the size of the target area, where the ratio of the size may refer to the size of the occupied area. For example, the size of the area occupied by the virtual object in the virtual space is 50m (meters) * 50m, and the size of the target area of Therefore, the virtual object and the target area may be any other size. The size of the virtual object may be determined after the virtual object model is built, and the size of the target area may be the product of the proportion of the real object occupied by the region selected by the user and the area of the pre-stored real object.

Calculating the scaling ratio according to the ratio between the virtual object and the target area may include taking the ratio of the area occupied by the virtual object and the target area as the scaling ratio, or multiplying the ratio of the area occupied by the virtual object and the target area by a preset coefficient. as the zoom ratio.

That is to say, the zoom ratio of the virtual object can be the ratio of the virtual object to the target area, so that the virtual object can be aligned with the target area, so that the virtual object can be superimposed on the target area; the zoom ratio of the virtual object can also be the virtual object and the target area. The ratio of the target area is multiplied by a preset coefficient, for example, the preset coefficient can be between 0.7 and 1.3, or an experience coefficient can be set according to the user's visual experience, allowing the alignment to have a certain deviation value while ensuring the user's experience. A flexible scaling ratio can be used to realize the self-adaptation of the virtual object and the size of the target area, so as to improve the user's viewing experience.

After the scaling ratio of the virtual object is obtained, the virtual object to be displayed is scaled according to the scaling ratio. The terminal device can read the data corresponding to the virtual object, and the data can include the model data of the virtual object. The model data is the data used to render the virtual object, and can include the color used to create the model corresponding to the virtual object, and the vertices in the 3D model. Coordinates, etc., the terminal device scales the size of the virtual object according to the data corresponding to the virtual object.

For example, if the terminal device displays a line segment of a certain color, data such as the 3D point queue (point coordinates in multiple virtual spaces) of the line segment, the thickness of the line segment, and the color of the line segment can be used as the model data of the virtual object.

When the terminal device calculates the display coordinates of the virtual object in the display space, it can obtain the coordinates of the target area in the real space from the third space position information of the terminal device relative to the target area, and then convert the coordinates of the target area in the real space into the terminal device. The coordinates in the display space of the device, that is, the display coordinates of the virtual object that needs to be displayed in the target area in the display space of the terminal device are obtained. The terminal device can render the virtual object at the calculated display coordinates, so that the real position of the virtual object displayed by the terminal device is the same as the real position of the target area in the field of view of the terminal device, that is, the display position of the virtual object corresponds to the real position in the real scene. target area. For example, referring to FIG. 6 , when the real object 300 is a plane, the marker 200 is arranged on the plane, and the virtual object 500 is displayed in the target area 400 of the plane.

The user can observe the virtual object through the terminal device, the user of the terminal device can move the position, and the virtual object will also change with the movement of the user's position. For example, when the terminal device is far away from the position of the virtual object in the real scene, the virtual object changes. On the contrary, when the virtual object is close to the position in the real scene, the virtual object becomes larger and so on.

In addition, when the user of the terminal device can observe the virtual object from his own perspective, he can also record the displayed content, so as to facilitate the observation and analysis after the current observation.

In the display method of a virtual object provided by the embodiment of the present application, after obtaining the spatial position information of the terminal device relative to the marker and the spatial position information of the marker relative to the physical target area, the spatial position information of the terminal device relative to the target area is determined, and finally the spatial position information of the terminal device relative to the target area is determined. According to the spatial position information of the terminal device relative to the target area, the display coordinates of the virtual object are determined, and the virtual object is displayed in the target area of the real object, so as to realize the combined display between the virtual content and the real object, and improve the user's viewing experience.

In one embodiment, please refer to FIG. 7 , which is a schematic flowchart of a method for displaying a virtual object provided by an embodiment of the present application. The flow shown in FIG. 7 will be described in detail below, and the method for displaying a virtual object may specifically include the following steps:

Step S210: Obtain first spatial position information of the terminal device relative to the marker.

Step S220: Determine the target area of the physical object according to the user's instruction for selecting the target area of the physical object.

In the embodiment of the present application, the target area of the physical object may be determined according to the selection of the target area of the physical object by the user using the control device. Specifically, step S220 may include:

According to the attitude information, spatial position information and control instructions sent by the control device, the target area of the object selected by the user is obtained.

It can be understood that, when the user uses the terminal device to watch, the virtual object can be displayed in the target area of the entity according to the selection requirement of the control device.

For example, when the user wears the head-mounted display device, he can observe the real object through the head-mounted display device, and then the user moves through the control device, and uses the control buttons of the control device to select the target area where the virtual object needs to be displayed. That is, the control device can send its attitude information and spatial position information to the terminal device in the process of being moved. In addition, when the user selects the target area of the object by using the buttons, the control device can send the control instruction, that is, the button of the control device. The information is sent to the terminal device. After receiving the attitude information, spatial position information and control instructions sent by the control equipment, the terminal equipment can determine the target area to be selected by the user according to the attitude information, spatial position information and control instructions of the control equipment.

Among them, according to the attitude information, spatial position information and control instructions of the control device, the starting point selected in the display space can be obtained, and then according to the change of the attitude and position of the control device, the movement trajectory of the starting point can be obtained, and finally according to the movement trajectory, the starting point can be obtained. Determines the selected target area in the display space. In addition, according to the position of the target area in the display space and the position of the physical object, the positional relationship between the target area and the physical object in the display space can be determined, that is, the fifth spatial position information of the target area relative to the physical object.

In this embodiment of the present application, the target area may also be selected by detecting the user's gesture. When a preset gesture in the display space is detected, the selection of the target area is triggered, and then the target area selected in the display space can be determined according to the movement trajectory of the detected gesture.

In the embodiment of the present application, the target area can also be selected by tracking the change of the focus of the eyeball. It can be understood that a camera can be set in the head-mounted display device to obtain the image of the user's eyes. When the user uses the head-mounted display device to view the real object, the user's eye image is obtained, and the target area will be captured in the process of determining the target area. Based on the image data of the user's retina and cornea, the terminal device builds a 3D model of the eye based on these data, and tracks the focus of the eyeball through the three-dimensional space to achieve the selection of the target area.

Step S230: Obtain fourth spatial position information of the marker relative to the real object, and fifth spatial position information of the target area relative to the real object.

In this embodiment of the present application, the terminal device obtains the second spatial position information of the marker relative to the target area, which may be the fourth spatial position information of the marker relative to the physical object stored in the terminal device, and the fifth spatial position of the target area relative to the physical object. Information (which can be obtained according to the target area of the object determined in step 220). The fifth spatial position information of the target area relative to the real object can be obtained according to the target area of the real object determined in step 220 . When the target area is a preset area of the real object, the fifth spatial position information of the target area relative to the real object may also be pre-stored in the terminal device. When the terminal device needs to obtain the spatial position information of the marker relative to the target area, it can read the fourth spatial position information of the marker relative to the real object and the fifth spatial position information of the target area relative to the real object.

Step S240: Based on the fourth spatial position information and the fifth spatial position information, determine the second spatial position information of the marker relative to the target area.

After the fourth spatial position information of the marker relative to the real object and the fifth spatial position information of the target area relative to the real object are obtained, it is possible to determine the relationship between the marker and the real object according to the fourth spatial position information and the fifth spatial position information, taking the real object as a reference. The positional relationship of the target area is obtained, and the second spatial position information of the marker relative to the target area is obtained.

Step S250: Based on the first spatial position information and the second spatial position information, determine third spatial position information of the terminal device relative to the target area.

Step S260: Display the virtual object in the target area of the real object based on the third spatial position information.

In this embodiment of the present application, after the virtual object is displayed in the physical target area based on the third spatial position information of the terminal device relative to the target area, the position and posture of the terminal device relative to the target area may also change, for example The distance of the terminal device relative to the target area changes, and the angle of the terminal device toward the target area changes. When the position and posture of the terminal device relative to the target area changes, the position and posture of the terminal device relative to the marker will also change, that is, the position and posture of the marker recognized by the terminal device according to the acquired image containing the marker will change. change. At this time, the displayed virtual object can be adjusted according to the movement of the terminal device. For example, when the terminal device is close to the target area, the virtual object displayed by the terminal device will become larger, and vice versa.

Specifically, in the embodiment of the present application, the display method of the virtual object may further include:

When it is detected that the gesture information of the marker changes, according to the gesture information of the marker, a virtual object corresponding to the gesture information is rendered in the target area.

It can be understood that, when the terminal device determines that the gesture information of the marker has changed according to the acquired image containing the marker, the displayed virtual object can be adjusted according to the gesture information of the marker. For example, the virtual object is a The house, previously shown as the front of the house, when the terminal device faces the other side of the marker, and the detected gesture information of the marker changes, the side of the house is rendered in the target area according to the model data of the virtual object. Therefore, when the user adjusts the orientation of the relative marker, the virtual object displayed in the target area of the object also changes accordingly, so that the user can feel a more realistic visual effect and improve the user's visual experience.

Of course, in this embodiment of the present application, the method for displaying the virtual object may also include:

When it is detected that the distance of the terminal device relative to the marker changes, according to the distance of the terminal device relative to the marker, a virtual object corresponding to the changed distance is rendered in the target area.

It can be understood that, when the terminal device determines that the distance between the terminal device and the marker changes according to the acquired image containing the marker, the displayed virtual object can be adjusted according to the distance between the terminal device and the marker. For example, if the virtual object is a house, when it is detected that the distance between the terminal device and the marker is smaller, a larger house is rendered in the target area according to the model data of the virtual object, so that the user can When a larger house is observed, when it is detected that the distance between the terminal device and the marker is larger, a smaller house is rendered in the target area according to the model data of the virtual object, so that the user can move away from the marker. Smaller houses observed. Therefore, when the user adjusts the distance relative to the marker, the virtual object displayed in the target area of the real object also changes, that is, the change in the position of the virtual object relative to the user can be felt, so that the user can feel a more realistic visual effect. , to enhance the user's visual experience.

In the embodiment of the present application, there is also a situation in which the target area for displaying the virtual object is adjusted. Therefore, the method for displaying the virtual object may further include:

Obtaining the user's selection instruction for the target area of the object again; adjusting the target area according to the selection instruction obtained again; displaying the virtual object in the adjusted target area.

Understandably, when the user needs to adjust the position of the virtual object displayed on the physical object, the target area of the physical object can be re-selected, and the terminal device can obtain the user's selection instruction for the physical target area using the control device, and then re-select the target area of the physical object. Determine the target area. After re-determining the target area, the terminal device may redisplay the virtual object in the new target area, and the specific display may be performed according to the above-mentioned display mode of the virtual object.

In the embodiment of the present application, the display method of the virtual object can be applied to some scenes where markers need to be masked, so that the terminal device can recognize the markers set at a position that does not affect the beauty of the real object, and the virtual object can be displayed on the A certain area of the real object allows users to observe the virtual objects in a certain area of the real object without affecting the overall appearance. For example, in some AR board games, markers can be placed on the edge of the table, and users can observe virtual game objects on the table by wearing a terminal device; when applied to themed restaurants, markers can also be placed on the edge of the table, and the terminal device After identifying the markers, virtual models of some dishes on the menu can be observed, so that users can better order food accordingly. For another example, the method for displaying virtual objects can also be applied to advertisement display. The marker can be placed at a position on the advertisement column. After the terminal device recognizes the marker, the advertisement content can be displayed at the target area on the advertisement column. , so that users can observe the advertising content.

In the display method of a virtual object provided by the embodiment of the present application, the first spatial position information of the terminal device relative to the marker is obtained, and then the target area of the physical object is determined according to the user's selection instruction for the target area of the physical object, and then the target area of the physical object is determined according to the target area of the physical object. The positional relationship of the physical object, the positional relationship between the physical object and the target area, determine the second spatial position information of the marker relative to the target area, and then determine the spatial position of the terminal device relative to the target area according to the first spatial position information and the second spatial position information information, and finally display the virtual object in the target area of the object according to the spatial position information. In addition, this embodiment also provides that the displayed virtual object is adjusted according to the position and posture of the terminal device relative to the marker, so that the virtual object seen by the user corresponds to its movement, and the user's visual experience is improved.

Of course, in addition to obtaining the target area for displaying virtual objects through the control device provided in the above embodiments, other electronic devices connected to the terminal device, such as mobile phones, tablets, etc., can also be used to obtain the target area. The specific method may be as follows: an electronic device connected to the terminal device stores the model of the physical object, the electronic device can display the physical model, and the user can select a certain block on the physical model from the physical model displayed by the electronic device. area. Then, the electronic device sends the area on the physical model selected by the user to the terminal device, and the terminal device determines the target area on the physical model for displaying the virtual object according to the area on the physical model.

In one embodiment, please refer to FIG. 8 , which shows a block diagram of a module of a display apparatus 400 for a virtual object provided by an embodiment of the present application. The display apparatus 400 of the virtual object is applied to a terminal device. The block diagram shown in FIG. 8 will be described below. The virtual object display device 400 includes: a first position acquisition module 410 , a second position acquisition module 420 , a third position acquisition module 430 , and a display execution module 440 . Wherein, the first position obtaining module 410 is used to obtain the first spatial position information of the terminal device relative to the marker; the second position obtaining module 420 is used to obtain the second position of the marker relative to the physical target area. spatial location information; the third location acquisition module 430 is configured to determine third spatial location information of the terminal device relative to the target area based on the first spatial location information and the second spatial location information; the The display execution module 440 is configured to display the virtual object in the target area based on the third spatial position information.

In this embodiment of the present application, the display execution module 440 may be specifically configured to: obtain a zoom ratio between the virtual object and the target area; determine the virtual object based on the zoom ratio and the third spatial position information Display coordinates of an object; displaying the virtual object in the target area based on the display coordinates.

In this embodiment of the present application, referring to FIG. 9 , the display apparatus 400 of the virtual object may further include: an object rendering module 450 . The object rendering module 450 is configured to render a virtual object corresponding to the gesture information in the target area according to the gesture information of the marker when it is detected that the gesture information of the marker changes.

In this embodiment of the present application, referring to FIG. 9 , the display apparatus for the virtual object may further include: a region determination module 460 . The area determination module 460 is configured to determine the target area of the physical object according to the user's instruction for selecting the target area of the physical object.

Further, the area determination module 460 may be specifically configured to: obtain the target area of the object selected by the user according to the attitude information, spatial position information and control instructions sent by the control device.

In this embodiment of the present application, referring to FIG. 9 , the display apparatus for the virtual object may further include: an instruction obtaining module 470 , an area adjustment module 480 , and an object display module 490 . Wherein, the instruction obtaining module 470 is used to obtain the user's selection instruction for the target area of the object again; the area adjustment module 480 is used to adjust the target area according to the selection instruction obtained again; the object display module 490 is used to The virtual object is displayed in the adjusted target area.

In this embodiment of the present application, the second position obtaining module 420 may be specifically configured to: obtain fourth spatial position information of the marker relative to the real object, and fifth spatial position information of the target area relative to the real object; Based on the fourth spatial position information and the fifth spatial position information, second spatial position information of the marker relative to the target area is determined.

In one embodiment, the present application further provides a terminal device, including a memory and a processor, where a computer program is stored in the memory, and when the computer program is executed by the processor, the processor implements the above method for displaying virtual objects.

In one embodiment, the present application further provides a computer storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned method for displaying a virtual object can be implemented.

To sum up, the above-mentioned display method, device, terminal device and storage medium for virtual objects provided by this application can obtain the first spatial position information of the terminal device relative to the marker, and then obtain the first position of the marker relative to the target area of the real object. Second spatial position information, and then based on the first spatial position information and the second spatial position information, determine the third spatial position information of the terminal device relative to the target area, and finally display the virtual object in the physical target area based on the third spatial position information, Therefore, the virtual object is displayed in the target area of the real object, and the combined display of the real object and the virtual object is completed.

It should be noted that, the functions of each device in the system of the embodiment of the present application may be specifically implemented according to the method in the foregoing method embodiment, and the specific implementation process may refer to the relevant description of the foregoing method embodiment, which is not repeated here.

It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. For the same and similar parts among the various embodiments, refer to each other Can. As for the apparatus type embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for the relevant part, please refer to the partial description of the method embodiment.

It should be noted that, herein, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements, It also includes other elements not expressly listed or inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element.

Those of ordinary skill in the art can understand that all or part of the steps of implementing the above embodiments can be completed by hardware, or can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. The storage medium mentioned may be a read-only memory, a magnetic disk or an optical disk, etc.

The above are only preferred embodiments of the present application, and are not intended to limit the present application in any form. Although the present application has been disclosed above with preferred embodiments, it is not intended to limit the present application. The technical personnel, within the scope of the technical solution of the present application, can make some changes or modifications to equivalent examples of equivalent changes by using the technical content disclosed above. Any simple modifications, equivalent changes and modifications made to the above embodiments still fall within the scope of the technical solutions of the present application.

Claims (10)

1, kinds of virtual object display method, characterized in that, applied to terminal equipment, the method includes:

spatial position information of the terminal equipment relative to the marker is obtained;

acquiring second spatial position information of the marker relative to a target area of the real object;

determining third spatial position information of the terminal device relative to the target area based on the th spatial position information and the second spatial position information;

and displaying a virtual object in the target area of the real object based on the third spatial position information.

2. The method of claim 1, wherein displaying a virtual object in a target area of the real object based on the third spatial location information comprises:

acquiring the scaling between the virtual object and the target area;

determining display coordinates of the virtual object based on the scaling and the third spatial location information;

displaying the virtual object in the target area based on the display coordinates.

3. The method of claim 1, wherein after the displaying a virtual object in the target area based on the third spatial location information, the method further comprises:

and when the change of the attitude information of the marker is detected, rendering a virtual object corresponding to the attitude information in the target area according to the attitude information of the marker.

4. The method of claim 1, wherein prior to said obtaining second spatial location information of the target region of the marker relative to the physical object, the method further comprises:

and determining the target area of the real object according to a selection instruction of a user on the target area of the real object.

5. The method according to claim 4, wherein the determining the target area of the real object according to the user's selection instruction for the target area of the real object comprises:

and obtaining a target area of the real object selected by the user according to the attitude information, the spatial position information and the control instruction sent by the control equipment.

6. The method of claim 4, wherein after displaying the virtual object in the target area based on the third spatial location information, the method further comprises:

obtaining a selection instruction of the user for the target area of the real object again;

adjusting the target area according to the obtained selecting instruction;

and displaying the virtual object in the adjusted target area.

7. The method of claim 1, wherein said obtaining second spatial position information of the marker relative to the target region comprises:

acquiring fourth spatial position information of the marker relative to the real object and fifth spatial position information of the target area relative to the real object;

determining second spatial position information of the marker relative to the target region based on the fourth spatial position information and the fifth spatial position information.

The device for displaying the virtual objects of 8 and types is applied to terminal equipment and comprises a position acquisition module, a second position acquisition module, a third position acquisition module and a display execution module, wherein,

the th position acquisition module is used for acquiring th spatial position information of the terminal device relative to a marker;

the second position acquisition module is used for acquiring second spatial position information of the marker relative to a target area of the real object;

the third position obtaining module is configured to determine third spatial position information of the terminal device relative to the target area based on the th spatial position information and the second spatial position information;

the display execution module is used for displaying the virtual object in the target area based on the third spatial position information.

Terminal device of the kind , comprising a memory and a processor, the memory having stored therein a computer program that, when executed by the processor, causes the processor to carry out the method of any of claims 1 to 7 and .

10, computer storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method of any of claims 1-7 to .

Images