CN118485754A - Display method, device, electronic device and medium for objects to be displayed - Google Patents

Abstract

The present disclosure provides a display method, device, electronic device and medium for objects to be displayed. The specific implementation scheme is: determine at least one target object to be displayed that belongs to the target display mode; render each target object to be displayed according to the preset attributes to obtain at least one first map corresponding to each target object to be displayed; wherein the preset attributes include at least one item of the position, angle, size and scale in the display area of the terminal device; based on the first posture of the terminal device and the attribute information corresponding to other objects to be displayed except the target object to be displayed, render other objects to be displayed to obtain at least one second map corresponding to other objects to be displayed; wherein the attribute information includes the position, angle and scale of other objects to be displayed in the reference coordinate system; based on at least one first map, at least one second map and the second posture of the terminal device, determine the screen to be displayed.

Description

Display method, device, electronic device and medium for objects to be displayed

Technical Field

The present disclosure relates to the technical field of virtual reality or augmented reality, and in particular to a display method, device, electronic device and medium for an object to be displayed.

Background Art

In the process of virtual reality (VR) applications, time warp is a technology that reduces the delay between motion and images. Usually, after the first rendering of the objects in the virtual world, the texture obtained by the first rendering is rendered a second time according to the latest posture to obtain the image to be displayed.

Summary of the invention

In view of this, the purpose of the present disclosure is to provide a display method, device, electronic device and medium for objects to be displayed, by rendering the target object to be displayed that needs to follow the viewing angle separately from other objects to be displayed, rendering the target object to be displayed according to its preset attributes in the display area of the terminal device, and rendering other objects to be displayed according to the posture of the terminal device and the attribute information of other objects to be displayed in the reference coordinate system, so that the target object to be displayed can be displayed more stably following the user's viewing angle, thereby effectively reducing the jitter of the picture.

In a first aspect, an embodiment of the present disclosure provides a display method for objects to be displayed, comprising: determining at least one target object to be displayed that belongs to a target display mode; rendering each target object to be displayed according to preset attributes to obtain at least one first map corresponding to each target object to be displayed; wherein the preset attributes include at least one item of position, angle, size, and scale in the display area of a terminal device; based on a first posture of the terminal device and attribute information corresponding to other objects to be displayed except the target object to be displayed, rendering other objects to be displayed to obtain at least one second map corresponding to other objects to be displayed; wherein the attribute information includes the position, angle, and scale of other objects to be displayed in a reference coordinate system; based on at least one first map, at least one second map, and a second posture of the terminal device, determining a picture to be displayed; wherein the second posture is obtained later than the first posture.

In a second aspect, an embodiment of the present disclosure provides a display device for objects to be displayed, comprising: a first processing module, used to determine at least one target object to be displayed that belongs to a target display mode; a second processing module, used to render each target object to be displayed according to preset attributes, and obtain at least one first map corresponding to each target object to be displayed; wherein the preset attributes include at least one item of position, angle, size, and scale in the display area of the terminal device; a third processing module, used to render other objects to be displayed based on the first posture of the terminal device and the attribute information corresponding to other objects to be displayed except the target object to be displayed, and obtain at least one second map corresponding to other objects to be displayed; wherein the attribute information includes the position, angle, and scale of other objects to be displayed in the reference coordinate system; a fourth processing module, used to determine the picture to be displayed based on at least one first map, at least one second map, and the second posture of the terminal device; wherein the second posture is obtained later than the first posture.

In a third aspect, an embodiment of the present disclosure provides a computer-readable storage medium, which stores a computer program, and the computer program is used to execute the display method for an object to be displayed provided by any of the above embodiments of the present disclosure.

In a fourth aspect, an embodiment of the present disclosure provides an electronic device, which includes: a processor; a memory for storing processor executable instructions; a processor, configured to read executable instructions from the memory and execute the instructions to implement a display method for an object to be displayed provided in any of the above embodiments of the present disclosure.

In a fifth aspect, an embodiment of the present disclosure provides a computer program product, the computer program product includes computer program instructions, and when the computer program instructions are executed by a processor, they implement the display method for an object to be displayed provided by any of the above embodiments of the present disclosure.

The technical solution of the present disclosure is further described in detail below through the accompanying drawings and embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other purposes, features and advantages of the present disclosure will become more apparent by describing the embodiments of the present disclosure in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. In the drawings, the same reference numerals generally represent the same components or steps.

FIG. 1 is an exemplary system architecture of an embodiment of a method or apparatus for displaying an object to be displayed that can be applied to the present disclosure;

FIG2 is a schematic flow chart of a method for displaying an object to be displayed provided by an exemplary embodiment of the present disclosure;

FIG3 is a schematic flow chart of a method for displaying an object to be displayed provided by another exemplary embodiment of the present disclosure;

FIG4 is a flow chart of step 50 provided by an exemplary embodiment of the present disclosure;

FIG5 is a flow chart of step 520 provided by an exemplary embodiment of the present disclosure;

FIG6 is a schematic structural diagram of a display device for displaying an object provided by an exemplary embodiment of the present disclosure;

FIG7 is a schematic structural diagram of a display device for displaying an object provided by another exemplary embodiment of the present disclosure;

FIG. 8 is a schematic diagram of the structure of an application embodiment of the electronic device disclosed herein.

DETAILED DESCRIPTION

Below, the exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described here.

In the description of the present disclosure, it should be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only simplified descriptions for the convenience of describing the present disclosure, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present disclosure.

It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

In the description of the present disclosure, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present disclosure can be understood according to specific circumstances.

Those skilled in the art can understand that the terms "first" and "second" in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor indicate the necessary logical order between them.

It should also be understood that in the embodiments of the present disclosure, “plurality” may refer to two or more than two, and “at least one” may refer to one, two, or more than two.

It should also be understood that any component, data or structure mentioned in the embodiments of the present disclosure can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

In addition, the term "and/or" in the present disclosure is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and/or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character "/" in the present disclosure generally indicates that the associated objects before and after are in an "or" relationship.

It should also be understood that the description of the various embodiments in the present disclosure focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.

At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

The embodiments of the present disclosure can be applied to electronic devices such as head-mounted display devices, terminal devices, servers, etc., which can operate with many other general or special computing system environments or configurations. Electronic devices such as head-mounted display devices, terminal devices, servers, etc. can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Generally, program modules can include routines, programs, target programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. The computer system/server can be implemented in a distributed cloud computing environment, in which tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located on a local or remote computing system storage medium including a storage device.

The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

FIG1 is an exemplary system architecture of an embodiment of a method or apparatus for displaying an object to be displayed that can be applied to the present disclosure. The system architecture may include a head mounted display device 1, a network 2, and a server 3. The network 2 may be a medium for providing a communication link between the head mounted display device 1 and the server 3. The network 2 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

The head-mounted display device 1 may be an electronic device with an image display function, including but not limited to AR (Augmented Reality) smart glasses, VR (Virtual Reality) smart glasses, and the like. The head-mounted display device 1 may be an all-in-one device, in which the function of a computing unit is integrated, and the computing unit may provide processing capabilities for the head-mounted display device, and various client applications installed on the head-mounted display device are actually installed in the computing unit. Alternatively, the head-mounted display device may also be a split device, in which the split head-mounted display device includes a head-mounted display and a computing unit that is separately arranged from the head-mounted display, and the display screen of the head-mounted display is used to display the image processed by the computing unit. The computing unit in the split head-mounted display device may be a terminal device, and the terminal device may be a mobile phone, a handle, an adapter, a PC, a tablet, a car computer, and the like. Here, for the convenience of representation, in the case where the execution subject is a head-mounted display device, the following text will introduce the scheme of the present disclosure using the all-in-one head-mounted display device as the action execution subject, and the head-mounted display device 1 is used to represent it in the figure. Those skilled in the art should understand that, for a split-type head-mounted display device, the execution entity is the computing unit.

The head-mounted display device 1 can provide various services, such as determining at least one target object to be displayed that belongs to the target display mode, rendering each target object to be displayed according to its preset attributes in the display area of the terminal device, and obtaining at least one first map corresponding to each target object to be displayed; rendering other objects to be displayed except the target object to be displayed according to the posture of the terminal device and the attribute information of other objects to be displayed in the reference coordinate system, and obtaining at least one second map corresponding to other objects to be displayed; determining the screen to be displayed based on at least one first map and at least one second map. The display method for the object to be displayed provided in the embodiment of the present disclosure can be executed by the head-mounted display device 1 above, and accordingly, the display device for the object to be displayed is usually arranged in the head-mounted display device 1, and the display screen can be located in the display device.

It should also be pointed out that the display method for objects to be displayed provided by the present disclosure can also be applied to server 3, which can be a background page server. In the case where the method of the present disclosure is applied to server 3, after server 3 determines at least one target object to be displayed belonging to the target display mode, each target object to be displayed can be rendered according to its preset attributes in the display area of the terminal device to obtain at least one first map corresponding to each target object to be displayed; for other objects to be displayed except the target object to be displayed, the terminal device and the attribute information of other objects to be displayed in the reference coordinate system are rendered to obtain at least one second map corresponding to other objects to be displayed; the screen to be displayed is determined based on at least one first map and at least one second map. Accordingly, the display device for the object to be displayed can also be set in server 3.

In addition to the server and the head mounted display device, the display method for the object to be displayed can also be applied to other terminal devices (not shown in the figure). The other terminal devices here can be mobile phones, tablets, computers, etc.

Exemplary Methods

FIG2 is a flow chart of a method for displaying an object to be displayed provided by an exemplary embodiment of the present disclosure. The embodiment of the present disclosure can be applied to an electronic device. As shown in FIG2 , the method includes the following steps:

Step 20: Determine at least one target object to be displayed that belongs to the target display mode.

In some optional embodiments of the present disclosure, the target display mode may refer to a follow-up (following perspective or following field of view) display mode, and the target object to be displayed belonging to the target display mode has a fixed relative posture relationship or a relative posture relationship with some degrees of freedom locked with the virtual camera observation posture. Among them, the virtual camera usually maintains a consistent posture relationship with the terminal device to which it belongs. When the movement of the terminal device causes the posture of the terminal device to change, the virtual camera usually also undergoes a corresponding posture change, so that the picture seen from the virtual camera can reflect the movement of the terminal device, and this picture will be presented on the display of the terminal device so that the display picture seen by the user is consistent with the picture seen from the virtual camera.

In some optional embodiments of the present disclosure, partial DOF locking means that some DOFs in three or six DOFs need to be displayed with a head-on display. For example, the yaw direction of the virtual camera is with a head-on display, but the pitch and roll directions of the virtual camera are not with a head-on display (this display mode can be called 2 DOF), or the yaw and pitch directions of the virtual camera are with a head-on display, but the roll direction of the virtual camera is not with a head-on display (this display mode can be called 1 DOF). Of course, which directions are with a head-on display and which directions are not with a head-on display can be set according to actual needs. The target object to be displayed has a fixed relative posture relationship with the virtual camera observation posture (this display mode can be called 0 DOF), which means that no matter how the observation posture of the virtual camera changes, the posture of the target object to be displayed in the virtual camera coordinate system remains unchanged. Similarly, the relative posture relationship of partial DOF locking means that the coordinate components of some degrees of freedom in the virtual camera coordinate system remain unchanged. For example, for a target object to be displayed with 2 degrees of freedom, its position and posture in the virtual camera coordinate system is expressed as (x, y, z, yaw, pitch, roll), where the x, y, z and yaw components remain unchanged, and the pitch and roll components correspond to the pitch angle and roll angle in a fixed reference coordinate system. When the virtual camera rotates relative to the reference coordinate system in the pitch and roll directions, the pitch and roll components of the target object to be displayed in the virtual camera coordinate system will change, that is, the observation angle in the pitch and roll directions will change.

In some optional embodiments of the present disclosure, the target object to be displayed belonging to the target display mode may include, for example, at least one of a video window when playing a video, a switching interface when switching between three-degree-of-freedom and six-degree-of-freedom modes, and a pop-up window for reminding the user during the use of the application.

In some optional embodiments of the present disclosure, the method for determining the target to-be-displayed object belonging to the target display mode can be set according to actual needs. For example, a preset type set belonging to the target to-be-displayed mode can be set, so that the target to-be-displayed object can be determined according to whether the type corresponding to each to-be-displayed object belongs to the preset type set. For another example, a target display mode identifier can be set for the to-be-displayed object that needs to be displayed according to the target display mode, so that the target to-be-displayed object can be determined according to the target display mode identifier carried by the to-be-displayed object, and so on.

An execution subject of the display method for an object to be displayed (which may be a terminal device, for example, a head-mounted display device, a mobile phone, a server, etc.) determines at least one target object to be displayed belonging to a target display mode.

Step 30: Render each target object to be displayed according to preset attributes to obtain at least one first texture corresponding to each target object to be displayed.

In some optional embodiments of the present disclosure, the preset attribute includes at least one of a position, an angle, a size, and a scaling in a display area of the terminal device.

In some optional embodiments of the present disclosure, the display area of the terminal device is the area for displaying the picture to be displayed. When the picture to be displayed is displayed, the picture to be displayed fills the display area, so that the user can view the content of the picture to be displayed in the display area.

In some optional embodiments of the present disclosure, the position in the display area of the terminal device included in the preset attributes indicates the position of the target object to be displayed when it is displayed in the display area, and also indicates the position of the target object to be displayed in the screen to be displayed. Similarly, the angle indicates the rotation angle of the target object to be displayed in the display area around at least one of the yaw angle, pitch angle, and roll angle of the virtual camera, the size indicates the size of the target object to be displayed in the display area, and the zoom indicates the zoom factor of the target object to be displayed when it is displayed in the display area. The preset attributes can be pre-set and stored. The preset attributes can also change according to the user's adjustments during the application process. For example, the user adjusts at least one of the position, angle, size, and zoom of the target object to be displayed in the display area.

In some optional embodiments of the present disclosure, the terminal device may be a device equipped with a camera, such as a head-mounted display device, a mobile phone, a tablet, etc.

Step 40: based on the first posture of the terminal device and the attribute information corresponding to the other objects to be displayed except the target object to be displayed, render the other objects to be displayed to obtain at least one second map corresponding to the other objects to be displayed.

In some optional embodiments of the present disclosure, the attribute information includes the position, angle, and scale of other objects to be displayed in the reference coordinate system.

In some optional embodiments of the present disclosure, the reference coordinate system is a relatively stable coordinate system, such as a world coordinate system, or a three-dimensional space coordinate system rigidly connected to the world coordinate system set based on application requirements. The attribute information may include the position, angle, and scale of other objects to be displayed. Among them, the position can be expressed as the coordinates of other objects to be displayed in the reference coordinate system, the angle can be expressed as the posture or orientation of other objects to be displayed in the reference coordinate system, and the scale can represent the zoom multiple of other objects to be displayed. The attribute information may be pre-set and stored for each other object to be displayed. The attribute information may also change according to the user's adjustment during the application process. For example, the user adjusts at least one of the position, angle, and scale of other objects to be displayed.

In some optional embodiments of the present disclosure, the first pose may include at least one of a position and a pose. The first pose may be a real-time pose of the terminal device obtained when at least one second map is about to be rendered, or may be an estimated pose of the terminal device at the next display moment, which may be set according to actual needs.

In some optional embodiments of the present disclosure, the terminal device may be configured with at least one of an IMU (Inertial Measurement Unit) and other sensors that can be used to measure the speed, acceleration, angle, angular velocity, etc. of the terminal device. The first posture can be obtained based on the data collected by the relevant sensors on the terminal device, for example, based on the IMU and the camera, combined with the IMU data and the camera image, through VSLAM (Visual Simultaneous Localization and Mapping) to achieve at least one of the real-time positioning and posture estimation of the posture. For the estimated posture of the terminal device, the specific posture estimation method can be set according to actual needs, for example, a preset posture estimation algorithm can be used for estimation. The preset posture estimation algorithm may include, for example, any filtering algorithm, optimization algorithm, and the like. Among them, the filtering algorithm may be, for example, a Euro filtering algorithm, a Kalman filtering algorithm, and the like. The optimization algorithm may include a least squares optimization algorithm and other possible algorithms.

In some optional embodiments of the present disclosure, other objects to be displayed may include all objects to be displayed except the target object to be displayed, and other objects to be displayed may include one or more objects to be displayed. For example, other objects to be displayed may include objects having at least one of a fixed position and a fixed posture in a reference coordinate system. When the observation posture of the virtual camera in the reference coordinate system changes, it means that at least one of the position and posture of other objects to be displayed in the virtual camera coordinate system has changed, thereby causing the situation of other objects to be displayed in the observation field to change.

In some optional embodiments of the present disclosure, based on the first posture of the terminal device and the attribute information corresponding to the other objects to be displayed, rendering the other objects to be displayed means: according to the relative posture relationship of the virtual camera observation posture in the first posture with respect to the other objects to be displayed, rendering the image (i.e., the second map) of the other objects to be displayed that should be seen in the first posture. At least one second map corresponding to the other objects to be displayed can be obtained according to actual needs. For example, one second map can be generated for multiple other objects to be displayed, or multiple second maps can be generated.

The virtual camera can be fixedly connected to the terminal device, so the posture of the terminal device can also reflect the posture of the virtual camera.

There is no restriction on the execution order of the above steps 30 and 40, and they can be executed sequentially or in parallel.

Step 50: determine a picture to be displayed based on at least one first map, at least one second map, and a second posture of the terminal device.

In some optional embodiments of the present disclosure, the second posture is obtained later than the first posture.

In some optional embodiments of the present disclosure, the time corresponding to the second posture is not earlier than the time corresponding to the first posture.

In some optional embodiments of the present disclosure, the second posture can be the latest posture of the terminal device, or it can be the estimated posture of the terminal device at the next display time. For example, after completing the rendering of other objects to be displayed and obtaining at least one second map, it is necessary to further determine the screen to be displayed. At this time (which can be called the current moment), the real-time posture of the terminal device at the current moment can be obtained as the second posture, or the posture of the terminal device at the next display moment can be estimated at the current moment as the second posture, without specific limitation.

In some optional embodiments of the present disclosure, the first posture is the real-time posture of the terminal device obtained when at least one second map is about to be rendered, and correspondingly, the second posture is the real-time posture of the terminal device obtained when the picture to be displayed needs to be further determined after obtaining at least one second map.

In some optional embodiments of the present disclosure, the first posture is the real-time posture of the terminal device obtained when at least one second map is about to be rendered, and the second posture is the estimated posture of the terminal device at the next display moment.

In some optional embodiments of the present disclosure, the first pose is a first estimated pose of the terminal device at the next display moment estimated at any moment before rendering at least one second map, and the second pose is a second estimated pose of the terminal device at the next display moment estimated at any moment later than the predicted moment of the first estimated pose.

In some optional embodiments of the present disclosure, if the second posture is the latest posture of the terminal device, the determined picture to be displayed is the picture to be displayed corresponding to the latest posture of the terminal device. The picture to be displayed can be displayed on the display screen of the terminal device in real time, or it can be displayed on the display screen when the display time arrives.

In some optional embodiments of the present disclosure, if the second posture is the estimated posture of the terminal device at the next display moment, the determined picture to be displayed is the picture to be displayed at the next display moment. After determining the picture to be displayed at the next display moment, the picture to be displayed can be displayed on the display screen of the terminal device when the next display moment arrives.

In some optional embodiments of the present disclosure, the determination principle of the second posture is similar to that of the first posture, and will not be repeated here. Since the second posture is obtained later than the first posture, relatively speaking, the second posture will be closer to the actual posture of the terminal device at the time to be displayed, and the second posture may have some changes relative to the first posture, such as a relative change in at least one of the position and posture.

In order to ensure low latency of other objects to be displayed, it is necessary to perform time warping (Time Warp) on at least one second map of other objects to be displayed based on the change in the second pose relative to the first pose, so as to adjust its display position, angle, etc. in the screen to be displayed. For the target object to be displayed, the rendering result of the target object to be displayed in the screen to be displayed can be determined directly based on at least one first map, without performing time warping based on the second pose, so as to avoid jitter of the target object to be displayed.

In the related art, all objects to be displayed are rendered according to a first pose to obtain a texture, and then the texture is time-distorted according to the pose change of the second pose relative to the first pose to obtain the image to be displayed. The inventors found that the target object to be displayed in this way will have a jitter problem.

The display method for objects to be displayed provided by the embodiments of the present disclosure renders each target object to be displayed according to its preset attributes in the display area of the terminal device to obtain at least one first map, instead of rendering the target object to be displayed according to the first pose to obtain a map as in the related art. Other objects to be displayed are rendered according to the first pose to obtain at least one second map. When determining the screen to be displayed, in order to ensure the low latency of other objects to be displayed, the second map can be further time-distorted based on the second pose of the terminal device, and the screen to be displayed is determined in combination with at least one first map corresponding to the target object to be displayed, thereby reducing the jitter of the target object to be displayed that needs to be displayed following the viewing angle while taking into account the low latency of other objects to be displayed, thereby improving the user experience.

In some optional embodiments of the present disclosure, a correspondence between pixels of the to-be-displayed image and pixels of the first map can be established based on preset attributes, so that sampling can be performed from the first map based on the correspondence to determine the pixel color value of each pixel in the to-be-displayed image. The correspondence can be determined by the relationship between the preset attribute and the display area of the terminal device and the relationship between the to-be-displayed image and the display area.

In some optional embodiments of the present disclosure, the attribute information of other objects to be displayed can be combined with the second posture to determine the correspondence between the pixels of the screen to be displayed and the pixels of the second map, so that the pixel color value of each pixel in the screen to be displayed can be determined by sampling from the second map based on the correspondence. The specific setting can be based on actual needs and is not limited in this embodiment.

In some optional embodiments of the present disclosure, when determining the picture to be displayed, each target object to be displayed and other objects to be displayed can be rendered as different levels, and then the sub-images rendered at each level can be fused to obtain the picture to be displayed. It can also be that each pixel of the picture to be displayed is sampled on the first map of each target object to be displayed and the second map of other objects to be displayed, and the pixel values corresponding to each pixel are collected, and the picture to be displayed is obtained by fusion according to the level, etc. It can be set specifically according to actual needs. The level of each target object to be displayed and the level of other objects to be displayed can be determined according to preset rules. The preset rules can be set according to actual needs. For example, the preset rules can include the positional relationship between each target object to be displayed and other objects to be displayed, and can also include the category of each target object to be displayed and other objects to be displayed, and so on.

In some optional embodiments of the present disclosure, the display screen of the terminal device may be, for example, a display screen of a head-mounted display device, a display screen of a mobile phone, a display screen of a tablet, etc., so that the user can view the content of the image to be displayed through the display screen.

The display method for objects to be displayed provided by the embodiments of the present disclosure can, for target objects to be displayed that need to be displayed following the viewing angle, be rendered according to their preset attributes in the display area of a terminal device to obtain at least one first map, and for other objects to be displayed, be rendered according to a first posture to obtain at least one second map, and then determine the picture to be displayed based on the first map, the second texture and the second posture of the terminal device, thereby reducing the jitter of the target objects to be displayed that need to be displayed following the viewing angle while taking into account the low latency of other objects to be displayed, thereby improving the user experience.

In some optional embodiments of the present disclosure, the step 20 of determining at least one target to-be-displayed object belonging to the target display mode includes:

At least one target object to be displayed belonging to the target display mode is determined according to the types corresponding to the objects to be displayed and the preset type set.

In some optional embodiments of the present disclosure, the preset type set stores types belonging to the target display mode.

In some optional embodiments of the present disclosure, the preset type set can be set according to actual needs. For example, the preset type set may include a pop-up window type, a video window type, a mode switching interface type, etc. The type of each object to be displayed is matched with the preset type set. If the type of a certain object to be displayed is in the preset type set, the object to be displayed is determined to be a target object to be displayed belonging to the target display mode.

In some optional embodiments of the present disclosure, the types in the preset type set can be adjusted according to the user's instructions, and the adjustment can include at least one of adding, deleting, and modifying. Then step 20 determines the target object to be displayed according to the adjusted preset type set.

The display method for objects to be displayed provided by the embodiments of the present disclosure sets a preset type set to store types belonging to the target display mode, so that during the rendering process, the target objects to be displayed belonging to the target display mode can be easily determined, thereby improving processing efficiency.

In some optional embodiments of the present disclosure, the step 20 of determining at least one target to-be-displayed object belonging to the target display mode includes:

In response to any to-be-displayed object carrying a target display mode identifier, it is determined that the to-be-displayed object belongs to the target display mode.

In some optional embodiments of the present disclosure, a target display mode identifier can be set in advance for the objects to be displayed that belong to the target display mode, so that the objects to be displayed that belong to the target display mode carry the target display mode identifier. During the rendering process of the picture to be displayed, the target objects to be displayed that belong to the target display mode can be determined by identifying the target display mode identifier carried by each object to be displayed.

In some optional embodiments of the present disclosure, the specific representation method of the target display mode identifier can be set according to actual needs, as long as the target display mode can be uniquely identified.

The display method for the object to be displayed provided by the embodiment of the present disclosure can further improve processing efficiency by identifying the target display mode identifier carried by the object to be displayed to determine the target object to be displayed belonging to the target display mode.

In some optional embodiments of the present disclosure, the target to-be-displayed object belonging to the target display mode may also be determined based on the user's operation. For example, the user may choose to display a certain to-be-displayed object in the target display mode, and then, in response to the user's operation, the to-be-displayed object may be used as the target to-be-displayed object belonging to the target display mode.

In an optional example, when a user long presses an object, such as a window, with a control handle, an option may pop up for the user to choose whether to display the object in target display mode. If the user chooses yes, the window will skip time warping during the rendering process to ensure the stability of the window.

In an optional example, the user may also pre-set an application (APP) as the target display mode. When rendering the application, the application will be rendered according to the rendering process corresponding to the target display mode to ensure the visual stability of the application.

In some optional embodiments of the present disclosure, when a user manually adds a target display mode identifier for certain windows or applications, in step 30, the original attribute information of these windows or applications will be shielded, and these windows or applications will be rendered using preset attributes to obtain a first map.

In some optional embodiments of the present disclosure, objects to be displayed that belong to a type in a preset type set and objects to be displayed that carry a target display mode identifier among the objects to be displayed may be taken together as target objects to be displayed.

FIG. 3 is a schematic flow chart of a method for displaying an object to be displayed provided by another exemplary embodiment of the present disclosure.

In some optional embodiments of the present disclosure, the method of the present disclosure further includes:

Step 610: Determine the level of each first texture and the level of each second texture according to a preset rule.

In some optional embodiments of the present disclosure, the preset rules may include the positional relationship between the first map of each target object to be displayed and the second map of other objects to be displayed, and may also include the categories of each target object to be displayed and other objects to be displayed, etc. The positional relationship may include at least one of depth and occlusion. The categories may include target categories and other categories. Among them, the target category corresponds to the target object to be displayed, and the other categories correspond to other objects to be displayed.

In some optional embodiments of the present disclosure, the levels of each first map and the levels of each second map may be represented in any manner, such as by numbers or other symbols, which may be specifically set according to actual needs.

In an optional example of the present disclosure, the level can be represented by a level number, such as 1, 2, 3, ..., and a larger level number can represent a higher level. The level number of each target to-be-displayed object can be determined according to the occlusion relationship, for example, the level of the occluded target to-be-displayed object is lower than the level of the target to-be-displayed object that occludes it.

In some optional embodiments of the present disclosure, the layer number of the target to-be-displayed object may be greater than that of other to-be-displayed objects. For example, when switching between three-degree-of-freedom and six-degree-of-freedom modes, the switching interface is at the top to ensure that the user can clearly see the switching interface.

In some optional embodiments of the present disclosure, each first map may be set at the same level, or may be set at different levels according to relative position relationships, and may be set specifically according to actual needs.

The display method for objects to be displayed provided by the embodiments of the present disclosure can handle the occlusion relationship between each target object to be displayed and other objects to be displayed when determining the picture to be displayed by determining the levels for each first map and each second map.

In some optional embodiments of the present disclosure, determining the level of each first map and the level of each second map according to a preset rule in step 610 includes:

Each first map is configured as a preset level; based on the positional relationship of all second maps when the terminal device is in a target position, the level of each second map is determined.

In some optional embodiments of the present disclosure, the target posture includes any one of the following: a first posture, a second posture. The position relationship includes at least one of the following: depth, occlusion.

In some optional embodiments of the present disclosure, the preset level can be set according to actual needs, such as configuring each first map to be a level with a preset number of the highest layer number, and the preset number is consistent with the number of first maps. For example, when there are three first maps, the layer numbers of the three first maps can be configured as 999, 998, and 997 respectively.

In some optional embodiments of the present disclosure, when the terminal device is in the target posture, each second map is observed through the camera or virtual camera of the terminal device, and the observable content will be affected by the positional relationship of each second map relative to the terminal device, such as the distance, angle, etc. Among them, the distance can characterize the depth of other objects to be displayed, and the angle can characterize the viewing angle. Different viewing angles may lead to different occlusion relationships between other objects to be displayed. Based on this principle, the level of each second map can be determined according to the positional relationship of all second maps when the terminal device is in the target posture. The depth information of the second map relative to the camera or virtual camera in the target posture can be determined based on the attribute information of other objects to be displayed corresponding to the second map and the target posture.

In actual applications, the levels of the objects to be displayed can also be adjusted according to the user's operations on the objects to be displayed. For example, if the user drags the second object to be displayed, which was originally blocked by the first object to be displayed, to a state where it blocks the first object to be displayed, the levels of the first object to be displayed and the second object to be displayed can be adjusted, and the screen to be displayed is determined based on the adjusted levels, so that the screen to be displayed is displayed accordingly in time following the user's operations.

When determining the levels for other objects to be displayed, the levels can be randomly assigned for the second maps that have no occlusion relationship. For example, there are two second maps A and B. If the terminal device is in the target posture and A and B do not occlude each other, the depth relationship between A and B can be ignored, and the layer numbers of A and B can be randomly assigned to 1 and 2, which will not affect the final display effect on the screen to be displayed.

The display method for the object to be displayed provided by the embodiment of the present disclosure can configure each first map as a preset level, and determine the level of each second map according to the positional relationship of all second maps when the terminal device is in a target position, so as to facilitate subsequent level fusion and improve processing efficiency.

FIG. 4 is a flowchart of step 50 provided by an exemplary embodiment of the present disclosure.

In some optional embodiments of the present disclosure, step 50 of determining the picture to be displayed based on at least one first map, at least one second map, and the second posture of the terminal device includes:

Step 510: Determine rendering results corresponding to each second texture based on at least one second texture and a second posture of the terminal device.

In some optional embodiments of the present disclosure, for each second map, the correspondence between each pixel of the to-be-displayed image and the pixel of the second map can be determined based on the second pose and the first pose, and based on this, the rendering result corresponding to the second map can be obtained. For details, please refer to the above content, which will not be repeated here.

Step 520 , determining a picture to be displayed based on each first texture, the level of each first texture, each rendering result and the level of each second texture.

In some optional embodiments of the present disclosure, the determination of the picture to be displayed can be implemented in any feasible manner. For example, sub-images of each level are superimposed layer by layer according to the level (for the target object to be displayed, the sub-image is the first map; for other objects to be displayed, the sub-image is the rendering result), and the picture to be displayed is cropped to obtain the picture to be displayed. For another example, the sub-images can be superimposed and cropped according to a certain number of levels to determine the picture to be displayed. All levels can also be superimposed and cropped together to determine the picture to be displayed. The specific setting can be based on actual needs.

The display method for the object to be displayed provided by the embodiment of the present disclosure renders the second map in combination with the second posture to obtain the rendering results corresponding to each second map, and then fuses the sub-images of each level to obtain the picture to be displayed, thereby achieving effective determination of the picture to be displayed and ensuring the stability of the target object to be displayed in the picture to be displayed.

In some optional embodiments of the present disclosure, determining the rendering results corresponding to the second maps respectively based on at least one second map and the second posture of the terminal device in step 510 includes:

For any second map, based on the second posture, the sampling coordinates of each pixel of the to-be-displayed picture in the second map are determined; based on the sampling coordinates of each pixel in the second map, the rendering result corresponding to the second map is determined.

In some optional embodiments of the present disclosure, the sampling coordinates of each pixel of the picture to be displayed in the second map of the other object to be displayed can be determined based on the conversion relationship between the pixel coordinates of the picture to be displayed and the pixel coordinates of the second map. The conversion relationship can be determined by the change in the second posture relative to the first posture.

The display method for an object to be displayed provided by the embodiment of the present disclosure determines the sampling coordinates of each pixel of the picture to be displayed in the second map based on the second posture. Based on the sampling coordinates of each pixel in the second map, when the picture to be displayed is subsequently determined, the pixel value corresponding to each pixel of the picture to be displayed after the time distortion of the second map can be determined, thereby providing an accurate rendering result for the determination of the picture to be displayed.

In some optional embodiments of the present disclosure, since the first map is obtained by rendering based on preset attributes of the target object to be displayed in the display area of the terminal device, that is, the first map has a corresponding correspondence with each pixel of the picture to be displayed, the sampling coordinates of each pixel of the picture to be displayed in the first map can be directly determined based on the first map.

FIG. 5 is a flowchart of step 520 provided by an exemplary embodiment of the present disclosure.

In some optional embodiments of the present disclosure, step 520 of determining the picture to be displayed based on each first map, the level of each first map, each rendering result and the level of each second map includes:

Step 5210, from among the levels whose processing status is unprocessed, determine the level with the greatest depth as the current level; wherein the initial processing status of all levels is unprocessed.

In some optional embodiments of the present disclosure, the processing status of all levels can be initialized to unprocessed. The level with the greatest depth is the level that may be blocked by other levels, so starting from the level with the greatest depth, the level with the greatest depth is used as the current level.

Step 5220, based on the sampling coordinates of each pixel of the screen to be displayed, merge the sub-image of the current level with the previous rendering result to obtain the current rendering result; wherein the previous rendering result is initially empty; the sub-image includes the following: a first map, a rendering result.

In some optional embodiments of the present disclosure, the previous rendering result refers to the rendering result obtained by fusion in the previous round of processing. The sub-image of the current level is fused with the previous rendering result, and the color value of each pixel of the sub-image of the current level is fused with the color value of each pixel of the previous rendering result on the basis of the previous rendering result. The color value may include the values and transparency of three color channels R (RED), G (GREEN), and B (BLUE). For the fusion of color values, the color value of the sub-image of the current level can be masked with the color value of the previous rendering result with corresponding transparency according to the transparency. For example, when the color value transparency of a pixel at the current level is 1, for the pixel, the fused result is the color value of the current level, based on which the current rendering result can be obtained. The transparency can be set to 1 by default, or it can be a value between 0 and 1, and can be set and adjusted as needed.

In some optional embodiments of the present disclosure, if the current level is the level corresponding to the first map, the sub-image of the current level is the first map; if the current level is the level corresponding to the second map, the sub-image of the current level is the rendering result corresponding to the second map.

In some optional embodiments of the present disclosure, the sub-image may be a result of cropping according to the picture to be displayed, or may be an uncropped result. For the uncropped result, cropping may be performed during fusion so that the current rendering result after fusion is a rendering result adapted to the picture to be displayed.

Step 5230, update the processing status of the current level to processed, and use the current rendering result to update the previous rendering result.

In some optional embodiments of the present disclosure, after obtaining the current rendering result, in order to avoid repeated processing, the processing status of the current level is updated to processed, and the current rendering result is used to update the previous rendering result.

In some optional embodiments of the present disclosure, the previous rendering result may be stored in a rendering buffer, and when the previous rendering result is updated using the current rendering result, the current rendering result may be used to replace the previous rendering result in the rendering buffer.

Step 5240, determine whether the processing status of all levels is processed.

In response to at least one level in all levels still having a processing status of unprocessed, step 5210 is repeatedly executed: determining, from among the levels in the processing status of unprocessed, the level with the greatest depth as the current level.

For the specific operation of this step, please refer to the above content and will not be repeated here.

In response to the processing status of all levels being processed, step 5250 is executed to use the previous rendering result as the picture to be displayed.

In some optional embodiments of the present disclosure, when the processing status of all levels is processed, it means that the fusion of sub-images of all levels is completed, and the final previous rendering result is used as the picture to be displayed, so that when the picture to be displayed needs to be displayed, the picture to be displayed can be displayed on the display screen of the terminal device.

The display method for the object to be displayed provided by the embodiment of the present disclosure realizes the determination of the picture to be displayed by fusing layer by layer in hierarchical order.

In some optional embodiments of the present disclosure, step 520 of determining the picture to be displayed based on each first map, the level of each first map, each rendering result and the level of each second map includes:

From the layers whose processing status is unprocessed, determine the layers with a preset number of the largest depths as the current target layers; wherein the initial processing status of all the layers is unprocessed; based on the sampling coordinates of each pixel of the picture to be displayed, merge the sub-images of each current target layer with the previous rendering result in the layer order to obtain the current rendering result; wherein the previous rendering result is initially empty; the sub-images include the following: a first map, a rendering result; update the processing status of each current target layer to processed, and use the current rendering result to update the previous rendering result; repeatedly execute the step of determining the layers with a preset number of the largest depths as the current target layers from the layers whose processing status is unprocessed; in response to the processing status of all the layers being processed, use the previous rendering result as the picture to be displayed.

In some optional embodiments of the present disclosure, the preset number can be set according to actual needs, for example, it can be set to 2 layers, 3 layers, and so on. For each sub-image of the current target level, the process of fusing with the previous rendering result in the hierarchical order can be layer-by-layer fusion or pixel-by-pixel fusion. The layer-by-layer fusion is similar to the aforementioned layer-by-layer fusion, and the current target level can be first fused in the hierarchical order to obtain a temporary fusion result, and then the temporary fusion result is fused with the previous rendering result to obtain the current rendering result. Pixel-by-pixel fusion means that for any pixel of the screen to be displayed, based on the sampling coordinates corresponding to the pixel in each current target level and the previous rendering result, the corresponding fused pixel value of the pixel is determined in the hierarchical order, which can be specifically set according to actual needs. The fusion of each pixel can be processed sequentially or in parallel, which can be specifically set according to actual needs.

The display method for the object to be displayed provided by the embodiment of the present disclosure can improve processing efficiency by performing hierarchical fusion by using a preset number of levels as the current target level to determine the picture to be displayed.

In some optional embodiments of the present disclosure, step 520 of determining the picture to be displayed based on each first map, the level of each first map, each rendering result and the level of each second map includes:

Based on the sampling coordinates of each pixel of the picture to be displayed, the sub-images of all levels are merged in a hierarchical order to obtain the picture to be displayed; the sub-image includes the following: a first map, a rendering result.

In some optional embodiments of the present disclosure, corresponding pixel values can be obtained from sub-images of each level based on the sampling coordinates of each pixel of the screen to be displayed, and the pixel values of all levels are fused in the order of the levels. During fusion, the sub-images can be cropped according to the screen to be displayed so that the fusion result can adapt to the screen to be displayed.

The display method for the object to be displayed provided by the embodiment of the present disclosure can fuse sub-images of all levels in a hierarchical order based on the sampling coordinates of each pixel of the picture to be displayed. The pixel value of each pixel of the picture to be displayed can be obtained by traversing each pixel of the picture to be displayed once, thereby improving the fusion efficiency.

In some optional embodiments of the present disclosure, step 50 of determining the picture to be displayed based on at least one first map, at least one second map, and the second posture of the terminal device includes:

For any pixel of the image to be displayed, the color value corresponding to the pixel is determined from each first map, and the color value corresponding to the pixel is determined in each second map based on the second posture, and the color values corresponding to the sampling coordinates of the pixel at each level are fused in the order of the levels to obtain the fused color value of the pixel.

In some optional embodiments of the present disclosure, after obtaining at least one first map corresponding to the target object to be displayed and at least one second map corresponding to other objects to be displayed, for any pixel of the screen to be displayed, the color value corresponding to the pixel can be directly determined from each first map, and the color value corresponding to the pixel can be determined in each second map based on the second posture, based on which the sampling coordinates corresponding to the pixel at each level can be obtained. The color values of the sampling coordinates corresponding to the pixel at each level are fused according to the hierarchical order to obtain the color value of the pixel. Based on this, the color values of all pixels of the screen to be displayed can be obtained, thereby determining the screen to be displayed. Since the step of rendering each second map based on the second posture to obtain the rendering result is omitted, and the pixel values of each pixel of the screen to be displayed can be obtained by traversing each pixel of the screen to be displayed once, the processing efficiency can be further improved.

In some optional embodiments of the present disclosure, the color value may include the values and transparency of three color channels: R (RED), G (GREEN), and B (BLUE).

The above-mentioned embodiments or optional examples of the present disclosure may be implemented separately, or may be implemented in combination in any manner without conflict. The specific configuration may be based on actual needs, and the present disclosure does not limit this.

Any display method for an object to be displayed provided in the embodiments of the present disclosure may be executed by any appropriate device with data processing capabilities, including but not limited to: a terminal device and a server, etc. Alternatively, any display method for an object to be displayed provided in the embodiments of the present disclosure may be executed by a processor, such as the processor executing any display method for an object to be displayed mentioned in the embodiments of the present disclosure by calling corresponding instructions stored in a memory. This will not be described in detail below.

Exemplary Devices

Fig. 6 is a schematic diagram of the structure of a display device for displaying an object provided by an exemplary embodiment of the present disclosure. The device of this embodiment can be used to implement the corresponding display method embodiment for displaying an object to be displayed of the present disclosure. The device shown in Fig. 6 includes: a first processing module 710, a second processing module 720, a third processing module 730 and a fourth processing module 740.

The first processing module 710 is used to determine at least one target to-be-displayed object belonging to a target display mode.

The second processing module 720 is used to render each target object to be displayed according to preset attributes to obtain at least one first map corresponding to each target object to be displayed; wherein the preset attributes include at least one of the position, angle, size, and scaling in the display area of the terminal device.

The third processing module 730 is used to render other objects to be displayed based on the first posture of the terminal device and the attribute information corresponding to other objects to be displayed except the target object to be displayed, and obtain at least one second map corresponding to other objects to be displayed; wherein the attribute information includes the position, angle, and scale of other objects to be displayed in the reference coordinate system.

The fourth processing module 740 is used to determine the picture to be displayed based on at least one first map, at least one second map, and a second posture of the terminal device; wherein the second posture is obtained later than the first posture.

FIG. 7 is a schematic structural diagram of a display device for displaying an object provided by another exemplary embodiment of the present disclosure.

In some optional embodiments of the present disclosure, the apparatus of the present disclosure further includes:

The fifth processing module 810 is used to determine the level of each first map and the level of each second map according to a preset rule.

In some optional embodiments of the present disclosure, the fourth processing module 740 is specifically used to:

Based on at least one second map and a second posture of the terminal device, rendering results corresponding to each second map are determined; based on each first map, the level of each first map, each rendering result and the level of each second map, a picture to be displayed is determined.

In some optional embodiments of the present disclosure, the fourth processing module 740 is specifically used to:

For any second map, based on the second posture, the sampling coordinates of each pixel of the to-be-displayed picture in the second map are determined; based on the sampling coordinates of each pixel in the second map, the rendering result corresponding to the second map is determined.

In some optional embodiments of the present disclosure, the fourth processing module 740 is specifically used to:

From the levels whose processing status is unprocessed, determine the level with the greatest depth as the current level; wherein the initial processing status of all levels is unprocessed. Based on the sampling coordinates of each pixel of the screen to be displayed, fuse the sub-image of the current level with the previous rendering result to obtain the current rendering result; wherein the previous rendering result is initially empty; the sub-image includes the following: a first map, a rendering result. Update the processing status of the current level to processed, and use the current rendering result to update the previous rendering result. Repeat the step of determining the level with the greatest depth as the current level from the levels whose processing status is unprocessed. In response to the processing status of all levels being processed, use the previous rendering result as the screen to be displayed.

In some optional embodiments of the present disclosure, the fourth processing module 740 is specifically used to:

From the levels whose processing status is unprocessed, determine the levels with the largest preset number of depths as the current target levels; wherein the initial processing status of all levels is unprocessed. Based on the sampling coordinates of each pixel of the picture to be displayed, merge the sub-images of each current target level with the previous rendering result in the order of the levels to obtain the current rendering result; wherein the previous rendering result is initially empty; the sub-images include the following: a first map, a rendering result. Update the processing status of each current target level to processed, and use the current rendering result to update the previous rendering result. Repeat the step of determining the levels with the largest preset number of depths as the current target level from the levels whose processing status is unprocessed. In response to the processing status of all levels being processed, use the previous rendering result as the picture to be displayed.

In some optional embodiments of the present disclosure, the fourth processing module 740 is specifically used to:

Based on the sampling coordinates of each pixel of the picture to be displayed, the sub-images of all levels are merged in a hierarchical order to obtain the picture to be displayed; the sub-image includes the following: a first map, a rendering result.

In some optional embodiments of the present disclosure, the fourth processing module 740 is specifically used to:

For any pixel of the image to be displayed, the color value corresponding to the pixel is determined from each first map, and the color value corresponding to the pixel is determined in each second map based on the second posture, and the color values corresponding to the sampling coordinates of the pixel at each level are fused in the order of the levels to obtain the fused color value of the pixel.

In some optional embodiments of the present disclosure, the fifth processing module 810 is specifically used to:

Each first map is configured as a preset level. Based on the positional relationship of all second maps in the terminal device in the target posture, the level of each second map is determined. The target posture includes any of the following: the first posture, the second posture. The positional relationship includes at least one of the following: depth, occlusion.

In some optional embodiments of the present disclosure, the first processing module 710 is specifically used to:

At least one target object to be displayed belonging to the target display mode is determined according to the types corresponding to the objects to be displayed and the preset type set; wherein the preset type set stores the types belonging to the target display mode.

In some optional embodiments of the present disclosure, the first processing module 710 is specifically used to:

In response to any to-be-displayed object carrying a target display mode identifier, it is determined that the to-be-displayed object belongs to the target display mode.

It should be noted that the specific implementation of the display device for the object to be displayed in the embodiment of the present disclosure is similar to the specific implementation of the display method for the object to be displayed in the embodiment of the present disclosure. Please refer to the part of the display method for the object to be displayed for details. In order to reduce redundancy, it will not be repeated here.

Exemplary Electronic Devices

An embodiment of the present disclosure further provides an electronic device, including: a processor, and a memory for storing instructions executable by the processor.

The processor is used to read the executable instructions from the memory and execute the executable instructions to implement the display method for the object to be displayed described in any of the above embodiments of the present disclosure.

Exemplary head mounted display device

The embodiment of the present disclosure also provides a head mounted display device, comprising: a processor, and a memory for storing instructions executable by the processor.

The processor is used to read the executable instructions from the memory and execute the executable instructions to implement the display method for the object to be displayed described in any of the above embodiments of the present disclosure.

Fig. 8 is a schematic diagram of the structure of an application embodiment of the electronic device disclosed in the present invention. The structure of the head mounted display device can refer to the structure of the electronic device. In this embodiment, the electronic device includes one or more processors 100 and a memory 200.

The processor 100 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

The memory 200 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and/or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 100 may run the program instructions to implement the methods of the various embodiments of the present disclosure described above and/or other desired functions. Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.

In one example, the electronic device may further include: an input device 300 and an output device 400 , and these components are interconnected via a bus system and/or other forms of connection mechanisms (not shown).

For example, the input device 300 may be the microphone or microphone array mentioned above, used to capture the input signal of the sound source.

In addition, the input device 300 may also include, for example, a keyboard, a mouse, and the like.

The output device 400 can output various information to the outside, including determined distance information, direction information, etc. The output device 400 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

Of course, for simplicity, only some of the components related to the present disclosure in the electronic device are shown in FIG8 , and components such as a bus, an input/output interface, etc. are omitted. In addition, the electronic device may further include any other appropriate components according to specific application scenarios.

Exemplary computer program products and computer-readable storage media

In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section of this specification.

The computer program product may be written in any combination of one or more programming languages to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present disclosure described in the above “Exemplary Method” section of this specification.

The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, rather than limitation, and the above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and/or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

The method and apparatus of the present disclosure may be implemented in many ways. For example, the method and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

It should also be noted that in the apparatus, device and method of the present disclosure, each component or each step can be decomposed and/or recombined. Such decomposition and/or recombination should be regarded as equivalent solutions of the present disclosure.

The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims (15)

1. A method for displaying an object to be displayed, comprising: Determining at least one target to-be-displayed object belonging to a target display mode; Rendering each of the target objects to be displayed according to preset attributes to obtain at least one first texture corresponding to each of the target objects to be displayed; wherein the preset attributes include at least one of a position, an angle, a size, and a scaling in a display area of a terminal device; Based on the first posture of the terminal device and the attribute information corresponding to the other objects to be displayed except the target object to be displayed, the other objects to be displayed are rendered to obtain at least one second map corresponding to the other objects to be displayed; wherein the attribute information includes the position, angle, and scale of the other objects to be displayed in the reference coordinate system; Based on the at least one first map, the at least one second map, and the second posture of the terminal device, a picture to be displayed is determined; wherein the second posture is obtained later than the first posture. 2. The method according to claim 1, further comprising: The level of each of the first maps and the level of each of the second maps are determined according to a preset rule. 3. The method according to claim 2, wherein the determining the picture to be displayed based on the at least one first map, the at least one second map, and the second posture of the terminal device comprises: Based on the at least one second map and the second posture of the terminal device, determining rendering results corresponding to each of the second maps; The picture to be displayed is determined based on each of the first maps, the level of each of the first maps, each of the rendering results, and the level of each of the second maps. 4. The method according to claim 3, wherein the determining, based on the at least one second map and the second posture of the terminal device, the rendering results corresponding to each of the second maps respectively comprises: For any second map, based on the second posture, determine the sampling coordinates of each pixel of the picture to be displayed in the second map; Based on the sampling coordinates of each pixel in the second map, a rendering result corresponding to the second map is determined. 5. The method according to claim 3, wherein the determining the to-be-displayed picture based on each of the first maps, the level of each of the first maps, each of the rendering results and the level of each of the second maps comprises: From the layers whose processing status is unprocessed, determine the layer with the largest depth as the current layer; wherein the initial processing status of all the layers is unprocessed; Based on the sampling coordinates of each pixel of the picture to be displayed, the sub-image of the current level is merged with the previous rendering result to obtain the current rendering result; wherein the previous rendering result is initially empty; the sub-image includes the following: a first map, a rendering result; Update the processing status of the current layer to processed, and use the current rendering result to update the previous rendering result; Repeat the step of determining the layer with the largest depth from the layers in the unprocessed state as the current layer; In response to the processing status of all levels being processed, the previous rendering result is used as the picture to be displayed. 6. The method according to claim 3, wherein the determining the to-be-displayed picture based on each of the first maps, the level of each of the first maps, each of the rendering results and the level of each of the second maps comprises: From the layers whose processing status is unprocessed, determine the layers with the largest preset number of depths as the current target layers; wherein the initial processing status of all the layers is unprocessed; Based on the sampling coordinates of each pixel of the picture to be displayed, the sub-images of each current target level are merged with the previous rendering result in the order of levels to obtain the current rendering result; wherein the previous rendering result is initially empty; the sub-image includes the following: a first map, a rendering result; Update the processing status of each current target layer to processed, and use the current rendering result to update the previous rendering result; Repeat the step of determining a preset number of levels with the largest depths from the levels in the unprocessed state as the current target levels; In response to the processing status of all levels being processed, the previous rendering result is used as the picture to be displayed. 7. The method according to claim 3, wherein the determining the to-be-displayed picture based on each of the first maps, the level of each of the first maps, each of the rendering results and the level of each of the second maps comprises: Based on the sampling coordinates of each pixel of the picture to be displayed, sub-images of all levels are merged in a hierarchical order to obtain the picture to be displayed; the sub-image includes the following: a first map, a rendering result. 8. The method according to claim 2, wherein the determining the picture to be displayed based on the at least one first map, the at least one second map, and the second posture of the terminal device comprises: For any pixel of the image to be displayed, the color value corresponding to the pixel is determined from each of the first maps, and the color value corresponding to the pixel is determined in each of the second maps based on the second posture, and the color values corresponding to the sampling coordinates of the pixel at each level are fused in the order of the levels to obtain the fused color value of the pixel. 9. The method according to claim 2, wherein determining the level of each of the first maps and the level of each of the second maps according to a preset rule comprises: configuring each of the first maps as a preset level; Based on the positional relationship of all the second maps when the terminal device is in a target posture, the level of each second map is determined; wherein the target posture includes any one of the following: the first posture, the second posture; the positional relationship includes at least one of the following: depth, occlusion. 10. The method according to claim 1, wherein the step of determining at least one target to-be-displayed object belonging to the target display mode comprises: At least one target object to be displayed belonging to the target display mode is determined according to the types corresponding to the objects to be displayed and the preset type set; wherein the preset type set stores the types belonging to the target display mode. 11. The method according to claim 1, wherein the step of determining at least one target to-be-displayed object belonging to the target display mode comprises: In response to any to-be-displayed object carrying a target display mode identifier, it is determined that the to-be-displayed object belongs to the target display mode. 12. A display device for an object to be displayed, comprising: A first processing module, used for determining at least one target to-be-displayed object belonging to a target display mode; A second processing module is used to render each of the target objects to be displayed according to preset attributes to obtain at least one first map corresponding to each of the target objects to be displayed; wherein the preset attributes include at least one of the position, angle, size, and scaling in the display area of the terminal device; A third processing module is used to render the other objects to be displayed based on the first posture of the terminal device and the attribute information corresponding to the other objects to be displayed except the target object to be displayed, so as to obtain at least one second map corresponding to the other objects to be displayed; wherein the attribute information includes the position, angle, and scale of the other objects to be displayed in the reference coordinate system; The fourth processing module is used to determine the picture to be displayed based on the at least one first map, the at least one second map, and the second posture of the terminal device; wherein the second posture is obtained later than the first posture. 13. A computer-readable storage medium storing a computer program, wherein the computer program is used to execute the display method for an object to be displayed according to any one of claims 1 to 11. 14. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the executable instructions to implement the display method for the object to be displayed as described in any one of claims 1-11 above. 15. A computer program product, comprising computer program instructions, which, when executed by a processor, implement the method for displaying an object to be displayed according to any one of claims 1 to 11.