CN115463412A - A display processing method and device for cloud games - Google Patents

Abstract

The embodiment of the application provides a display processing method and device of a cloud game, which are applied to the technical field of cloud computing and comprise the steps of respectively carrying out vertex processing on a first marked object and a second marked object at a frame time to obtain the position relation of the first marked object and the second marked object; performing rendering pipeline processing on the first marked object according to the position relation to obtain a first layer, and sending frame time information to a server; receiving a return picture sent by a server, and decoding the return picture to obtain a second picture layer; the return picture is obtained by rendering pipeline processing on the second marked object according to the position relation obtained by the server side according to the frame time information; and obtaining a frame time image according to the first image layer and the second image layer. By rendering the interactive content and the real-time content which are relatively concerned by the user at the client, the picture delay of the relatively concerned content of the user is effectively reduced, and the game experience of the user is improved.

Description

A display processing method and device for cloud games

technical field

The present invention relates to the field of cloud computing, in particular to a display processing method and device for cloud games.

Background technique

Cloud gaming is a gaming method based on cloud computing. In the operating mode of cloud gaming, all games run on the server side, and the rendered game screen is compressed and encoded and then transmitted to the user client through the network. In the cloud game mode, the user client needs to send an operation instruction to the cloud server, and the cloud server transmits the corresponding game screen to the client according to the operation instruction.

However, the rendering of cloud games is all done by the server side, which will cause excessive rendering pressure on the server. In addition, the game logic calculation time, audio and video rendering time, compression coding time, and network transmission time in the server will all cause delays in the game screen of the user client and reduce the user experience of the game.

Contents of the invention

Embodiments of the present application provide a cloud game display processing method and device, which are used to solve the delay problem of the client game screen and improve the user experience of the game.

In the first aspect, the embodiment of the present application provides a cloud game display processing method, which is applied to the client, including:

Perform vertex processing on the first marked object and the second marked object at the frame time respectively to obtain the positional relationship between the first marked object and the second marked object; the first marked object and the second marked object is the object to be displayed in the image at the frame time corresponding to the frame time;

According to the positional relationship, perform rendering pipeline processing on the first marker object to obtain a first layer, and send frame time information to the server;

receiving the returned image sent by the server, and decoding the returned image to obtain a second layer; the returned image is the positional relationship obtained by the server according to the frame time information, and the second Marked objects are obtained through rendering pipeline processing;

According to the first layer and the second layer, the image at the frame time is obtained.

By dividing the content of the game screen into two categories according to user needs, the interactive content and strong real-time content that users are more concerned about are rendered on the client side, which greatly reduces the rendering pressure on the server side, balances the load on the server side, and makes the server The online consumption is reduced, and more clients can be supported at the same time. At the same time, it avoids the slow transmission speed of the server and the network delay, which causes the screen delay of the content that the user pays attention to, and improves the user's game experience.

Optionally, the first marked object is interactive content determined based on the user's game operation on the client; the second marked object is background content other than the interactive content determined by the client's game operation.

The interactive content and real-time content that users are more concerned about are rendered on the client side, so that the delay of this part of the content is low, and the user's game experience is improved.

Optionally, the first marked object and the second marked object are obtained by marking objects defined in the cloud game through a software development kit.

Mark the objects defined in the cloud game through the software development kit, which is convenient for the client and server to identify.

Optionally, the frame time information includes camera information of the client at the frame time, operation event information corresponding to the user's game operation on the client, and game event information obtained based on the operation event information .

Optionally, according to the positional relationship, performing rendering pipeline processing on the first marker object to obtain a first layer, including:

determining the unoccluded portion of the first marked object according to the positional relationship;

Perform rendering pipeline processing on the unoccluded part of the first marked object to obtain the first layer.

Through the positional relationship, the client specifies the parts that need to be rendered, and completes the rendering pipeline processing for the unoccluded parts, but does not perform the complete rendering pipeline processing for the occluded parts.

Optionally, the client obtains the frame moment image according to the first layer and the second layer, including:

determining a superimposition mode between the first layer and the second layer according to a preset superposition mode between each first marked object and each second marked object;

According to the superimposition mode between the first layer and the second layer, the image at the frame time is obtained after superimposition processing.

By presetting the overlay mode between each first marker object and each second marker object, it is possible to enable the client to perform overlay processing according to the overlay mode of each marker object when superimposing layers, which can improve the game screen of the client quality and improve the user's visual experience.

In the second aspect, the embodiment of the present application provides a cloud game display processing method applied to the server, including:

Receive the frame time information under the frame time sent by the client;

According to the frame time information, perform vertex processing on the first marked object and the second marked object at the frame time respectively, to obtain the positional relationship between the first marked object and the second marked object; the first marked object The marked object and the second marked object are objects to be displayed in the image at the frame time corresponding to the frame time;

Perform rendering pipeline processing on the second marker object according to the positional relationship to obtain a returned image and send it to the client; the returned image is used by the client to generate the frame time image.

By dividing the content of the game screen into two categories according to user needs, the interactive content and strong real-time content that users are more concerned about are rendered on the client side, which greatly reduces the rendering pressure on the server side, balances the load on the server side, and makes the server The consumption on the network becomes smaller, and more clients can be supported at the same time.

Optionally, the frame time information includes camera information of the client at the frame time, operation event information corresponding to the user's game operation on the client, and game event information obtained based on the operation event information ;

According to the frame time information, performing vertex processing on the first mark object and the second mark object under the frame time respectively, including:

determining the camera information of the server at the frame time according to the camera information of the client at the frame time;

According to the operation event information and/or the game event information, determine the first marker object and the second marker object at the frame moment;

Perform vertex processing on the first marked object and the second marked object respectively according to the camera information of the server at the frame moment.

The camera information of the server and the client is kept unified, and the server can obtain a more accurate positional relationship between the first marked object and the second marked object according to the frame time information.

In the third aspect, the embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored on the memory and operable on the processor, so that the processor executes the cloud program described in the first aspect above. The display handling method for the game.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program executable by a computer device, and when the program runs on the computer device, the computer device executes the above-mentioned first aspect The display processing method of the cloud game.

Render the interactive content and real-time content that users pay more attention to on the client side, which reduces the rendering pressure on the server side, balances the load on the server side, reduces the consumption on the server, and can support more clients at the same time. At the same time, it avoids the slow transmission speed of the server and the delay of the network, which causes the screen delay of the content that the user pays attention to, and improves the user's game experience.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings that need to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For Those of ordinary skill in the art can also obtain other drawings based on these drawings without any creative effort.

FIG. 1 is a schematic diagram of a cloud game system architecture provided by an embodiment of the present invention;

FIG. 2 is a schematic flowchart of a display processing method for a cloud game provided by an embodiment of the present invention;

FIG. 3A is a schematic diagram of a game viewing angle provided by an embodiment of the present invention;

FIG. 3B is a schematic diagram of a game viewing angle provided by an embodiment of the present invention;

4 is a schematic flowchart of a cloud game display processing method provided by an embodiment of the present invention;

FIG. 5A is a schematic diagram of a game viewing angle provided by an embodiment of the present invention;

FIG. 5B is a schematic diagram of a game viewing angle provided by an embodiment of the present invention;

Fig. 6 is a schematic diagram of a cloud game device provided by an embodiment of the present invention;

FIG. 7 is a schematic structural diagram of a computing device provided by an embodiment of the present invention.

detailed description

In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some of the embodiments of the present invention, rather than all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

Referring to FIG. 1 , it is a diagram of a cloud game system architecture applicable to the embodiment of the present application. The system architecture may include a client 101 and a server 102 . The client 101 and the server 102 are connected through a wired network or a wireless network. The client 101 may be a computer, a mobile phone, or a tablet, and the server 102 may be a high-performance ARM cluster server or a processor of a Complex Instruction Set Computer (CISC) architecture.

Based on the above description, FIG. 2 exemplarily shows the flow of a cloud game display processing method. The flow of the method is executed by the client 101, including the following steps:

Step S201, perform vertex processing on the first marker object and the second marker object at the frame time respectively, to obtain the positional relationship between the first marker object and the second marker object; the first marker object and the second marker object are corresponding to the frame time The object to be displayed in the image at the frame time.

Specifically, the first tagged object may be game content that has a small rendering workload, is strongly associated with user game operations, and has high timeliness requirements for users, and the second tagged object may be game content that has a large rendering workload, is weakly associated with user game operations, Users do not need high real-time game content.

In some embodiments, the first marked object and the second marked object are obtained by marking objects defined in the cloud game through a software development kit. The first marked object is the interactive content determined based on the user's game operation on the client; the second marked object is the background content other than the interactive content determined by the client's game operation.

Specifically, in a cloud game, it contains multiple first objects and multiple second objects, where the game developer defines the first object as the interactive content determined by the user during the game operation, or it can be the game that the user is more concerned about. The content includes the character to be operated, the enemy who is close to the character, and the user interface (User Interface, referred to as UI) related to the character. Therefore, the first object has strong real-time performance; the developer defines the second object as Background content other than the interactive content determined by the user during the game operation, including buildings, ground, vegetation, trees, etc. in the game scene. The developer uses a software development kit (Software Development Kit, referred to as SDK for short) to mark the first object and the second object to obtain the first marked object and the second marked object respectively.

For example, in a multiplayer combat competitive game, the first marked object can be a character operated by the player, an enemy at close range, and a user interface related to the character. These user interfaces include the following: map UI, energy bar UI, and team UI , skill UI, etc.; the second marking objects are houses, mountains, land, etc. in the game screen.

Taking a first marker object and a second marker object in a frame as an example, the client will use a simple model to perform vertex processing on the first marker object and the second marker object to obtain the first marker object and the second marker object The positional relationship between the two marked objects is to correctly handle the influence of the second marked object on the first marked object.

The possible positional relationship between the first marked object and the second marked object is that the second marked object completely covers the first marked object, the second marked object partially covers the first marked object, and the second marked object overlaps with the first marked object. collision events etc.

For example, if the first marked object is a character operated by the user, and the second marked object is a pillar, at frame n, when the character runs behind the pillar, the pillar completely blocks the character, that is, the second marked object completely block the first marked object;

If the first marking object is a character operated by the user, and the second marking object is a stone platform, at frame n+1, when the character runs behind the stone platform, the stone platform only covers the character's legs, and the character's The upper body and head are not blocked by the stone platform, that is, the second marked object partially covers the first marked object;

If the first mark object is the machete held by the character operated by the user, and the second mark object is a wooden barrel, at frame n+2, the character performs a chopping action on the barrel, the machete falls into the barrel, and the blade of the machete Collision with the barrel, that is, a collision event occurs between the second marked object and the first marked object.

The client uses a simple model to process the vertices of the first marker object and the second marker object, which can correctly handle the occlusion and collision events that appear in the game. Steps to prepare. Among them, simple models can be line graphs, point-and-line graphs, etc.

Step S202, according to the positional relationship, perform rendering pipeline processing on the first marker object to obtain the first layer, and send the frame time information to the server.

Specifically, rendering pipeline processing is a process of converting a 3D scene into a 2D image on the screen. The complete rendering pipeline process is: vertex data input, vertex shader, tessellation process, geometry shader, primitive assembly, clipping and culling, rasterization, fragment shader, blending test. Therefore, based on the result of vertex processing in step S201, the client performs rendering pipeline processing after vertex data input for all first marked objects, including vertex shader, surface subdivision process, geometry shader, primitive assembly, clipping and culling, Rasterization, fragment shader, blend testing, and finally the first layer.

In the above step S202, it also includes: determining the unoccluded part of the first marker object according to the positional relationship; performing rendering pipeline processing on the unoccluded part of the first marker object to obtain the first layer.

For example, if the first marker object is a character operated by the user, and the second marker object is a pillar, if the character runs behind the pillar at the nth frame, the pillar completely blocks the character, that is, the second marker object Completely cover the first marker object, at this time, the client does not need to perform rendering pipeline processing after vertex data input for the character;

If the first marking object is a character operated by the user, the second marking object is a stone platform. If the character runs behind the stone platform at frame n+1, the stone platform only covers the legs of the character. The upper body and head of the character are not blocked by the stone platform, that is, the second marker object partially covers the first marker object; at this time, the client needs to perform rendering pipeline processing after vertex data input for the part of the character that is not blocked by the stone platform;

If the first mark object is the machete held by the character operated by the user, and the second mark object is a wooden barrel, at frame n+2, the character performs a chopping action on the barrel, the machete falls into the barrel, and the blade of the machete When it collides with the barrel, the machete part of the first mark object is covered by the barrel of the second mark object. At this time, the client performs the rendering pipeline processing after the vertex data input for the part not covered by the machete.

After the client completes the rendering pipeline processing on the first marked object, it sends the frame time information to the server.

By dividing the content of the game screen into two categories according to user needs, the client is allowed to undertake the rendering work of part of the content in the cloud game screen, which greatly reduces the rendering pressure on the server; Rendering on the client side effectively reduces the screen delay of content that users are more concerned about, and improves the user's gaming experience.

Step S203, receiving the returned picture sent by the server, and decoding the returned picture to obtain the second layer; the returned picture is obtained by the server according to the positional relationship obtained by the frame time information, and rendering pipeline processing on the second marker object.

Specifically, the server receives the frame time information sent by the client, and the server uses a simple model to perform vertex processing on the first marked object and the second marked object to obtain the positional relationship between the first marked object and the second marked object, and the service The terminal performs rendering pipeline processing after inputting vertex data on the second marker object according to the positional relationship, to obtain a returned image. The server encodes the returned picture and sends it to the client, and the client receives the encoded returned picture and decodes it to obtain the second layer.

Wherein, the frame time information includes the camera information of the client at the frame time, the operation event information corresponding to the user's game operation on the client, and the game event information obtained based on the operation event information.

The camera information is the angle information of the imaging camera of the client, the camera coordinate system, and the like.

The operation event information is the coordinate information of the user's touch point and the data information generated by the touch generated after the user clicks a certain place on the interface (game screen) of the client.

The game event information is the data information generated by the user manipulating the game character on the client terminal, or the data information generated after the user manipulates the character character to interact with the surrounding scene, non-player character (Non Player Character, NPC for short), etc.

For example, if the user clicks on a gem that falls on the game screen, the player character will bend down to pick it up; where the user clicks is a game operation, and the coordinate information of the clicked position is the operation event information, based on the game operation and operation event information, The bending action of the player character, the pick-up action of the player character, the number of gems and the total value of the gems are the game event information.

If the user continuously clicks the attack button on the game screen, the player character will draw the gun and shoot the enemy, and the enemy's HP will decrease; among them, the user continuously clicks the attack button is the game operation, and the coordinate information of the attack button and the coordinate information of the attack object are the operation Event information, based on the game operation and operation event information, the player's attack caused damage to the enemy and the enemy's remaining HP are the game event information.

The frame time information changes with different operations of the user, and also affects the positional relationship between the first marked object and the second marked object, so the positional relationship between the first marked object and the second marked object corresponding to different frame time information Also different.

As shown in FIG. 3A, the first marking object is a character 301 operated by the user, and the second marking object is a pillar 302. The user uses a third-person perspective. At frame n, the camera is located behind the character 301, and the pillar 302 is located at Right in front of the character 301, at this time, the second marking target column 302 is partly covered by the first marking target character 301, the server needs to input the vertex data for the part of the column 302 that is not covered by the character 301 Rendering pipeline processing;

If at frame n+1, as shown in FIG. 3B , the user slides the game viewing angle and rotates the camera viewing angle 90 degrees to the left, the camera is located on the right side of the character 301, that is, at this time, the second marking object column 302 is not blocked. The first marking object character 301 is blocked, and the server performs rendering pipeline processing after inputting vertex data to the pillar 302 .

Step S204, according to the first layer and the second layer, the image at the frame time is obtained.

The client renders the first marker object to obtain the first layer. The client receives the return image sent by the server and decodes it to the second layer. The client combines the first layer and the second layer. Overlay (Overlay) to obtain the game image at the frame time.

By dividing the content of the game screen into two categories according to user needs, the client is allowed to undertake the rendering work of part of the content in the cloud game screen, which greatly reduces the rendering pressure on the server; Rendering on the client side avoids the slow transmission speed of the server side and network delay caused by the screen delay of the content that the user cares about, and improves the user's game experience.

In the above step S204, the client obtains the image at the frame time according to the first layer and the second layer, including: determining the first The overlay mode between the layer and the second layer; according to the overlay mode between the first layer and the second layer, the image at the frame time is obtained after overlay processing.

In some embodiments, the effect of directly overlaying the first layer and the second layer (Overlay) is not ideal, so the optimal overlay mode of each first marked object and each second marked object is preset, and the customer The terminal performs marking to determine the superimposition mode of the first layer and the second layer, and the client superimposes the first layer and the second layer according to the determined superimposition mode to obtain the frame time image.

For example, in a cloud game, the developer uses a software development kit to mark the shadow of the game character as the first mark object, and mark the ground as the second mark object. The best overlay mode for the shadow of the game character and the ground is Alpha aliasing. The developer uses the software development kit to mark the overlay mode of the shadow of the game character and the ground. The client performs complete rendering pipeline processing on all first marked objects to obtain the first layer, the client marks the pixel position of the shadow of the game character in the first layer, and the server completes all second marked objects The second layer is obtained after processing by the rendering pipeline. When the client superimposes the first layer where the first marker object is located and the second layer where the second marker object is located, it will perform an Alpha aliasing operation on the pixel where the shadow of the game character is located and the ground, and the effect is better than that of the first layer The layer and the second layer are directly superimposed. By performing an Alpha aliasing operation on the pixel where the shadow of the game character is located and the ground, the material of the ground can be seen clearly after superimposition.

Mark the overlay mode between each first marker object and each second marker object through the software development kit, so that the client can perform overlay processing according to the overlay mode marked by each marker object when performing layer overlay , can improve the game picture quality of the client, and improve the user's visual experience.

Figure 4 exemplarily shows the flow of a cloud game display processing method, the flow of the method is executed by the server, including the following steps:

Step S401, receiving frame time information at the frame time sent by the client.

The server receives the frame time information sent by the client. The frame time information includes the camera information of the client at the frame time, the operation event information corresponding to the user's game operation on the client, and the game event information obtained based on the operation event information.

The camera information is the angle information of the imaging camera of the client, the camera coordinate system, and the like.

The operation event information is the coordinate information of the user's touch point and the data information generated by the touch generated after the user clicks a certain place on the interface (game screen) of the client.

The game event information is the data information generated by the user manipulating the game character on the client terminal, or the data information generated after the user manipulates the character character to interact with the surrounding scene, non-player character (Non Player Character, NPC for short), etc.

Step S402, according to the frame time information, perform vertex processing on the first marked object and the second marked object at the frame time respectively, to obtain the positional relationship between the first marked object and the second marked object; the first marked object and the second marked object is the object to be displayed in the image at the frame time corresponding to the frame time.

In the above step S402, according to the frame time information, respectively perform vertex processing on the first mark object and the second mark object at the frame time, including: according to the camera information of the client at the frame time, determine that the server is at the frame time camera information; according to the operation event information and/or game event information, determine the first marker object and the second marker object at the frame time; according to the server’s camera information at the frame time, determine the first marker object and the second marker object Objects are vertex processed separately.

Specifically, the camera information of the server and the client must be consistent, that is, the angle information of the camera, the camera coordinate system, etc., in order to ensure that when the second layer generated by the server and the first layer generated by the client are superimposed, the generated There will be no problem with frame moment images. Therefore, according to the camera information of the client in frame n, the camera information of the server in frame n is determined.

The frame time information changes with different operations of the user, and also affects the positional relationship between the first marked object and the second marked object, so the positional relationship between the first marked object and the second marked object corresponding to different frame time information It is also different. As shown in FIGS. 3A and 3B , according to the camera information of the server at the frame time, a simple model is used to perform vertex processing on the first marker object and the second marker object.

Step S403, according to the positional relationship, perform rendering pipeline processing on the second marker object to obtain a returned image and send it to the client; the returned image is used by the client to generate a frame time image.

Specifically, according to the positional relationship between the first marker object and the second marker object obtained by the vertex processing in step S402, the server performs a complete rendering pipeline processing on the second marker object to obtain a returned image, and the server performs the returned image Compress the code and send it to the client.

By dividing the content of the game screen into two categories according to user needs, let the client undertake the rendering of part of the content in the cloud game screen, effectively utilize the performance redundancy of the client, reduce the rendering pressure on the server, and balance the load on the server. This reduces the consumption on the server and supports more clients at the same time.

A variety of cloud games can realize the presentation of the game screen through the technical solutions in the embodiments of this application. The following uses cloud game C as an example to illustrate. The client of this cloud game C is a mobile phone, and the server is a server.

Cloud Game C is an adventurous level-breaking mobile game. Players need to operate characters to move, and to level up by fighting monsters and doing tasks.

In the development stage of cloud game C, the developer uses the software development kit (Software Development Kit, referred to as SDK) to first mark the characters, enemy characters and user interface (User Interface, referred to as UI) controlled by the client, and obtain Several first marking objects; performing second marking on houses, grounds, trees, non-player characters (Non Player Character, NPC for short) etc. in the game scene to obtain several second marking objects.

Taking a certain frame in cloud game C as an example, the mobile phone performs simple modeling and vertex processing on several first marked objects and several second marked objects, so as to correctly handle the second marked object blocking the first marked object. question. The mobile phone performs rendering pipeline processing on each first marked object according to the positional relationship between each first marked object and each second marked object, and generates layer 1 on the mobile phone, and at the same time, the camera information, Operation event information and game event information are sent to the server.

For example, as shown in FIG. 5A , one of the first marking objects is a character 501 controlled by a player, and one of the second marking objects is a wall 502 . At this time, the character 501 stands in front of the wall 502 , and the character 501 covers part of the wall 502 , and the mobile phone terminal will perform rendering pipeline processing on the complete character 501 .

If at a certain frame time, the character 501 stands behind the wall 502, and the wall 502 covers part of the area of the character 501, as shown in Figure 5B, the mobile phone does not render the part of the character 501 covered by the wall 502 Pipeline processing, rendering pipeline processing is performed on the part of the wall 502 that does not cover the character 501 .

The server receives the camera information, operation event information and game event information sent by the mobile phone, and performs simple modeling and vertex processing on several first marked objects and several second marked objects respectively, so as to correctly process the first marked objects The problem of occluding the second marker object; the server performs rendering pipeline processing on each second marker object according to the positional relationship between each first marker object and each second marker object, encodes the rendered picture, and obtains the return screen, and send the returned screen to the mobile phone.

For example, as shown in FIG. 5A , the first marking object is a character 501 controlled by a player, and the second marking object is a wall 502 . At this time, the character 501 stands in front of the wall 502, and the character 501 covers part of the wall 502, and the server will perform a complete rendering pipeline processing on the wall 502 that is not covered by the character 501, and will not be blocked by the character 501. Complete rendering pipeline processing for the walls.

If at a certain frame time, the character 501 stands behind the wall 502 and the wall 502 covers part of the character 502 , as shown in FIG. 5B , the server will perform a complete rendering pipeline process on the entire wall 502 .

The mobile terminal receives and decodes the returned screen sent by the server to obtain layer 2, and the mobile terminal superimposes layer 1 and layer 2, according to the overlay mode of each first marked object and each second marked object Superimpose in the superimposition mode, and finally get the actual game screen of the frame.

By dividing the content of the game screen into two categories according to user needs, the mobile phone end can undertake the rendering work of part of the content in the cloud game screen, reducing the rendering pressure on the server end; The content is rendered on the mobile phone, which avoids the problem of image delay caused by slow server transmission speed and network delay, and also avoids the problem of blurred content caused by video encoding, which improves the image quality of cloud games and improves the game experience of users. sense of experience.

Based on the same technical concept, the embodiment of the present application provides a schematic structural diagram of a cloud game device, as shown in Figure 6, the cloud game device 600 includes:

The processing module 601 performs vertex processing on the first marked object and the second marked object at the frame time respectively, to obtain the positional relationship between the first marked object and the second marked object; the first marked object and the The second marked object is an object to be displayed in the frame time image corresponding to the frame time;

The processing module 601 is further configured to perform rendering pipeline processing on the first marker object according to the positional relationship to obtain a first layer, and send frame time information to the server;

The obtaining module 602 is configured to receive the returned picture sent by the server, and decode the returned picture to obtain the second layer; the returned picture is the positional relationship obtained by the server according to the frame time information , obtained by performing rendering pipeline processing on the second marker object;

The processing module 601 is configured to obtain the frame time image according to the first layer and the second layer.

Optionally, the processing module 601 is further configured to:

determining the unoccluded portion of the first marked object according to the positional relationship;

Perform rendering pipeline processing on the unoccluded part of the first marked object to obtain the first layer.

Optionally, the processing module 601 is further configured to:

determining a superimposition mode between the first layer and the second layer according to a preset superposition mode between each first marked object and each second marked object;

According to the superimposition mode between the first layer and the second layer, the image at the frame time is obtained after superimposition processing.

Optionally, the obtaining module 602 is also used for:

Receive the frame time information under the frame time sent by the client;

According to the frame time information, perform vertex processing on the first marked object and the second marked object at the frame time respectively, to obtain the positional relationship between the first marked object and the second marked object; the first marked object The marked object and the second marked object are objects to be displayed in the image at the frame time corresponding to the frame time;

Perform rendering pipeline processing on the second marker object according to the positional relationship to obtain a returned image and send it to the client; the returned image is used by the client to generate the frame time image.

Optionally, the processing module 601 is further configured to:

determining the camera information of the server at the frame time according to the camera information of the client at the frame time;

According to the operation event information and/or the game event information, determine the first marker object and the second marker object at the frame moment;

Perform vertex processing on the first marked object and the second marked object respectively according to the camera information of the server at the frame moment.

By dividing the content of the game screen into two categories according to user needs, the client is allowed to undertake the rendering work of part of the content in the cloud game screen, which greatly reduces the rendering pressure on the server; Rendering on the client side avoids the problem of screen delay caused by slow transmission speed on the server side and network delay, and improves the user's game experience.

Based on the same technical concept, an embodiment of the present application provides a computer device, which may be the client 101 and/or server 102 shown in FIG. 1 , as shown in FIG. 7 , including at least one processor 701, As well as the memory 702 connected to at least one processor, the specific connection medium between the processor 701 and the memory 702 is not limited in the embodiment of the present application, and the bus connection between the processor 701 and the memory 702 in FIG. 7 is taken as an example. The bus can be divided into address bus, data bus, control bus and so on.

In the embodiment of the present application, the memory 702 stores instructions executable by at least one processor 701, and at least one processor 701 can execute the steps of the above cloud game display processing method by executing the instructions stored in the memory 702.

Among them, the processor 701 is the control center of the computer equipment, and various interfaces and lines can be used to connect various parts of the computer equipment, and by running or executing instructions stored in the memory 702 and calling data stored in the memory 702, the frame Display of time images. Optionally, the processor 701 may include one or more processing units, and the processor 701 may integrate an application processor and a modem processor. The tuner processor mainly handles wireless communication. It can be understood that the foregoing modem processor may not be integrated into the processor 701 . In some embodiments, the processor 701 and the memory 702 can be implemented on the same chip, and in some embodiments, they can also be implemented on independent chips.

The processor 701 may be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array or other programmable logic devices, discrete gates or transistors Logic devices and discrete hardware components can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of the methods disclosed in connection with the embodiments of the present application may be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor.

The memory 702, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs and modules. The memory 702 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (Random Access Memory, RAM), static random access memory (Static Random Access Memory, SRAM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Magnetic Memory, Disk, discs and more. Memory 702 is, but is not limited to, any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer device. The memory 702 in the embodiment of the present application may also be a circuit or any other device capable of implementing a storage function, and is used for storing program instructions and/or data.

Based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program executable by a computer device, and when the program is run on the computer device, the computer device executes the above cloud game display processing method A step of.

Those skilled in the art should understand that the embodiments of the present application may be provided as methods, systems, or computer program products. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the present application. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a An apparatus for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions The device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby The instructions provide steps for implementing the functions specified in the flow diagram procedure or procedures and/or block diagram procedures or blocks.

Obviously, those skilled in the art can make various changes and modifications to the application without departing from the spirit and scope of the application. In this way, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these modifications and variations.

Claims (10)

1. A display processing method for a cloud game, applied to a client, characterized in that it comprises: Perform vertex processing on the first marked object and the second marked object at the frame time respectively to obtain the positional relationship between the first marked object and the second marked object; the first marked object and the second marked object is the object to be displayed in the image at the frame time corresponding to the frame time; According to the positional relationship, perform rendering pipeline processing on the first marker object to obtain a first layer, and send frame time information to the server; receiving the returned image sent by the server, and decoding the returned image to obtain a second layer; the returned image is the positional relationship obtained by the server according to the frame time information, and the second Marked objects are obtained through rendering pipeline processing; According to the first layer and the second layer, the image at the frame time is obtained. 2. The method according to claim 1, wherein the first marking object is an interactive content determined based on the user's game operation on the client side; the second marking object is the game operation other than the client side Background content outside of identified interactive content. 3. The method according to claim 1, wherein the first marked object and the second marked object are obtained by marking objects defined in the cloud game through a software development kit. 4. The method according to claim 1, wherein the frame time information includes camera information of the client at the frame time, operation event information corresponding to the user's game operation on the client, and Game event information obtained based on the operation event information. 5. The method according to claim 1, wherein, according to the positional relationship, performing rendering pipeline processing on the first marker object to obtain the first layer, comprising: determining the unoccluded portion of the first marked object according to the positional relationship; Perform rendering pipeline processing on the unoccluded part of the first marked object to obtain the first layer. 6. The method according to any one of claims 1 to 5, wherein the client obtains the frame moment image according to the first layer and the second layer, comprising: determining a superimposition mode between the first layer and the second layer according to a preset superposition mode between each first marked object and each second marked object; According to the superimposition mode between the first layer and the second layer, the image at the frame time is obtained after superimposition processing. 7. A display processing method for a cloud game, applied to a server, characterized in that it comprises: Receive the frame time information under the frame time sent by the client; According to the frame time information, perform vertex processing on the first marked object and the second marked object at the frame time respectively, to obtain the positional relationship between the first marked object and the second marked object; the first marked object The marked object and the second marked object are objects to be displayed in the image at the frame time corresponding to the frame time; Perform rendering pipeline processing on the second marker object according to the positional relationship to obtain a returned image and send it to the client; the returned image is used by the client to generate the frame time image. 8. The method according to claim 7, wherein the frame time information includes camera information of the client at the frame time, operation event information corresponding to the user's game operation on the client, and Game event information obtained based on the operation event information; According to the frame time information, performing vertex processing on the first mark object and the second mark object under the frame time respectively, including: determining the camera information of the server at the frame time according to the camera information of the client at the frame time; According to the operation event information and/or the game event information, determine the first marker object and the second marker object at the frame moment; Perform vertex processing on the first marked object and the second marked object respectively according to the camera information of the server at the frame moment. 9. A computer device, comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, characterized in that, when the processor executes the program, any one of claims 1 to 8 is realized The steps of the method are claimed. 10. A computer-readable storage medium, characterized in that it stores a computer program executable by a computer device, and when the program is run on the computer device, the computer device executes any one of claims 1-8. steps of the method described above.

Images