CN117274030A - A mobile-side Vulkan rendering process optimization method - Google Patents

Abstract

The invention discloses a mobile terminal Vulkan rendering process optimization method, which is designed based on the characteristics of the Vulkan rendering API and the characteristics of the mobile terminal GPU. Through a series of methods, reduce the number of times the rendering pass is enabled and try to discard unnecessary frame buffers, thereby reducing the read and write overhead of the frame buffer when using the rendering pass. The purpose of the present invention is to reduce the data bandwidth required when the mobile terminal performs drawing, and designs a series of methods of encapsulating the graphics API using Vulkan to optimize the performance of the mobile terminal without increasing the complexity of upper layer calls.

Description

A mobile-side Vulkan rendering process optimization method

Technical field

The invention belongs to the field of rendering in computer graphics, and in particular relates to a mobile terminal Vulkan rendering process optimization method.

Background technique

The IMR (Immediate Mode Rendering) process applied on the PC side requires reading and writing the frame buffer and depth buffer once for each triangle during rendering, so such an architecture requires a lot of bandwidth or a large cache. The effect of large bandwidth is high power consumption, high heat dissipation requirements, and the mobile terminal is limited by size requirements and cannot design a large cache, so IMR obviously cannot be applied on the mobile terminal. Therefore, for the mobile terminal, people invented the TBR (Tile-BasedRendering) architecture, which aims to reduce the external memory access required by the GPU during shading. The logic of TBR is to divide the screen into small blocks (tiles). For example, 16x16 or 32x32 pixels can be used as a tile. Allocate graphics elements to each small block for calculation. Each small block has a separate cache. After all calculations are completed locally, the frame buffer of the small block is written back to the main memory. After all tiles have completed their work, You get the final frame buffer. Therefore, it is not difficult to see that when implementing mobile rendering, it is necessary to minimize the read and write operations between the GPU and external memory in the entire rendering process.

Frame Graph is a new rendering framework. It is a technology that completes optimization operations by obtaining all the information in a complete frame and analyzing the dependencies between each rendering channel node.

Contents of the invention

In view of the shortcomings of the existing technology, the present invention provides a method for optimizing the Vulkan drawing process on the mobile terminal. This method automatically merges compatible Render Passes into one by reducing the number of rendering passes (Render Pass) in one frame; For the same render target, reduce the operation of writing it back to the video memory and then reading it into Tile again by Render Pass. Then it automatically determines whether each texture corresponding to the rendering target of the Render Pass needs to save the calculation results, and whether it needs to read the previous existing results; automatically sets unnecessary access textures to not access to save bandwidth overhead. Finally, for the rendering target that needs to turn on MSAA, it is automatically determined whether its texture in the Multiple Sample state is needed as the rendering target in the future, otherwise it is specified to resolve to the texture of the Single Sample in place at the end of the Render Pass.

This eliminates the multiple storage overhead caused by MSAA and optimizes the Vulkan drawing process on the mobile terminal to cause low performance problems caused by the overhead of reading and writing the frame buffer (Frame Buffer).

The main technical solutions of the present invention are as follows:

A mobile-side Vulkan drawing process optimization method encapsulates the VkRenderPass-related calls provided by Vulkan, and optimizes its reading and writing frame buffer operations in the encapsulated function body. The method includes the following steps:

(1) Encapsulate the three function interfaces of the BeginRenderPass function, the Draw function, and the EndRenderPass function;

The start rendering pass function BeginRenderPass obtains the Vulkan rendering pass VkRenderPass and the frame buffer object VkFrameBuffer by passing in a set of rendering targets RenderTarget and the bound depth map, and saves them. Check whether the objects saved when BeginRenderPass was called last time. Compatible with the rendering channel and framebuffer obtained by the current call;

If it is not compatible, call the corresponding vkCmdEndRenderPass function provided by Vulkan to terminate the last enabled rendering pass, and save the new object for actual use;

If compatible, there is no need to terminate the enabled rendering pass, and the rendering passes will be automatically merged;

After the start rendering pass function BeginRenderPass obtains and saves the rendering pass and the frame buffer, the drawing function Draw is used to determine whether it is necessary to call Vulkan's API to start a new rendering pass and start; the function interface of the end rendering pass function EndRenderPass It is used to check the semantic clarity when the upper layer calls this set of functions, and does not execute the termination rendering pass function provided by Vulkan;

(2) The premise for calling the start rendering pass function in step (1) to perform compatibility determination to form a merger is that before calling the start rendering pass function, the homogeneous rendering pass objects enabled by the last drawing are not affected by other circumstances. Termination; specifically:

If the frame buffer objects written by a group of rendering passes are the same, they can be merged. If each rendering pass requires different resources when performing drawing, the resources need to be updated before the rendering pass starts;

Therefore, in a group of homogeneous rendering passes, in order to prevent the update operation of the rendering pass executed later from interrupting the enabled homogeneous rendering pass, the update operation corresponding to the resources required by the rendering pass executed later is advanced. Frame Graph needs to be used to achieve this goal before entering this group of rendering passes; (3) When determining the dependencies between rendering passes connected by resources in step (2), for resources that do not have dependencies, they can be passed at the end of the rendering pass Discard resources to prevent redundant writeback operations; specifically, use Frame Graph to abstract each rendering pass into a rendering node, and treat the merged rendering passes in step (1) as the same rendering node; for each rendering node, Use Frame Graph to determine whether each rendering target in its bound frame buffer is dependent on subsequent nodes or depends on previous nodes; for rendering targets that do not have corresponding dependencies, set the corresponding flags when creating the rendering pass. bit to avoid redundant read and write operations.

(4) For the rendering pass that turns on multi-sampling anti-aliasing MSAA, it needs to be parsed and converted into a normal texture without multi-sampling turned on, so as to produce the final required result, and the multi-sampling target is written back at the end of the rendering pass. The cost of the operation is too high, so when determining the rendering target in step (3), special processing is required to avoid redundant reading and writing; specifically, when creating a rendering pass, specify the corresponding multi-sampling target. The parsed rendering target means that an additional rendering target is bound to the rendering pass, so that the parsing operation can be completed in place at the end of the rendering pass, thereby discarding the multi-sampling target and only writing the parsed target back to the video memory.

Further, the step (1) includes the following sub-steps:

(1.1) Maintain the cache of rendering channels and frame buffers. Each time the start rendering channel function is called, the corresponding rendering channels and frame buffers are searched for in the cache based on the rendering target and depth map in the passed parameters. If the search fails, a generated New render passes and framebuffers and saving them in their respective caches;

(1.2) In the start rendering pass function, the compatibility of the rendering pass is determined. At this time, the keywords for finding and creating rendering passes and frame buffers are only related to the parameters of the rendering target and depth template map, and have nothing to do with the operation of the rendering pass itself;

(1.3) In the drawing function, determine whether it is necessary to call Vulkan's API to start a new rendering pass and start;

(1.4) In the end rendering pass function, ensure the integrity of the rendering hardware interface semantics and ensure that each initial call has a corresponding end call.

Further, the step (2) includes the following sub-steps:

(2.1) Utilize the architectural design of Frame Graph to place most of the resource update operations required for the rendering pass in the Frame

The construction phase of Graph, and the actual drawing process occurs in the execution phase after construction;

(2.2) For the situation where the result of a certain rendering node is dependent on subsequent rendering nodes, in order to avoid its state switching occurring between subsequent homogeneous rendering passes and causing interruption to the merged homogeneous rendering passes, the analysis through Frame Graph is used to perform advance analysis. Record the state switching that the resource needs to perform, and place the execution timing of the state switching operation after the actual execution of Vulkan's termination rendering pass function, that is, between non-homogeneous rendering nodes, thus preventing the results from being used as resources in homogeneous areas that should be merged. Switch states between rendering passes.

The beneficial effects of the present invention are as follows:

The present invention provides a method for optimizing the Vulkan drawing process on the mobile terminal. By adding judgment logic in the Frame Graph layer, the rendering channel nodes are sorted, and at the same time, each color and depth in each rendering channel node is determined based on the sorted results. Read and write operations on attachments, and perform special judgment on texture objects with MSAA turned on, thus saving the overhead of reading and writing the frame buffer when drawing with Vulkan on the mobile terminal, and reducing the performance caused by bandwidth limitations on the mobile terminal to a certain extent. bottleneck problem and improve overall performance.

As can be seen from the above embodiments, the present application should be understood that the above general description and the following detailed description are only illustrative and explanatory, and do not limit the present application.

Description of the drawings

Figure 1 shows the underlying implementation of the merged Render Pass logic;

Figure 2 is a logical flow chart for determining Frame Graph and calculating Load Store Flag;

Figure 3 is an example diagram after Frame Graph calculates Load Store Flag.

Detailed ways

Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this application.

The purpose of the embodiments of this application is to provide a method to optimize the low performance problem caused by the overhead of reading and writing the Frame Buffer in the Vulkan drawing process on the mobile terminal.

Step 1: First encapsulate the function interfaces of BeginRenderPass, Draw, and EndRenderPass. The function of the BeginRenderPass function is to pass in a set of render targets and bound depth maps (Render Targets). to get VkRenderPass and VkFrameBuffer objects.

As shown in Figure 1, at this time, check whether the object set when BeginRenderPass was called last time is compatible with the currently obtained VkRenderPass and VkFrameBuffer. If it is not compatible, call vkCmdEndRenderPass to terminate the last enabled VkRenderPass, and save the new object for actual use. ; If it is compatible, there is no need to terminate the enabled VkRenderPass, thereby reducing the calls to vkCmdBeginRenderPass and vkCmdEndRenderPass, and achieving the effect of automatic merging of VkRenderPass.

(a) Maintain the cache of VkFrameBuffer and VkRenderPass. Each time the BeginRenderPass function is called, the corresponding VkFrameBuffer and VkRenderPass are searched for in the cache according to the Render Targets in the passed parameters. If the search fails, new VkFrameBuffer and VkRenderPass are generated and saved in in their respective caches;

(b) The VkRenderPass obtained from the BeginRenderPass function will not start immediately, but will start by calling vkCmdBeginRenderPass before calling the vkCmdDraw function within the encapsulated Draw function;

(c) The EndRenderPass function will not call vkCmdEndRenderPass to terminate the rendering pass. In order to make the rendering passes merged as much as possible, a mark is just added here. The real vkCmdEndRenderPass call will appear when the BeginRenderPass function finds that the rendering target is switched;

(d) The keywords for finding and creating VkFrameBuffer and VkRenderPass are only related to the parameters of Render Targets, and have nothing to do with the operation of Render Pass itself (such as whether to read in, clear, etc. Render Targets), thus ensuring compatibility with VkRenderPass sexual judgment.

Step 2: In addition to the automatic merging logic in the first step, some API calls need to be outside the scope of VkRenderPass, which will interrupt the VkRenderPass that should be merged. There are mainly two situations:

(a) Some updates to the data or status of resources required for rendering (such as vkCmdCopyBuffer, vkCmdCopyImage, conversion to VkImageLayout, etc.). These operations need to be advanced as much as possible before all VkRenderPass starts, while also reducing unnecessary status updates to resources. This can be achieved through Frame Graph. Determine the dependency relationship between resources such as Buffer and Image and Render Pass nodes in Frame Graph. For resources that do not depend on all Render Pass of the current frame, the update operation will be advanced to the beginning of frame drawing. ; For resources that have dependencies (for example, the resource result output by a node is used as a resource input by a subsequent node), find the position where VkRenderPass will be switched between the two nodes and insert it;

(b) Since the operation of Compute Pass will inevitably interrupt VkRenderPass, the Render Pass nodes that are allowed to be merged need to be merged as much as possible according to the dependency relationship between Compute Pass and Render Pass in Frame Graph (for example, for rendering channel C, which also depends on calculation Pass A and rendering pass B, but when calculation pass A and rendering pass B are independent of each other, Frame Graph can put the execution of calculation pass A before rendering pass B).

The third step is to determine in the Frame Graph whether each rendering result of each Render node is dependent on the subsequent Render node or Compute node. What should also be considered here is: because the real merging operation of the above Render nodes is performed at the bottom, the merging The logic is not visible to the Frame Graph, so the Frame Graph needs to determine whether the adjacent Render nodes will be merged. Here, it is determined whether the two Render nodes are merged by judging whether the output results of the two nodes are consistent (that is, the rendering targets are consistent). If If there is a merge, the merged whole will be considered as one node. Here you need to determine each output resource associated with each Render node (that is, each Attachment in the Frame Buffer). The decision logic is shown in Figure 2, and the specific process is as follows:

(a) For each Render node, for each Attachment of the Render node, traverse to determine whether subsequent nodes (except for other Render nodes that will be merged with the current Render node) depend on its results. If other subsequent Render nodes do not need it, Current Attachment, the rendering result can be discarded after the current Render node ends without writing back to the video memory. This is accomplished by specifying VkAttachmentStoreOp as VK_ATTACHMENT_STORE_OP_DONT_CARE in VkAttachmentDescription, otherwise it is set to VK_ATTACHMENT_STORE_OP_STORE.

(b) In the same way, each Render node also needs to determine for each bound Attachment whether the previous Render node (except for other Render nodes that will be merged with the current Render node) has written operations to it. If a previous Render node has written and saved the current Attachment, then you need to set VkAttachmentLoadOp to VK_ATTACHMENT_LOAD_OP_LOAD in VkAttachmentDescription, otherwise set it to VK_ATTACHMENT_LOAD_OP_DONT_CARE.

The fourth step is to manage the texture resource with MSAA turned on and the corresponding Resolve texture as the same object. In the third step, use Frame Graph to determine the rendering target of the Render node to determine whether multi-sampling anti-aliasing (MSAA) is turned on for the rendering target object. For objects with MSAA turned on, by default, when used as the input resource of a node, the result of its resolution (Resolve) is required, and when used as the output result of the node, it is required to be used as the object of multiple sampling (Multiple Sample). From this logic, it can be concluded that if there is no subsequent node that uses the current MSAA rendering target as a rendering target again, but only as an input resource required by subsequent nodes, the MultipleSample object can be discarded to greatly reduce storage overhead. Specifically See Figure 3, which is a simple example of the effect obtained through this logical calculation.

(a) When using Frame Graph to determine the texture of MSAA in the third step, a special judgment is made when determining whether to write it back to the video memory. If there is no node to be written again in the future, the VkAttachmentStoreOp of the MSAA object can be set to VK_ATTACHMENT_STORE_OP_DONT_CARE.

(b) The corresponding read and write operations of the Resolve texture are fixed to VK_ATTACHMENT_LOAD_OP_DONT_CARE and VK_ATTACHMENT_STORE_OP_STORE.

(c) For MSAA's Depth Stencil Attachment, you can also use the structure VkSubpassDescriptionDepthStencilResolve provided by the extension named: VK_KHR_depth_stencil_resolve to complete the in-place Resolve operation.

Other embodiments of the present application will be readily apparent to those skilled in the art from consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary technical means in the technical field that are not disclosed in this application. .

It is to be understood that the present application is not limited to the precise structures described above and illustrated in the accompanying drawings, and that various modifications and changes may be made without departing from the scope thereof.

Claims (3)

1. A mobile terminal Vulkan drawing process optimization method, characterized by encapsulating the VkRenderPass related calls provided by Vulkan, and optimizing its read and write frame buffer operations in the encapsulated function body. The method includes the following steps: (1) Encapsulate the three functions of BeginRenderPass, Draw and EndRenderPass for the upper layer to call, and implement calls to Vulkan related APIs inside the encapsulated function body; The start rendering pass function BeginRenderPass obtains the Vulkan rendering pass VkRenderPass and the frame buffer object VkFrameBuffer by passing in a set of rendering targets Render Target and the bound depth map, and saves them. Check the objects saved when BeginRenderPass was last called. Whether it is compatible with the rendering channel and frame buffer obtained by the current call; If it is not compatible, call the corresponding vkCmdEndRenderPass function provided by Vulkan to terminate the last enabled rendering pass, and save the new object for actual use; If compatible, there is no need to terminate the enabled rendering pass, and the rendering passes will be automatically merged; After the start rendering pass function BeginRenderPass obtains and saves the rendering pass and the frame buffer, the drawing function Draw is used to determine whether it is necessary to call Vulkan's API to start a new rendering pass and start; the function interface of the end rendering pass function EndRenderPass It is used to check the semantic clarity when the upper layer calls this set of functions, and does not execute the termination rendering pass function provided by Vulkan; (2) The premise for calling the start rendering pass function in step (1) to perform compatibility determination to form a merger is that before calling the start rendering pass function, the homogeneous rendering pass objects enabled by the last drawing are not affected by other circumstances. Termination; specifically: If the frame buffer objects written by a group of rendering passes are the same, they can be merged. If each rendering pass requires different resources when performing drawing, the resources need to be updated before the rendering pass starts; Therefore, in a group of homogeneous rendering passes, in order to prevent the update operation of the rendering pass executed later from interrupting the enabled homogeneous rendering pass, the update operation corresponding to the resources required by the rendering pass executed later is advanced. Before entering this set of rendering passes, Frame Graph needs to be used to achieve this goal; (3) When determining the dependencies between rendering passes connected by resources in step (2), for resources that do not have dependencies, redundant writeback operations can be prevented by discarding the resources at the end of the rendering pass; specifically, using Frame Graph Abstract each rendering channel into a rendering node, and treat the merged rendering channels in step (1) as the same rendering node; for each rendering node, use Frame Graph to determine each rendering in its bound frame buffer. Whether the target is dependent on subsequent nodes or depends on previous nodes; for rendering targets that do not have corresponding dependencies, set the corresponding flag bits when creating the rendering pass to avoid redundant read and write operations; (4) For the rendering pass that turns on multi-sampling anti-aliasing MSAA, it needs to be parsed and converted into a normal texture without multi-sampling turned on, so as to produce the final required result, and the multi-sampling target is written back at the end of the rendering pass. The cost of the operation is too high, so when determining the rendering target in step (3), special processing is required to avoid redundant reading and writing; specifically, when creating a rendering pass, specify the corresponding multi-sampling target. The parsed rendering target means that an additional rendering target is bound to the rendering pass, so that the parsing operation can be completed in place at the end of the rendering pass, thereby discarding the multi-sampling target and only writing the parsed target back to the video memory. 2. A method for optimizing the Vulkan rendering process of a mobile terminal according to claim 1, characterized in that the step (1) includes the following sub-steps: (1.1) Maintain the cache of rendering channels and frame buffers. Each time the start rendering channel function is called, the corresponding rendering channels and frame buffers are searched for in the cache based on the rendering target and depth map in the passed parameters. If the search fails, a generated New render passes and framebuffers and saving them in their respective caches; (1.2) In the start rendering pass function, the compatibility of the rendering pass is determined. At this time, the keywords for finding and creating rendering passes and frame buffers are only related to the parameters of the rendering target and depth template map, and have nothing to do with the operation of the rendering pass itself; (1.3) In the drawing function, determine whether it is necessary to call Vulkan's API to start a new rendering pass and start; (1.4) In the end rendering pass function, ensure the integrity of the rendering hardware interface semantics and ensure that each initial call has a corresponding end call. 3. A method for optimizing the Vulkan rendering process of a mobile terminal according to claim 1, characterized in that the step (2) includes the following sub-steps: (2.1) Using the architectural design of Frame Graph, most of the resource update operations required for the rendering channel are placed in the construction phase of Frame Graph, and the actual drawing process occurs in the execution phase after construction; (2.2) For the situation where the result of a certain rendering node is dependent on subsequent rendering nodes, in order to avoid its state switching occurring between subsequent homogeneous rendering passes and causing interruption to the merged homogeneous rendering passes, the analysis through Frame Graph is used to perform advance analysis. Record the state switching that the resource needs to perform, and place the execution timing of the state switching operation after the actual execution of Vulkan's termination rendering pass function, that is, between non-homogeneous rendering nodes, thus preventing the results from being used as resources in homogeneous areas that should be merged. Switch states between rendering passes.