CN116703693A - Image rendering method and electronic device - Google Patents

Abstract

The embodiment of the application discloses an image rendering method and electronic equipment, relates to the field of image processing, and aims to reduce rendering overhead through a variable rate coloring mechanism, and meanwhile, the influence on user experience caused by the reduction of coloring rate is avoided. The specific scheme is as follows: and acquiring a first rendering command issued by the application program, wherein the first rendering command is used for drawing a first model. A first rate of coloration of the first model is determined. The first model is drawn according to the first rendering command and the first shading rate.

Description

Image rendering method and electronic device

This application is a divisional application, the application number of the original application is 202110951444.3, and the original application date is August 18, 2021. The entire content of the original application is incorporated in this application by reference.

technical field

The present application relates to the technical field of image processing, in particular to an image rendering method and electronic equipment.

Background technique

When an electronic device renders an image, it includes coloring the image. Exemplarily, a graphics processing unit (graphics processing unit, GPU) of the electronic device may color each pixel of the image separately, and then complete the coloring process of the entire image.

With the increase of the pixels of the image, the coloring process of the image will generate a higher rendering load on the electronic device, such as increasing the computing power and power consumption during the rendering process.

Contents of the invention

Embodiments of the present application provide an image rendering method and an electronic device, which can reduce rendering overhead through a variable rate shading mechanism without affecting user experience due to the reduction of shading rate.

In order to achieve the above purpose, the embodiment of the present application adopts the following technical solutions:

In a first aspect, an image rendering method is provided, which is applied to an electronic device, and an application program is installed in the electronic device. The method includes: acquiring a first rendering command issued by the application program, and the first rendering command is used to draw the first a model. A first shading rate for the first model is determined. The first model is rendered according to the first rendering command and the first shading rate.

Based on this scheme, an example of a scheme for coloring different models at different rates is provided. In this example, the electronic device can determine the shading rate of each model according to rendering commands for different models. Then, with the rendering command (Drawcall) as the granularity, the rendering operations such as model drawing are shaded at different rates, and some models are shaded at a low rate, thereby saving the rendering overhead during the shader process, such as computing power overhead, heat generation, etc. .

In a possible design, the determining the first shading rate of the first model includes: determining the depth of the first model according to the first rendering command, where the depth of the first model is used to identify the first model In the observation space or the clipping space, the greater the depth of the first model is, the farther the distance between the first model and the observer is. The first shading rate is determined according to the depth of the first model. Based on this scheme, a mechanism is provided to determine the shading rate of different models. Exemplarily, the corresponding shading rate can be determined according to the depth of different models. It can be understood that the greater the depth, the farther the model is from the user in the image, so lower rate shading can be used to save rendering overhead.

In a possible design, the determining the depth of the first model according to the first rendering command includes: determining the depths of n vertices of the first model according to the first rendering command. Determine the depth of the first model according to the depths of the n vertices. Wherein, n vertices are included in the vertices of the first model. Based on this scheme, an example scheme for determining the depth of a model is provided. Exemplarily, the depth of the model may be determined according to the depth of the vertices of the model. Wherein, the depth of the vertex of the model may be the observation space or the clipping space or other depths that can be used to identify the distance from the vertex to the user. In this example, n can be equal to the maximum number of vertices of the model. In this way, the depth of the model can be determined according to the depth of all vertices of the model. In some other embodiments, n may be less than the maximum number of vertices of the model, for example, n vertices may be randomly selected n vertices from all vertices of the model. In this way, the calculation amount in the process of calculating the vertices of the model can be saved.

In a possible design, the method further includes: acquiring a second rendering command issued by the application program, where the second rendering command is used to draw the second model. A second shading rate for the second model is determined. The second model is rendered according to the second rendering command and the second shading rate. The first model and the second model are included in the same frame image, and the first shading rate and the second shading rate are different. Based on this scheme, the electronic device can also perform variable rate shading of the second model. In some cases, the relevant parameters of the second model (such as depth information) are different from those of the first model, and the corresponding shading rates of the two models are also different.

In a possible design, the electronic device is configured with an interception module and a data processing module, and the acquisition of the first rendering command issued by the application program includes. The interception module intercepts the first rendering command. The interception module transmits the intercepted first function and second function to the data processing module. The first function is a function carrying a first parameter, the second function is a function carrying a second parameter, the first parameter is a parameter carried in the process of transferring vertex data by the application program, and the second parameter is a function passed by the application program Parameters carried in the MVP matrix process. Based on this solution, a specific example of a solution for intercepting commands is provided. In this example, the interception module can intercept all the first rendering commands, and transmit the instruction (or function) carrying the first parameter and the second parameter in the first rendering command to the data processing module, so as to realize the interception of useful data. The useful data may be vertex related data and MVP matrix.

In a possible design, the method further includes: the intercepting module calling back functions other than the first function and the second function in the first rendering command to a graphics library of the electronic device. Based on this solution, a callback mechanism is provided. It can be understood that, in the native logic, the first rendering command can be sent to the graphics library for processing. In this example, after intercepting the first function and the second function, the interception module can transmit other part of commands to the graphics library, so as to ensure the normal operation of the rendering logic.

In a possible design, after the interception module transmits the intercepted first function and second function to the data processing module, the method further includes: the data processing module, according to the first function and the second function, Determine the first storage location of the vertex coordinates corresponding to the first model, and the second storage location of the first MVP matrix, the first storage location and the second storage location are included in the first storage area, the first storage A region is a memory region that can be accessed by the processor of the electronic device. Based on this scheme, an example scheme for determining vertex coordinates and MVP matrix is provided. For example, the first function carrying the first parameter may be used to indicate the cache ID of the vertex coordinates of the model corresponding to the current Drawcall. Then the data processing module can analyze the cache ID of the vertex coordinates of the first model according to the first function. Further, the specific storage location of the vertex coordinates is determined according to parameters such as preset or attributes indicated by the first function and offsets. The data processing module can also obtain the MVP matrix based on similar logic. Wherein, the storage location determined by the data processing module may be a storage location of backup stored data, thereby ensuring normal calling of other modules.

In a possible design, the method further includes: the data processing module calling back the first function and the second function to a graphics library of the electronic device. Based on this solution, another callback mechanism is provided. Exemplarily, the interception module has already called back the functions in the first command except the first function and the second function, then the data processing module can call back the first function and the second function to realize the full callback of the first rendering command , so as to ensure the accurate execution of the rendering logic.

In a possible design, before acquiring the first rendering command issued by the application, the method further includes: the interception module intercepts a third rendering command issued by the application, and the third rendering command is used to The first data used during the running of the application program is stored in the second storage area, the second storage area is the storage area used by the GPU of the electronic device, and the first data includes the vertex coordinates of the first model and the second storage area. An MVP matrix. The interception module transmits the third function and the fourth function to the data processing module, the third function carries the first parameter, and the fourth function carries the second parameter. The data processing module stores data corresponding to the third function and the fourth function in the first storage area. Based on the solution, a mechanism for backing up stored data is provided. It can be understood that, in the first rendering command (or the second rendering command), the corresponding rendering instruction can be implemented by invoking the data that has been loaded into the GPU by the third rendering command. The data loaded to the GPU is invisible to other modules. Therefore, in this example, through the backup storage mechanism, the loaded data is backed up and stored in the CPU schedulable storage space, thus ensuring that each A smooth call to the loaded data when the module needs to determine the depth of the model.

In a possible design, the method further includes: the intercepting module calling back functions other than the third function and the fourth function in the third rendering command to a graphics library of the electronic device. Based on this solution, another callback mechanism is provided. Therefore, during the loading process, the electronic device can correctly load instructions other than the third function and the fourth function.

In a possible design, the method further includes: the data processing module calling back the third function and the fourth function to a graphics library of the electronic device. Based on this solution, another callback mechanism is provided. Therefore, during the loading process, the electronic device can correctly load the third function and the fourth function. Exemplarily, the execution of the callback mechanism may be performed after the data processing module completes backup storage of the third function and the fourth function.

In a possible design, the method further includes: the data processing module stores a first correspondence, and the first correspondence is used to indicate the storage address of the same data in the first storage area and the address of the same data in the second storage area. Correspondence between storage addresses. Based on the scheme, a mechanism for correct data calling is provided. It is understandable that during backup storage, data cannot be stored in the area of memory configured for the GPU. In this example, by storing the correspondence between the address of the backup storage and the address indicated in the native command, the corresponding data of the backup storage can be accurately found subsequently according to the native command.

In a possible design, the data processing module determines the first storage location of the vertex coordinates corresponding to the first model and the second storage location of the first MVP matrix according to the first function and the second function, The method includes: the data processing module determines the first storage location of the vertex coordinates corresponding to the first model according to the cache location indicated by the first function and the first corresponding relationship. The data processing module determines a second storage location of the first MVP matrix according to the cache location indicated by the second function and the first correspondence. Based on this solution, an example of a solution for determining the vertex coordinates and MVP matrix corresponding to the current Drawcall is provided. For example, in combination with the cache location indicated by the current command and the corresponding relationship, the corresponding data can be found in the backup storage. The data may be the vertex coordinates and the MVP matrix of the first model indicated by the current Drawcall.

In a possible design, the electronic device is further equipped with a calculation module, and determining the depths of n vertices of the first model according to the first rendering command includes: the interception module intercepts the first rendering In the case of the drawing element Drawelement in the command, the calculation module calculates the depths of the n vertices according to the vertex coordinates corresponding to the first model and the first MVP matrix. Wherein, the vertex data corresponding to the first model is obtained by the calculation module from the first storage location, and the first MVP matrix is obtained by the calculation module from the second storage location. Based on this scheme, a trigger mechanism for computing model depth is provided. For example, it can be determined that the instruction of data such as vertex coordinates and MVP matrix has been completed when the Drawelement issued by the application is intercepted, so that the vertex coordinates indicated by the current Drawcall and the storage location of the MVP matrix determined in the above scheme can be determined. , call the corresponding data for calculation.

In a possible design, the determining the depth of the first model according to the depths of the n vertices includes: the calculation module determining the depth of the first model as an average value of the depths of the n vertices. Based on this solution, a specific solution example for determining the depth of a model according to the depth of a vertex is provided. For example, the model depth may be the mean value of the depths of the selected n vertices.

In a possible design, a decision module is further configured in the electronic device, and the method further includes: the calculation module transmits the depth of the first model to the decision module. According to the depth of the first model, the decision-making module searches for an entry matching the depth of the first model from the preset second correspondence. The decision module determines the shading rate indicated by the matching entry as the first shading rate. Based on this solution, an example of a solution for determining the corresponding shading rate according to the model depth is provided. For example, according to the preset depth range of the model depth, a matching entry can be found in the second correspondence, and the representation can indicate a preset shading rate. Then the preset shading rate may be the first shading rate corresponding to the depth of the current model.

In a possible design, the method further includes: the decision module transmits a shading instruction indicating the first shading rate to the graphics library. Based on this scheme, a scheme mechanism that triggers shading according to the first shading rate is provided. For example, after determining that the first model uses the first shading rate, the decision module may call an API corresponding to the first shading rate in the graphics library. In some implementations, calling the API may be implemented by sending a shading instruction corresponding to the first shading rate.

In a possible design, the rendering of the first model according to the first rendering command and the first shading rate includes: the graphics library calls the function corresponding to the first The first application programming interface API corresponding to the rendering command. The graphics library calls a first variable rate shading API corresponding to the first shading rate according to the shading instruction. The graphics library sends a rendering instruction to the GPU of the electronic device according to the first API and the first variable-rate shading API, so that the GPU performs a rendering operation on the first model according to the first shading rate. Based on the solution, a solution mechanism for GPU to perform variable rate shading on the first model is provided. Exemplarily, based on the aforementioned callback mechanism, the GPU may receive all instructions of the first rendering command (second rendering command). In addition, the GPU may also receive instructions indicated by corresponding APIs invoked according to the first shading rate. Therefore, the GPU can render the first model indicated by the first rendering command according to the first shading rate. Realize the flexible adjustment of the shading rate of the drawing model.

In a second aspect, an electronic device is provided, and the electronic device includes one or more processors and one or more memories; the one or more memories are coupled to the one or more processors, and the one or more memories store computer instructions; When one or more processors execute computer instructions, the electronic device is made to execute the image rendering method according to the first aspect above and any one of various possible designs.

In a third aspect, the embodiment of the present application provides an image rendering apparatus, and the apparatus may include: a processor and a memory. The memory is used to store computer-executable program code, which includes instructions. When the processor executes the instructions, the instructions cause the electronic device to execute the image rendering method according to the first aspect and any one of various possible designs.

In a fourth aspect, a computer-readable storage medium is provided, the computer-readable storage medium includes computer instructions, and when the computer instructions are run, the image rendering method in the above-mentioned first aspect and any one of various possible designs is executed.

It should be understood that the technical features of the technical solutions provided by the second aspect, the third aspect, and the fourth aspect can all correspond to the image rendering method provided in the first aspect and its possible design, so the beneficial effects that can be achieved Similar and will not be repeated here.

Description of drawings

Fig. 1 is a schematic diagram of a coordinate space;

Fig. 2 is a schematic diagram of a variable rate shading;

FIG. 3 is a schematic diagram of the composition of an electronic device provided in an embodiment of the present application;

FIG. 4 is a schematic diagram of the software composition of an electronic device provided in an embodiment of the present application;

FIG. 5 is a schematic diagram of a rendering process provided by an embodiment of the present application;

FIG. 6 is a schematic flowchart of an image rendering method provided by an embodiment of the present application;

FIG. 7 is a schematic flowchart of an image rendering method provided by an embodiment of the present application;

FIG. 8 is a schematic flowchart of an image rendering method provided by an embodiment of the present application;

FIG. 9 is a schematic flowchart of an image rendering method provided by an embodiment of the present application;

FIG. 10 is a schematic flowchart of an image rendering method provided by an embodiment of the present application;

FIG. 11 is a schematic flowchart of an image rendering method provided by an embodiment of the present application;

FIG. 12 is a schematic flowchart of an image rendering method provided by an embodiment of the present application;

FIG. 13 is a schematic diagram of a comparison of image rendering results provided by the embodiment of the present application;

FIG. 14 is a schematic composition diagram of an electronic device provided by an embodiment of the present application.

Detailed ways

The electronic device can render different frames of images according to the rendering commands issued by the installed application program, so as to obtain the display data corresponding to each frame of images, and then control the display to display each frame of images according to the display data.

During the image rendering process, the electronic device needs to determine the vertex positions of one or more objects included in the current frame image.

Exemplarily, the rendering command issued by the application may include the vertex coordinates of the object. In some implementations, the vertex coordinates included in the rendering command may be coordinates based on the object's own local coordinate system. In this application, the distribution space of objects based on the local coordinate system may be referred to as a local space (Local Space). In order to determine the coordinates of each vertex of the object on the display screen, the electronic device may perform matrix transformation based on the coordinates of the object in the local space. In this way, the coordinates of the object in the coordinate system based on the display screen space (such as called screen space (Screen Space)) are obtained.

As an example, the electronic device can process each vertex of the object in Local coordinates in local space are converted to coordinates in screen space.

Exemplarily, referring to FIG. 1 , it shows a logical process of matrix transformation of coordinates from local space to world space to observation space to clipping space. In this example, the rendering command issued by the application may include the rendering of object 1. As shown in FIG. 1 , in local space, the coordinate system may be based on an object 1 . For example, the origin of the coordinate system in the local space may be set at the center of the object 1, or where a vertex is located, and so on. The application program may carry the coordinates of each vertex of the object 1 in the coordinate system of the local space, that is, the local coordinates, in the rendering command issued to the object 1 . The electronic device can convert the coordinates in the local space to the coordinates in the world space through the M matrix issued by the application program. Wherein, the world space may be a larger area than the local space. For example, take a rendering command issued by an application program for rendering a game image as an example. The local space may correspond to a smaller area that can cover a certain object (such as object 1). The world space may correspond to an area of a map including object 1 and other objects (such as object 2) in the game. The electronic device can combine the local coordinates in the local space with the M matrix to perform M matrix transformation, so as to obtain the coordinates of the object 1 in the world space. Similarly, when the application program issues a rendering command for the object 2 in the image frame, the electronic device can also obtain the coordinates of the object 2 in the world space through the above-mentioned M matrix transformation.

After acquiring the coordinates of vertices of each object in the world space in the current frame image, the electronic device can convert the coordinates in the world space into coordinates in the observation space according to the V matrix issued by the application program. It can be understood that the coordinates in world space may be coordinates in three-dimensional space. When the electronic device displays frame images to the user, various objects (such as object 1, object 2, etc.) are displayed on a two-dimensional display screen. When observing objects in world space with different viewing angles, you will see different two-dimensional pictures. The viewing angle may be relative to the position of the camera (or observer) set in world space. In this example, the coordinate space corresponding to the camera position may be referred to as the viewing space. As an example, take the camera set in the positive direction of the y-axis in the world space as an example. Then based on the transformation of the V matrix, the coordinates of the vertices of the object 1 and object 2 in the observation space corresponding to the camera position can be obtained. As shown in FIG. 1 , since the camera is located in the positive direction of the y-axis and shoots downwards, the objects 1 and 2 corresponding to the observation space can be presented as a top view effect.

After the electronic device acquires the coordinates of each object in the viewing space, it can project them to the clipping coordinates. The coordinate space corresponding to the clipping coordinates may be called clipping space. It can be understood that, in the process of performing the V matrix transformation, it may be a transformation of a larger area in the world space, so the acquired image range may be relatively large. However, due to the limited size of the display screen of the electronic device, it may not be possible to display all objects in the observation space at the same time. In this example, the electronic device can project the coordinates of each object in the observation space into the clipping space. After projection into clip space, the coordinates of objects that can be displayed on the display can be in the range of -1.0 to 1.0. And for the coordinates of some objects that cannot be displayed on the display screen, they can be outside the range of -1.0 to 1.0. In this way, the electronic device can perform corresponding display according to the vertex coordinates whose coordinates are within the range of -1.0 to 1.0. Exemplarily, the electronic device may perform P-matrix transformation on each coordinate in the viewing space according to the P-matrix issued by the application program, so as to obtain the clipping coordinates corresponding to each coordinate in the clipping space.

It can be understood that, through the transformation of the above MVP matrix (ie M matrix transformation, V matrix transformation, and P matrix transformation), the electronic device can obtain the coordinates (ie clipping coordinates) of the vertices of each object displayed on the display screen. Next, the electronic device can also transform the clipping coordinates into screen coordinates, for example, use a viewport transform (Viewport Transform) to transform the coordinates in the range of -1.0 to 1.0 into the coordinate range defined by the glViewport function. The final transformed coordinates will be sent to the rasterizer, which will be converted into fragments, and then the display data corresponding to each pixel will be obtained. Based on these display data, the electronic device can control the display screen to perform corresponding display.

In the image rendering process of the electronic device, in addition to determining the vertex coordinates of each object according to the above scheme, it is also necessary to color each pixel in the current frame image, that is, determine the color data of each pixel. Therefore, according to the color data of each pixel, the display is controlled to display the corresponding color at the corresponding pixel position.

In some implementations, the electronic device may color each pixel in units of one pixel during the rendering process, so as to achieve coloring of the entire frame image. With the improvement of the resolution and refresh rate of the display screen of the electronic device, and the scene of the frame image that needs to be rendered becomes more and more complex, the coloring based on one pixel will cause a large memory and power consumption in the rendering process of the electronic device. Overhead, resulting in heat generation or frame drop phenomenon, affecting user experience. It should be noted that, in some implementations of the present application, the rendering process and the coloring process may be implemented by components with graphics processing functions such as a GPU set in the electronic device.

In order to deal with the above problems, some electronic devices can reduce memory and power consumption overhead during the coloring process by providing a variable rate coloring function.

Exemplarily, under a general coloring mechanism, an electronic device may use a shader to color a pixel. After the pixel is shaded, the electronic device can use the shader to shade another pixel. For example, in combination with (a) in FIG. 2 , the electronic device can use a shader to color the pixels located in the first row and the first column. After the coloring of the pixels in the first row and the first column is completed, the electronic device may use a shader to color other pixels, such as the pixels in the first row and the second column. In this way, to complete the shading of 5*5 pixels as shown in (a) in FIG. 2 , the electronic device needs to use the shader to perform at least 25 shading operations. It should be noted that, with reference to the foregoing description, the above-mentioned process of performing coloring through a shader may be executed by a GPU in an electronic device. When the GPU has a strong parallel processing capability, for example, the GPU can simultaneously color three pixels through the shader, and then the electronic device can also implement the coloring operation of multiple pixels (such as three pixels) in parallel through the shader. However, although the multi-issue parallel processing process can save processing time, it will not reduce the load of the electronic device during the coloring operation. For the convenience of description, the GPU in the electronic device uses a shader to color one pixel at the same time as an example.

As opposed to performing a shading operation in units of a single pixel, when the electronic device uses a variable-rate shading function, the electronic device can use a shader to complete the shading of multiple pixels through a single shading operation. For example, take a coloring operation that can complete the coloring of 4 pixels as an example. In combination with (b) in FIG. 2 , the electronic device can color the first pixel in the first column to the second pixel in the second column through one coloring operation. In this way, through variable rate shading, the electronic device can complete the shading in the image rendering process with fewer shading operations.

It can be understood that the color fineness of the image composed of pixels after variable rate shading is lower than that of an image composed of pixels obtained by a general shading mechanism (ie, a shading operation with a granularity of one pixel). Then, how to reasonably use variable rate shading without significantly affecting the user's perception of the image becomes the key to using the variable rate shading function.

In order to solve the above problems, the rendering solution provided by the embodiment of the present application can reasonably select the area in the frame image that needs to use the variable rate shading function, so that the electronic device can reduce the power consumption and heat generation in the rendering process through the variable rate shading function At the same time, rendering the acquired image will not have a significant impact on the user's perception. In this way, the user experience can be improved while reducing the power consumption and heat generation of the electronic device.

The solutions provided by the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings.

It should be noted that the rendering method provided in the embodiment of the present application may be applied in a user's electronic device. For example, the electronic device may be a portable mobile device such as a mobile phone, a tablet computer, a personal digital assistant (personal digital assistant, PDA), an augmented reality (augmented reality, AR)/virtual reality (virtual reality, VR) device, a media player, etc. The electronic device may also be a wearable electronic device capable of providing shooting capabilities, such as a smart watch. The embodiment of the present application does not specifically limit the specific form of the device.

Please refer to FIG. 3 , which is a schematic structural diagram of an electronic device 300 provided in an embodiment of the present application.

As shown in Figure 3, the electronic device 300 may include a processor 310, an external memory interface 320, an internal memory 321, a universal serial bus (universal serial bus, USB) interface 330, a charging management module 340, a power management module 341, a battery 342, antenna 1, antenna 2, mobile communication module 350, wireless communication module 360, audio module 370, speaker 370A, receiver 370B, microphone 370C, earphone jack 370D, sensor module 380, button 390, motor 391, indicator 392, camera 393, a display screen 394, and a subscriber identification module (subscriber identification module, SIM) card interface 395, etc. Wherein, the sensor module 380 may include a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.

It can be understood that the structure shown in this embodiment does not constitute a specific limitation on the electronic device 300 . In other embodiments, the electronic device 300 may include more or fewer components than shown, or combine certain components, or separate certain components, or arrange different components. The illustrated components can be realized in hardware, software or a combination of software and hardware.

The processor 310 may include one or more processing units, for example: the processor 310 may include a central processing unit (Central Processing Unit, CPU), an application processor (application processor, AP), a modem processor, a GPU, an image signal processor (image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural network processor (neural-network processing unit, NPU) and so on. Wherein, different processing units may be independent devices, or may be integrated in one or more processors 310 . As an example, in this application, the ISP can process the image, such as the processing can include automatic exposure (Automatic Exposure), automatic focus (Automatic Focus), automatic white balance (Automatic White Balance), denoising, backlight compensation , color enhancement and other processing. Among them, the processing of automatic exposure, automatic focus, and automatic white balance can also be called 3A processing. After processing, the ISP can obtain the corresponding photos. This process may also be referred to as the slice operation of the ISP.

In some embodiments, processor 310 may include one or more interfaces. The interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuitsound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous receiver (universal asynchronous receiver) /transmitter, UART) interface, mobile industry processor interface (mobile industry processor interface, MIPI), general-purpose input and output (general-purpose input/output, GPIO) interface, subscriber identity module (subscriber identity module, SIM) interface, and/or A universal serial bus (universal serial bus, USB) interface, etc.

The electronic device 300 can realize the shooting function through an ISP, a camera 393 , a video codec, a GPU, a display screen 394 , and an application processor.

The ISP is used for processing the data fed back by the camera 393 . For example, when taking a picture, open the shutter, the light is transmitted to the photosensitive element of the camera 393 through the lens, and the light signal is converted into an electrical signal, and the photosensitive element of the camera 393 transmits the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on image noise, brightness, and skin color. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be located in the camera 393 .

Camera 393 is used to capture still images or video. The object generates an optical image through the lens and projects it to the photosensitive element. The photosensitive element may be a charge coupled device (charge coupled device, CCD) or a complementary metal-oxide-semiconductor (complementary metal-oxide-semiconductor, CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. DSP converts digital image signals into standard RGB, YUV and other image signals. In some embodiments, the electronic device 300 may include 1 or N cameras 393, where N is a positive integer greater than 1.

Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. For example, when the electronic device 300 selects a frequency point, the digital signal processor is used to perform Fourier transform on the energy of the frequency point.

Video codecs are used to compress or decompress digital video. The electronic device 300 may support one or more video codecs. In this way, the electronic device 300 can play or record videos in various encoding formats, for example: moving picture experts group (moving picture experts group, MPEG) 1, MPEG2, MPEG3, MPEG4 and so on.

The NPU is a neural-network (NN) computing processor. By referring to the structure of biological neural networks, such as the transfer mode between neurons in the human brain, it can quickly process input information and continuously learn by itself. Applications such as intelligent cognition of the electronic device 300 can be realized through the NPU, such as image recognition, face recognition, speech recognition, text understanding, and the like.

The charging management module 340 is configured to receive charging input from the charger. Wherein, the charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 340 can receive charging input from a wired charger through the USB interface 330 . In some wireless charging embodiments, the charging management module 340 may receive wireless charging input through a wireless charging coil of the electronic device 300 . While the charging management module 340 is charging the battery 342 , it can also supply power to the electronic device 300 through the power management module 341 .

The power management module 341 is used for connecting the battery 342 , the charging management module 340 and the processor 310 . The power management module 341 receives the input of the battery 342 and/or the charging management module 340, and supplies power for the processor 310, the internal memory 321, the external memory, the display screen 394, the camera 393, and the wireless communication module 360, etc. The power management module 341 can also be used to monitor parameters such as the capacity of the battery 342 , the number of cycles of the battery 342 , and the state of health of the battery 342 (leakage, impedance). In some other embodiments, the power management module 341 may also be disposed in the processor 310 . In some other embodiments, the power management module 341 and the charging management module 340 may also be set in the same device.

The wireless communication function of the electronic device 300 can be realized by the antenna 1 , the antenna 2 , the mobile communication module 350 , the wireless communication module 360 , the modem processor 310 and the baseband processor 310 .

Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 300 may be used to cover single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example: Antenna 1 can be multiplexed as a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 350 can provide wireless communication solutions including 2G/3G/4G/5G applied on the electronic device 300 . The mobile communication module 350 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA) and the like. The mobile communication module 350 can receive electromagnetic waves through the antenna 1, filter and amplify the received electromagnetic waves, and send them to the modem processor for demodulation. The mobile communication module 350 can also amplify the signal modulated by the modem processor, convert it into electromagnetic wave and radiate it through the antenna 1 . In some embodiments, at least part of the functional modules of the mobile communication module 350 may be set in the processor 310 . In some embodiments, at least part of the functional modules of the mobile communication module 350 and at least part of the modules of the processor 310 may be set in the same device.

A modem processor may include a modulator and a demodulator. Wherein, the modulator is used for modulating the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low frequency baseband signal. Then the demodulator sends the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal is passed to the application processor after being processed by the baseband processor. The application processor outputs sound signals through audio equipment (not limited to speaker 370A, receiver 370B, etc.), or displays images or videos through display screen 394 . In some embodiments, the modem processor may be a stand-alone device. In some other embodiments, the modem processor may be independent from the processor 310, and be set in the same device as the mobile communication module 350 or other functional modules.

The wireless communication module 360 can provide applications on the electronic device 300 including wireless local area networks (wireless local area networks, WLAN) (such as wireless fidelity (wireless fidelity, Wi-Fi) network), bluetooth (bluetooth, BT), global navigation satellite system (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions. The wireless communication module 360 may be one or more devices integrating at least one communication processing module. The wireless communication module 360 receives electromagnetic waves via the antenna 2 , frequency-modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 310 . The wireless communication module 360 can also receive the signal to be sent from the processor 310 , frequency-modulate it, amplify it, and convert it into electromagnetic waves through the antenna 2 for radiation.

In some embodiments, the antenna 1 of the electronic device 300 is coupled to the mobile communication module 350, and the antenna 2 is coupled to the wireless communication module 360, so that the electronic device 300 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (general packet radio service, GPRS), code division multiple access (codedivision multiple access, CDMA), wideband code Wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM , and/or IR technology, etc. The GNSS may include a global positioning system (global positioning system, GPS), a global navigation satellite system (global navigation satellite system, GLONASS), a Beidou satellite navigation system (beidounavigation satellite system, BDS), a quasi-zenith satellite system (quasi- zenith satellite system (QZSS) and/or satellite based augmentation systems (SBAS).

The electronic device 300 implements a display function through a GPU, a display screen 394 , and an application processor 310 . The GPU is a microprocessor for image processing, connected to the display screen 394 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. Processor 310 may include one or more GPUs that execute program instructions to generate or alter display information.

The display screen 394 is used to display images, videos and the like. Display 394 includes a display panel. The display panel can be a liquid crystal display 394 (liquid crystal display, LCD), an organic light-emitting diode (organic light-emitting diode, OLED), an active matrix organic light-emitting diode or an active-matrix organic light emitting diode (active-matrix organic light emitting diode). diode, AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diodes (quantum dot light emitting diodes, QLED), etc. In some embodiments, the electronic device 300 may include 1 or N display screens 394, where N is a positive integer greater than 1.

The external memory interface 320 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 300 . The external memory card communicates with the processor 310 through the external memory interface 320 to implement a data storage function. Such as saving music, video and other files in the external memory card.

The internal memory 321 may be used to store computer-executable program code, which includes instructions. The processor 310 executes various functional applications and data processing of the electronic device 300 by executing instructions stored in the internal memory 321 . The internal memory 321 may include an area for storing programs and an area for storing data. Wherein, the stored program area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.) and the like. The storage data area can store data (such as audio data, phone book, etc.) created during the use of the electronic device 300 . In addition, the internal memory 321 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (universal flash storage, UFS) and the like.

The electronic device 300 can realize the audio function through the audio module 370 , the speaker 370A, the receiver 370B, the microphone 370C, the earphone interface 370D, and the application processor 310 . Such as music playback, recording, etc.

The audio module 370 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal. The audio module 370 may also be used to encode and decode audio signals. In some embodiments, the audio module 370 can be set in the processor 310 , or some functional modules of the audio module 370 can be set in the processor 310 . Speaker 370A, also called "horn", is used to convert audio electrical signals into sound signals. Electronic device 300 can listen to music through speaker 370A, or listen to hands-free calls. Receiver 370B, also called "earpiece", is used to convert audio electrical signals into audio signals. When the electronic device 300 receives a call or a voice message, the receiver 370B can be placed close to the human ear to receive the voice. The microphone 370C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, or needing to trigger the electronic device 300 to perform certain functions through the voice assistant, the user can make a sound by approaching the microphone 370C with a human mouth, and input the sound signal into the microphone 370C. The electronic device 300 may be provided with at least one microphone 370C. In some other embodiments, the electronic device 300 may be provided with two microphones 370C, which may also implement a noise reduction function in addition to collecting sound signals. In some other embodiments, the electronic device 300 can also be provided with three, four or more microphones 370C, so as to collect sound signals, reduce noise, identify sound sources, and realize directional recording functions, etc. The earphone interface 370D is used to connect wired earphones. The earphone interface 370D may be a USB interface 330, or a 3.5mm open mobile terminal platform (open mobile terminal platform, OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface. .

Touch sensor, also known as "touch panel". The touch sensor can be arranged on the display screen 394, and the touch sensor and the display screen 394 form a touch screen, also called “touch screen”. The touch sensor is used to detect a touch operation on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. In some embodiments, the visual output related to the touch operation can be provided through the display screen 394 . In some other embodiments, the touch sensor can also be disposed on the surface of the electronic device 300 , which is different from the position of the display screen 394 .

The pressure sensor is used to sense the pressure signal and convert the pressure signal into an electrical signal. In some embodiments, a pressure sensor may be located on display screen 394 . There are many types of pressure sensors, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may be comprised of at least two parallel plates with conductive material. When a force is applied to the pressure sensor, the capacitance between the electrodes changes. The electronic device 300 determines the intensity of pressure according to the change in capacitance. When a touch operation acts on the display screen 394, the electronic device 300 detects the intensity of the touch operation according to the pressure sensor. The electronic device 300 may also calculate the touched position according to the detection signal of the pressure sensor. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities may correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view short messages is executed. When a touch operation whose intensity is greater than or equal to the first pressure threshold acts on the icon of the short message application, the instruction of creating a new short message is executed. The gyro sensor can be used to determine the motion posture of the electronic device 300 . The acceleration sensor can detect the acceleration of the electronic device 300 in various directions (generally three axes). Distance sensor for measuring distance. The electronic device 300 can measure the distance by infrared or laser. The electronic device 300 can use the proximity light sensor to detect that the user holds the electronic device 300 close to the ear to make a call, so as to automatically turn off the screen to save power. The ambient light sensor is used to sense the ambient light brightness. The fingerprint sensor is used to collect fingerprints. A temperature sensor is used to detect temperature. In some embodiments, the electronic device 300 uses the temperature detected by the temperature sensor to implement a temperature processing strategy. The audio module 370 can analyze the voice signal based on the vibration signal of the vibrating bone mass of the vocal part acquired by the bone conduction sensor, so as to realize the voice function. The application processor can analyze the heart rate information based on the blood pressure beating signal acquired by the bone conduction sensor, so as to realize the heart rate detection function.

The keys 390 include a power key, a volume key and the like. The motor 391 can generate a vibrating prompt. The indicator 392 can be an indicator light, which can be used to indicate the charging status, the change of the battery capacity, and can also be used to indicate messages, missed calls, notifications and the like. The SIM card interface 395 is used for connecting a SIM card. The electronic device 300 may support 1 or N SIM card interfaces 395, where N is a positive integer greater than 1. SIM card interface 395 can support Nano SIM card, Micro SIM card, SIM card and so on. Multiple cards can be inserted into the same SIM card interface 395 at the same time. The SIM card interface 395 is also compatible with different types of SIM cards. The SIM card interface 395 is also compatible with external memory cards. The electronic device 300 interacts with the network through the SIM card to implement functions such as calling and data communication. In some embodiments, the electronic device 300 adopts an eSIM, that is, an embedded SIM card. The eSIM card can be embedded in the electronic device 300 and cannot be separated from the electronic device 300 .

All the rendering methods provided in the embodiments of the present application can be applied to an electronic device having the composition as shown in FIG. 3 .

It should be noted that the foregoing FIG. 3 and its description are only an example of an application carrier of the solution provided by the embodiment of the present application. The composition in FIG. 3 does not constitute a limitation to the solution described in the embodiment of the present application. In other embodiments, the electronic device may also have more or fewer components than those shown in FIG. 3 .

In the example shown in FIG. 3 , the hardware composition of the electronic device is provided. In some embodiments, the electronic device can also run an operating system through its various hardware components (the hardware composition shown in FIG. 3 ). In the operating system, different software layers can be set, so as to realize the running of different programs.

Exemplarily, FIG. 4 is a schematic diagram of a software composition of an electronic device provided in an embodiment of the present application. As shown in FIG. 4 , the electronic device may include an application layer 401 , a framework layer 402 , a system library 403 , and a hardware layer 404 .

Wherein, the application layer 401 may also be called an application program layer, or an application (application, APP) layer. In some implementations, the application layer can include a series of application packages. Application packages can include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and SMS. The application package can also include applications that need to display pictures or videos to users by rendering images. For example, the application program included in the application layer 401 may be a game application program, such as and wait.

Framework layer 402 may also be called an application framework layer. The framework layer 402 can provide an application programming interface (application programming interface, API) and a programming framework for the application program of the application layer 401 . Framework layer 402 includes some predefined functions.

Exemplarily, the framework layer 402 may include a window manager, a content provider, a view system, a resource manager, a notification manager, an activity manager, an input manager, and the like. The window manager provides a window management service (Window Manager Service, WMS). WMS can be used for window management, window animation management, surface management, and as a transfer station for the input system. Content providers are used to store and retrieve data and make it accessible to applications. This data can include videos, images, audio, calls made and received, browsing history and bookmarks, phonebook, etc. The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, and so on. The view system can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text message notification icon may include a view for displaying text and a view for displaying pictures. The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, and so on. The notification manager enables the application to display notification information in the status bar, which can be used to convey notification-type messages, and can automatically disappear after a short stay without user interaction. For example, the notification manager is used to notify the download completion, message reminder, etc. The notification manager can also be a notification that appears on the top status bar of the system in the form of a chart or scroll bar text, such as a notification of an application running in the background, or a notification that appears on the screen in the form of a dialog window. For example, prompting text information in the status bar, issuing a prompt sound, vibrating the electronic device, and flashing the indicator light, etc. The activity manager can provide activity management service (Activity Manager Service, AMS), and AMS can be used for system components (such as activities, services, content providers, broadcast receivers) to start, switch, schedule, and manage and schedule application processes . The input manager may provide an input management service (Input Manager Service, IMS), and the IMS may be used to manage system inputs, such as touch screen input, key input, sensor input, and the like. IMS fetches events from input device nodes, and distributes events to appropriate windows through interaction with WMS.

In the embodiment of the present application, one or more functional modules may be set in the framework layer 402 to implement the rendering solution provided in the embodiment of the present application. Exemplarily, the framework layer 402 may be provided with an interception module, a data processing module, a calculation module, and a decision-making module. In subsequent examples, the functions of the above modules will be described in detail.

The system library 403 may include multiple functional modules. For example: surface manager (surface manager), media framework (Media Framework), standard C library (Standard C library, libc), open graphics library for embedded systems (OpenGL for Embedded Systems, OpenGL ES), Vulkan, SQLite, Webkit wait.

Among them, the surface manager is used to manage the display subsystem, and provides the fusion of 2D and 3D layers for multiple applications. The media framework supports playback and recording of various commonly used audio and video formats, as well as still image files. The media library can support multiple audio and video encoding formats, such as: Moving Pictures Experts Group 4 (Moving Pictures Experts Group, MPEG4), H.264, Moving Picture Experts Group Audio Layer 3 (Moving Picture Experts Group Audio Layer3, MP3), advanced audio Coding (Advanced Audio Coding, AAC), adaptive multi-code decoding (Adaptive Multi-Rate, AMR), Joint Photographic Experts Group (Joint Photographic Experts Group, JPEG, or called JPG), portable network graphics (Portable Network Graphics, PNG) wait. OpenGLES and/or Vulkan provide drawing and manipulation of 2D graphics and 3D graphics in applications. SQLite provides a lightweight relational database for applications of the electronic device 400 . In some implementations, OpenGL ES in system library 403 can provide variable rate shading functionality. Then, when the electronic device needs to perform variable-rate shading for a drawing command (Drawcall), it can call the variable-rate shading API in OpenGL ES to implement variable-rate shading for the current Drawcall together with other instructions. For example, the electronic device may use a lower rate (such as 2×1, 2×2, 4×2, 4×4, etc.) to render the current Drawcall, thereby reducing the overhead caused by rendering the current Drawcall.

In the example shown in FIG. 4 , the electronic device may further include a hardware layer 404 . The hardware layer 404 may include a processor (such as a CPU, a GPU, etc.), and a component with a storage function (such as the internal memory 321 shown in FIG. 3 , etc.). In some implementations, the CPU can be used to control each module in the framework layer 402 to realize its respective functions, and the GPU can be used for the graphics library (such as OpenGL ES) that is called according to the instructions processed by each module in the framework layer 402. The API performs the corresponding rendering processing.

In order to illustrate the functions of each layer in the software architecture provided by the embodiment of the present application more clearly, image rendering is taken as an example below to illustrate the function implementation of each component with software components as shown in FIG. 4 .

For example, please refer to FIG. 5 . When an application in the application layer needs to render an image, it can issue a rendering command. In the following description, a rendering command issued by an application may also be called a Drawcall. In different examples, the rendering command may include different content. For example, in some embodiments, an application program needs to render graphics in a frame image as an example. Vertex data of the graphics to be rendered may be included in the rendered command issued. In some implementations, this vertex data can be used to indicate the coordinates of the vertices of the graph to be rendered. The coordinates may be local space based coordinates. In the rendering command, the MVP matrix in the description as shown in FIG. 1 and one or more drawing elements (Drawelement) may also be included. After receiving the rendering command, the framework layer 402 may convert the rendering command into a rendering instruction, and the rendering instruction may carry the above-mentioned vertex data, MVP matrix, and one or more Drawelements. In some implementations, the framework layer 402 can also obtain the API required by the current Drawcall from the graphics library of the system library 403 according to the instruction of the application program, so as to instruct other modules (such as GPU) to perform rendering operations using the functions corresponding to the API. Exemplarily, the electronic device may determine the parameters to be used in the variable rate shading process before the Drawelement. The electronic device can also send a variable shading command by calling the variable rate shading API in combination with the aforementioned parameters. Implements variable rate shading of subsequent Drawelements. Take the rendering performed by the GPU in the hardware layer 404 as an example. The GPU may acquire a variable shading instruction, and in response to the variable shading instruction, execute the Drawelement using a shading rate indicated by a corresponding parameter.

The rendering method provided by the embodiment of the present application may also be applied to an electronic device with software components as shown in FIG. 4 . The solution provided by the embodiment of the present application will be described below in conjunction with the software composition shown in FIG. 4 .

It should be noted that in the following examples, in order to more clearly describe the solution provided by this application, the electronic equipment is divided into modules according to different functions, and the module division can be understood as having Another form of division of composed electronic equipment. Whether a certain function is executed by hardware or computer software drives hardware depends on the specific application and design constraints of the technical solution. Skilled artisans may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present application.

The embodiment of the present application can divide the functional modules of the electronic equipment involved according to the solution provided by the embodiment of the present application. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software function modules. It should be noted that the division of modules in the embodiment of the present application is schematic, and is only a logical function division, and there may be other division methods in actual implementation.

Exemplarily, the framework layer 402 of the electronic device may be provided with an interception module, a data processing module, a calculation module, and a decision-making module.

Based on the rendering method provided by the embodiment of the present application, the electronic device can intercept a rendering command from an application program (such as a game application) through various modules set therein. The electronic device can also filter and obtain the vertex data of each vertex included in the current rendering command from these rendering commands. Vertex coordinates may be included in this vertex data. The electronic device may also acquire the MVP matrix corresponding to the current rendering command. The electronic device can determine the depth of each vertex according to the vertex coordinates and the MVP matrix. Further, the depth of the model to be drawn by the current Drawcall is determined according to the depth of each vertex. The greater the depth of the model, the farther the model is from the observer in the current frame image. Correspondingly, the smaller the depth of the model, the closer the distance between the model and the observer. It can be understood that the models that are far away from the observer (that is, the user) are generally not followed by the user, while the models that are close to the user may be the models that the user pays attention to. Therefore, in the solution provided by the embodiment of the present application, a model with a greater depth can be rendered at a lower rate, thereby reducing the rendering overhead corresponding to the rendering without affecting the user experience. In contrast, for a model with a small depth, the electronic device can color it at a higher rate, so as to achieve accurate coloring of the model, improve the display quality of the model in the current frame image, and achieve the effect of improving user experience .

The specific implementation of the rendering method provided by the embodiment of the present application will be described in detail below in combination with the above module division. For ease of description, in the following examples, the application program that issues rendering commands is a game application, and the game application uses the OpenGL graphics library as an example for illustration. It should be understood that, in other different rendering engines/rendering environments, the implementation mechanisms are similar, and only the corresponding functions may be different.

When the game starts running (or when the game loads), the game application can load data that may be used during the rendering of subsequent frame images. In some implementations, the data loaded when the game is loaded may include vertex data of all models that may be used in a subsequent rendering process and MVP matrices of one or more frame images. In other implementations, only a portion of the model's vertex data may be loaded during a game load. In this way, when a new model needs to be used, the electronic device can execute the loading process again to load the vertex data of the new model into the GPU. Alternatively, the electronic device may load the vertex data by carrying the vertex data of the new model in the issued rendering instruction. In other implementations, the game application can only transmit vertex data during the game loading process, and for the MVP matrix that may be different for each frame of image, the game application can transmit the MVP matrix during the game running process. In the embodiment of the present application, the vertex data may include vertex coordinates, and the vertex coordinates may be coordinates based on local space.

For the sake of illustration, in the following example, the game is loaded once to realize the vertex coordinates of all models and the loading of the MVP matrix as an example.

In this example, through game loading, the game application can transmit all possible vertex coordinates of the model and one or more MVP matrices through a command including multiple instructions. Through these instructions, the vertex coordinates of the model and the MVP matrix can be stored in the memory space that the GPU can call.

Exemplarily, when the game is started, the game application may issue command 1 to implement the above loading process. In some embodiments, one or more of glGenbuffers function, glBindbuffer function, glBufferdata function and glBufferSubData function are included in the command 1 as an example.

Among them, the glGenbuffers function can be used to create a buffer. That is, one or more storage spaces are divided in the memory of the electronic device, and each storage space may have an identifier (ID). The divided storage space can be used to store various data in the rendering process. For example, some caches can be used to store the vertex coordinates of the model, some caches can be used to store the MVP matrix, etc.

The glBindbuffer function can be used to bind buffers. Through the binding of this function, subsequent operations can be bound to the corresponding cache. For example, take the created caches including cache 1, cache 2, and cache 3 as an example. Through glBindbuffer(1), subsequent operations can be bound to buffer 1. For example, if the subsequent operation includes an operation of writing data (such as vertex coordinates), then the electronic device can write the vertex coordinates into the buffer 1 for storage.

The glBufferdata function can be used to pass data. Exemplarily, if the data carried by the glBufferdata function is not empty (NULL), the electronic device can store the data (or data pointer) carried by the glBufferdata function in the bound buffer. For example, when the glBufferdata function carries vertex coordinates, the electronic device can store the vertex coordinates in the bound frame buffer. For another example, when the glBufferdata function carries the index of the vertex coordinates, the electronic device can store the index of the vertex coordinates in the bound frame buffer.

glBufferSubData function can be used for data update. For example, a game application can update part or all of the vertex coordinates through the glBufferSubData function. In this way, an effect of instructing an electronic device (such as a GPU) to draw and render according to the new vertex coordinates is achieved.

In the embodiment of the present application, the electronic device may intercept the instruction stream of command 1, thereby obtaining the instruction for transmitting the vertex data and the MVP matrix. The electronic device can also backup and store the acquired instructions. For example, an electronic device may store these data in an area in the memory that the CPU can call. Therefore, before the game is run, the memory of the electronic device can store the vertex data (such as vertex coordinates) and the MVP matrix that may be used in the subsequent rendering process. It can be understood that the native command (such as the instruction stream in command 1) is used to transfer data to the storage area that the GPU can call. Therefore, through the backup storage in this example, the CPU can also have vertex data and the calling ability of the MVP matrix, thereby ensuring the realization of subsequent solutions.

Exemplarily, as shown in FIG. 6 , the interception module in the electronic device can intercept the glGenbuffers function, the glBindbuffer function, the glBufferdata function, and the glBufferSubData function included in the command 1. The interception module can also transmit these functions to the data processing module for analysis and processing. For example, the data processing module can filter the function with parameter 1 among the functions from the interception module. Wherein, parameter 1 may be a parameter indicating to transmit vertex-related data. In this way, the data processing module can obtain instructions related to transmitting vertex data. Then, based on the functions obtained through these screenings, the data processing module can perform backup storage of vertex data.

Wherein, the parameter 1 may be obtained through offline analysis. In some embodiments, the parameter 1 may be pre-stored in an electronic device (such as a data processing module), so that the data processing module may perform screening of instructions related to vertex data based on the parameter 1 . As a possible implementation, the parameter 1 may include GL_ELEMENT_ARRAY_BUFFER and/or GL_ARRAY_BUFFER.

Similar to the interception and backup storage for vertex data instructions, the glGenbuffers function, glBindbuffer function, glBufferdata function, and glBufferSubData function included in the interception command 1 intercepted by the interception module can also be used to transmit the MVP matrix.

Then, the data processing module can also filter the function carrying parameter 2 among the functions from the interception module to obtain the function for transmitting the MVP matrix. Wherein, parameter 2 may be a parameter indicating to perform MVP matrix transmission. In this way, the data processing module can obtain instructions related to transmitting the MVP matrix. Then, based on these functions obtained through screening, the data processing module can perform backup storage of the MVP matrix.

Wherein, the parameter 2 may be obtained through offline analysis. In some embodiments, the parameter 2 may be pre-stored in an electronic device (such as a data processing module), so that the data processing module may perform screening of instructions related to the MVP matrix based on the parameter 2 . As a possible implementation, the parameter 2 may include GL_UNIFORM_BUFFER.

In the above example, the interception module directly transmits all intercepted instructions to the data processing module for analysis and processing without processing. In some other embodiments of the present application, the interception module may also have basic analysis capabilities. For example, the interception module can only intercept the glGenbuffers function, glBindbuffer function, glBufferdata function, and glBufferSubData function that carry parameter 1 and parameter 2. Wherein, parameter 1 may be a parameter indicating to transmit vertex-related data. Parameter 2 may be a parameter indicating to perform MVP matrix transmission.

In this way, the data processing module can directly back up and store the instructions from the interception module. In this way, the data processing pressure of the data processing module can be reduced.

In the following example, as shown in Figure 6, the interception module intercepts vertex-related instructions (such as instructions carrying parameter 1) and MVP-related instructions (such as instructions carrying parameter 2) and transmits them to the data processing module for backup storage. illustrate.

The backup storage involved in the embodiment of the present application may be implemented in the form of a jump table. The jump table can be used to indicate the correspondence between the original ID and the backup ID. Wherein, the original ID may be the ID of the cache that needs to be operated as indicated by the function carried in command 1. The backup ID may be a cache ID configured in the memory and called by the CPU for backing up and storing data.

Exemplarily, the functions of the vertex data-related instructions intercepted by the interception module include the following functions as an example:

glGenbuffers(GL_ARRAY_BUFFER, 1)//Create a buffer for vertex data, the buffer ID is 1;

glBindbuffer(GL_ARRAY_BUFFER, 1)//Bind the buffer whose ID is 1 for the vertex data;

glBufferdata(GL_ARRAY_BUFFER, data1)//write data1 to the bound buffer;

glBufferSubData(GL_ARRAY_BUFFER, data2)//Update the data in the bound buffer to data2.

Then, the original ID in this example can be 1. Take the corresponding backup ID as 11 as an example.

According to the intercepted glGenbuffers(GL_ARRAY_BUFFER, 1), the data processing module can create a cache whose ID is 11 corresponding to 1 in the backup cache.

According to the intercepted glBindbuffer(GL_ARRAY_BUFFER, 1), the data processing module can control subsequent operations to be performed on the cache whose ID is 11 corresponding to 1.

According to the intercepted glBufferdata (GL_ARRAY_BUFFER, data1), the data processing module can write data1 into the storage space with ID 11 in the backup cache.

According to the intercepted glBufferSubData(GL_ARRAY_BUFFER, data2), the data processing module can update data2 to the storage space with ID 11 in the backup cache.

Among them, data1 and data2 can include vertex data, such as vertex coordinates, normal vectors of vertices, etc.

In this way, the backup storage of the vertex data-related instructions carried in the command 1 can be realized.

Similar to the backup storage of vertex data, the data processing module can also backup and store the MVP matrix.

Exemplarily, the functions of the MVP matrix-related instructions intercepted by the interception module include the following functions as an example:

glGenbuffers(GL_UNIFORM_BUFFER, 2)//Create a cache for uniform variables (such as MVP matrix), and the cache ID is 2;

glBindbuffer(GL_UNIFORM_BUFFER, 2)//Bind the buffer with ID 2 for the uniform variable (such as MVP matrix);

glBufferdata(GL_UNIFORM_BUFFER, data3)//write data3 to the bound buffer;

glBufferSubData(GL_UNIFORM_BUFFER, data4)//Update the data in the bound buffer to data4.

Wherein, data3 and data4 may include the MVP matrix.

Then, the original ID in this example can be 2. Take the corresponding backup ID as 22 as an example.

According to the intercepted glGenbuffers(GL_UNIFORM_BUFFER, 2), the data processing module can create a cache whose ID is 22 corresponding to 2 in the backup cache. The backup cache whose ID is 22 may be used to store data corresponding to the uniform variable, for example, the uniform variable may include an MVP matrix.

According to the intercepted glBindbuffer(GL_UNIFORM_BUFFER, 2), the data processing module can control the subsequent operations to be performed on the cache whose ID is 22 corresponding to 2.

According to the intercepted glBufferdata (GL_UNIFORM_BUFFER, data3), the data processing module can write data3 into the storage space whose ID is 22 in the backup cache.

According to the intercepted glBufferSubData(GL_UNIFORM_BUFFER, data4), the data processing module can update data4 to the storage space with ID 22 in the backup cache.

Wherein, data3 and data4 may include MVP matrix, such as M matrix, VP matrix and so on.

In this way, the backup storage of the MVP matrix-related instructions carried in the command 1 can be realized.

In addition to backing up instructions and related data, the data processing module can also store a jump table including the correspondence between the original ID and the backup ID, so that the backup storage can be accurately found according to the ID in the command issued by the subsequent application. The ID of the corresponding data in .

For example, Table 1 below shows an example of a jump table.

Table 1

original ID backup ID 1 11 2 twenty two ... ...

Based on Table 1, when the game application issues an instruction to perform an operation on the cache whose ID is 1, the electronic device can determine that corresponding data can be stored in the storage space whose backup ID is 11. Similarly, when the game application issues an instruction to perform an operation on the cache whose ID is 2, the electronic device can determine that corresponding data can be stored in the storage space whose backup ID is 22.

It should be noted that, in order to ensure the smooth execution of command 1, in this example, the interception module can also call back the command that does not carry parameter 1 or parameter 2 (such as callback command a) to the graphics library, so as to call the graphics library The corresponding interface controls components in the hardware layer (such as GPU) to perform corresponding functions. In other implementations, the data processing module can also call back the instructions from the interception module (such as callback instruction b) to the graphics library after completing the backup storage of the vertex data and the MVP matrix, so that by calling the corresponding interface in the graphics library , to control components in the hardware layer (such as GPU) to execute corresponding functions.

In this way, while realizing the backup storage of the vertex data and the MVP matrix, the complete execution of the command 1 can also be realized, so that the execution of the commands issued by the subsequent game application will not be affected.

In the embodiment of the present application, according to the data stored in the backup during the loading process, the electronic device can realize the related processing of the commands during the running of the game, so as to determine the vertex coordinates of the model to be drawn by the current command (that is, the current Drawcall) and the corresponding vertex coordinates of the current Drawcall. The MVP matrix.

For example, refer to FIG. 7 . While the game is running, the game application can issue commands 2 . Among them, the command 2 can be used to implement including drawing the model.

It can be understood that, in combination with the foregoing description, the vertex data and the MVP matrix of the model to be drawn may have been loaded through command 1. That is to say, these data may already be stored in the memory space that the GPU can call. Then, in command 2, you can draw the corresponding model by indicating the vertex data to be used and the relevant parameters of the MVP matrix, so that the GPU can obtain the corresponding vertex data and MVP matrix from the loaded data.

In this example, the command 2 issued by the game may include an instruction stream composed of multiple instructions (that is, functions). In order to realize the above functions, the command 2 may include the function of binding the cache, the function indicating the vertex data parsing method, and the function indicating the related parameters of the MVP matrix.

As a possible implementation, at least the following instructions can be included in command 2:

Binding cache functions, such as glBindbuffer function;

The glVertexAttribPoint function used to indicate how the vertex data is parsed;

The glBindBufferRange function used to indicate the parameters related to the MVP matrix.

Then, the interception module can be used to intercept the above instructions during the running of the game. In some embodiments, the interception module can intercept vertex-related instructions including glBindbuffer function and glVertexAttribPoint function. The interception module can also intercept MVP-related instructions including the glBindBufferRange function. The interception module can also transmit the vertex-related instructions and the MVP-related instructions to the data processing module for analysis.

Similar to the above description of data interception in the loading process, in this example, the interception module may also have certain data parsing capabilities. Then, the interception module can intercept the glBindbuffer function carrying parameter 1 (such as GL_ELEMENT_ARRAY_BUFFER and/or GL_ARRAY_BUFFER), and the glVertexAttribPoint function. The interception module can also intercept the glBindBufferRange function carrying parameter 2 (such as GL_UNIFORM_BUFFER).

Then, the data processing module can determine the vertex coordinates to be used by the current Drawcall after receiving the glBindbuffer function and the glVertexAttribPoint function carrying parameter 1.

In some embodiments of the present application, the data processing module may determine the vertex coordinates to be used by the current Drawcall in combination with the locally stored parameter 3 . The parameter 3 may be determined through static analysis of the current game.

It can be understood that the vertex data may include multiple items of vertex related data. Different data can be stored in different attributes. For example, attitude0 can be used to store vertex coordinates, attribute1 can be used to store vertex normal vectors, etc. For a game application, the ID (such as 0) of the attribute used to store the vertex coordinates generally does not change during the working process. Therefore, in this example, parameter 3 may include the ID (eg 0) of the attribute used by the current game to store the vertex coordinates.

In this way, after receiving the instruction from the interception module, the data processing module can determine whether the glVertexAttribPoint function is used for vertex data transmission according to whether the attribute ID of the stored data indicated by the glVertexAttribPoint function matches parameter 3. When the attribute ID indicated by the glVertexAttribPoint function matches parameter 3, the data processing module can determine that the glVertexAttribPoint function is a function related to vertex coordinates.

Then, the data processing module can determine the storage location of the vertex data indicated by the current Drawcall according to the cache ID bound by the glBindbuffer function before the glVertexAttribPoint function. Based on this, the data processing module can determine the storage location of the vertex coordinates indicated by the current Drawcall in the cache of the backup storage.

For example, the instructions received by the data processing module from the interception module include:

glBindbuffer(GL_ARRAY_BUFFER, 1)//Bind buffer with ID 1;

glVertexAttribPoint(0, 3, GL_FLOAT, GL_FALSE, 3*sizeof(float), (void*_0))// Each parameter is indicated in sequence: the attribute value is 0, and each group of data includes 3 values (such as XYZ), The type is a floating point type and does not need to be standardized. The step size is 3*4 and the starting address is 0.

Take the property ID indicated by parameter 3 as 0 as an example.

The data processing module can determine that the attribute ID (such as 0) indicated by the glVertexAttribPoint function matches the parameter 3, then the ID bound to the glVertexAttribPoint function, that is, the buffer with ID 1 bound to glBindbuffer(GL_ARRAY_BUFFER, 1) before the glVertexAttribPoint function , which is used to transfer the vertex data cache of the model to be drawn by the current Drawcall.

Next, the data processing module can determine, according to the correspondence between the original ID and the backup ID (as shown in Table 1), that in the data stored in the backup, the cache ID for storing the vertex coordinates of the model to be drawn by the current Drawcall is 11. Next, determine the vertex coordinates of each vertex of the model according to the attribute ID indicated by the glVertexAttribPoint function (that is, the ID indicated by parameter 3) and the offset.

The above description elaborates in detail how to obtain the vertex coordinates of the model to be drawn by the current Drawcall (referred to as the current model for short). The acquisition of the MVP matrix corresponding to the current model is described below.

Combined with the aforementioned game running process, the interception module can also intercept the glBindBufferRange function with parameter 2 (such as GL_UNIFORM_BUFFER).

The glBindBufferRange function carrying parameter 2 can be transmitted to the data processing module for processing.

Exemplarily, in some embodiments of the present application, the data processing module may combine the locally stored parameter 4 to determine the MVP matrix to be used by the current Drawcall. The parameter 4 may be determined by static analysis of the current game.

In some embodiments, the parameter 4 may include an offset to store the M matrix, and/or the VP matrix. It can be understood that, for a certain game application, the offset of the M matrix and the offset of the VP matrix are generally not changed in the corresponding cache. Therefore, in this example, the data processing module can determine whether the intercepted glBindBufferRange function is used to transmit the MVP matrix corresponding to the current Drawcall according to the parameter 4 and the parameters carried by the intercepted glBindBufferRange function.

When the data processing module determines that the offset carried by the glBindBufferRange function matches parameter 4, the data processing module can determine that the glBindBufferRange function is used to transmit the MVP matrix corresponding to the current Drawcall.

Then the data processing module can determine the cache ID of the MVP matrix corresponding to the current Drawcall stored in the backup storage according to the cache ID indicated by the glBindBufferRange function and the above jump table (such as Table 1). In addition, the specific storage locations of the M matrix and the VP matrix in the cache can also be determined according to the offset indicated by the glBindBufferRange function (or the offset indicated by parameter 4).

For example, the instructions received by the data processing module from the interception module include:

glBindBufferRange(GL_UNIFORM_BUFFER, 2, 0, 152)//The meanings indicated by each parameter in sequence are: the target is GL_UNIFORM_BUFFER, the buffer ID is 2, the first offset address is 0, and the data size is 152.

Take the first offset address indicated by parameter 4 as 0 and the data size as 152 as an example.

The data processing module can determine that the offset indicated by the currently intercepted glBindBufferRange function matches parameter 4, then the glBindBufferRange function is used to transfer the MVP matrix corresponding to the current Drawcall. Therefore, according to the ID (such as 2) indicated by the glBindBufferRange function, the data processing module can determine that the native ID corresponding to the MVP matrix corresponding to the current Drawcall is 2.

Next, the data processing module can determine, according to the correspondence between the original ID and the backup ID (as shown in Table 1), that the cache ID of the MVP matrix storing the model to be drawn by the current Drawcall is 22 in the data stored in the backup. According to the offset indicated by the glBindBufferRange function (that is, the offset indicated by parameter 4), the data processing module can determine the MVP matrix of the model.

It should be noted that, in this example, similar to the aforementioned backup and storage process, the interception module can also implement the callback of commands not related to vertex data and MVP matrix in command 2 by calling back command c. The data processing module can call back instructions related to vertex data and MVP matrix in command 2 by calling back instruction d.

In order to enable those skilled in the art to more clearly understand the solution provided by the embodiment of the present application, regarding the backup storage process of data during the game loading process, and the determination process of vertex data and MVP matrix during the game running process, the following is from the instruction flow From a perspective, the functions of each module in the process are illustrated.

For example, please refer to FIG. 8 , which is a schematic flowchart of data backup and storage when the game is loaded (or started) according to the embodiment of the present application. As shown in Figure 8, the process can include:

S801. After receiving an instruction P, determine whether the instruction P is a vertex-related instruction.

In this example, when the game is loaded, the game application can issue an instruction P, and the instruction P can be used to load vertex data.

If the instruction P is a vertex-related instruction, the following S802 is executed. If the instruction P is not a vertex-related instruction, the following S811 is executed.

With reference to the foregoing example, the vertex-related instruction may include a function carrying parameter 1. For example, the glGenbuffers function, glBindbuffer function, glBufferdata function, glBufferSubData function, etc. that carry parameter 1.

S802. The interception module sends a vertex-related instruction to the data processing module. Wherein, the vertex-related instruction may include instruction P.

S803. The data processing module controls the memory backup to store vertex-related data.

Among them, the memory in this example can correspond to the cache in the above example. The memory (or cache) may be a part of the storage space in the internal storage medium of the electronic device that can be called by the CPU.

In the above S801-S803, the interception, analysis, storage and other processes of vertex data are similar to the specific implementation in the foregoing description, and will not be repeated here. In this way, the backup storage of vertex data can be realized.

In some embodiments of the present application, during the execution of S803, the data processing module may also store the correspondence between the original ID and the backup ID for subsequent data call.

In the process of executing S801-S803, the electronic device can also implement the normal operation of the command issued by the game application through the command callback, such as realizing the normal loading of data. Exemplarily, the process may include:

S811. The interception module calls back the instruction 1-1 to the graphics library.

Exemplarily, in the case that the instruction P is not a vertex-related instruction, the interception module may call back the instruction to the graphics library. For example, the instruction 1-1 may include instruction P.

S812. The graphics library calls related APIs 1-1. Wherein, the relevant API 1-1 may be an API called to realize the function of the callback instruction 1-1.

S813. The graphics library sends an instruction 1-1 to the GPU. The code corresponding to API 1-1 may be carried in the instruction 1-1.

S814. The GPU executes the operation corresponding to the instruction 1-1.

Similar to the interception module, the data processing module can also call back vertex-related instructions. Exemplarily, the process may include:

S815. The data processing module calls back the instruction 1-2 to the graphics library. The instructions 1-2 may include instructions intercepted by the interception module and transmitted to the data processing module. For example, the instruction may include a vertex-related instruction in instruction P.

S816. The graphics library calls related APIs 1-2. The API 1-2 may be an API called to realize the function of the callback instruction 1-2.

S817. The graphics library sends instructions 1-2 to the GPU. The code corresponding to API 1-2 can be carried in the instruction 1-2.

S818. The GPU executes the operation corresponding to the instruction 1-2.

Thus, through S811-S818, the callback of all the data in the instruction P is realized, thereby successfully realizing the loading of the data in the instruction P.

In this example, the electronic device may also implement backup storage of the MVP matrix through the following process. Exemplarily, the process may include:

S804. After receiving the instruction Q, determine whether the instruction Q is an MVP-related instruction.

In this example, when the game is loaded, the game application can issue an instruction Q, which can be used to load MVP data. The interception module may intercept the function carrying parameter 2 included in the instruction Q.

For example, the glGenbuffers function, glBindbuffer function, glBufferdata function, glBufferSubData function, etc. that carry parameter 2.

In the case that the command Q is an MVP-related command, the following S805 is executed. If the instruction P is not an MVP-related instruction, the following S821 is executed.

S805. The interception module sends MVP related instructions to the data processing module. Wherein, the MVP-related instruction may include instruction Q.

S806. The data processing module controls the memory to back up and store MVP-related data.

In the above S804-S806, the interception, analysis, storage and other processes of the MVP matrix are similar to the specific implementation in the foregoing description, and will not be repeated here. In this way, the backup storage of the MVP matrix can be realized.

Similar to the aforementioned callback process on vertex data, in this example, the electronic device can also call back the MVP instruction to implement normal loading of the MVP matrix. Exemplarily, the process may include:

S821. The interception module calls back the instruction 2-1 to the graphics library.

Exemplarily, in the case that the instruction P is not an MVP-related instruction, the interception module may call back the instruction to the graphics library. For example, the instruction 2-1 may include instruction Q.

S822. The graphics library calls related APIs 2-1. Wherein, the relevant API 2-1 may be an API called to realize the function of the callback instruction 2-1.

S823. The graphics library sends the instruction 2-1 to the GPU. The code corresponding to API 2-1 may be carried in the instruction 2-1.

S824. The GPU executes the operation corresponding to the instruction 2-1.

Similar to the interception module, the data processing module can also call back MVP-related instructions. Exemplarily, the process may include:

S825. The data processing module calls back the instruction 2-2 to the graphics library. The instruction 2-2 may include an instruction intercepted by the intercepting module and transmitted to the data processing module. For example, the instruction may include an MVP-related instruction in instruction Q.

S826. The graphic library calls related API 2-2. The API 2-2 may be an API called to realize the function of the callback instruction 2-2.

S827. The graphics library sends the instruction 2-2 to the GPU. The code corresponding to API 2-2 may be carried in the instruction 2-2.

S828. The GPU executes the operation corresponding to the instruction 2-2.

Thus, through S821-S828, the callback of all the instructions in the instruction Q is realized, thereby successfully realizing the loading of the data in the instruction Q.

In the following, from the perspective of instruction flow, an example will be given to illustrate the process of determining the vertex coordinates and MVP matrix corresponding to the model to be drawn by the current Drawcall during the running of the game.

Exemplarily, with reference to FIG. 9 , it is taken as an example that a game application issues an instruction N to indicate vertex data of the current model during running. The process can include:

S901. After receiving the instruction N, the interception module determines whether the instruction N is a vertex-related instruction.

In this example, the vertex-related instruction may be an instruction carried in the instruction N for indicating the vertex data corresponding to the model to be drawn by the current Drawcall. In some embodiments, these instructions may carry a parameter 1 related to vertex data. With reference to the foregoing description, the vertex-related instructions may include a glVertexAttribPoint function, a corresponding glBindbuffer function, and the like.

In a case where the instruction N is a vertex-related instruction, the following S902 may be executed. In the case that the instruction N is not a vertex-related instruction, a callback to the instruction N may be executed, such as executing S911.

S902. The interception module sends a vertex-related instruction to the data processing module. Wherein, the vertex-related instruction may include instruction N.

S903. The data processing module determines a buffer ID for transmitting vertex data.

Exemplarily, the data processing module may determine that the currently intercepted function is used to indicate the vertex data corresponding to the model to be drawn by the current Drawcall when the attribute ID indicated by the glVertexAttribPoint function intercepted by the interception module matches the preset parameter 3.

Then the data processing module can determine the buffer ID for transmitting vertex data according to the glBindbuffer function before the glVertexAttribPoint function. The cache ID can be an original ID.

S904. The data processing module determines a storage location in the backup storage of the vertex data.

Exemplarily, the data processing module may determine the backup ID corresponding to the current original ID according to a jump table including the correspondence between the original ID and the backup ID. Thus, the buffer ID of the vertex data corresponding to the model to be drawn by the current Drawcall in the backup storage can be determined. In addition, the storage location of each vertex coordinate in the backup storage can be accurately obtained according to the attribute ID and the offset of the vertex coordinate.

In some embodiments of the present application, after determining the storage location in the backup storage of the vertex data, the data processing module may transfer the vertex coordinates to the preset location 1 for subsequent calling. In some other embodiments, after determining the storage location in the backup storage of the vertex data, the data processing module may mark the location in the backup storage that stores the coordinates of the vertex corresponding to the current Drawcall for subsequent calls.

It should be noted that, in order to ensure the normal operation of the instruction N, in the embodiment of the present application, the interception module and the data processing module may also perform instruction callback. Exemplarily, the process may include:

S911. The interception module calls back the instruction 3-1 to the graphics library.

Exemplarily, in the case that the instruction N is not a vertex-related instruction, this step may be performed to implement a callback to the instruction N. In some embodiments, the instruction 3-1 may include instruction N.

S912. The graphics library calls related APIs 3-1. The API 3-1 may be an API in a graphics library for realizing the function corresponding to the instruction 3-1.

S913. The graphics library sends an instruction 3-1 to the GPU. The instruction 3-1 may include codes corresponding to the API 3-1.

S914. The GPU executes the operation corresponding to the instruction 3-1.

It should be noted that, in some embodiments, the execution of the above S911-S914 may be executed after S902.

Similar to the interception module, the data processing module can also execute instruction callbacks. Exemplarily, the process may include:

S915. The data processing module calls back the instruction 3-2 to the graphics library.

Exemplarily, the instruction 3-2 may include the vertex-related instructions intercepted by the interception module in the instruction N.

S916. The graphic library calls related API 3-2. The API 3-2 may be an API in a graphics library for realizing the function corresponding to the instruction 3-2.

S917. The graphics library sends the instruction 3-2 to the GPU. The instructions 3-2 may include codes corresponding to the API 3-2.

S918. The GPU executes the operation corresponding to the instruction 3-2.

In some embodiments, the above S915-S918 may be performed after S904.

Thus, the full callback of instruction N is realized, thereby ensuring the normal execution of instruction N.

In this example, the electronic device may also determine the storage location of the MVP matrix corresponding to the current Drawcall in the backup storage through the following process. Exemplarily, with reference to FIG. 9 , it is taken as an example that a game application issues an instruction M to indicate the MVP matrix of the current model during running. The process can include:

S905. After receiving the instruction M, the interception module determines whether the instruction N is an MVP-related instruction.

In this example, the MVP-related instruction may be an instruction carried in the instruction M for indicating the MVP matrix corresponding to the model to be drawn by the current Drawcall. In some embodiments, these instructions may carry a parameter 2 related to the MVP matrix. With reference to the foregoing description, the MVP-related instructions may include a glBindBufferRange function and the like.

In the case that the instruction M is an MVP-related instruction, the following S906 may be executed. In the case that the instruction M is not an MVP-related instruction, a callback to the instruction M may be executed, such as executing S921.

S906. The interception module sends MVP related instructions to the data processing module.

S907. The data processing module determines the cache ID of the transmission MVP matrix.

Exemplarily, the data processing module can determine that the currently intercepted function is used to indicate the MVP matrix corresponding to the model to be drawn by the current Drawcall when the offset indicated by the glBindBufferRange function intercepted by the interception module matches the preset parameter 4 .

Then the data processing module can determine the cache ID for transmitting the MVP matrix according to the cache ID indicated by the glBindBufferRange function. The cache ID can be an original ID.

S908. The data processing module determines a storage location in the backup storage of the MVP matrix.

Exemplarily, the data processing module may determine the backup ID corresponding to the current original ID according to a jump table including the correspondence between the original ID and the backup ID. From this, the cache ID of the MVP matrix corresponding to the model to be drawn by the current Drawcall in the backup storage can be determined. In addition, the M matrix and/or the storage location of the VP matrix in the backup storage can be accurately acquired according to the offset of the MVP matrix.

In some embodiments of the present application, similar to the processing of the aforementioned vertex data, after determining the storage location in the backup storage of the MVP matrix, the data processing module can dump the MVP matrix to the preset location 2 for subsequent recall. In some other embodiments, after determining the storage location in the backup storage of the MVP matrix, the data processing module may mark the location in the backup storage that stores the MVP matrix corresponding to the current Drawcall for subsequent calling.

It should be noted that, in order to ensure the normal operation of the instruction M, in the embodiment of the present application, the interception module and the data processing module may also perform instruction callback. Exemplarily, the process may include:

S921. The interception module calls back the instruction 4-1 to the graphics library.

Exemplarily, in the case that the instruction M is not an MVP-related instruction, this step may be performed to implement a callback to the instruction M. In some embodiments, the instruction 4-1 may include an instruction M.

S922. The graphic library calls related API 4-1. The API 4-1 may be an API in the graphics library for realizing the function corresponding to the instruction 4-1.

S923. The graphics library sends the instruction 4-1 to the GPU. The instruction 4-1 may include codes corresponding to the API 4-1.

S924. The GPU executes the operation corresponding to the instruction 4-1.

It should be noted that, in some embodiments, the execution of the above S921-S924 may be executed after S906.

Similar to the interception module, the data processing module can also execute instruction callbacks. Exemplarily, the process may include:

S925. The data processing module calls back the instruction 4-2 to the graphics library.

Exemplarily, the instruction 4-2 may include an MVP-related instruction intercepted by the interception module in the instruction M.

S926. The graphic library calls related API 4-2. The API 4-2 may be an API in the graphics library for realizing the function corresponding to the instruction 4-2.

S927. The graphics library sends the instruction 4-2 to the GPU. The instructions 4-2 may include codes corresponding to the API 4-2.

S928. The GPU executes the operation corresponding to the instruction 4-2.

In some embodiments, the above S925-S928 may be performed after S908.

Thus, the full callback of instruction N is realized, thereby ensuring the normal execution of instruction M.

Through the above example, the electronic device can obtain the vertex coordinates and the MVP matrix of the model corresponding to the current Drawcall.

In the embodiment of the present application, the electronic device can also calculate the depth of the current model based on this, and then use a reasonable shading rate to render the model.

Exemplarily, in conjunction with the calculation module in FIG. 10 , it can be used to determine the depth of the graph (or model) to be drawn by the current drawcall according to the vertex coordinates and the MVP matrix stored in the memory.

Wherein, the depth of the model can be determined by the depth of some or all vertices on the model.

The method of determining the depth of vertices will be firstly described below.

Exemplarily, in some embodiments, when the interception module intercepts the Drawelement, the calculation module may be triggered to calculate the depth of each vertex.

It can be understood that, in the embodiment of the present application, the calculation module may cooperate with the decision-making module to determine whether to perform low-rate shading on some vertices by calculating the depth of each vertex. Therefore, in different implementations of the present application, the triggering mechanism for the calculation module to calculate the depth of each vertex may be different. As long as the calculation module and/or the decision module can know the vertex depth corresponding to the drawelement before delivering the corresponding drawelement to the GPU. For example, in this example, the calculation of the depth of each vertex by the calculation module may be triggered by intercepting the drawelement by the interception module. In other implementations, the calculation module can also trigger the calculation of the depth of each vertex after the vertex coordinates and MVP matrix parameters are cached in the memory, so that the calculation module can store the depth information obtained by calculation in the memory, so that Subsequent use.

In some embodiments of the present application, the calculation module may call corresponding data according to the vertex coordinates corresponding to the current Drawcall determined by the data processing module and the storage address of the MVP matrix, and calculate and obtain the depth information of each vertex.

Exemplarily, take the local coordinates of vertex 1 indicated by the vertex coordinates as (x1, y1, z1) as an example. The calculation module can perform MVP transformation on the local coordinates of the vertex 1 according to the following formula (1), and then determine the depth of the vertex 1 in the clipping space (or screen space).

Clipping coordinates=P·V·M·(x1, y1, z1)...Formula (1).

Take the obtained clipping coordinates of vertex 1 as (x2, y2, z2) as an example. Then the calculation module can determine that the depth of the current vertex 1 is z2.

Similar to the above vertex 1, the calculation module can perform depth calculation on other vertices of the model corresponding to the current drawcall to obtain the depth of each vertex. It can be understood that the depth may be the depth in the clipping space. The deeper the depth, the farther the vertex (or model) is from the user's observation position during the current frame image display process, so when performing lower rate shading on the vertex (or model) with a larger depth, it will not be perceived by the user.

In combination with the above calculation method of vertex depth, in some embodiments of the present application, the electronic device can determine the depth of the model corresponding to the current drawcall accordingly.

Exemplarily, as a possible implementation manner, the electronic device may calculate the depths of all vertices of the current model, and use an average of these depths as the depth of the model. For example, when the depth calculation is triggered, the calculation module can retrieve all the currently stored vertex coordinates from the memory, and perform matrix transformation according to the above formula (1) according to the MVP matrix, so as to obtain the clipping coordinates of each vertex, and then obtain The depth of each vertex. The electronic device can take the average of these vertex depths as the depth of the model.

It should be noted that, generally speaking, the number of vertices of a model corresponding to a drawcall is very large. Therefore, in some implementation manners of the present application, the electronic device may select a part of the calculated depth from the vertex coordinates stored in the memory, and determine the depth of the current model accordingly. So as to achieve the effect of reducing the calculation cost.

Exemplarily, the electronic device may randomly select n vertices from the vertices of the model corresponding to the current drawcall according to a preset number of vertices (such as n). The depth of the current model is obtained by determining the depth of the n vertices and taking the mean value.

As a possible implementation, the calculation module can recall n random vertex coordinates among all currently stored vertex coordinates from the memory when the depth calculation is triggered. The calculation module can perform matrix transformation on the coordinates of the n vertices according to the above formula (1) according to the MVP matrix, so as to obtain the clipping coordinates of the n vertices. The calculation module can obtain the depth of each vertex according to the clipping coordinates of the n vertices. The calculation module can also perform weighted average calculation according to the obtained depths of the n vertices, so as to obtain the depth of the model. Wherein, the weight involved in the weighted average may be preset or flexibly adjusted.

It should be noted that, in the above example, the random selection of n vertices is realized as an example when the calculation module retrieves the coordinates of vertices from the memory. In other implementations, the calculation module can also recall all stored vertex coordinates from the memory. When performing depth calculations, the calculation module can select n vertex coordinates from all the recalled vertex coordinates for depth calculation. Other vertices The coordinates are not processed or discarded. In this way, the random selection of n vertex coordinates can also be realized.

In some other embodiments of the present application, the electronic device may adopt different strategies to determine the depth of the model according to the size of the model corresponding to the current drawcall. Among them, the size of the model can be identified by the number of vertices.

Take the interception module to intercept the depth calculation of the drawelement trigger calculation module as an example.

The electronic device can determine the number of vertices of the current model according to the GLsizeel count value indicated in the intercepted drawelement.

The electronic device may determine that the current model is a large model or a small model according to a size relationship between the determined number of vertices and a preset threshold of the number of vertices. For example, the vertex quantity threshold may be 5000. Then, when the count value is greater than 5000, the current model is considered to be a large model. Correspondingly, when the count value is less than 5000, the current model is considered to be a small model.

The embodiments of the present application provide different depth calculation strategies for large models and small models, so as to calculate the model depth more reasonably.

Exemplarily, for a large model, the electronic device may select 6 vertices with the largest or smallest x, y, and z components among all vertices of the current model. The depth of the 6 vertices is calculated, and the average value is taken to determine the depth of the current model.

For example, among the vertices of the current model, the coordinates of the vertex P1 with the largest x component are (x _max , y1, z1), the coordinates of the vertex P2 with the smallest x component are (x _min , y2, z2), and the coordinates of the vertex P3 with the largest y component is (x3, y _max , z3), the coordinate of the vertex P4 with the smallest y component is (x4, y _min , z4), the coordinate of the vertex P5 with the largest z component is (x5, y5, z _max ), and the vertex P6 with the smallest x component The coordinates are (x6, y6, z _min ) as an example.

After MVP matrix transformation, the coordinates of P1-P6 under the clipping coordinates are: P1(x ^' _max , y ^' 1, z ^' 1), P2(x ^' _min , y ^' 2, z ^' 2), P3( ^x'3 , ^y'max , ^z'3 ), P4( ^x'4 _, ^y'min _, ^z'4 ), P5(x'5 ^{, y'5} , ^z'max ), P6( ^x'6 , _y ^' 6, ^z'min ) _.

Then, the electronic device can determine that the depth of the current model is (z ^' 1+z ^' 2+z ^' 3+z ^' 4+z ^' _max +z ^' _min )/6.

In other implementations, for a small model, the electronic device may randomly select n vertices from all vertices of the current model. Calculate the depth of these n vertices, and take the mean value as the depth of the current model. For example, the n may be pre-configured, for example, n may be any integer between 20-100.

It can be understood that, in the above example, for a large model, the electronic device can select the edgemost vertices to calculate the depth of the model, so that when the depth of different vertices of the model differs greatly, the calculation method can be used to truly Accurately identify the depth of large models. For a small model, due to the limitation of the model size, the depth of different vertices of the model will not be greatly different. Therefore, in this example, a preset number (or a random number) of vertices can be selected from the model. Calculate their depths respectively and take the mean value to determine the depth of the model.

It should be noted that, in the above example, the z coordinate value under the clipping coordinates is taken as the depth of the corresponding vertex as an example for illustration. In other embodiments of the present application, the depth of the vertex may also be determined according to the z coordinate value under the clipping coordinates. Exemplarily, the vertex depth may be determined according to the z-coordinate of the vertex under normalized device coordinates (Normalized Device Coordinate, NDC). For example, after obtaining the coordinates of the vertex in the clip space, the z coordinate in the NDC space can be obtained by perspective division. In this way, the z-coordinate in the NDC space can be used to identify the depth of the vertex. In addition, in the above example, determining the depth of the model according to the depth z of the clipping space is used to identify the distance of the model in the image displayed to the user, which is only a possible implementation. In other examples of the present application, the distance of the model (that is, the depth of the model) can also be identified by the model length based on the observation coordinates of the model in the observation space. That is to say, the distance of the model can also be determined according to the distance from the model to the camera in the observation space.

It can be seen that in the above description of the present application, each module in the electronic device can obtain the depth of the model by obtaining the vertex coordinates of the model and processing these vertex coordinates. In some other implementations of the present application, each module in the electronic device may also determine the depth of the model by obtaining information of other models. Exemplarily, take obtaining the bounding box information of the model to determine the depth of the model as an example. The interception module can be used to intercept bounding box information-related instructions in the rendering command, and transmit the information to the data processing module. The data processing module and/or the calculation module can be used to determine the depth of the current model according to the bounding box information obtained by interception.

In this way, the electronic device can obtain the depth of the model corresponding to the current drawcall through the calculation of the calculation module.

In this embodiment of the present application, the calculation module may cooperate with the decision module to determine whether to perform variable rate shading on the model corresponding to the current drawcall, for example, perform lower rate shading on the model corresponding to the current drawcall.

Exemplarily, the decision module may determine the shading rate performed on the model according to the depth of the model calculated and obtained by the calculation module. In the embodiment of the present application, the decision module may be configured with (or retrieved from memory) shading rates corresponding to different model depths. If the shading rate corresponding to the current model depth is a lower shading rate (that is, the shading rate is less than the shading rate in units of 1 pixel), the decision-making module can transfer the drawelement corresponding to the drawcall to the hardware layer, from the system library The corresponding variable rate shading API is called in the graphics library of .

As an example, Table 2 below shows a schematic representation of the correspondence relationship between model depth and shading rate.

Table 2

model depth shader rate [1, 10) 1×1 (10, 50] 2×1 (50, 100] 2×2 (100, 200] 4×2 greater than 200 4×4

As shown in Table 2, when the depth of the model is less than 10, the decision-making module can determine that the current model uses a shading rate of 1×1 for shading. That is, the depth of the current model is small, and there is no need to use the variable rate shading mechanism. Then the decision-making module does not need to call the variable rate shading API, but directly sends the rendering command corresponding to the rendering command issued by the game application to the hardware layer (sent to the GPU of the hardware layer as follows), so that the GPU uses 1 pixel as the Unit to perform rendering operations for the current rendering command.

In the case that the model depth is greater than 10 and less than or equal to 50, the decision module may determine that the current model uses 2×1 shading rate for shading. Then the decision-making module can call the variable rate shading API and set the shading rate to 2×1 so that the GPU can pass the 2×1 The shading rate performs the rendering operations of the current rendering command.

In the case that the model depth is greater than 50 and less than or equal to 100, the decision module may determine that the current model uses 2×2 shading rate for shading. Then the decision-making module can call the variable rate shading API and set the shading rate to 2×2, so that the GPU can pass the 2×2 The shading rate performs the rendering operations of the current rendering command.

In the case that the model depth is greater than 100 and less than or equal to 200, the decision module may determine that the current model uses 4×2 shading rate for shading. Then the decision-making module can call the variable rate shading API and set the shading rate to 4×2, so that the GPU can pass the 4×2 The shading rate performs the rendering operations of the current rendering command.

If the depth of the model is greater than 200, the decision module may determine that the current model uses 4×4 shading rate for shading. Then the decision-making module can call the variable rate shading API and set the shading rate to 4×4, so that the GPU can pass the 4×4 The shading rate performs the rendering operations of the current rendering command.

In addition, the interception module can also call back other instructions of command 2 to the graphics library, so that by calling the corresponding API in the graphics library, the corresponding rendering operation can be performed at the shading rate determined in the above solution.

In order to enable those skilled in the art to more clearly understand the solutions provided by the embodiments of the present application, the following continues to describe the solutions provided by the embodiments of the present application from the perspective of instruction flow. Exemplarily, referring to FIG. 11, the process may include:

S1101. The interception module and the data processing module determine the vertex coordinates and the MVP matrix of the current model in the backup storage.

Exemplarily, the interception module may determine the storage location of the vertex coordinates of the current model in the backup storage according to the instruction N issued by the game application. The data processing module can determine the storage location of the MVP matrix of the current model in the backup storage according to the instruction M issued by the game application.

For the specific execution process of S1101 , reference may be made to the relevant description in FIG. 9 , which will not be repeated here.

S1102. The calculation module obtains the vertex coordinates and the MVP matrix from the memory.

Exemplarily, in the case that the data processing module stores the vertex coordinates and the MVP matrix in a specific location, the calculation module can obtain the vertex coordinates and the MVP matrix from the specific location. In some other embodiments, when the data processing module identifies the storage locations of the vertex coordinates and the MVP matrix, the calculation module can obtain the vertex coordinates and the MVP matrix from the memory according to the identification.

In some embodiments of the present application, the computing module may obtain the vertex coordinates and the MVP matrix after the game application issues the instruction R. Wherein, the instruction R may include one or more Drawelements.

Exemplarily, the interception module may instruct the calculation module to execute S1102 when receiving the instruction R.

S1103. The calculation module calculates the model depth.

S1104. The calculation module sends the model depth to the decision module.

S1105. The decision module determines the shading rate 1 of the current model.

Wherein, the process of determining the model depth and determining the corresponding shading rate (such as shading rate 1) according to the model depth can refer to the above description, and will not be repeated here.

S1106. The decision-making module calls the corresponding API of the graphics library. Exemplarily, the corresponding API may be variable rate shading API 1 corresponding to shading rate 1.

S1107. The graphics library correspondingly calls the variable rate shading API 1.

In this way, the electronic device can call the corresponding API in the graphics library according to the depth of the model, so as to control the components in the hardware layer (such as GPU) to perform corresponding variable-rate shading.

With reference to the foregoing description, in this example, the electronic device may, in addition to executing the above S1101-S1107, perform an instruction callback on the received command of the game application, so that the instruction is correctly executed.

Exemplarily, the process may include:

S1108, the interception module and the data processing module call back related instructions.

Wherein, the relevant instructions may include a callback instruction 3-1 and a callback instruction 3-2 corresponding to the instruction N, and a callback instruction 4-1 and a callback instruction 4-2 corresponding to the instruction M. Please refer to the description of FIG. 9 for the callback process and content, and details are not repeated here.

In this example, the related instruction may also include an instruction R.

S1109. The graphics library calls the corresponding API. The corresponding API may include an API for realizing the functions indicated by the instruction N, the instruction M, and the instruction R.

S1110. The graphics library sends instruction 2 to the GPU. The instruction 2 may include the above-mentioned corresponding code obtained by calling the API. Exemplarily, the instruction 2 may include the corresponding code obtained by calling the variable rate shading API 1, and the variable rate shading of the shading rate 1 can be realized based on the code.

S1111. The GPU executes a rendering operation corresponding to the shading rate 1.

In this way, the electronic device can flexibly adjust the use mechanism of variable rate shading according to the depth of the model in the current rendering command at the granularity of drawcall, so as to achieve the goal of reducing the rendering overhead in the shading process without affecting the user experience. Effect.

It should be understood that, in the above example, a game application sends a command to draw a model as an example, and the specific implementation details thereof are described. In the actual implementation process, the game application can issue multiple commands to draw multiple models. In the drawing process of each model, the above solution can also be used to implement adaptive variable rate shading according to the depth of each model.

Exemplarily, command 1 is used to load data, command 2 is used to draw model 1, and command 3 is used to draw model 2 as an example.

Referring to FIG. 12 , the game application can issue command 1, command 2, and command 3 respectively.

For command 1, each module set at the framework layer can perform backup storage of vertex data and/or MVP matrix according to the scheme shown in FIG. 6 or FIG. 8 . At the same time, each module in the framework layer can also combine the callback mechanism shown in Figure 8 to realize the loading of data corresponding to command 1. For example, the framework layer can call the corresponding API in the graphics library according to the command 1, so as to issue the command 1 corresponding to the command 1 to the GPU, so as to complete the data loading.

For command 2, each module set in the framework layer can determine the corresponding shading rate of model 1 according to the depth of model 1 to be drawn by the command 2 according to the scheme shown in Fig. 7, Fig. 9, Fig. 10 and Fig. 11 (such as shader rate 1). Combined with the callback mechanism, the framework layer can call the corresponding API in the graphics library based on command 2 and shading rate 1, so as to issue command 2 to the GPU. According to the instruction 2, the GPU can draw the model 1 according to the shading parameters of the shading rate 1. Thus, model 1 as shown in FIG. 12 can be obtained. As an example, the model 1 may be stored in a pre-allocated frame buffer corresponding to the command 2 (such as frame buffer 1).

Similar to command 2, for command 3, each module set at the framework layer can be determined according to the scheme shown in Figure 7, Figure 9, Figure 10 and Figure 11, and then determine the model according to the depth of the model 2 to be drawn by the command 3 2 corresponds to a shading rate (such as shading rate 2). Combined with the callback mechanism, the framework layer can call the corresponding API in the graphics library based on command 3 and shading rate 2, so as to issue command 3 to the GPU. According to the instruction 3, the GPU can draw the model 2 according to the shading parameters of the shading rate 2. Thus, the model 2 shown in FIG. 12 can be obtained. As an example, the model 1 may be stored in a pre-allocated frame buffer corresponding to the command 2 (such as frame buffer 2).

Take model 1 and model 2 belonging to the same frame image as an example. Before displaying the frame image, the GPU can also render the images on frame buffer 1 and frame buffer 2 to a basic frame buffer (such as frame buffer 0), so as to obtain the full content of the frame image. Exemplarily, referring to FIG. 13 , the GPU can render model 1 and model 2 on the same canvas.

It should be understood that due to the adoption of the variable rate shading mechanism provided by the embodiment of the present application, the depths of models with different depths may be different. Take the example where the depth of model 1 is less than that of model 2. That is, model 1 is closer to the user, while model 2 is farther away from the user. In combination with the foregoing description, for model 1 with a smaller depth, it can be shaded with a higher shading rate. For example, take model 1 with a shading rate of 1×1 as an example. Then, in the coloring process of the model 1, each pixel included in the model 1 may be colored in units of one pixel. Correspondingly, for the model 2 with a larger depth, it can be shaded with a lower shading rate. For example, take model 2 with a shading rate of 2×2. Then, in the coloring process of the model 2, the pixels included in the model 2 can be colored in units of 2×2 pixels.

Thus, after the coloring operation is completed (for example, after all rendering operations are completed and the two models are rendered on a canvas), among the pixels included in model 1, it can be seen that the color of each pixel can be different. For the pixels included in model 2, there will be a large number of adjacent 2×2 pixels with the same or similar colors. As a result, the color of the model closer to the user can be rendered and displayed more accurately. Correspondingly, the color granularity of the model farther away from the user can be larger, thereby saving the corresponding rendering overhead.

The foregoing mainly introduces the solutions provided by the embodiments of the present application from the perspective of electronic devices. In order to realize the above functions, it includes corresponding hardware structures and/or software modules for performing various functions. Those skilled in the art should easily realize that the present application can be implemented in the form of hardware or a combination of hardware and computer software in combination with the units and algorithm steps of each example described in the embodiments disclosed herein. Whether a certain function is executed by hardware or computer software drives hardware depends on the specific application and design constraints of the technical solution. Skilled artisans may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present application.

The embodiments of the present application may divide the involved devices into functional modules according to the above method example, for example, each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software function modules. It should be noted that the division of modules in the embodiment of the present application is schematic, and is only a logical function division, and there may be other division methods in actual implementation.

Exemplarily, FIG. 14 shows a schematic composition diagram of an electronic device 1400 . As shown in FIG. 14 , the electronic device 1400 may include: a processor 1401 and a memory 1402 . The memory 1402 is used to store computer-executable instructions. Exemplarily, in some embodiments, when the processor 1401 executes the instruction stored in the memory 1402, the electronic device 1400 may be made to execute the image rendering method shown in any one of the above embodiments.

It should be noted that all relevant content of the steps involved in the above method embodiments can be referred to the function description of the corresponding function module, and will not be repeated here.

The functions or actions or operations or steps in the above-mentioned embodiments may be fully or partially implemented by software, hardware, firmware or any combination thereof. When implemented using a software program, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application will be generated in whole or in part. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website, computer, server, or data center Transmission to another website site, computer, server or data center by wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer, or may include one or more data storage devices such as servers and data centers that can be integrated with the medium. The available medium may be a magnetic medium (such as a floppy disk, a hard disk, or a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (solid state disk, SSD)) and the like.

Although the application has been described in conjunction with specific features and embodiments thereof, it will be apparent that various modifications and combinations can be made thereto without departing from the spirit and scope of the application. Accordingly, the specification and drawings are merely illustrative of the application as defined by the appended claims and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of this application. Obviously, those skilled in the art can make various changes and modifications to the application without departing from the spirit and scope of the application. In this way, if these modifications and variations of the application fall within the scope of the claims of the application and their equivalent technologies, the application also intends to include these modifications and variations.

Claims (19)

1. An image rendering method, characterized in that it is applied to an electronic device, and an application program is installed in the electronic device, and the method comprises: Acquiring a first rendering command issued by the application, where the first rendering command is used to draw the first model; determining a first shading rate for the first model; Rendering the first model according to the first rendering command and the first shading rate. 2. The method according to claim 1, wherein the determining the first shading rate of the first model comprises: According to the first rendering command, determine the depth of the first model, the depth of the first model is used to identify the distance between the first model and the observer in the observation space or clipping space, the first model the greater the depth of a model, the greater the distance between the first model and the observer; The first shading rate is determined according to the depth of the first model. 3. The method according to claim 2, wherein the determining the depth of the first model according to the first rendering command comprises: According to the first rendering command, determine the depth of n vertices of the first model; determining the depth of the first model according to the depths of the n vertices; Wherein, n vertices are included in the vertices of the first model. 4. The method according to any one of claims 1-3, wherein the method further comprises: Acquiring a second rendering command issued by the application, where the second rendering command is used to draw a second model; determining a second shading rate for the second model; rendering the second model according to the second rendering command and the second shading rate; The first model and the second model are included in the same frame image, and the first shading rate and the second shading rate are different. 5. The method according to any one of claims 1-4, wherein an interception module and a data processing module are configured in the electronic device, The acquiring the first rendering command issued by the application includes; The interception module intercepts the first rendering command; The interception module transmits the intercepted first function and second function to the data processing module; The first function is a function carrying a first parameter, the second function is a function carrying a second parameter, the first parameter is a parameter carried in the process of transferring vertex data by the application program, and the second parameter is the parameter carried in the process of the application transferring the MVP matrix. 6. The method according to claim 5, further comprising: The interception module calls back functions other than the first function and the second function in the first rendering command to the graphics library of the electronic device. 7. The method according to claim 5 or 6, wherein, after the interception module transmits the intercepted first function and the second function to the data processing module, the method further comprises: The data processing module determines the first storage position of the vertex coordinates corresponding to the first model and the second storage position of the first MVP matrix according to the first function and the second function, and the first A storage location and the second storage location are included in a first storage area that is accessible to a processor of the electronic device. 8. The method according to claim 7, further comprising: The data processing module calls back the first function and the second function to the graphics library of the electronic device. 9. The method according to claim 7 or 8, wherein before obtaining the first rendering command issued by the application, the method further comprises: The interception module intercepts a third rendering command issued by the application, the third rendering command is used to store the first data used during the running of the application in the second storage area, and the second The storage area is a storage area used by the GPU of the electronic device, and the first data includes vertex coordinates of the first model and the first MVP matrix; The interception module transmits the third function and the fourth function to the data processing module, the third function carries the first parameter, and the fourth function carries the second parameter; The data processing module stores data corresponding to the third function and the fourth function in the first storage area. 10. The method according to claim 9, further comprising: The interception module calls back functions other than the third function and the fourth function in the third rendering command to the graphics library of the electronic device. 11. The method according to claim 9 or 10, further comprising: The data processing module calls back the third function and the fourth function to the graphics library of the electronic device. 12. The method according to any one of claims 9-11, further comprising: The data processing module stores a first correspondence, and the first correspondence is used to indicate a correspondence between storage addresses of the same data in the first storage area and storage addresses in the second storage area. 13. The method according to any one of claims 7-12, wherein the data processing module determines the coordinates of the vertices corresponding to the first model according to the first function and the second function The first storage location, and the second storage location of the first MVP matrix include: The data processing module determines a first storage location of vertex coordinates corresponding to the first model according to the cache location indicated by the first function and the first correspondence; The data processing module determines a second storage location of the first MVP matrix according to the cache location indicated by the second function and the first corresponding relationship. 14. The method according to claim 13, wherein a calculation module is further configured in the electronic device, and determining the depths of n vertices of the first model according to the first rendering command includes : When the interception module intercepts the drawing element Drawelement in the first rendering command, The calculation module calculates the depths of the n vertices according to the vertex coordinates corresponding to the first model and the first MVP matrix; wherein, the vertex data corresponding to the first model is obtained by the calculation module from the The first storage location is obtained, and the first MVP matrix is obtained by the calculation module from the second storage location. 15. The method according to claim 14, wherein the determining the depth of the first model according to the depths of the n vertices comprises: The calculation module determines the depth of the first model as an average value of the depths of the n vertices. 16. The method according to claim 14 or 15, wherein a decision-making module is also configured in the electronic device, and the method further comprises: The calculation module transmits the depth of the first model to the decision module; The decision-making module searches for an entry matching the depth of the first model from the preset second correspondence according to the depth of the first model; The decision module determines the shading rate indicated by the matched entry as the first shading rate. 17. The method of claim 16, further comprising: The decision module transmits shading instructions to the graphics library indicating the first shading rate. 18. The method according to claim 17, wherein the drawing the first model according to the first rendering command and the first shading rate comprises: The graphics library calls the first application programming interface API corresponding to the first rendering command according to the callback function of the interception module and the data processing module; The graphics library calls a first variable rate shading API corresponding to the first shading rate according to the shading instruction; The graphics library sends a rendering instruction to the GPU of the electronic device according to the first API and the first variable rate shading API, so that the GPU executes the first shading rate according to the first shading rate. A model rendering operation. 19. An electronic device, characterized in that the electronic device comprises one or more processors and one or more memories; the one or more memories are coupled to the one or more processors, and the one or more or a plurality of memories storing computer instructions; When the one or more processors execute the computer instructions, the electronic device is made to execute the image rendering method according to any one of claims 1-18.