CN112783584B - Layer display method and device of terminal system, vehicle equipment and storage device - Google Patents
Layer display method and device of terminal system, vehicle equipment and storage device Download PDFInfo
- Publication number
- CN112783584B CN112783584B CN201911082445.8A CN201911082445A CN112783584B CN 112783584 B CN112783584 B CN 112783584B CN 201911082445 A CN201911082445 A CN 201911082445A CN 112783584 B CN112783584 B CN 112783584B
- Authority
- CN
- China
- Prior art keywords
- layer
- layers
- display
- image
- displayed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
- G06F3/1407—General aspects irrespective of display type, e.g. determination of decimal point position, display with fixed or driving decimal point, suppression of non-significant zeros
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- General Engineering & Computer Science (AREA)
- Controls And Circuits For Display Device (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The application discloses a layer display method and device of a terminal system, a vehicle machine device and a storage device, wherein the method comprises the following steps: the layer management module creates a layer with a Z sequence value for each application program to be displayed; acquiring N first layers in all layers to be displayed, wherein the data format of the first layers is not identified by a graphic processor, and the layer of the layers to be displayed, of which the data format can be identified by the graphic processor, is a second layer; forming M image layer groups by a plurality of second image layers with Z-order values smaller than the maximum Z-order value in the N first image layers, respectively combining the M image layer groups through an image processor to obtain M special image layers, and outputting the M special image layers to M overlay display layers for displaying; and respectively sending the N first image layers to different layers of overlay display layers for display through the HWC module. By the scheme, the display performance of the layer to be displayed, of which the data format is not identified by the graphic processor, can be guaranteed.
Description
Technical Field
The present application relates to the field of layer display technologies, and in particular, to a layer display method and apparatus for a terminal system, a vehicle device, and a storage apparatus.
Background
With the continuous iteration of the terminal system version, functions such as a "floating window mode", a "multi-window" and the like are added successively, taking an android system as an example, the number of layers (layers) which can be displayed simultaneously in the android system is more and more, and the number of hardware resources (overlayers) which can be actually displayed on a chip is limited. There are generally two options available in the design of display schemes: the GPU (graphics processing unit) synthesizes each layer and sends it to the overlay display, or the HWC (software module for assisting the synthesis function in the HardWare company, android system) directly uses the overlay display. The comparison between the two cases is as follows:
| mode of Synthesis | Power consumption | Performance of | Other limitations |
| HWC | Is low in | Height of | Limited number of synthesis layers |
| GPU | Height of | Is low in | The number of synthesis layers is not limited |
Taking a back-up (back) function on a car machine as an example, for the back car function on the car machine, video display generally requires high performance, so that the occurrence of frame loss can be avoided as far as possible, and therefore, the synthesis mode of the HWC used for displaying the video layer of the back car is reasonable. While for other layers the synthesis can in principle be used either with HWC synthesis or with HWC synthesis, otherwise with GPU synthesis. Then a problem arises: when the number of layers to be displayed by the system is too many and the z-order of the video layer of the baccarat (the value is larger and closer to the human eye as the arrangement hierarchical relationship of the layers from top to bottom is higher), the situation that the video layer of the baccarat is synthesized by using the GPU occurs due to the limited overlay resources, and at this time, the problem that the GPU cannot identify the format (data format) of the video layer of the baccarat or cannot ensure the performance after the GPU can identify the format through other hardware resources conversion occurs.
Disclosure of Invention
The technical problem mainly solved by the application is to provide a layer display method and device of a terminal system, a vehicle-mounted device and a storage device, which can ensure the display performance of a layer to be displayed, the data format of which is not identified by a graphic processor.
In order to solve the above technical problem, a first aspect of the present application provides a layer display method for a terminal system, where a layer management module is disposed in the terminal system; the method comprises the following steps: the layer management module creates a layer for each application program to be displayed, wherein the layer of each application program has a Z-order value; acquiring N first layers in all layers to be displayed, wherein the data format of the first layers is not identified by a graphic processor, and the layer of which the data format can be identified by the graphic processor in the layers to be displayed is a second layer; forming M image layer groups by a plurality of second image layers with Z-order values smaller than the maximum Z-order value in the N first image layers, respectively combining the M image layer groups through a graphics processor to obtain M special image layers, and outputting the M special image layers to M overlay display layers for displaying; m is more than or equal to 1 and less than or equal to N, each image layer group refers to a set of all continuous second image layers of each section of Z-order value in the Z-order value sequence; and respectively sending the N first image layers to different layers of overlay display layers for display through the HWC module.
In order to solve the foregoing technical problem, a second aspect of the present application provides a layer display apparatus of a terminal system, including a graphics processor, an HWC module, a layer management module, and a display module, which are coupled to each other: the layer management module is used for: creating a layer for each application program to be displayed, wherein the layer of each application program has a Z-order value; acquiring N first layers in all layers to be displayed, wherein the data format of the first layers is not identified by a graphic processor, and the layer of which the data format can be identified by the graphic processor in the layers to be displayed is a second layer; forming M image layer groups by a plurality of second image layers with Z-order values smaller than the maximum Z-order value in the N first image layers, respectively combining the M image layer groups through a graphics processor to obtain M special image layers, and outputting the M special image layers to M overlay display layers for display, wherein M is larger than or equal to 1 and smaller than or equal to N, each image layer group refers to a set of all continuous second image layers with each section of Z-order value in a Z-order value sequence; and respectively sending the N first layers to overlay display layers of different layers for display through the HWC module.
In order to solve the above technical problem, a third aspect of the present application provides a layer display apparatus of a terminal system, including a memory and a processor coupled to each other; the processor is configured to execute the program data stored by the memory to implement the method as described above.
In order to solve the foregoing technical problem, a fourth aspect of the present application provides a car machine device, including any one of the above image layer display devices of a terminal system.
In order to solve the above technical problem, a fifth aspect of the present application provides a storage device storing program data capable of being executed by a processor, the program data being used for implementing the method as described above.
The invention has the beneficial effects that: different from the situation of the prior art, the layer display method of the terminal system comprises the following steps: the layer management module creates a layer for each application program to be displayed, wherein the layer of each application program has a Z-order value; acquiring N first layers in all layers to be displayed, wherein the data format of the first layers is not identified by a graphic processor, and the layer of which the data format can be identified by the graphic processor in the layers to be displayed is a second layer; forming N image layer groups by a plurality of second image layers with Z-order values smaller than the maximum Z-order value in the N first image layers, respectively combining the N image layer groups through an image processor to obtain N special image layers, and outputting the N special image layers to N overlay display layers for displaying; respectively sending the N first layers to overlay display layers of different layers through an HWC module for displaying; and M is more than or equal to 1 and less than or equal to N, and each image layer group refers to the set of all continuous second image layers of each section of Z-order value in the Z-order value sequence. According to the method and the device, when the layer management module acquires that at least one first layer with a data format which is not identified by the graphics processor exists, a plurality of second layers with continuous Z-sequence values under the at least one first layer are combined through the graphics processor to form a special layer, the special layer is output to the overlay display layer to be displayed, the data format of the second layer can be identified by the graphics processor, and each first layer is respectively sent to the overlay display layers of different layers through the HWC module to be displayed, so that each first layer with the data format which is not identified by the graphics processor can be sent to a single overlay display layer through the HWC module to be displayed, and the display performance of the layer to be displayed, the data format of which is not identified by the graphics processor, can be guaranteed.
Drawings
Fig. 1 is a schematic structural diagram of a layer display method of a terminal system in the prior art;
fig. 2 is a schematic flowchart of an embodiment of a layer display method of a terminal system according to the present application;
fig. 3a is a schematic structural diagram in an application scene of the layer display method of the terminal system provided in the present application;
FIG. 3b is a schematic diagram of a structure in an application scenario of a layer display method of a terminal system in the prior art;
fig. 4 is a schematic flowchart of another embodiment of a layer display method of a terminal system provided in the present application;
fig. 5 is a schematic structural diagram in another application scenario of the layer display method of the terminal system provided in the present application;
fig. 6 is a schematic structural diagram of an embodiment of a layer display apparatus of a terminal system provided in the present application;
fig. 7 is a schematic structural diagram of another embodiment of a layer display apparatus of a terminal system provided in the present application;
fig. 8 is a schematic structural diagram of an embodiment of a vehicle-mounted device provided in the present application;
fig. 9 is a schematic structural diagram of an embodiment of a memory device provided in the present application.
Detailed Description
The following describes in detail the embodiments of the present application with reference to the drawings attached hereto.
In the following description, for purposes of explanation and not limitation, specific details are set forth such as particular system structures, interfaces, techniques, etc. in order to provide a thorough understanding of the present application.
The terms "system" and "network" are often used interchangeably herein. The term "and/or" herein is merely an association describing an associated object, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship. Further, the term "plurality" herein means two or more than two.
Regarding the display scheme of the terminal system, an application program requests a WindowManager (desktop window manager) to display or create a window, the WindowManager initiates a Layer creation request to a Layer management service (for example, a surfefinger service in an Android system, which is responsible for synthesizing layers) according to the request of the application program, and the Layer management service creates a Layer; the method comprises the steps that a WindowManager calculates a proper Z-Order value (Z-Order) according to an operation state of an application in a terminal system and Layer information of other applications, and the Z-Order value is set in a Layer management service and represents the arrangement hierarchical relationship of each Layer from top to bottom, the larger the value is, the closer the value is to the human eye, namely the position of each Layer on a Z axis is determined by the Z-Order value of each Layer; it can be understood that the sequence of the layer layers in each layer, that is, the sequence of the layer layers when displayed on the screen, is the sequence of the layer layers when the layer layers in each layer are displayed farther forward, and the sequence of the layer layers when the layer layers in each layer are displayed farther forward is the sequence of the layer layers when the layer layers in each layer are displayed farther forward, which is the sequence of Z-Order. The layer with larger Z-Order value can block the layer with smaller Z-Order value; when an area on a screen needs to be refreshed (Update), the layer with the larger Z-Order in each layer is refreshed first, and the layer with the smaller Z-Order is refreshed later. Overlay is a display technology of digital video, which allows digital signals to be directly output to a display screen through a video memory without being processed by a display chip. The most important purpose of the Overlay display mode is to optimize video playback. Because different videos have different reference hues, brightness, contrast and saturation, for different computers and different video files, various display attributes need to be adjusted in order to obtain the best display effect, and the common display mode obviously cannot be competent, so that the Overlay display mode is used for independent adjustment. The Overlay display mode has the characteristics of high speed, good image quality, less occupied system resources and the like, and is very suitable for video playing. With the continuous iteration of the terminal system version, the number of layers which can be displayed simultaneously in the terminal system is more and more, and the number of hardware resources which can be actually displayed on a chip is limited. Therefore, in an overlay display mode of a terminal system, please refer to fig. 1, where fig. 1 is a schematic structural diagram of a layer display method of a terminal system in the prior art, where a car machine system is an Android system, the car machine system currently has 10 layer layers to be displayed, and the current layer layers are, in order of a z-order value from small to large, a wall paper layer, a launcher layer, a status bar layer, a PIP activity layer, a PIP UI (picture in picture interface) layer, a PIP video layer, a backhaul UI layer, a hotel video layer, and a bluetooth layer, and the number of the overlay layers actually available for display on a chip is only 5, and the current layer, the overlay display layer, the overlay layer2, the overlay layer, and the overlay layer are, respectively, from bottom to top, layer0, overlay 1, overlay layer2, overlay 3, and overlay layer4; at this time, part of the layer layers need to be synthesized and then sent to the overlay for display, since the synthesis of the layer layers can use HWC synthesis in principle and HWC synthesis, and in other cases, GPU synthesis is used, 4 layers to be displayed with smaller z-order value can use HWC synthesis, and 6 layers to be displayed with larger z-order value are synthesized into a buffer layer (FrameBuffer Target) by the GPU, so that of 10 layer layers to be displayed, the wall paper layer, the launcher layer, the status bar layer and the PIP activity layer with smaller z-order value are respectively directly displayed by using overlay 0, overlay 1, overlay 2 and overlay 3 through the HWC, while the PIP UI layer, the PIP video layer, the backup UI layer, the wechat video layer and the bluetoothth layer with larger z-order value are combined by the GPU to form a buffer layer and then sent to the overlay 4 for display, namely, all the layer layers beyond the hardware resource limit are combined by the SurfaceFlinger to form a buffer layer by the GPU and then sent to the overlay together with the bottom 4 layer layers for display. However, for the function of the back car on the car, the video display generally requires high performance, so that it is possible to avoid frame loss as much as possible, so the way of synthesizing the video layer of the back car using HWC is reasonable, but as shown in fig. 1, the back car video layer uses GPU to synthesize the video layer of the back car because of limited overlay resources, and the format of the back car video is generally a format of YUV (a color coding method), such as YUYV/uy/YV 12, etc., while the GPU, such as ARM's mali (a high-end GPU), supports YUV only for both formats, such as 12 and NV21, so that the back car video display is black because the GPU cannot recognize the format of the back car video layer, or converts the format of the back car video into NV12 through other hardware, and the probability of frame loss is greatly reduced.
Based on the principle, the application provides a layer display method of a terminal system. Referring to fig. 2, fig. 2 is a schematic flowchart illustrating an embodiment of a layer displaying method of a terminal system according to the present application. The terminal system in the present application is provided with a layer management module therein, and the layer display method of the terminal system in this embodiment includes the following steps:
s201: the layer management module creates a layer for each application program to be displayed, wherein the layer of each application program has a Z-order value.
It can be understood that a layer management module and a plurality of application programs are arranged in the terminal system, and the layer management module is used for realizing layer management service. In an embodiment, each application program may initiate a window display request to a WindowManager service in the terminal system, and the WindowManager service initiates a Layer creation request to the Layer management module after receiving the window display request, so that the Layer management module may create a Layer for each application program to be displayed. In addition, the Z-order value refers to a Z-order value corresponding to each Layer of each application program, the Z-order value represents the front-back order of each Layer when displayed on a terminal display screen, and the Layer with the larger Z-order value is more front when displayed; the WindowManager can calculate the Z-order value of the Layer of each application program according to the running states of all application programs in the terminal system and the Layer information of other application programs.
S202: and acquiring N first layers in all the layers to be displayed, wherein the data format of the first layers is not identified by the graphic processor, and the layer of which the data format can be identified by the graphic processor in the layers to be displayed is a second layer.
It can be understood that the layer management module may detect whether a first layer exists in all layers to be displayed, where the first layer is a layer to be displayed whose data format is not recognized by the graphics processor, and the second layer is a layer to be displayed whose data format is recognized by the graphics processor. Therefore, the layer management module may obtain N first layers in all layers to be displayed, where N may be an integer greater than or equal to 0.
S203: forming M image layer groups by a plurality of second image layers with Z-order values smaller than the maximum Z-order value in the N first image layers, respectively combining the M image layer groups through a graphics processor to obtain M special image layers, and outputting the M special image layers to M overlay display layers for displaying; and M is more than or equal to 1 and less than or equal to N, and each graph layer group refers to the set of all second graph layers with continuous Z-order values in each segment in the Z-order value sequence.
It can be understood that, in the existing design, when the number of layers that need to be displayed simultaneously in the terminal system is large and the number of overlay display layers that can be displayed is small, part of the layer layers to be displayed need to be synthesized into one buffer layer by the graphics processor and then sent to the overlay display layer for display, so as to reduce the requirement for the number of overlay display layers. Because the first layer is a layer to be displayed, the data format of which is not identified by the graphics processor, and in order to avoid the situation that the first layer cannot occupy one overlayer display layer alone and needs to be displayed after being synthesized into a buffer layer by the graphics processor, a plurality of second layers, the Z-order values of which are smaller than the maximum Z-order value in the N first layers, can be formed into M layer groups, then the M layer groups are respectively combined by the graphics processor to obtain M special layers, so that the M special layers can be displayed only by occupying the M overlayer display layers, a condition can be provided for each first layer to independently occupy one overlayer display layer for displaying, that is, the first layer does not need to be synthesized into the buffer layer by the graphics processor, and the situation that the data format of the first layer is not identified can be avoided. It can be understood that, in the present application, a special layer may be preset in the graphics management module, and after a graphics processor combines a plurality of second layers into a layer group and combines the layer group, the layer group may be stored in the special layer. In this embodiment, when there are more than two first layers, in a corresponding plurality of map layer groups, the Z-order values of a plurality of second layers of each map layer group may be smaller than the minimum Z-order value of the first layer, or smaller than the maximum Z-order value of the first layer and larger than the minimum Z-order value of the first layer.
S204: and respectively sending the N first image layers to overlay display layers of different layers for display through an HWC module.
The method comprises the steps that M image layer groups are formed by a plurality of second image layers with Z-sequence values smaller than the maximum Z-sequence value in N first image layers, the M image layer groups are combined through a graphics processor to obtain M special image layers and output to M overlay display layers for display, meanwhile, the N first image layers can be sent to the overlay display layers of different layers through a HWC module to be displayed, at the moment, each first image layer can occupy one overlay display layer independently for display, and therefore the display performance of the first image layers can be guaranteed. It can be understood that, in practical applications, the operations of sending the special layer and the first layer to the overlay display layer for displaying are performed synchronously, and steps S203 and S204 in the present application do not limit the order of the operations of sending the special layer and the first layer to the overlay display layer for displaying.
It can be understood that, in the present application, the Z-order value of the layer to be displayed by the overlay display layer located on the upper layer is greater than the Z-order value of the layer to be displayed by the overlay display layer located on the lower layer. Because the Z-order value of the layer to be displayed, which is displayed by the overlay display layer located on the upper layer, is greater than the Z-order value of the layer to be displayed, which is displayed by the overlay display layer located on the lower layer, the sequence displayed by each layer to be displayed is not disturbed, that is, the original display sequence of each layer is not destroyed, that is, the layer located below the first layer can be covered on the first layer to cause the first layer to be completely or partially shielded.
In this embodiment, a layer management module creates a layer for each application program to be displayed, then obtains N first layers in all the layers to be displayed, combines a plurality of second layers with Z-order values smaller than the maximum Z-order value in the N first layers into M layer groups, respectively combines the M layer groups through a graphics processor to obtain M special layers, outputs the M special layers to M overlay display layers for display, and sends each of the N first layers to overlay display layers of different layers through an HWC module for display; the Z-order value of the layer to be displayed by the overlay display layer positioned on the upper layer is larger than the Z-order value of the layer to be displayed by the overlay display layer positioned on the lower layer. When the layer management module acquires that at least one first layer with a data format which is not identified by the graphics processor exists, a plurality of second layers with continuous Z-sequence values under the at least one first layer are combined through the graphics processor to form a special layer, the special layer is output to an overlay display layer to be displayed, the data format of the second layer can be identified by the graphics processor, meanwhile, the first layers are respectively sent to overlay display layers of different layers through the HWC module to be displayed, each first layer with the data format which is not identified by the graphics processor can be sent to a single overlay display layer through the HWC module to be displayed, and the display performance of the layer to be displayed, of which the data format is not identified by the graphics processor, can be guaranteed.
Referring to fig. 3a, fig. 3a is a schematic structural diagram of an application scenario of the layer display method of the terminal system according to the present application. As shown in fig. 3a, in a specific application scenario, the car machine system respectively includes layer layers to be displayed, such as a wall paper layer, a launcher layer, a status bar layer, a PIP activity layer, a PIP UI layer, a PIP video layer, a back car UI layer, a wechat video layer, and a blue button layer, in order of z-order value from small to large, and the number of layers actually available for display on the chip is only 5, and is respectively overlay 0, overlay 1, overlay 2, overlay 3, and overlay 4 in order from bottom to top. Different from the application scenario shown in fig. 1, in the implementation scenario, a special layer is created in advance in a layer management module (for example, a surfefinger module) by default, and when the layer management module detects that a backhaul video layer exists, the layer management module synthesizes all layer layers having z-order values smaller than that of the backhaul video layer into one layer in advance by using a GPU, and then sends the layer layers to the overlay along with other layer layers for display. That is, the wall paper layer, the launcher layer, the status bar layer, the PIP activity layer, the PIP UI layer, and the PIP video layer are merged by the GPU to form a specific layer, and the baccaravideo layer, the baccara UI layer, the wechat video layer, and the bluetoothlayer are respectively synthesized by the HWC, and then the specific layer is sent to the display of the specific layer together with the baccara video layer, the baccara UI layer, the wechat video layer, and the bluetoothlayer, and at this time, the number of the specific layers actually available for display on the chip is 5, and does not exceed the hardware limit, so that it can be guaranteed that the usage of the specific layers does not exceed the hardware limit, and the video can be displayed by the synthesized video layer (hwplayer) with high performance. In addition, as shown in fig. 3a, if the car machine system further includes a plurality of other layer layers having z-order values larger than the z-order value of the bluetooth layer, in order to avoid exceeding the limitation of the hardware resource, the bluetooth layer and the other layer layers having z-order values larger than the z-order value of the bluetooth layer need to be merged by the GPU to form a buffer layer (FrameBuffer Target), and then the specific layer is sent to the overlay for display together with the back video layer, the back ui layer, the feature video layer and the buffer layer, so that the high performance requirement of the video display of the back (vehicle backing) function can be ensured by using the HWC to synthesize and display the video layer of the back without exceeding the limitation of the hardware resource.
It should be noted that, in the application, all layer layers with z-order values smaller than that of the bacckari video layer are synthesized into one special layer by using the GPU in advance through the layer management module, and while the video layer of the bacckari is synthesized by using the HWC under the condition that the limitation of hardware resources is not exceeded, the display sequence of each layer originally can be prevented from being damaged, and the situation that all or part of the bacckari video is shielded as a result of the layer originally located below the bacckari video covering the bacckari video is avoided. As shown in fig. 3b, fig. 3b is a schematic structural diagram in an application scenario of the layer display method of the terminal system in the prior art, the car machine system includes 10 layers to be displayed, namely, a wall layer, a launcher layer, a status bar layer, a PIP activity layer, a PIP UI layer, a PIP video layer, a backup UI layer, a feature video layer, and a bluetooth layer, respectively, in order of a z-order value from small to large, and the number of layers actually available for display on the chip is only 5, and it can be seen that, in a case where the backup video layer occupies one layer by default, the wall layer, the launcher layer, and the status bar layer can occupy one layer of layers, respectively, and the remaining layers of layers can occupy one layer of layers by a GPU (GPU) when the layers of layers can be used, the layers of layers can be combined into one buffer layer (buffer layer), so that the original content of layers is less than the original map layer, and the original content of layers can be combined into a buffer layer by the GPU layer (buffer video layer), and the original layer, so that the original content of the map layer and the buffer layer can be combined into a PIP activity layer. And the layer management module synthesizes all layer layers with the z-order values smaller than that of the back video layer into a special layer in advance by using the GPU, so that the original display sequence of each layer cannot be damaged.
Referring to fig. 4, fig. 4 is a schematic flowchart illustrating a layer display method of a terminal system according to another embodiment of the present application. The layer display method of the terminal system in the embodiment includes the following steps:
s401: the layer management module creates a layer for each application program to be displayed, wherein the layer of each application program has a Z-order value.
S402: and acquiring N first layers in all the layers to be displayed, wherein the data format of the first layers is not identified by the graphic processor, and the layer of which the data format can be identified by the graphic processor in the layers to be displayed is a second layer.
S403: forming M image layer groups by a plurality of second image layers with Z-order values smaller than the maximum Z-order value in the N first image layers, respectively combining the M image layer groups through a graphics processor to obtain M special image layers, and outputting the M special image layers to M overlay display layers for displaying; and M is more than or equal to 1 and less than or equal to N, and each graph layer group refers to the set of all second graph layers with continuous Z-order values in each segment in the Z-order value sequence.
In this implementation scenario, steps S401 to S403 provided in this embodiment are substantially similar to steps S201 to S203 in the previous embodiment of the layer display method of the terminal system provided in this application, and are not described herein again.
In this embodiment, the Z-order value of the nth first layer is greater than the Z-order value of the N-1 st first layer, where N is greater than or equal to 1. It can be understood that when N =1, that is, there is only one first image layer, the Z-order value of the first image layer is greater than the Z-order value of the 0 th first image layer, and at this time, the Z-order value of the 0 th first image layer is 0. Further, the present embodiment is also different from the previous embodiment in that the overlay display layer in the present embodiment has Q number. The embodiment further comprises the following steps:
s404: and judging whether the number P of all second layers with the Z-order value larger than that of the Nth first layer is larger than Q- (M + N) or not.
S405: if the Z-sequence value is larger than the preset value, the graphics processor merges the P- (Q- (M + N)) +1 second layers with the largest Z-sequence value to obtain a buffer layer, and sends the buffer layer to the uppermost layer of the overlay display layer for displaying.
It can be understood that, through steps S401 to S403, N first layers and M special layers are obtained, so that the N first layers and the N special layers need to occupy (M + N) overlay display layers, and the number of the overlay display layers that can also be used at this time is Q- (M + N). In step S403, all second layers with a Z-order value smaller than that of the nth first layer are merged into M special layers, at this time, if there is a second layer with a Z-order value larger than that of the nth first layer, it is determined whether the number P of all second layers with a Z-order value larger than that of the nth first layer is larger than Q- (M + N), and if P is larger than Q- (M + N), it indicates that P second layers cannot be respectively sent to overlay display layers of different layers through the HWC module for display, and then, the P- (Q- (M + N)) +1 second layers with the largest Z-order value need to be merged by the graphics processor to obtain a buffer layer, and sent to the overlay display layer on the uppermost layer for display.
S406: and respectively sending the N first layers and each second layer, of which the Z sequence value is larger than that of the Nth first layer and is not combined to form a buffer layer, to an overlay display layer through the HWC module for display.
It can be understood that, after P- (Q- (M + N)) +1 second layers with the largest Z-order value are merged to obtain one buffer layer, Q- (M + N) -1 second layers in all second layers with Z-order values larger than that of the nth first layer are not merged to form the buffer layer, at this time, each of the Q- (M + N) -1 second layers is regarded as one buffer layer, and then the N first layers, the M special feature layers, and the Q- (M + N) buffer layers may be respectively sent to an overlay display layer for display.
Further, this embodiment further includes:
s407: and if the Z-order value of the second image layer is not greater than the Z-order value of the Nth first image layer, respectively sending the N first image layers and each second image layer with the Z-order value greater than the Z-order value of the Nth first image layer to an overlay display layer through an HWC module for display.
In the step S404, it is determined whether the number P of all second layers having Z-sequence values greater than the Z-sequence value of the nth first layer is greater than Q- (M + N), and if P is not greater than Q- (M + N), it indicates that P second layers can all be respectively sent to overlay display layers of different layers through the HWC module for display, so that a buffer layer does not need to be obtained by merging multiple second layers through the graphics processor, and then N first layers and each second layer having Z-sequence values greater than the Z-sequence value of the nth first layer can be respectively sent to overlay display layers through the HWC module for display.
It is understood that, in another embodiment, the layer display method of the terminal system may also include the following steps:
s501: the layer management module creates a layer for each application program to be displayed, wherein the layer of each application program has a Z-order value.
S502: and acquiring N first layers in all the layers to be displayed, wherein the data format of the first layers is not identified by the graphic processor, and the layer of which the data format can be identified by the graphic processor in the layers to be displayed is a second layer.
S503: forming M image layer groups by a plurality of second image layers with Z-order values smaller than the maximum Z-order value in the N first image layers, respectively combining the M image layer groups through a graphics processor to obtain M special image layers, and outputting the M special image layers to M overlay display layers for displaying; and M is more than or equal to 1 and less than or equal to N, and each graph layer group refers to the set of all second graph layers with continuous Z-order values in each segment in the Z-order value sequence.
S504: and respectively sending the N first image layers to different layers of overlay display layers for display through the HWC module.
In this implementation scenario, steps S501 to S504 provided in this embodiment are substantially similar to steps S201 to S204 in an embodiment of the layer display method of the terminal system provided in this application, and are not described herein again.
S505: and combining all second layers with Z-order values larger than that of the Nth first layer through the graphics processor to obtain a buffer layer, and sending the buffer layer to an overlay display layer for displaying.
It can be understood that, in the present application, a sequence of operations for merging the special layer and the buffer layer is not limited, for example, the operations may be performed synchronously, and a sequence of operations for sending the special layer, the first layer, and the buffer layer to the overlay display layer to be displayed is also not limited, for example, the operations may also be performed synchronously, so that a sequence of steps S503, S504, and S505 in this embodiment is not limited. Through steps S501-S504, N first layers and M special layers may be obtained, and the N first layers and the M special layers are respectively sent to overlay display layers of different layers for display, and at this time, a problem how to send and display all second layers having Z-order values larger than that of the nth first layer needs to be considered; therefore, in order to avoid the problem that all second layers with Z-order values larger than that of the nth first layer may not be individually sent to overlay display layers of different layers through the HWC module for display in the previous embodiment, in this embodiment, all second layers with Z-order values larger than that of the nth first layer may be directly combined by the graphics processor to obtain a buffer layer, and then the buffer layer is sent to an overlay display layer for display, so that the resource amount of the overlay display layer can be ensured to meet the display requirements of all layers.
As an implementation manner, the first image layer may include at least one of a media video layer and a reverse video layer; and/or, the second layer may include at least one of a wallpaper layer, a pip layer, a wechat video layer, and a bluetooth charging layer.
As an implementation manner, the terminal system in the present application may be an android system, and correspondingly, the layer management module is a surfaceflag module; and/or the terminal system is a vehicle-mounted system. It is understood that the terminal system of the present application may also be an embedded system such as other Linux (a UNIX operating system available for free).
The method of the application can be used for setting a plurality of special layers, the position of the special layers is not limited, and the special layers can be set as required. For example, in a case that no format of the layer with two layers in the middle can be identified by the GPU, please refer to fig. 5, where fig. 5 is a schematic structural diagram in another application scenario of the layer display method of the terminal system provided in the present application. In another specific application scenario, the car machine system is an android system, the car machine system respectively includes a wall paper layer, a launcher layer, a status bar layer, a media video layer, a PIP UI layer, a back car video layer, a back car UI layer, and a layer to be displayed of a blue layer in the order of z-order value from small to large, and the number of overlays actually available for display on the chip is only 5, and the overlays are respectively overlay 0, overlay 1, overlay 2, overlay 3, and overlay 4 in the order from bottom to top. In the implementation scenario, both the media layer and the backhaul video layer cannot be recognized by the GPU, so two special layers, namely special layer0 and special layer1, are created in advance in the surfafinger by default; when the SurfaceFlinger detects that a media video layer and a back video layer exist, and the z-order value of the media video layer is smaller than that of the back video layer, the SurfaceFlinger synthesizes all layer layers with z-order values smaller than that of the media video layer into one layer in advance by using the GPU, synthesizes all layer layers with z-order values larger than that of the media video layer and smaller than that of the back video layer into one layer in advance by using the GPU, and then sends the layer layers to the overlay to display together with other layer layers. The method includes the steps that a wall paper layer, a launcher layer and a status bar layer are combined through a GPU to form special layer0, a PIP video layer and a PIPUI layer are also combined through the GPU to form special layer1, in addition, as the number of the special layers which can be actually displayed on a chip is 5, in order to guarantee that the limitation of hardware resources is not exceeded, a backscar UI layer, a bluetooth layer and other layer layers with z-order values larger than that of the bluetooth layer are combined through the GPU to form a buffer layer (FrameBuffer Target), the medium video layer and the backscar video layer are respectively combined through the HWC, then the medium layer0, the medium video layer, the special layer1, the backspace layer and the backspace layer are sent together to the HWC, and the video layer is not subjected to display together, so that the hardware resources can be displayed on the chip and the video cannot exceed the limitation of the hardware resources, and the hardware layers can be displayed on the chip, and the video can be displayed on the chip, and the monitor layer can be displayed on the chip, and the hardware resources are not exceeded, and the hardware resources are 5.
Referring to fig. 6, fig. 6 is a schematic structural diagram of an embodiment of a layer display apparatus of a terminal system according to the present application. The layer display apparatus 60 of the terminal system in this embodiment includes a graphics processor 601, an HWC module 602, a layer management module 603, and a display module 604, which are coupled to each other: the layer management module 603 is configured to: creating a layer for each application program to be displayed, wherein the layer of each application program has a Z-order value; acquiring N first layers in all layers to be displayed, wherein the data format of the first layers is not identified by the graphic processor 601, and the layer in the layers to be displayed, of which the data format can be identified by the graphic processor 601, is a second layer; forming N image layer groups by using a plurality of second image layers with Z-order values smaller than the maximum Z-order value in the N first image layers, respectively combining the N image layer groups through the graphic processor 601 to obtain N special image layers, and outputting the N special image layers to N overlay display layers for displaying; sending the N first layers and each second layer which is not combined to form a special layer to an overlay display layer of a different layer through the HWC module 602 for display; the display module is provided with a plurality of overlay display layers, and the overlay display layers are used for displaying the layers to be displayed; the Z-order value of the layer to be displayed by the overlay display layer positioned on the upper layer is larger than the Z-order value of the layer to be displayed by the overlay display layer positioned on the lower layer. For a specific process of the layer display device 60 of the terminal system of this embodiment to specifically implement the above functions, reference may be made to the above method embodiment.
Referring to fig. 7, fig. 7 is a schematic structural diagram of a layer display apparatus of a terminal system according to another embodiment of the present application. The layer display apparatus 70 of the terminal system in this embodiment includes a memory 701 and a processor 702 coupled to each other; the memory 701 is configured to store program data, and the processor 702 is configured to execute the program data stored in the memory 701 to implement the layer display method of the terminal system as described above. For the processor 702 and other components in the image layer display device 70 of the terminal system of this embodiment to implement specific functional contents, reference may be made to the above method embodiments.
Referring to fig. 8, fig. 8 is a schematic structural diagram of an embodiment of a vehicle-mounted device provided in the present application. The car machine equipment 80 of the present application includes a layer display device 800 of the terminal system, and the layer display device 800 may be the layer display device 60 of the terminal system or the layer display device 70 of the terminal system.
Referring to fig. 9, fig. 9 is a schematic structural diagram of a memory device according to an embodiment of the present disclosure. The application storage device 90 stores therein program data 900, and the program data 900 can be executed to implement the layer display method of the terminal system as described above. The storage device 90 may be a storage chip in a server, a readable and writable storage tool such as an SD card, or a server.
In the several embodiments provided in the present application, it should be understood that the disclosed method, apparatus, and device may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of a module or a unit is merely a logical division, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, a network device, or the like) or a processor (processor) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk, and various media capable of storing program codes.
The above description is only an embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes performed by the present specification and drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. The layer display method of a terminal system is characterized in that a layer management module is arranged in the terminal system; the method comprises the following steps:
the layer management module creates a layer for each application program to be displayed, wherein the layer of each application program has a Z-order value;
acquiring N first layers in all layers to be displayed, wherein the data format of the first layers is not recognized by a graphic processor, and the layer of which the data format can be recognized by the graphic processor in the layers to be displayed is a second layer;
forming M image layer groups by a plurality of second image layers with Z-order values smaller than the maximum Z-order value in the N first image layers, respectively combining the M image layer groups through a graphics processor to obtain M special image layers, and outputting the M special image layers to M overlay display layers for displaying; m is more than or equal to 1 and less than or equal to N, each graph layer group refers to a set of all second graph layers with continuous Z-order values in each segment in the Z-order value sequence;
and respectively sending the N first image layers to different layers of overlay display layers for display through the HWC module.
2. The method according to claim 1, wherein a Z-order value of the Nth first image-layer is greater than a Z-order value of the (N-1) th first image-layer, and N ≧ 1.
3. The method of claim 2, wherein the overlay display layer has Q;
after the step of obtaining N first layers of all layers to be displayed, the method further includes:
and judging whether the number P of all second layers with the Z-order value larger than that of the Nth first layer is larger than Q- (M + N) or not.
4. A method according to claim 3, wherein after said step of determining whether the number P of all second layers having Z-order values greater than the Z-order value of the nth first layer is greater than Q- (M + N), the method includes:
if the Z-sequence value is larger than the preset value, combining the P- (Q- (M + N)) +1 second layers with the largest Z-sequence value through the graphics processor to obtain a buffer layer, and sending the buffer layer to the uppermost layer of the overlay display layer for displaying;
the step of sending the N first layers to different layers of overlay display layers through the HWC module for display includes:
and respectively sending the N first image layers and each second image layer which has Z-sequence values larger than that of the Nth first image layer and is not combined to form the buffer image layer to an overlay display layer through an HWC module for displaying.
5. A method according to claim 3, wherein after said step of determining whether the number P of all second layers having Z-order values greater than the Z-order value of the nth first layer is greater than Q- (M + N), the method includes:
and if not, respectively sending the N first layers and each second layer with the Z-order value larger than that of the Nth first layer to an overlay display layer through an HWC module for displaying.
6. The method according to claim 2, wherein after the step of obtaining N first layers of all layers to be displayed, the method further comprises:
and combining all second layers with Z-order values larger than that of the Nth first layer through the graphics processor to obtain a buffer layer, and sending the buffer layer to an overlay display layer for displaying.
7. The layer display device of the terminal system is characterized by comprising a graphics processor, a HWC module, a layer management module and a display module which are coupled with each other:
the layer management module is used for: creating a layer for each application program to be displayed, wherein the layer of each application program has a Z-order value; acquiring N first layers in all layers to be displayed, wherein the data format of the first layers is not identified by a graphic processor, and the layer of which the data format can be identified by the graphic processor in the layers to be displayed is a second layer; forming M image layer groups by a plurality of second image layers with Z-order values smaller than the maximum Z-order value in the N first image layers, respectively combining the M image layer groups through a graphics processor to obtain M special image layers, and outputting the M special image layers to M overlay display layers for display, wherein M is larger than or equal to 1 and smaller than or equal to N, each image layer group refers to a set of all continuous second image layers with each section of Z-order value in a Z-order value sequence; and respectively sending the N first layers to overlay display layers of different layers for display through the HWC module.
8. The layer display device of the terminal system is characterized by comprising a memory and a processor which are coupled with each other;
the processor is configured to execute the program data stored by the memory to implement the method of any one of claims 1 to 6.
9. A vehicle-mounted equipment, characterized by comprising the layer display device of the terminal system according to claim 7 or 8.
10. A storage device, characterized in that program data are stored which can be executed by a processor for implementing the method as claimed in any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911082445.8A CN112783584B (en) | 2019-11-07 | 2019-11-07 | Layer display method and device of terminal system, vehicle equipment and storage device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911082445.8A CN112783584B (en) | 2019-11-07 | 2019-11-07 | Layer display method and device of terminal system, vehicle equipment and storage device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112783584A CN112783584A (en) | 2021-05-11 |
| CN112783584B true CN112783584B (en) | 2023-04-14 |
Family
ID=75747885
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911082445.8A Active CN112783584B (en) | 2019-11-07 | 2019-11-07 | Layer display method and device of terminal system, vehicle equipment and storage device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112783584B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114071212B (en) * | 2021-11-15 | 2024-07-16 | 北京字节跳动网络技术有限公司 | Information display processing method and device and electronic equipment |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1503344A2 (en) * | 2003-07-31 | 2005-02-02 | Seiko Epson Corporation | Layered presentation system utilizing compressed-domain image processing |
| CN105278911A (en) * | 2015-12-01 | 2016-01-27 | 上海兆芯集成电路有限公司 | Method and device for displaying map layers |
| CN105447813A (en) * | 2015-12-01 | 2016-03-30 | 上海兆芯集成电路有限公司 | Graphic layer combination method and apparatus |
| CN106201550A (en) * | 2016-07-22 | 2016-12-07 | 青岛海信移动通信技术股份有限公司 | A kind of figure layer display packing and equipment |
| CN108648249A (en) * | 2018-05-09 | 2018-10-12 | 歌尔科技有限公司 | A kind of image rendering method, device and intelligent wearable device |
| CN109697739A (en) * | 2018-12-25 | 2019-04-30 | 掌阅科技股份有限公司 | The inverse display methods of hand-written arrangement for reading and hand-written arrangement for reading |
| CN110020300A (en) * | 2017-11-07 | 2019-07-16 | 华为技术有限公司 | A kind of browser page synthetic method and terminal |
| CN110377257A (en) * | 2019-07-17 | 2019-10-25 | Oppo广东移动通信有限公司 | Layer synthesis method, device, electronic device and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| AU2013211451A1 (en) * | 2013-07-30 | 2015-02-19 | Canon Kabushiki Kaisha | Method of processing graphics with limited memory |
| US10198674B2 (en) * | 2016-12-22 | 2019-02-05 | Canon Kabushiki Kaisha | Method, apparatus and system for rendering a graphical representation within limited memory |
-
2019
- 2019-11-07 CN CN201911082445.8A patent/CN112783584B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1503344A2 (en) * | 2003-07-31 | 2005-02-02 | Seiko Epson Corporation | Layered presentation system utilizing compressed-domain image processing |
| CN105278911A (en) * | 2015-12-01 | 2016-01-27 | 上海兆芯集成电路有限公司 | Method and device for displaying map layers |
| CN105447813A (en) * | 2015-12-01 | 2016-03-30 | 上海兆芯集成电路有限公司 | Graphic layer combination method and apparatus |
| CN106201550A (en) * | 2016-07-22 | 2016-12-07 | 青岛海信移动通信技术股份有限公司 | A kind of figure layer display packing and equipment |
| CN110020300A (en) * | 2017-11-07 | 2019-07-16 | 华为技术有限公司 | A kind of browser page synthetic method and terminal |
| CN108648249A (en) * | 2018-05-09 | 2018-10-12 | 歌尔科技有限公司 | A kind of image rendering method, device and intelligent wearable device |
| CN109697739A (en) * | 2018-12-25 | 2019-04-30 | 掌阅科技股份有限公司 | The inverse display methods of hand-written arrangement for reading and hand-written arrangement for reading |
| CN110377257A (en) * | 2019-07-17 | 2019-10-25 | Oppo广东移动通信有限公司 | Layer synthesis method, device, electronic device and storage medium |
Non-Patent Citations (4)
| Title |
|---|
| An FPGA implementation of multi-channel video processing and 4K real-time display system;Xin Guo等;《2017 10th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI)》;20180226;全文 * |
| 基于ArcGIS Server的地质数据可视化平台构建与优化;吴珍云等;《实验室研究与探索》;20100715(第07期);全文 * |
| 基于ARINC661规范的平视显示器显示技术研究;周德新等;《计算机测量与控制》;20150325(第03期);全文 * |
| 移动学习平台的多图层管理技术研究;邢邗等;《计算机应用与软件》;20170815(第08期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112783584A (en) | 2021-05-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110377257B (en) | Layer composition method and device, electronic equipment and storage medium | |
| US20220139353A1 (en) | Display method, electronic device, and non-transitory computer-readable storage medium | |
| CN110377264B (en) | Layer synthesis method, device, electronic equipment and storage medium | |
| CN110363831B (en) | Layer composition method and device, electronic equipment and storage medium | |
| EP3311565B1 (en) | Low latency application streaming using temporal frame transformation | |
| CN102196034B (en) | The Application share of occlusion removal | |
| WO2021008424A1 (en) | Method and device for image synthesis, electronic apparatus and storage medium | |
| US7995003B1 (en) | System and method for rendering and displaying high-resolution images | |
| CN103686304B (en) | Method, device and terminal device for layer composition | |
| JP2019512749A (en) | Application program processing method and terminal device | |
| US9883137B2 (en) | Updating regions for display based on video decoding mode | |
| US10949226B2 (en) | Display method of multi-application based on Android system, and terminal device | |
| CN112057852B (en) | A game screen rendering method and system based on multiple graphics cards | |
| US11594194B2 (en) | Video timing for display systems with variable refresh rates | |
| CN112783584B (en) | Layer display method and device of terminal system, vehicle equipment and storage device | |
| CN113064728A (en) | High-load application image display method, terminal and readable storage medium | |
| CN115543244A (en) | Multi-screen splicing method and device, electronic equipment and storage medium | |
| CN105378645B (en) | The system and method shown for information | |
| US8436851B2 (en) | Systems and methods for rendering three-dimensional graphics in a multi-node rendering system | |
| US12353708B2 (en) | Display method, terminal device and non-transitory storage medium | |
| US20140104242A1 (en) | System and method for concurrent display of a video signal on a plurality of display devices | |
| CN109859328B (en) | Scene switching method, device, equipment and medium | |
| CN111785229B (en) | Display method, device and system | |
| KR102587866B1 (en) | Interoperable 3D image content handling | |
| EP0803798A1 (en) | System for use in a computerized imaging system to efficiently transfer graphics information to a graphics subsystem employing masked direct frame buffer access |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |