Disclosure of Invention
The embodiment of the invention provides a video coding optimization method and device for an Android terminal, which can improve the picture quality of important areas of a game in a targeted manner.
In a first aspect, an embodiment of the present invention provides an Android-end video coding optimization method, where the optimization method includes:
obtaining an AI training model through training;
identifying an important area or a non-important area through the application of the AI training model to the cloud game picture;
reserving more detail contents for the important area, and reducing the detail contents of the non-important area;
wherein the important area comprises equipment, characters, objects of attack or important props in the game; the non-significant area includes the background or sky in the game.
Optionally, the obtaining an AI training model through training includes:
recording game pictures and marking important areas or non-important areas;
the model of the important area or the non-important area is generated through computer learning.
Optionally, the detailed content of the non-important area is reduced under the condition of insufficient bandwidth to improve the experience effect of the cloud game.
Optionally, the reducing the detail content of the non-important region includes reducing the contrast of the non-important region; reducing the image quality of the non-important area.
Optionally, the more detail content is reserved for the important area, and the detail content of the unimportant area is reduced in the same game picture at the same compression ratio.
Optionally, the cloud game screen needs to be rendered by a hardware renderer before being identified by the AI model.
Optionally, after the operation of reserving more detail content for the important region and reducing the detail content for the non-important region is performed, data needs to be compressed by using a video coding algorithm (H264, HEVC, AV1) for the cloud game picture.
In a second aspect, an embodiment of the present invention provides an Android-end video coding optimization apparatus, where the optimization apparatus includes:
the training module is used for obtaining an AI training model through training;
the recognition module is used for recognizing an important area or a non-important area through the application of the AI training model to the cloud game picture;
and the processing module reserves more detail contents for the important area and reduces the detail contents of the non-important area.
In a third aspect, an embodiment of the present invention provides an electronic device, including a memory and a processor, where the memory stores a computer program thereon, and the processor implements the method according to any one of the first aspect when executing the program.
In a fourth aspect, an embodiment of the invention provides a computer-readable storage medium on which is stored a computer program which, when executed by a processor, implements the method of any one of the first aspects.
Advantageous effects
The invention provides a video coding optimization method of an Android terminal, electronic equipment and a storage medium, wherein the optimization method comprises the steps of obtaining an AI training model through training; identifying an important area or a non-important area through the application of the AI training model to the cloud game picture; reserving more detail contents for the important area, and reducing the detail contents of the non-important area; wherein the important area comprises equipment, characters, objects of attack or important props in the game; the non-significant area includes, a background or sky in a game; the method can improve the picture quality of important areas of the game in a targeted manner and improve the game experience of users.
It should be understood that the statements herein reciting aspects are not intended to limit the critical or essential features of any embodiment of the invention, nor are they intended to limit the scope of the invention. Other features of the present invention will become apparent from the following description.
Detailed Description
In order to make those skilled in the art better understand the technical solutions in one or more embodiments of the present disclosure, the technical solutions in one or more embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in one or more embodiments of the present disclosure, and it is obvious that the described embodiments are only a part of the embodiments of the present disclosure, and not all embodiments. All other embodiments that can be derived by a person skilled in the art from one or more of the embodiments described herein without making any inventive step shall fall within the scope of protection of this document.
In the related art, GPU22242 is a graphics processor, also called a display core, a visual processor, and a display chip, and is a microprocessor dedicated to image and graphics related operations on a personal computer, a workstation, a game console, and some mobile devices (e.g., a tablet computer, a smart phone, etc.); the GPU reduces the dependence of the display card on the CPU, and performs part of the work of the original CPU, particularly the core technologies adopted by the GPU in the 3D graphics processing comprise hardware T & L (geometric transformation and illumination processing), cubic environment material mapping and vertex mixing, texture compression and concave-convex mapping, a dual-texture four-pixel 256-bit rendering engine and the like, and the hardware T & L technology is a mark of the GPU; it should be said that there are graphics processors (commonly known as graphics cards) in display systems, but earlier graphics cards contained only simple memory and frame buffers, which actually served only one graphics storage and transfer function, all operations having to be controlled by the CPU; this is sufficient for text and some simple graphics, but this system alone cannot do its task when dealing with complex scenes, especially some realistic three-dimensional scenes; therefore, the later developed display cards all have the function of graphic processing; the method not only stores the graphics but also can complete most of the graphics functions, thereby greatly lightening the burden of a CPU and improving the display capability and the display speed; with the development of electronic technology, the technical content of the graphics card is higher and higher, the functions are stronger and stronger, many professional graphics cards already have strong 3D processing capability, and the 3D graphics cards are gradually moved to personal computers; some professional display cards have even more transistors than contemporary CPUs.
cgroups2225 is a mechanism provided by Linux kernel, which can integrate (or separate) a series of system tasks and their subtasks into different groups according to resource classification according to specific behaviors, thereby providing a uniform framework for system resource management; the cgroups can limit, record and isolate physical resources (including CPU, memory, IO and the like) used by a process group, essentially, a system kernel is attached to a program, a series of hooks are provided for realizing virtualization of a container, and corresponding hooks are triggered by scheduling of the resources during program operation, so that the purposes of resource tracking and limitation are achieved. The basic guarantee is provided, and the method is a foundation stone for constructing a series of virtualization management tools such as Docker and the like; the API of the cgroups is realized in a mode of a pseudo file system, namely, a user can realize organization management of the cgroups through file operation; the organization management operation unit of the cgroups can reach a thread level in a fine granularity mode, and the user mode codes can also establish and destroy the cgroups aiming at the resources distributed by the system, so that the resource reallocation and management are realized; all the functions of resource management are realized in a subsystem mode; a child process creates a control group that is initially in the same cgroups as its parent process.
The present invention is described below with specific examples, and it should be noted that the descriptions in the examples of the present application are only for clearly illustrating the technical solutions in the examples of the present application, and do not limit the technical solutions provided in the examples of the present application.
Fig. 1 shows a flowchart of a video coding optimization method of an Android end according to an embodiment of the present invention. Referring to fig. 1, the optimization method includes:
s20, obtaining an AI training model through training;
firstly, recording a game picture, then manually marking an important area, learning and generating a model of the important area or a non-important area through a computer to obtain an AI training model and outputting the identification model obtained by training for identifying the cloud game picture later.
S40, identifying important areas or non-important areas through the application of the AI training model to the cloud game picture;
rendering the cloud game picture through a hardware renderer, judging an important area or a non-important area through application of the rendered cloud game picture through the identification model, and processing the picture of the cloud game according to a judgment result.
S60, reserving more detail contents for the important area and reducing the detail contents of the non-important area;
under the condition of the same game picture and the same compression ratio, the detailed contents of more important areas are reserved, the detailed contents of unimportant areas are reduced, and the picture quality of important areas of the cloud game is improved in a targeted manner; reducing the detail content of the non-important area can reduce the image quality by reducing the contrast of the non-important area after the non-important area is determined, so that a video coding algorithm can compress the area data at a larger rate; wherein the data is compressed using a video coding algorithm (H264, HEVC, AV 1); wherein the important area comprises equipment, characters, objects of attack or important props in the game; the non-significant area includes the background or sky in the game.
The invention provides a video coding optimization method of an Android terminal, electronic equipment and a storage medium, wherein the optimization method comprises the steps of obtaining an AI training model through training; identifying an important area or a non-important area through the application of the AI training model to the cloud game picture; reserving more detail contents for the important area, and reducing the detail contents of the non-important area; wherein the important area comprises equipment, characters, objects of attack or important props in the game; the non-significant area includes, a background or sky in a game; the method can improve the picture quality of important areas of the game in a targeted manner and improve the game experience of users.
Fig. 2 shows a software architecture diagram of a video coding optimization method on an Android side according to an embodiment of the present invention; as shown in fig. 2, the software architecture includes a user end 21 and a cloud end 22;
wherein the user terminal comprises a mobile phone/tablet 211, a notebook computer 212, a television 213 and a personal computer 214; the cloud comprises a cloud game management platform 221 and a Linux server 222;
the Linux server 222 comprises an Android container manager 2221, an Android container server 2222, an Android system container 2223, hardware 2224 and cgroups 2225;
the Android container server 2222 includes four modules, which are an audio encoding module 22221 for encoding audio in a cloud game, a hardware graphics renderer 22222 for rendering pictures in the cloud game, a real-time communication module 22223 for communicating with a network controller 22241 in hardware 2224, and an AI video encoding module 22224 for encoding cloud game pictures rendered by the hardware graphics renderer 22222, so as to improve the quality of the cloud game pictures and improve the user game experience;
the Android system container 2223 contains an Android game application module 22231 and an Android hardware simulator 22232;
hardware 2224 includes a network controller 22241 and a GPU22242, where GPU22242 is a graphics processor, also called a display core, a visual processor, and a display chip, and is a microprocessor dedicated to image and graphics related operations on a personal computer, a workstation, a game machine, and some mobile devices (e.g., a tablet computer, a smart phone, etc.); the GPU reduces the dependence of the display card on the CPU, and performs part of the work of the original CPU, particularly the core technologies adopted by the GPU in the 3D graphics processing comprise hardware T & L (geometric transformation and illumination processing), cubic environment material mapping and vertex mixing, texture compression and concave-convex mapping, a dual-texture four-pixel 256-bit rendering engine and the like, and the hardware T & L technology is a mark of the GPU;
cgroups2225 is a mechanism provided by Linux kernel, which can integrate (or separate) a series of system tasks and their subtasks into different groups according to resource classification according to specific behaviors, thereby providing a uniform framework for system resource management; the cgroups can limit, record and isolate physical resources (including CPU, memory, IO and the like) used by a process group, essentially, a system kernel is attached to a program, a series of hooks are provided for realizing virtualization of a container, and corresponding hooks are triggered by scheduling of the resources during program operation, so that the purposes of resource tracking and limitation are achieved. The basic guarantee is provided, and the method is a foundation stone for constructing a series of virtualization management tools such as Docker and the like;
in the software architecture, the user 21 is connected to the network controller 22241 in the hardware 2224 to achieve the effect of connecting to the cloud; the Android container manager 2221 and the Android system container 2223 are respectively connected with an Android container server 2222; the real-time communication modules 22223 in the Android container manager 2221 and the Android container server 2222 are respectively connected with the network controller 22241 in the hardware 2224; an AI video coding module 22224 in the Android container server 2222 is connected to a GPU22242 in the hardware 2224; the Android container manager 2223 is connected with cgroups 2225; the network controller 22241 in the hardware 2224 is further connected to the cloud game management platform 221;
in the software architecture, an Android container manager 2221, an Android container server 2222 and an Android system container 2223 belong to a user space; hardware 2224 and cgroups belong to kernel space.
Based on the same inventive concept, an embodiment of the present invention further provides an Android-side video coding optimization apparatus, which can be used to implement the Android-side video coding optimization method described in the foregoing embodiment, as described in the following embodiment: the principle of the video coding optimization device for the Android end for solving the problems is similar to that of the video coding optimization method for the Android end, so that the implementation of the video coding optimization device for the Android end can refer to the implementation of the video coding optimization method for the Android end, and repeated parts are not repeated. As used hereinafter, the term "unit" or "module" may be a combination of software and/or hardware that implements a predetermined function. While the system described in the embodiments below is preferably implemented in software, implementations in hardware, or a combination of software and hardware are also possible and contemplated.
Fig. 3 shows a block diagram of a video coding optimization device on an Android side according to an embodiment of the present invention. As shown in fig. 3, the optimizing apparatus includes:
a training module 20 for obtaining an AI training model through training;
the recognition module 40 is used for recognizing important areas or non-important areas through the application of the AI training model to the cloud game picture;
and the processing module 60 reserves more detail contents for the important area and reduces the detail contents of the non-important area.
The embodiment of the invention provides a video coding optimization device of an Android terminal, which comprises a training module 20, an AI training model is obtained through training; the recognition module 40 is used for recognizing important areas or non-important areas through the application of the AI training model to the cloud game picture; and the processing module 60 reserves more detail contents for the important area and reduces the detail contents of the non-important area.
Fig. 4 is a schematic structural diagram of an electronic device to which an embodiment of the present invention can be applied, and as shown in fig. 4, the electronic device includes a Central Processing Unit (CPU)401 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)402 or a program loaded from a storage section 408 into a Random Access Memory (RAM) 403. In the RAM403, various programs and data necessary for system operation are also stored. The CPU 401, ROM 402, and RAM403 are connected to each other via a bus 404. An input/output (I/O) interface 405 is also connected to bus 404.
The following components are connected to the I/O interface 405: an input section 406 including a keyboard, a mouse, and the like; an output section 407 including a display device such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 408 including a hard disk and the like; and a communication section 409 including a network interface card such as a LAN card, a modem, or the like. The communication section 409 performs communication processing via a network such as the internet. A driver 410 is also connected to the I/O interface 405 as needed. A removable medium 411 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 410 as necessary, so that a computer program read out therefrom is mounted into the storage section 408 as necessary.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units or modules described in the embodiments of the present invention may be implemented by software, or may be implemented by hardware. The described units or modules may also be provided in a processor, and may be described as: a processor includes a training module, a recognition module, and a processing module, wherein the names of these modules do not constitute a limitation on the module itself in some cases, for example, a recognition module can also be described as "recognition module that recognizes a significant area or a non-significant area through the application of the AI training model to a cloud game screen".
As another aspect, the present invention further provides a computer-readable storage medium, where the computer-readable storage medium may be a computer-readable storage medium included in the video coding optimization apparatus of the Android end in the foregoing embodiment; or it may be a computer-readable storage medium that exists separately and is not built into the electronic device. The computer readable storage medium stores one or more programs, which are used by one or more processors to execute a video coding optimization method on the Android side described in the present invention.
The foregoing description is only exemplary of the preferred embodiments of the invention and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention herein disclosed is not limited to the particular combination of features described above, but also encompasses other arrangements formed by any combination of the above features or their equivalents without departing from the spirit of the invention. For example, the above features and (but not limited to) features having similar functions disclosed in the present invention are mutually replaced to form the technical solution.