CN101977322A - Screen coding system based on universal video coding standard - Google Patents

Abstract

A screen coding system based on a general video coding standard in the technical field of video image processing, including: a screen information acquisition module, a color space conversion module and an improved general encoder, the screen information acquisition module is connected with the color space conversion module and transmits the original The screen image and the changed area information, the color space conversion module is connected with the improved universal encoder to transmit the changed area information and encoded input image, and the improved universal encoder outputs encoded and compressed data. The invention can effectively reduce the CPU occupation rate.

Description

Screen Coding System Based on Universal Video Coding Standard

technical field

The present invention relates to a system in the technical field of video image processing, in particular to a screen coding system based on general video coding standards.

Background technique

In computer remote network teaching, monitoring system, network conference, etc., computer screen video compression is an important link. Due to the high resolution and complex content of the screen image, the screen image requires high network bandwidth. Video coding refers to a method of storing digital video stream sequences with fewer data bits, and screen video coding is hereinafter referred to as screen coding, which means that digital video stream sequences are generated from screen images. This technology can effectively reduce the amount of data that needs to be transmitted, thereby reducing the pressure on the network.

Compared with ordinary video, screen video has the characteristics of high resolution and insignificant frame-to-frame changes. As for the general video coding standard, considering the complex image content and object motion in general video, the general video coding standard provides a wealth of compression tools, so in order to achieve a better compression effect, a large number of calculations are usually required. If the general video coding standard is directly used for screen coding, it will take up a lot of CPU time, especially for real-time live broadcast scenarios, such a high CPU usage is unacceptable.

Therefore, for screen video compression, a dedicated screen video coding method is usually sampled for compression. However, the screen encoding method has great limitations on the screen video content. When the screen image changes drastically, especially when a video is played, the compression effect will become worse.

After searching the literature of the prior art, it is found that the remote desktop synchronization method in the patented pervasive environment of the Chinese Patent Document No. CN101014127A proposes a screen video coding scheme based on motion prediction and compensation of changing sub-regions, which has certain advantages , but the continuous and drastic changes of the screen image will cause the compression rate to drop, and the use has certain limitations.

Contents of the invention

The present invention aims at the above-mentioned deficiencies in the prior art, and provides a screen coding system based on a general video coding standard, which can effectively reduce CPU usage.

The present invention is achieved through the following technical solutions, and the present invention includes: a screen information acquisition module, a color space conversion module and an improved universal encoder, wherein: the screen information acquisition module is connected with the color space conversion module and transmits the original screen image and the changed The area information, color space conversion module is connected with the improved universal encoder and transmits the changed area information and encodes the input image, and the improved universal encoder outputs encoded and compressed data.

The coded input image refers to an image that meets the requirements of the input image format of the improved general encoder;

The screen information acquisition module is used to obtain current screen image information and change area information from the computer display subsystem, wherein: change area information refers to: the current screen image described by the coordinates of the rectangle changes compared with the previous frame image The area enclosed by the smallest circumscribed rectangle of the image area composed of pixels.

The color space conversion module is used to perform color space conversion on the original screen image obtained in the screen information acquisition module and output an image that meets the requirements of an improved universal video coding standard-based encoder input image format;

The general video coding standard refers to: dividing an image into pixel blocks of a specified size, that is, macroblocks, and encoding and compressing them in units of macroblocks; and supporting encoding with a "skip" compression mode, that is, when compressed There is a macroblock completely consistent with the content of the current macroblock in the previous frame of the image, that is, the predicted macroblock. The predicted macroblock and the current macroblock have the same position in the image. When compressing the current macroblock, only the position information of the predicted macroblock is recorded. and "skip" mode flags, no other information about the current macroblock is recorded.

The improved general encoder is used to encode and compress the encoded input image and output the compressed data. The improved general encoder includes: an encoder based on the general video coding standard and a newly-added embedded in the general video encoder The "skip" mode fast detection unit, wherein: the "skip" mode fast detection unit is used to directly select the "skip" mode for the macroblock that is not in the changed area, that is, the unchanged macroblock, for compression .

The invention can solve the problem that the traditional screen-specific encoding scheme has poor compression effect when playing video on the screen, and the system can effectively reduce the CPU occupation rate compared with the use of a general video encoding standard encoder for screen encoding. The system of the invention can be widely used in the fields of computer remote network teaching, monitoring system, network conference, computer remote control system and the like.

Description of drawings

Fig. 1 is a system structure diagram of the present invention.

Fig. 2 is a system structure diagram in Embodiment 2.

FIG. 3 is a schematic diagram of the operation of the color space conversion module in Embodiment 2.

FIG. 4 is an operation flow chart of the improved h.264 general-purpose video encoder in Embodiment 2.

Detailed ways

The embodiments of the present invention are described in detail below. This embodiment is implemented on the premise of the technical solution of the present invention, and detailed implementation methods and specific operating procedures are provided, but the protection scope of the present invention is not limited to the following implementation example.

Example 1

As shown in Figure 1, this embodiment includes: a screen information acquisition module 110, a color space conversion module 120 and an improved universal encoder 130, wherein: the screen information acquisition module 110 is connected with the color space conversion module 120 and transmits the original screen image 101 and change area information 102, the color space conversion module is connected to the improved general encoder, and transmits the change area information, as well as an image that meets the requirements of the input image format of the improved general encoder, hereinafter referred to as the encoded input image 103, The modified universal encoder outputs coded compressed data 104 .

The screen information obtaining module 110 is used to obtain the current screen image information and the change area information 102 from the computer display subsystem, wherein: the change area information 102 refers to: the current screen image described by the coordinates of the rectangle is the same as the previous frame image The area enclosed by the smallest circumscribed rectangle of the image area composed of pixels whose ratio has changed.

The color space conversion module 120 is used to perform color space conversion on the original screen image acquired in the screen information acquisition module 110 and output an image that meets the requirements of the input image format of the encoder based on the improved general video coding standard. The video coding standard refers to: first divide the image into pixel blocks of a specified size, that is, macroblocks, encode and compress in units of macroblocks, and then encode with a "skip" compression mode, that is, when the compressed previous frame There is a macroblock in the image that is completely consistent with the content of the current macroblock, that is, the predicted macroblock, and the position of the predicted macroblock and the current macroblock in the image is the same. When compressing the current macroblock, only the position information of the predicted macroblock and the "jump" are recorded Over" mode flag, do not record other information about the current macroblock.

The improved general encoder 130 is used to encode and compress the encoded input image and output compressed data. The improved general encoder 130 includes: an encoder based on the general video coding standard and a new embedded general video The "skip" mode fast detection unit in the encoder, wherein: the "skip" mode fast detection unit is used to directly select the "skip" mode for the macroblocks that are not in the changed area, that is, the unchanged macroblocks to compress.

In the screen coding system based on the universal coding standard, the screen information acquisition module 110 first obtains information such as the current screen image and the changed area, and puts these information into the buffer, and the color space conversion module 120 takes out the screen image from the buffer, Convert the color space according to the change area information 102, and output an image that meets the requirements of the input format of the encoder based on the improved universal video coding standard, that is, the so-called encoded input image. The improved general encoder 130 uses the changed area to encode the input image. compression. A "skip" mode fast detection device 131 is included in the general video encoder, and its function is to quickly judge whether the current macroblock can be compressed in the "skip" mode according to the change area provided by the screen information acquisition module 110, that is, for the non-changing The macroblocks in the area should use the "skip" mode directly. Finally, the general video encoder outputs the compressed data. The above process is called screen recording.

Example 2

As shown in Figure 2 is an embodiment of the screen coding system based on the general coding standard, this embodiment system uses the Mirror Driver mechanism under the Windows operating system to obtain screen information, adopts the general video coding standard h.264 to screen image sequence The resulting video is compressed. The h.264 standard cuts video images into macroblocks with a size of 16*16 for compression, and includes a P_Skip mode to indicate that the content of the current macroblock is the same as that of the predicted macroblock.

The format of the Windows operating system screen image is a color bitmap, called RGB format; the advanced profile high profile of h.264 is adopted by the present embodiment system, and the input image color format supported by this profile is YUV, so the color space conversion module 120 , responsible for converting RGB to YUV.

This embodiment system includes following 5 modules: Mirror driver screen information acquisition module 221, RGB to YUV conversion module 222, improved h.264 universal video encoder 223, timer 224 and user interface 225. The timer in it is used to control the screen encoding system to perform screen recording according to the specified frequency until the screen recording process is ended artificially. The user interface in it enables the user to specify the parameters of the screen encoding as well as control the start and end of the screen recording.

First, the screen information is obtained by the Mirror driver screen information acquisition module 221, including the screen image, the changed area, and the resolution and color format of the current screen.

The resolution of the computer screen is 1024*768 or higher. Before acquiring the screen image, the system will automatically change the screen resolution to 800*600. This is to avoid too much data during the encoding process. When the screen recording ends, the system will The resolution is automatically adjusted back to the original value. The acquired color format is generally a 32-bit RGB bitmap, specifically a 32-bit BGRA.

The embodiment system uses Mirror Driver to obtain screen information. Mirror Driver is a virtual display driver technology provided by Windows, which simulates the physical operation of actual video equipment such as a display screen in a piece of memory space. Therefore, the accurate rectangular change area of the screen image can be obtained through this mechanism. The embodiment system divides the changing screen image into 16 rectangular sub-areas of 4*4, and obtains the smallest rectangular changing area for each rectangular area, and each rectangular changing area is represented by the coordinates of the upper left corner and the lower right corner, a total of 16 groups coordinate.

The general encoder 130 used in the embodiment system is an open source encoder x264 that follows the h.264 standard. This encoder is fast and has a good compression effect, and is widely used in the field of video encoding. The input image color space supported by this encoder is specifically YUV420. Therefore, the RGB to YUV conversion module converts the screen image in BGRA format into an encoded input image in YUV420 format. As shown in Figure 3, during operation, color space conversion is performed on the pixels in the changed area Figure 3a, and these pixels are copied to the corresponding position Figure 3b of the previous frame image, covering the invalid area Figure 3c due to the change.

The x.264 video encoder includes the following main modules:

1) The macroblock segmentation device is used to segment the input image into 16*16 pixel macroblocks.

2) A frame type determiner, which determines the frame type of the image to be encoded. In x264, there are three frame types, key frames are called I frames, unidirectional predictive frames are called P frames, and bidirectional predictive frames are B frames. Considering that this embodiment is mainly oriented to real-time applications, the function of the B frame is disabled. Wherein, the I frame cannot include the modules of the "skip P_Skip" mode. X264 judges whether the current image is a key I frame according to the image change degree scenecut and the preset GOP length. The system of this embodiment sets scenecut=40, GOP=100.

3) A compression mode decider, used to determine the compression mode of the current macroblock. Which includes a "skip P_Skip" mode fast detection device, its function is to quickly judge whether the current macroblock can be compressed in the "skip P_Skip" mode according to the change area provided by the screen information acquisition module 110, that is, for the area not in the change area Macroblocks should use the "Skip P_Skip" mode directly.

4) A compressor, configured to perform data compression according to the selected prediction mode, and write the compressed data into a video file.

As shown in Figure 4, after the coded input image is segmented into macroblocks and judged by the frame type, the compression mode is selected by the compression mode decider. The change area information 102 detects whether the current macroblock is in the change area, and when the macroblock is not in the change area, select the "Skip P_Skip" mode for it. For macroblocks in the change area, x264's macroblock compression mode decider will select the appropriate compression mode for it. For key frames, the x264 compression mode decider will make mode decisions according to the default strategy. Finally, the compressor performs data compression according to the compression mode, and writes the compressed data into a video file.

In this embodiment, on the basis of the general video encoder x264, a "Skip P_Skip" mode fast detection module is added, and this module uses the changed area information 102 to make a fast decision on the compression mode of the unchanged macroblock. For the color space conversion module 120 , this embodiment uses the change area information 102 to reduce the image area that needs color space conversion. Therefore, compared with the general screen recording system based on the universal encoder 130 that does not use the change region information 102, the system of this embodiment has significantly reduced computation.

Use vTune9.0 to test the CPU usage of the screen recording system on a notebook computer configured with CPU core 2duo2.0GHz and memory 1G. The obtained data shows that compared with the operation time without optimization, the operation time of color space conversion and encoding after using the change area is reduced by 99.58% and 54.84%, respectively, and the number of CPU clocks occupied by the whole system is reduced. 62%. In addition, the CPU usage of the x264 screen recording system based on the "skip" mode fast detection given by XP has also been reduced from 44% of the unimproved system to 25%.

The above experimental results show that the screen coding system based on the general video coding standard can effectively reduce the CPU usage compared with the screen coding using the general video coding standard encoder.

Claims (6)

1. A screen encoding system based on general video encoding standards, characterized in that it comprises: a screen information acquisition module, a color space conversion module and an improved general encoder, wherein: the screen information acquisition module is connected with the color space conversion module and The original screen image and the changed area information are transmitted, the color space conversion module is connected with the improved general encoder and transmits the changed area information and encoded input image, and the improved general encoder outputs encoded and compressed data. 2. The screen coding system based on the universal video coding standard according to claim 1, characterized in that, said coded input image refers to an image conforming to the input image format requirements of the improved universal coder. 3. The screen coding system based on the universal video coding standard according to claim 1, wherein the screen information acquisition module is used to obtain current screen image information and change area information from the computer display subsystem, wherein: change The area information refers to the area enclosed by the smallest circumscribed rectangle of the image area composed of pixels whose current screen image has changed compared with the previous frame image described by the coordinates of the rectangle. 4. The screen coding system based on the universal video coding standard according to claim 1, wherein the color space conversion module is used to perform color space conversion on the original screen image obtained in the screen information acquisition module and output a A picture that conforms to the requirements of the input picture format of the modified common video coding standard-based encoder. 5. The screen coding system based on the general video coding standard according to any one of the preceding claims, wherein the general video coding standard refers to: dividing an image into pixel blocks of a specified size, i.e. macroblocks, to Encoding and compression is performed in units of macroblocks; and encoding is supported by the "skip" compression mode, that is, when the compressed previous frame image has a macroblock that is completely consistent with the content of the current macroblock, that is, the predicted macroblock, the predicted macroblock It is the same as the position of the current macroblock in the image. When compressing the current macroblock, only the position information of the predicted macroblock and the flag of the "skip" mode are recorded, and other information about the current macroblock is not recorded. 6. The screen coding system based on the universal video coding standard according to claim 1 or 2, wherein the improved universal encoder is used to encode and compress the encoded input image and output compressed data, and the improved universal encoder The encoder includes: an encoder based on general video coding standards and a newly added "skip" mode fast detection unit embedded in the general video coder, wherein: the "skip" mode fast detection unit is used for For the macroblocks in the changed area, that is, the unchanged macroblocks, the "skip" mode is directly selected for compression.

Images