CN101163237A - Image processing method and device - Google Patents

Abstract

The present invention provides an image processing method, comprising: decoding the bitstream to reconstruct the B frames and generate data corresponding to the first field and data corresponding to the second field, respectively; and temporarily storing the data corresponding to the first field and the data corresponding to the second field in a frame buffer respectively, for storing a self-use part and a second area of a first area in a buffer area of the B frame, or temporarily storing the data corresponding to the first field and the data corresponding to the second field in the second area respectively at different times, wherein the storage capacity of the second area is large enough to store a complete field, and the storage capacity of the self-use part of the first area is small enough to store a complete field.

Description

Image processing method and device

technical field

The present invention relates to the MPEG2 decoding function of low-end video products, especially an image processing method and device.

Background technique

The image processing device with MPEG2 decoding function on the market, such as: Digital Versatile Disc player (Digital Versatile Disc player, DVD player) or set-top box (Set Top Box, STB), usually has a memory used as a buffer, so that Store the I frame (I frame), P frame (P frame), and B frame (B frame) generated during MPEG2 decoding.

Since both the I frame and the P frame can be used as reference frames, they are usually kept in memory until they are no longer referenced. On the other hand, the B frame is not used as a reference frame but for display and needs to be temporarily stored in the memory. According to the known technology, under the premise of maintaining the image quality, if three I-frames, P-frames, and B-frames with a size of (720*576) pixels are all temporarily stored in the memory , then the storage space required for each frame is about 608 kilobytes (608 kilobytes), that is, a total storage space of about 1824 kilobytes is required.

In a low-end digital versatile disc player (low-end DVD player) or a low-end set-top box (low-end STB), the storage capacity of the memory used as a buffer is usually small, for example: storage capacity The size is less than or equal to two megabytes (2megabytes). If some functions and features in higher-end products are to be selectively implemented in the low-end digital versatile disc player or the low-end set-top box, it is often necessary to consume central processor resources (central processor resources) to The functions and features to be added need sufficient memory storage capacity to facilitate high-speed computing. In addition, additional data corresponding to these new functions and features also require additional storage space. Therefore, even though the above-mentioned I frame, P frame, and B frame are all compressed data (for example: compressed to about 66.7% of the size), the storage capacity of the memory is still very insufficient.

Contents of the invention

Therefore, one object of the present invention is to provide an image processing method and device to solve the above problems.

Another object of the present invention is to provide an image processing method and device for selectively implementing certain functions and features of higher-end products in lower-end products.

Another object of the present invention is to provide an image processing method and device, so that while selectively implementing certain functions and features in higher-end products in lower-end products, sufficient memory storage can still be maintained. The capacity is to facilitate the central processor resource to perform high-speed calculations, and to ensure the clarity and smoothness of the displayed images.

Another object of the present invention is to provide an image processing method and device, so as to realize low-end products with low price and good quality, and benefit many users; because the product itself is cheap and high-quality, its market share (market share) is will increase accordingly.

An embodiment of the present invention provides an image processing method. The image processing method includes: decoding a bit stream (bit stream) to reconstruct (reconstruct) a B frame (B frame), and respectively generating data corresponding to a first field (field) and data corresponding to a second field , wherein the first field represents the field where the B frame is to be displayed first, and the second field represents the field where the B frame is displayed secondarily; and the data corresponding to the first field and The data corresponding to the second image field is temporarily stored in the frame buffer (frame buffer), which is used to store the private use portion of the first area and the second area in the buffer area of the B frame , or temporarily store the data corresponding to the first image field and the data corresponding to the second image field in the second area at different times, wherein the storage capacity of the second area is sufficient to store a complete image field, and the size of the storage capacity of the self-use part of the first area is not enough to store a complete image field.

While providing the above-mentioned image processing method, the present invention also provides a corresponding image processing device. The image processing device includes: a central processor (central processor), used to control the operation of the image processing device; a frame buffer, used to temporarily store data; a video processor (video processor), coupled to the image The frame buffer and the central processing unit are used to decode the bit stream to reconstruct the B frame, and respectively generate data corresponding to the first field and data corresponding to the second field, and data corresponding to the first field The data and the data corresponding to the second field are temporarily stored in the frame buffer respectively, wherein the first field represents the field where the B frame is first to be displayed, and the second field represents the B The picture frame is displayed next to the image field, and the video processor includes a core circuit (core circuit), used to control the operation of the video processor; and a display processor (display processor), coupled to the video processor , the frame buffer, and the central processing unit are used to control the display of the B frame, and the display processor includes a core circuit, which is used to control the operation of the display processor. Wherein the operation of the video processor temporarily storing the data corresponding to the first image field and the data corresponding to the second image field in the frame buffer also includes: storing the data corresponding to the first image field and the data corresponding to the second image field respectively. The data corresponding to the second field is temporarily stored in the frame buffer, and is used to store the self-use part of the first area and the second area in the buffer area of the B frame, wherein the storage of the second area The size of the capacity is enough to store a complete image field, but the size of the storage capacity of the self-use part of the first area is not enough to store a complete image field; or at different times, the data corresponding to the first image field and The data corresponding to the second field are temporarily stored in the second area respectively.

Description of drawings

FIG. 1 is a schematic diagram of an image processing device provided according to an embodiment of the present invention.

FIG. 2 is a schematic diagram of a macroblock (MB) of a frame processed by the image processing device shown in FIG. 1 and a macroblock row (MB row) formed therefrom.

FIG. 3 is a schematic diagram of a buffering region used to temporarily store a B frame (B frame) in the frame buffer (frame buffer) shown in FIG. 1 .

FIG. 4 is a schematic diagram of the cooperative operation of the video processor and the display processor shown in FIG. 1 according to an embodiment of the present invention.

[Description of main component labels]

100 Image processing device 106 bus 108 frame buffer 108I, 108P, 108B Buffer area 110 CPU 120 video processor 122,132 core circuit 130 display processor MB(1, 1), MB(1, 2), ..., MB(P, Q) macro block R(1), R(2), ..., R(P) macro block column 310, 310U, 310V, 320 A region or part of a buffer zone Vsync Vertical sync

t _B1-DIS , t _B2-DIS , t _B2-DEC point in time O, E, O, E,... Display sequence of odd and even fields T _B1-DEC , T _D , T, T ₁ , T ₂ , T ₃ time interval

Detailed ways

Please refer to FIG. 1, FIG. 1 is a schematic diagram of an image processing device 100 provided according to an embodiment of the present invention, wherein the image processing device 100 can be an image processing device with MPEG2 decoding function, for example: a digital versatile disc player (Digital Versatile Disc player, DVDplayer), Set Top Box (Set Top Box, STB)...etc.

As shown in Figure 1, the image processing device 100 includes a bus (bus) 106, a frame buffer (framebuffer) 108, a central processing unit (central processor) 110, a video processor (videoprocessor) 120, and a display processor (display processor) 130, wherein the video processor 120 includes a core circuit (core circuit) 122, and the display processor 130 includes a core circuit 132, and the core circuit 122 and the core circuit 132 respectively control the operation of the video processor 120 and display processing The operation of device 130. According to this embodiment, the frame buffer 108 is a random access memory (random access memory, RAM), and its storage capacity is less than or equal to 2 megabytes (2megabytes), wherein the temporary The stored data is not limited to frame data. In addition, the video processor 120 in this embodiment is a video engine (video engine, VE), and the display processor 130 in this embodiment is a display engine (display engine, DE).

In this embodiment, the CPU 110 can control the operation of the image processing device 100 . For example: the bit stream to be decoded is controlled to enter the video processor 120 for the video processor 120 to perform MPEG2 decoding. Therefore, after the video processor 120 performs MPEG2 decoding on the bit stream, it successively reconstructs the I frame (I frame), the P frame (P frame), and the B frame (B frame). Fig. 2 depicts the macro block (macro block, MB) MB(p, q) (p=1, 2, . . . , P, q=1, 2 , ..., Q), and a macroblock row (MB row) R(p) (p=1, 2, ..., P) formed by each column (row) of macroblocks. According to this embodiment, each frame has (720*576) pixels, and each macroblock has a size of (16*16) pixels, so P=36 and Q=45. In addition, the height of each macroblock column is the height of 16 lines, which corresponds to the height of 16 pixels.

According to this embodiment, the video processor 120 may temporarily store all or part of the data of a frame in one or some buffering regions in the frame buffer 108 . For example: the video processor 120 can temporarily store the data of an entire I frame in the buffer area 108I in the frame buffer 108, and can also temporarily store the data of an entire P frame in the frame buffer successively. Buffer area 108P in 108 . Another example: the video processor 120 can temporarily store a part of the data of the B frame in the buffer area 108B of the frame buffer 108, and can also overwrite (overwrite) the subsequent decoded data of the B frame in the buffer area 108B. In, at least a part of where the part of the data is located; more specifically, the video processor 120 replaces the decoded data in the buffer area 108B with the new decoded data after being utilized (utilized) by the display processor 130, wherein the core circuit Controlled by 122 and 132 , the video processor 120 and the display processor 130 can reuse at least a part of the data in the frame buffer in the same buffer area (eg, buffer area 108I, 108P, or 108B).

The size of the storage space used to store the B frame in the buffer area 108B of this embodiment can be reduced. As shown in FIG. 3, the buffer area 108B of this embodiment includes a first area 310 and a second area 320, corresponding to a first field (field) and a second field respectively, wherein the first field represents the B The field where the frame is to be displayed first, and the second field represents the field where the B frame is displayed secondarily. In addition, the first area 310 can be divided into a private use portion 310U and a virtual portion 310V. According to this embodiment, the size of the storage space used to store the B frame in the buffer area 108B can be dynamically adjusted to allocate additional storage space, for example: the virtual frame in the first area 310 shown in FIG. 3 Part 310V. Therefore, the present invention can use these additional storage spaces (for example: the virtual part 310V) as the storage space required for other operations (non-B frame buffer storage operations), especially the newly added functions and features of the image processing device 100 The storage space required for the operation of the computer, such as: adding entertainment games, making the OSD more colorful,...etc. In this way, the above-mentioned additional storage space can be used to temporarily store the buffer data generated when the central processing unit 110 performs higher-speed or more complex calculations and the original data required for the calculations.

In this embodiment, through the control of the core circuits 122 and 132, the operation of the video processor 120 and the operation of the display processor 130 can cooperate with each other, so that in the buffer area 108B, the storage space used to store the B frame When the size is substantially reduced (for example, at least the virtual part 310V is deducted), the display processor 130 can still normally control the display of the B frame to maintain its smoothness and correctness of display.

According to this embodiment, the display processor 130 performs display control in units of macroblock columns. For example: every time the display processor 130 finishes processing 16 lines of data, the storage space occupied by the 16 lines of data is released. The video processor 120 can immediately complete the decoding of the data of a slice to generate the data of a macroblock column, and write the data of the macroblock column into the buffer area 108B. If the decoding speed of the video processor 120 exceeds the display speed controlled by the display processor 130, the video processor 120 waits for the display processor 130 until the next macroblock sequence is released. After the video processor 120 fills the buffer storage space set in the buffer area 108B (for example: the above-mentioned self-use part 310U, and/or the above-mentioned second area 320), the video processor 120 will be written according to the previous data. According to the sequence when entering the buffer storage space, the decoding results of the next macroblock sequence are overwritten from the beginning of the buffer storage space, that is, they are written from the beginning according to the same sequence.

It should be noted that the display processor 130 is correspondingly designed to repeatedly display at least a part of the data stored in the buffer area 108B sequentially. According to different implementation options of this embodiment, the size of the storage space used to temporarily store the B frame may be different, so the above-mentioned times of repeatedly displaying at least a part of the data stored in the buffer area 108B will be correspondingly Change.

According to the present embodiment, for the frame picture, the video processor 120 will simultaneously decode to reconstruct the data of two fields, namely the data of the top field and the data of the bottom field. The video processor 120 divides the data into the above-mentioned first field and the above-mentioned second field according to the order in which the data will be displayed, respectively representing the first field to be displayed and the second field to be displayed.

It should be noted that, for the frame image, when the video processor 120 decodes any B frame, it will temporarily store the data of the second field into the complete field buffer storage space, for example: as shown in FIG. 3 The second region 320 has a storage capacity sufficient to store a complete image field. On the other hand, when the video processor 120 decodes the B frame, it will temporarily store the data corresponding to the first frame into another buffer storage space that has only a partial size and is not enough to store a complete frame, for example : the self-use part 310U of the first area 310 shown in FIG. 3 , the size N (N=0, 1, . Adjust dynamically. For frame images, N≧2; that is, under typical conditions, the size of the storage capacity of the self-use part 310U can store more than two complete macroblock rows. Under the condition that the data of the above-mentioned I frame, P frame, and B frame are all compressed to 66.7% of the size, the buffer space corresponding to the B frame in the storage capacity of the memory can further save about 11,000 space. Bytes (11 kilobytes) to 191 kilobytes.

In addition, according to this embodiment, for the field picture, the size of the self-use part 310U of the first area 310 is set to zero, that is, N=0, and the extra storage space provided by the virtual part 310V is set to zero. Expanded, it can be used as an additional function and characteristic operation of the image processing device 100 . For example: the size of the extra storage space provided by the virtual part 310V is about 203 kilobytes, and the size of the second area 320 is also about 203 kilobytes.

FIG. 4 is a schematic diagram of the cooperative operation of the video processor 120 and the display processor 130 shown in FIG. 1 according to an embodiment of the present invention, wherein this embodiment is used to illustrate various possible problems faced by the embodiment shown in FIG. 1 Among the conditions, one of the stricter conditions; if the stricter condition can be implemented, the implementation of the looser condition will be easy.

As shown in FIG. 4, the time point t _B1-DIS and the time point t _B2-DIS respectively represent the time points when the display processor 130 controls the two B picture frames B1 and B2 to start displaying. Frames do not necessarily appear continuously, but are decoded according to a predetermined sequence composed of I, P, and B. Therefore, the situation shown in Figure 4 is one of the more severe conditions among the various situations that may occur. That is, the situation where two B frames appear consecutively. In addition, the sequence {O, E, 0, E, ...} shown in the upper part of Figure 4 represents the display sequence of the odd field (odd field) and the even field (even field), and the lower part of Figure 4 The time points and time intervals related to decoding are shown.

According to this embodiment, the video processor 120 first decodes the data of the first field corresponding to the B frame B1 and stores the data into the second area 320 . Here, the data of the first field corresponding to the B frame B1 represents the field data to be displayed first among the data of the B frame B1, and the time interval for decoding the data corresponding to the first field does not exceed that shown in FIG. 4 Outside the time interval T _B1-DEC shown. From the time point t _B1-DIS , the display processor 130 starts to display the data that has just been decoded, that is, the data stored in the second area 320 .

At the time point t _B1-DIS , the video processor 120 does not immediately proceed to subsequent decoding of the B frame B1. The video processor 120 will wait until the display processor 130 has used up the time T _D of at least one macroblock column in the second area 320 before starting to decode the data of the second field corresponding to the B frame B1, and start to decode the data of the second field of the frame B1. The data generated by the second image field is written into the second area 320 successively, and the data corresponding to the second image field gradually replaces the part of the data corresponding to the first image field that is no longer used by the display processor 130 . Therefore, the video processor 120 sequentially overwrites the data corresponding to the second field in the storage space originally used to temporarily store the data corresponding to the first field.

Similarly, at the time point t _B2-DEC , the video processor 120 does not decode the B frame B2 immediately. The video processor 120 will also wait for the display processor 130 to use up the time T _D of at least one macroblock column in the second area 320 before starting to decode the first field of the B frame B2, and start decoding the B frame The data generated in the first field of B2 is written into the second area 320 one after another, gradually replacing the data corresponding to the second field of the B frame B1 with the data corresponding to the first field of the B frame B2. The display processor 130 parts no longer used. Therefore, the video processor 120 sequentially overwrites the data of the first field corresponding to the B frame B2 in the storage space originally used to temporarily store the data of the second field corresponding to the B frame B1. Then, from the time point t _B2-DIS , the display processor 130 starts to display the B frame B2. Those skilled in the art should be able to understand the decoding and display operations after the time point t _{B2 -DIS} after learning the above disclosed content, so the details will not be repeated here.

Compared with the known technology, the present invention saves more buffer storage space required for the MPEG2 decoding function of low-end video products, so that some functions in higher-end products, Features are implemented in low-end products.

Another benefit of the present invention is that the image processing method and device of the present invention can maintain sufficient memory storage capacity while selectively implementing certain functions and features in higher-end products in lower-end products. To facilitate high-speed computing with central processor resources, and ensure the clarity and fluency of displayed images.

The above descriptions are only preferred embodiments of the present invention, and all equivalent changes and modifications made according to the claims of the present invention shall fall within the scope of the present invention.

Claims (12)

1. image processing method, it includes:

Decoding bit stream is to rebuild the B picture frame, and produce respectively corresponding to the data of the first figure field and corresponding to the data of the second figure field, the figure field of representing this B picture frame at first will be shown, this first figure field wherein, and its less important figure field that is shown of this B picture frame is represented in this second figure field; And

To temporarily be stored in the middle of the picture frame buffer respectively, be used for storing the personal part and the second area of the first area in the buffer area of this B picture frame to data that should the first figure field and to data that should the second figure field, or will temporarily be stored in this second area respectively to data that should the first figure field and to data that should the second figure field in the different time, wherein the size of the storage volume of this second area is enough to store a complete figure field, and the size of the storage volume of the part of should using by oneself of this first area then is not enough to store a complete figure field.

2. image processing method according to claim 1, it is at the picture frame image, be with to data that should the first figure field and to data that should the second figure field temporarily be stored in the middle of this picture frame buffer respectively, be used for storing this first area in the buffer area of this B picture frame should personal part and this second area.

3. image processing method according to claim 2, the big I of the storage volume of the part of wherein should using by oneself stores macro block column complete more than two.

4. image processing method according to claim 1, it is at the figure field picture, is will temporarily be stored in this second area respectively to data that should the first figure field and to data that should the second figure field in the different time.

5. image processing method according to claim 4, the size of the storage volume of the part of wherein should using by oneself is set to zero.

6. image processing method according to claim 1, wherein this personal part and virtual part can be divided in this first area, and this image processing method also includes:

By adjusting the size of this personal part, dynamically adjust the size of the storage area that is used for storing this B picture frame; Or

Utilize this virtual part as the required storage area of other running.

7. image processing method according to claim 1, wherein the size of the storage volume of this picture frame buffer is to be less than or equal to two Mbytes.

8. image processing apparatus, it includes:

Central processing unit is used for controlling the running of this image processing apparatus;

The picture frame buffer is used for temporarily storage data;

Video processor, be coupled to this picture frame buffer and this central processing unit, be used for decoding bit stream to rebuild the B picture frame, and produce respectively corresponding to the data of the first figure field and corresponding to the data of the second figure field, and will temporarily be stored in respectively in the middle of this picture frame buffer to data that should the first figure field and to data that should the second figure field, the figure field of representing this B picture frame at first will be shown, this first figure field wherein, and its less important figure field that is shown of this B picture frame is represented in this second figure field, and this video processor includes core circuit, is used for controlling the running of this video processor; And

Video-stream processor, be coupled to this video processor, this picture frame buffer, with this central processing unit, be used for controlling the demonstration of this B picture frame, and this video-stream processor includes core circuit, be used for controlling the running of this video-stream processor;

Wherein this video processor will also include data that should the first figure field and the running that data that should the second figure field temporarily are stored in respectively in the middle of this picture frame buffer:

To temporarily be stored in the middle of this picture frame buffer respectively, be used for storing the personal part and the second area of the first area in the buffer area of this B picture frame to data that should the first figure field and to data that should the second figure field, wherein the size of the storage volume of this second area is enough to store a complete figure field, and the size of the storage volume of the part of should using by oneself of this first area then is not enough to store a complete figure field; Perhaps

To temporarily be stored in this second area respectively to data that should the first figure field and to data that should the second figure field in the different time.

9. image processing apparatus according to claim 8, wherein at the picture frame image, this video processor be with to data that should the first figure field and to data that should the second figure field temporarily be stored in the middle of this picture frame buffer respectively, be used for storing this first area in the buffer area of this B picture frame should personal part and this second area.

10. image processing apparatus according to claim 8, wherein at the figure field picture, this video processor is will temporarily be stored in this second area respectively to data that should the first figure field and to data that should the second figure field in the different time.

11. image processing apparatus according to claim 8, wherein this video processor and this video-stream processor can reuse in the middle of this picture frame buffer, the data of at least a portion in the same buffer area.

12. image processing apparatus according to claim 8, wherein the size of the storage volume of this picture frame buffer is to be less than or equal to two Mbytes.

Images