CN1535029A - A reference image buffer management method - Google Patents

Abstract

A reference image buffer management method, including: according to the encoding method, retaining reference images for encoding/decoding in the buffer; each reference image is marked by a reference index; the current image determines the reference image according to the reference index image, complete encoding and decoding; if the current encoded/decoded image is an I frame/field or a P frame/field, replace the reference frame/field farthest from the current image in the buffer with the decoded I/P frame/field, and encode /Decode the next image; if the current encoding/decoding image is a B frame/field, encode/decode the next image without changing the reference index of the reference image; if all images have been encoded/decoded, delete the buffer in the buffer All images. The present invention improves the access efficiency of the codec to the reference images by effectively managing the reference images, only retains the images used as references, and reduces the demand for buffering. At the same time, the unified management of these reference images in the memory can improve Access speed.

Description

A reference image buffer management method

technical field

The invention relates to a buffer management method, in particular to a reference image buffer management method in a video encoding and decoding process.

Background technique

With the rapid development of science and technology, high-definition digital TV broadcasting, multimedia network communication, video monitoring, remote monitoring, etc. have increasingly prominent demands on efficient video codec technology. Efficient video codec technology is the key to realize high-quality, low-cost multimedia data storage and transmission. The more popular international standards for video coding now generally adopt a hybrid coding method of prediction/transformation. The time, space and subjective visual redundancy of video images are removed through prediction/transformation, so that the amount of data can be minimized while maintaining a certain visual effect. This prediction can be performed within the current picture or between pictures. For inter-frame prediction, a motion search strategy based on block matching is generally used now, that is, for each block in the current image, find the best matching position in its reference image according to a certain strategy. In order to improve the accuracy of prediction, it is usually possible to match in several reference images, such as B frames using two reference frames forward and backward, so the reference images need to be effectively managed in the codec to improve codec The access efficiency of the reference image in the device. Of course, the multi-frame reference prediction has a certain complexity in implementation, for example, more buffers need to be allocated to store reference images, more calculations need to be performed to select reference images, and so on.

In previous coding standards such as MPEG-1, MPEG-2, MPEG-4, etc., there are two types of inter-frame prediction, one is the P frame that only comes from the forward prediction, and the other is the B frame that performs bidirectional prediction. frame. The P frame prediction here is based on a reference image, while the B frame is predicted using two forward and backward reference images. The inter-frame prediction here is relatively simple, and the corresponding reference image buffer management is also relatively simple. It is necessary to keep the last two reference frames to perform inter-frame prediction on P and B frames.

With the rapid development of computing technology, the realization of multi-frame reference prediction is not very difficult, and can bring significant gains, because the use of multi-reference frame prediction technology is a very effective way to improve compression efficiency. However, since the complexity of too many reference frame predictions far outweighs the gains it brings, and since B frames usually need to use at least two reference frames before and after, in order to make full use of the reference image buffer , the P frame is generally restricted to use at most two reference frames, so that the size of the image buffer does not have to be increased under the condition of performing multi-frame prediction on the P frame.

An important part of the multi-reference frame prediction technology is how to effectively manage the reference image buffer. Multi-frame reference technology is also used in the latest coding standard H.264, but the reference image buffer management is very complicated. First, it saves each decoded image, no matter whether the image is used as a reference image or not, and uses The index marks its order, and there is another mark indicating whether the image is used as a reference image; in addition, there are also very complicated regulations for reference images in and out of the buffer, for example, a frame image can only be used if it is not used as a reference image and is used for It is only removed from the buffer when it is displayed. Such management is complicated, and takes up a large amount of storage space, and buffer usage is inefficient.

Contents of the invention

The present invention proposes an effective reference image buffer management method, which only saves all reference images used as references and a buffer of an image currently being reconstructed, and realizes operations such as accessing and updating of reference images.

Technical scheme of the present invention, comprises the steps:

Step 1. According to the encoding method, the reference image used for encoding/decoding is reserved in the buffer;

Step 2, each reference image is marked by a reference index;

Step 3, the current image determines the reference image according to the reference index, and completes encoding and decoding;

Step 4. If the current encoding/decoding image is an I frame/field or a P frame/field, replace the reference frame/field farthest from the current image in the buffer with the decoded I/P frame/field, and encode/decode One image, re-execute step 2;

Step 5. If the current encoding/decoding image is a B frame/field, the reference index of the reference image is not changed, and the next image is encoded/decoded, and step 3 is performed again;

Step 6, if all the images have been encoded/decoded, then delete all the images in the buffer.

The present invention improves the access efficiency of the codec to the reference images by effectively managing the reference images, only retains the images used as references, and reduces the demand for buffering, and at the same time, the unified management of these reference images in the memory can improve Access speed.

Description of drawings

Fig. 1 is when the current image is an I frame and is encoded as two fields, the current decoded field is a schematic diagram of the marking method of the second field in the display order;

Fig. 2 is a schematic diagram of a marking method in which the current image is a P frame and is coded as a frame;

Fig. 3 is a schematic diagram of a marking method in which the current decoded field is the first field in the display order when the current image is a P frame and the frame is coded as two fields;

Fig. 4 is a schematic diagram of a marking method in which the current image is a P frame and the frame is coded as two fields, and the current decoding field is the second field in the display order;

Fig. 5 is a schematic diagram of a marking method in which the current image is a B frame and is coded as a frame;

Fig. 6 is a schematic diagram of a marking method in which the current decoded field is the first field in the display order when the current image is a B frame and the frame is coded as two fields;

Fig. 7 is a schematic diagram of a marking method in which the current decoded field is the second field in the display order when the current image is a B frame and the frame is coded as two fields;

Fig. 8 is a flow chart of updating the reference image in the image buffer according to the present invention.

Detailed ways

The reference image buffer management mechanism of the present invention enables the P/B frame or field to efficiently access the reference image, and at the same time, for the macroblock of the special coding mode in the P/B frame/field, such as skipping the macroblock mode (Skip mode), direct mode (Direct mode), and symmetric mode (Symmetric mode) can directly find its forward and backward reference images when accessing reference images.

Technical scheme of the present invention, comprises the steps:

Step 1. According to the encoding method, the reference image used for encoding/decoding is reserved in the buffer;

Step 2, each reference image is marked by a reference index;

Step 3, the current image determines the reference image according to the reference index, and completes encoding and decoding;

Step 4. If the current encoding/decoding image is an I frame/field or a P frame/field, replace the reference frame/field farthest from the current image in the buffer with the decoded I/P frame/field, and encode/decode One image, re-execute step 2;

Step 5. If the current encoding/decoding image is a B frame/field, the reference index of the reference image is not changed, and the next image is encoded/decoded, and step 3 is performed again;

Step 6, if all the images have been encoded/decoded, then delete all the images in the buffer.

Both image encoding and decoding involve image encoding/decoding in the buffer, and both involve buffer management in the present invention.

In step 1 of the present invention, in the image buffer, only images used as references are reserved, and the reference images can only be I images or P images. According to different encoding methods, the number of reserved reference images is also different. For frame-based encoding, in the reference image buffer, keep the last two reference images for reference; for field-based encoding, in the reference image buffer In the area, keep the latest 4 reference field images for reference.

The marking of the reference image is a key step of the present invention. After the image is accessed in the buffer and the encoding/decoding is completed, the updating of the reference image is carried out according to the marking of the image. The invention realizes convenient image buffer management through a simple marking method.

Each reference image is marked with a reference index. The principle of marking is: the forward reference image is in the display order, the reference index is in descending order, and the reference index of the reference image closest to the current image in the display order is 0; the backward reference The images are arranged in ascending order according to the display order, and the reference index of the reference image closest to the current image in the display order is 0.

For the frame-based encoding method, if the current image is a P frame, its two forward reference frames are the two reference images in the buffer, and the image closest to the current image in the display order is marked as 0, compared with The other mark far away is 1; if the current frame is a B frame, its backward reference frame is the reference frame after the current image in the display order, marked as 0 in the backward reference order, and the forward reference frame is Reference frames preceding the current image in display order, marked with 0 in forward reference order.

For the field-based encoding method, if the current field is a P field, its four forward references are the four reference images in the buffer, and the image closest to the current image in the display order is marked as 0; if the current The field is the B field, and its backward reference field is two reference fields located after the current image in the display order, where the image closest to the current image in the display order is marked as 0, and the other is marked as 1, forward reference The field is the reference image before the current image in the display sequence, the field closest to the current image is marked as 0, and the other is marked as 1.

Below, through a series of specific examples, the marking process of the present invention is illustrated in the marking mode in different situations in decoding: the numbers in the figure represent the reference index value, the image indicated by the arrow is the reference image,

As shown in Figure 1, if the current image is an I image, and the image is coded as two fields, and the current decoding field is the second field in the display order, mark the first field above the current decoding field as 0;

As shown in Figure 2, if the current image is a P image and frame coding is performed, mark the forward reference image closest to the current image as 0; the frontmost forward reference image is 1;

As shown in Figure 3, if the current picture is a P picture, and the picture is coded as two fields, and the current decoding field is the first field in the display order, the second field marking the nearest forward reference picture of the current field is 0, In the direction away from the current image, mark 4 fields in sequence.

As shown in Figure 4, if the current picture is a P picture, and the frame is coded as two fields, and the current decoded field is the second field in the display order, the first field above the current field is marked as 0, and the field farther away from the current picture is Direction, mark 4 fields in sequence.

As shown in Figure 5, if the current picture is a B picture, and it is coded as a frame, the forward reference picture is marked as 0, and the backward reference picture is also marked as 0. These two pictures are distinguished according to the difference in time sequence from the current picture .

As shown in Figure 6, if the current picture is a B picture, and it is coded as two fields, and the current decoding field is the first field in the display order, the second field marking the forward reference picture is 0, and the first field is 1; The first field of the marked backward reference image is 0, and the second field is 1; the B image can only index the forward and backward reference images, so that under the restriction of not marking the second field below the current field, it is also marked according to the distance from the current field There are 4 reference fields.

As shown in Figure 7, if the current picture is a B picture, and it is coded as two fields, and the current decoding field is the second field in the display order, the second field marking the forward reference picture is 0, and the first field is 1; The first field of the marked backward reference image is 0, and the second field is 1, for the above reasons.

Special coding modes in some P/B frames/fields, which are also considered by the present invention:

When encoding the bottom field of an I-frame image, the bottom field is always coded as a P field, and there is only one reference image, that is, the top field, which is also the field marked as 0 in the reference image buffer.

For Direct mode in B frame or B field, the default forward and backward reference image is the nearest reference image, that is, the reference image marked as 0.

For the Symmetric mode in the B frame, its default backward reference image is derived from the forward reference image. If the forward reference image is the reference image marked 0, then the backward reference image is the reference image marked 1; otherwise, if The forward reference picture is the reference picture marked 1, and the backward reference picture is the reference picture marked 0.

After marking the image in the buffer, you can encode/decode the image according to the mark. During the encoding/decoding process of the current image, the data in the image buffer remains unchanged until the encoding/decoding of the current image is completed, and the reference image in the reference image buffer needs to be updated.

In step 3 of the present invention, the method of encoding the current image based on the reference image may be any method of predictive encoding using the reference image.

As shown in Figure 8, it is the flow chart of the present invention to update the reference image in the image buffer, decode an image, if the current image is I or P frame/field, then replace the farthest distance from the current image in the reference image buffer with this image frame/field, re-index marking.

For frame-based encoding, if the currently decoded frame is an I or P frame, replace the reference frame in the buffer with the farthest from the current image with the decoded I/P frame, and then mark in step 2.

For example, if the current decoded frame is an I or P frame, replace the decoded frame with the reference frame marked 1 in the original buffer, so that the newly inserted reference frame is the latest reference frame of the next image to be decoded , marked as 0, another image was originally marked as 0, and now it is updated to be marked as 1.

For field-based coding, if the current decoded frame is an I or P field, replace the reference field in the buffer with the farthest from the current image with the decoded I/P field, and then mark in step 2.

For example, if the currently decoded picture is an I or P field, replace the decoded picture with the reference field marked 3 in the original buffer, so that the newly inserted reference field is the nearest reference field of the next picture to be decoded , marked as 0, another image was originally marked as 2, and now it needs to be updated to be marked as 3, the original marked as 1 is first updated to 2, and the original marked as 0 is updated to 1.

Decoding B frame or B field, the buffer state remains unchanged.

If all images have been encoded/decoded, the entire encoding/decoding process has been completed, and all images in the buffer are deleted to release resources.

Finally, it should be noted that the above embodiments are only used to illustrate and not limit the technical solutions of the present invention, although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: the present invention can still be modified Or an equivalent replacement, any modification or partial replacement without departing from the spirit and scope of the present invention shall fall within the scope of the claims of the present invention.

Claims (10)

1, a kind of reference picture buffer management method is characterized in that: comprise the steps:

Step 1, according to the mode of coding, in buffering area, be preserved for the reference picture of coding/decoding;

Step 2, each reference picture all come mark by a reference key;

Step 3, present image are determined reference picture according to reference key, finish encoding and decoding;

If the current coding/decoding image of step 4 is I frame/field or P frame/field, then decoded I or P frame/field to be replaced in the buffering area from present image reference frame/field farthest, coding/decoding next one image re-executes step 2;

If the current coding/decoding image of step 5 is B frame/field, do not change the reference key of reference picture, the next image of coding/decoding re-executes step 3;

If all images of step 6 have all been finished coding/decoding, then all images in the delete buffer.

2, reference picture buffer management method according to claim 1 is characterized in that: adopt following mode in the described step 2, and the mark reference picture,

The forward direction reference picture is according to DISPLAY ORDER, and reference key is by descending, and the reference key from the nearest reference picture of present image on the DISPLAY ORDER is 0; According to DISPLAY ORDER, reference key is arranged by ascending order to reference picture in the back, and the reference key from the nearest reference picture of present image on the DISPLAY ORDER is 0.

3, reference picture buffer management method according to claim 1 is characterized in that: adopt following mode in the described step 2, and the mark reference picture,

For the coded system based on frame, if present image is the P frame, then its two forward reference frame are these two reference pictures in the buffering area, are 0 from the nearest image tagged of present image on DISPLAY ORDER wherein, and far away another is labeled as 1; For if present frame be the B frame, the back of it is the reference frame be positioned at present image on DISPLAY ORDER after to reference frame, be labeled as 0 in the back in reference sequence, forward reference frame is labeled as 0 for the reference frame before DISPLAY ORDER is positioned at present image in the forward direction reference sequence.

4, according to claim 1 or 3 described reference picture buffer management methods, it is characterized in that: adopt following mode in the described step 2, the mark reference picture, for coded system based on the field, if when the front court is the P field, its 4 forward direction reference pictures are these four reference pictures in the buffering area, are 0 from the nearest image tagged of present image on DISPLAY ORDER wherein; For if when the front court be the B field, the back of it is two reference field be positioned at present image on DISPLAY ORDER after to reference field, on DISPLAY ORDER, be 0 wherein from the nearest image tagged of present image, another is labeled as 1, the forward direction reference field is the reference picture before DISPLAY ORDER is positioned at present image, the field flag nearest from present image is 0, and another is labeled as 1.

5, reference picture buffer management method according to claim 1 is characterized in that: in the described step 3, for the I two field picture encode in the field at the end time, the field into P is always compiled in field, the end, its reference picture is to be labeled as 0 field in the reference picture buffering area.

6, reference picture buffer management method according to claim 1 is characterized in that: in the described step 3, under the Direct Model in B frame or the B field, its default front and back are labeled as 0 reference picture to being referenced as.

7, reference picture buffer management method according to claim 1 is characterized in that: in the described step 3, for the symmetric pattern in the B frame, if the forward direction reference picture is the reference picture of mark 0, then the back is to be labeled as 1 reference picture to reference picture; If the forward direction reference picture is the reference picture of mark 1, then the back is to be labeled as 0 reference picture to reference picture.

8, reference picture buffer management method according to claim 1, it is characterized in that: in the described step 4, the decoding piece image is if present image is I or P frame/field, then replace reference picture buffering area middle distance present image frame/field farthest, carry out index marker again with this image.

9, reference picture buffer management method according to claim 1, it is characterized in that: in the described step 4, for coding based on frame, if current decoded frame is I or P frame, then decoded I/P frame is replaced in the buffering area from present image reference frame farthest, carried out the mark in the step 2 again.

10, reference picture buffer management method according to claim 1, it is characterized in that: in the described step 4, for coding based on the field, if current decoded frame is I or P field, then decoded I/P field is replaced in the buffering area from present image reference field farthest, carry out the mark in the step 2 again.

Images