TWI862229B - Video decoding method and decoder device - Google Patents

Abstract

A video decoding method is applied to a decoder device to decode an encoded stream stored in a memory. The decoder device includes a video decoder, a memory controller, and a processor, and the video decoding method includes: setting, by the memory controller, a decoded picture buffer in the memory based on a maximum number X of reference frames in the encoded stream, in which the decoded picture buffer includes X+1 picture buffers, each picture buffer is configured to store a decoded image, and X is a positive integer; determining an idle picture buffer in the decoded picture buffer based on state information of the decoded picture buffer; and controlling the video decoder to decode a current frame in the encoded stream to obtain a decoded image of the current frame and store the decoded image of the current frame to the idle picture buffer.

Description

Video decoding method and decoding device

This application relates to the field of video encoding and decoding technology, and specifically to a video decoding method and decoding device.

The decoded picture buffer (DPB) is a storage area used to store multiple decoded frames during video decoding and used as reference frames for subsequent images. Since the decoding order and display order of backward reference frames are different, if the characteristics of such reference frames are not fully considered when setting the DPB, the DPB will occupy a large amount of storage space and increase the cost of the decoding device.

In view of the shortcomings of the prior art, this application provides a video decoding method and decoding device to improve the shortcomings of the prior art.

An embodiment of the present application provides a video decoding method, which is applied to a decoding device to decode a coded stream stored in a memory. The decoding device includes a video decoder, a memory controller, and a processor. The video decoding method includes: based on the maximum reference frame number X of the coded stream, the memory controller sets a decoded image buffer in the memory, and the decoded image buffer The buffer includes X+1 image buffers, each image buffer is used to store a decoded image, and X is a positive integer; Based on the status information of the decoded image buffer, determine the free image buffer of the decoded image buffer; control the video decoder to decode the current frame in the coded stream to obtain a decoded image, and store the decoded image of the current frame in the image buffer.

Another embodiment of the present application provides a decoding device, the decoding device includes a video decoder, a memory controller and a processor, the decoding device is used to decode a coded stream stored in a memory, wherein: the processor is used to set a decoded image buffer in the memory through the memory controller based on the maximum number of reference frames X of the coded stream, the decoded image buffer includes X+1 images Image buffer, each image buffer is used to store a decoded image, X is a positive integer; the processor is also used to determine the free image buffer of the decoded image buffer based on the status information of the decoded image buffer; the processor is also used to control the video decoder to decode the current frame in the coded stream to obtain a decoded image, and store the decoded image of the current frame in the image buffer.

The video decoding method and decoding device of this application can efficiently allocate buffers during the decoding process. Compared with the previous technology, the video decoding method and decoding device of this application can more efficiently allocate buffers during the decoding process, effectively improving the buffer utilization rate.

The features, implementation and effects of this application are described in detail below with reference to the accompanying drawings.

The technical terms used in the following descriptions refer to the customary terms in this technical field. If this manual explains or defines some of the terms, the interpretation of these terms shall be based on the explanation or definition in this manual.

The disclosure of this application includes a video decoding method and a decoding device. Since some of the components included in the decoding device of this application may be known components individually, the following description will omit the details of the known components without affecting the full disclosure and feasibility of the device application. In addition, part or all of the process of the video decoding method of this application may be in the form of software and/or firmware, and can be executed by the decoding device of this application or its equivalent device. Without affecting the full disclosure and feasibility of the method application, the following description of the method application will focus on the steps rather than the hardware.

FIG1 is a schematic diagram of the structure of a decoding device provided in an embodiment of the present application. The decoding device 101 includes a video decoder (VD) 102, a memory controller 103 and a processor 104. The decoding device 101 may be a decoding chip, which may decode the encoded stream (ES) data stored in the memory 111. The memory controller 103 may be a memory controller (Memory Controller), which may be implemented by a memory interface unit (MIU) circuit and/or a direct memory access (DMA) circuit. The memory 111 stores a video decoder driver (VDD) program 115. The processor 104 reads and executes the VDD program 115 from the memory 111 through the memory controller 103 to control the video decoder 102 to decode the encoded stream 112.

The decoding process of the coded stream 112 by the decoding device 101 will be described in detail below with reference to FIG. 2 and FIG. 3.

FIG2 is a flow chart of a video decoding method provided by an embodiment, including the following steps.

In step 201, the processor 104 sets a decoded image buffer in the memory 111 through the memory controller 103 based on the maximum reference frame number X of the encoded stream.

Specifically, the processor 104 reads the sequence parameter set (SPS) data of the coded stream 112 from the memory (e.g., dynamic random access memory (DRAM) 111) through the memory controller 103, parses the SPS data to obtain the maximum number of reference frames (denoted as X) of the coded stream 112, and allocates a decoded picture buffer (denoted as Decoded Picture Buffer) for the decoded picture in the memory 111 through the memory controller 103 according to the maximum number of reference frames. Buffer (DPB) area 113. In one embodiment, the DPB area includes at least X+1 image buffers, each of which is used to store a decoded image. The SPS data includes reference frame information of each frame in the coded stream and display order information of each decoded image.

In step 202, the processor 104 determines whether there is an idle image buffer in the decoded image buffer based on the status information of the decoded image buffer. If yes, step 203 is executed; if not, step 204 is executed.

The processor 104 obtains the status information of the decoded image buffer based on the mark information of each stored image buffer. The mark information includes the display order (Picture Order Count, POC) mark and the reference frame mark of the decoded image. The display order mark is used to indicate that the decoded image is a frame to be displayed and its display order. When a decoded image is to be displayed, the display order mark of the decoded image is added; when the decoded image is displayed, the display order mark is cleared or changed to a null value. When a decoded image needs to be used as a reference image for decoding other images, the decoded image includes a reference frame mark; when the decoded image does not need to be used as a reference image for decoding other images, the reference frame mark is cleared or changed to a null value. The status information of the decoded image buffer includes marking information of each image buffer, and the marking information includes a display order mark and a reference frame mark of the decoded image in the image buffer. When the display order mark and reference frame mark of the decoded image in the image buffer are both cleared (for example, the mark values are all null values), the processor 104 determines that the image buffer storing the decoded image is in an idle state; when one of the display order mark and reference frame mark of the decoded image exists (for example, one of the mark values is a non-null value), the processor 104 determines that the image buffer storing the decoded image (whose mark information is non-null) is in a non-idle state.

Before decoding a frame in the coded stream, the processor 104 determines whether there is an empty image buffer in the decoded image buffer based on the status information of the decoded image buffer (ie, the marking information of each image buffer) to determine a target image buffer for storing the decoded image of the frame. The target image buffer can be an empty image buffer in the DPB area (i.e., there is an empty image buffer in the DPB area), or it can be an image buffer newly added on the basis of the DPB area (i.e., if there is no empty image buffer in the DPB area, then an image buffer is newly added to expand the DPB area so that the decoded image can be stored in the newly added image buffer in the DPB area).

In a specific embodiment, the encoded stream 112 includes the reference picture control information (RPCI) of each frame (Encoded Frame, EF) and the display order of the decoded image (Picture Order Count, POC). The RPCI and POC of each frame are preset in the SPS data of the encoded stream 112. The processor 104 obtains the RPCI of the current frame from the coded stream 112 (i.e., which decoded images in the decoded image buffer are needed to decode the current frame) and updates the RPCI mark of the decoded images in the decoded image buffer according to the RPCI (e.g., clears the RPCI mark of one or more decoded images) to identify the free image buffer in the decoded image buffer according to the updated RPCI mark and POC mark. The processor 104 also obtains the POC of the current frame from the coded stream 112 to add a POC mark to the target image buffer. The processor 104 updates its POC mark in time according to the display status of the decoded image in the DPB area.

In step 203, the processor 104 selects an idle image buffer from the decoded image buffer as the target image buffer.

In step 204, the processor 104 adds a new image buffer as a target image buffer to expand the decoded image buffer.

In step 205, the processor 104 controls the video decoder 102 to decode the current frame in the encoded stream to obtain a decoded image, and stores the decoded image in the target image buffer via the memory controller 103.

After the target image buffer is determined, the video decoder 102 decodes the current frame to obtain the decoded image of the current frame, and stores the decoded image of the current frame in the target image buffer through the memory controller 103.

In some implementations, the processor 104 may be a central processing unit (CPU) or a microcontroller unit (MCU), which is not limited in this application.

In some implementations, the decoding device 101 further includes a post-processing circuit (PPC) 105. The post-processing circuit 105 is controlled by the processor 104 and is used to perform further post-processing on the decoded image decoded by the video decoder 102, such as adjusting the resolution, rotating, scaling, etc., and storing the decoded image in the memory 111 through the memory controller 103 after the processing is completed. Correspondingly, the memory 111 also includes a post-processing circuit buffer (PPCB) area 114, and the PPCB area 114 is used to store the image to be displayed. The image to be displayed is the image obtained by the post-processing circuit 105 performing post-processing on the decoded image. The memory 111 also stores a post-processing circuit driver (PPC Driver, PPCD) program 116. The processor 104 reads and executes the PPCD program 116 from the memory 111 through the memory controller 103 to control the post-processing circuit 105 to read the decoded image from the DPB area 113 of the memory 111 through the memory controller 103, and perform post-processing on the decoded image, and then the memory controller 103 stores the image obtained by the post-processing in the PPCB area 114.

Specifically, the processor 104 can also obtain the maximum delayed display number (denoted as Y) of the coded stream 112 by parsing the SPS data. During the decoding process, the processor 104 obtains the number of image buffers with display order marks (or whose values are not null) in the DPB area of the memory 111 through the memory controller 103. When the number is greater than the maximum delayed display number, the processor 104 will display the image buffers with display order marks (or whose values are not null). When the number of images displayed is Y, the image buffer with the smallest display order (e.g., the smallest mark value) indicated by the display order mark in the DPB area is determined, and the post-processing circuit 105 is controlled to read the decoded image from the image buffer, and the decoded image is post-processed, and the image obtained by the post-processing is then stored in the PPCB area 114 through the memory controller 103.

In some feasible implementations, SPS data is usually the first data of a coding stream, or the same coding stream may also contain multiple sequences, and the SPS data of each sequence is recorded in the first data of the coding stream. Each SPS data includes syntax applicable to all frames under the sequence, mainly including: SPS identification (ID), resolution, maximum number of reference frames, and maximum number of delayed display (max reorder delay) frames.

It can be seen that the decoding device 101 provided in the embodiment of the present application dynamically manages the buffer space size of the decoded image buffer according to the status information of the decoded image buffer. The present application can achieve more efficient buffer allocation while ensuring normal decoding, reduce buffer waste, and reduce the hardware cost of the decoding device.

FIG3 is a schematic diagram of the architecture of a video decoding system provided by an embodiment of the present application. In the video decoding system, the processor 104 reads and executes the VDD program 115 in the memory 111 to send a decoding control instruction to the video decoder 102. The video decoder 102 decodes the frame in the coded stream in response to the instruction and stores the decoded image in the DPB area of the memory 111. The microcontroller MCU 104 reads and executes the PPCD program 116 in the memory 111 to send a display processing instruction to the post-processing circuit 105. The post-processing circuit 105 responds to the instruction to perform post-processing on the decoded image stored in the DPB area, such as adjusting the resolution, rotating, scaling, etc. of the decoded image. The microcontroller MCU 104, the video decoder 102, and the post-processing circuit 105 can read/write the data of the memory 111 through the memory controller 103.

Among them, the initial space size of the DPB area 113 of the memory 111 is set to (X+1) image buffers, X is the maximum number of reference frames, and the image buffers in the DPB area 113 are used to store the decoded images of each frame in the coded stream. During the decoding process, it can be determined whether to increase the image buffer according to the status information of the DPB area 113. For example, the DPB area 113 can be increased to (X+Y) image buffers, and Y is the maximum number of delayed display frames. During the decoding process, the marking information of each image buffer in the DPB area 113 can be updated in time according to the reference frame information of each frame in the coded stream and whether the decoded image in the DPB area 113 has been processed by the later stage.

In Figure 3, N, N+1, N+2, ..., N+16 represent decoded images obtained by decoding a frame in the coded stream according to a decoding order, and POC 0, POC 2, POC 1, ..., POC 8 represent the display order of the decoded images. For example, (N, POC 0) in DPB indicates that the decoded image is decoded from the Nth frame in the coded stream, and the display order of the decoded image is 1, and (N+16, POC 8) indicates that the decoded image is decoded from the (N+16)th frame, and the display order of the decoded image is 9. In addition, before decoding the (N+16)th frame, it is determined that there is no free image buffer in the DPB area 113, so an additional image buffer is added to expand the DPB area 113 (such as the rightmost image buffer in the DPB area 113 in FIG. 3), and the newly added image buffer is used to store the decoded image of the (N+16)th frame, and a display order mark is added according to the display order of the decoded image, and a reference frame mark is added according to whether the decoded image is used as a reference frame for decoding other frames.

In another embodiment, the processor 104 obtains the maximum delayed display number of the coded stream 112 (counted as Y) by parsing the SPS data, and sets the DPB area according to the maximum reference frame number X of the coded stream and the maximum delayed display number Y. The DPB area may include X+1 to X+Y image buffers, that is, the number of image buffers included in the initially allocated DPB area is greater than or equal to X+1, and less than or equal to X+Y, 1<Y<=X. During the decoding process, the processor 104 makes full use of the buffer of the DPB area according to the status information of the DPB, and improves the utilization rate of the buffer space in the DPB area while ensuring that the decoding is carried out normally.

In another embodiment, if the post-processing circuit 105 needs to process multiple decoded images to be displayed at the same time, the post-processing circuit 105 will perform post-processing on the multiple decoded images after the video decoder 102 decodes the multiple images. In this way, the DPB area needs to store V decoded images that can be processed by the post-processing circuit PPC105 at the same time. In this case, the processor 104 initially sets the number of image buffers in the DPB area to the sum of the maximum number of reference frames X, the number of decoded images to be processed simultaneously V and 1, that is, the DPB area includes (X+V+1) image buffers, and during the decoding process, the buffer space size of the DPB area can be dynamically managed according to the status information of the decoded image buffer, so as to achieve more efficient buffer allocation while ensuring normal decoding.

In another embodiment, if the post-processing circuit 105 needs to process multiple decoded images to be displayed at the same time, so that the post-processing of the previous decoded images cannot be completed at the same time as the decoder outputs the decoded images, assuming that the post-processing circuit 105 needs to buffer V decoded images additionally, that is, the buffer number of frames to be displayed is V, then the processor 104 can set the initial size of the DPB area to Set to the sum of the maximum number of reference frames X, the maximum number of delayed display frames Y, and the number of decoded images to be processed simultaneously V, that is, the DPB area includes (X+Y+V) image buffers, and during the decoding process, the buffer space size of the DPB area can be dynamically managed according to the status information of the decoded image buffer, while ensuring that the decoding is carried out normally, to achieve more efficient buffer allocation.

The following is an example to illustrate the process of dynamically managing the decoded image buffer during the decoding process of the present application. FIG4a is a schematic diagram of a coding stream sequence provided by an embodiment of the present application. The coding stream sequence includes 21 frames, and its SPS records the maximum number of reference frames of the coding stream sequence (such as max_ref_num in the figure) as 4, and the maximum number of delayed display frames (such as reorder delay in the figure) as 4. Figure 4a shows the decoding order (decode order), image display order (POC) and reference frame information (reference POC, i.e., the reference relationship based on the display order of the decoded image) of each frame. For example, the first frame in the coded stream has a decoding order of 1, a POC of 0 (indicating that the decoded image is displayed first), and a reference POC of NA (no reference frame, i.e., no reference to other frames when decoding this frame); the fifth frame in the coded stream has a decoding order of 5, a POC of 2 (indicating that the decoded image is displayed third), and a reference POC of 0, 16, 8, 4 (i.e., the four decoded images with display orders of 0, 16, 8 and 4 need to be referenced when decoding this frame). Other frames will not be described one by one.

Compared with FIG4a, FIG4b shows the reference relationship between frames according to the POC of the decoded image. For example, in FIG4b, the B frame with a POC of 8 includes directed arrows pointing to the I frame with POC 0 and the I frame with POC 16, indicating that the decoded images of the two frames with POC 0 and POC 16 need to be referred to when decoding the frame; the B frame with a POC of 4 includes directed arrows pointing to the I frame with POC 0 and the B frame with POC 8. At the same time, since the B frame with POC 8 includes directed arrows pointing to the I frame with POC 0 and the I frame with POC 16, the decoded images of the three frames with POC 0, 16 and 8 need to be referred to when decoding the frame. The display order and reference frames of other frames will not be described one by one.

In the example shown in FIG. 4a and FIG. 4b, the processor 104 initially sets the buffer space of the DPB area to 5 (i.e., 4+1) image buffers based on the maximum number of reference frames 4. Since the maximum number of delayed display frames is 4, the coded stream needs to decode at least 4 decoded images before displaying its decoded images, and the decoded images with POC of 0 are not displayed until the 5th frame in the coded stream is decoded. That is, the decoding process of the 5th frame and the display processing of the decoded images with POC of 0 are performed simultaneously, and FIG. 4c uses this as the starting point for illustrating the decoding process. FIG. 4c to FIG. 4e are schematic diagrams showing the management process of the decoded image buffer when decoding the coded stream.

Decode the 5th frame: Take the 5th frame as the current frame. Before decoding the 5th frame, the first 4 frames of decoded images have occupied 4 image buffers in the DPB. According to the reference relationship shown in Figure 4a or 4b, the marking information of the first 4 image buffers includes the display order mark and the reference frame mark. Obtain the compressed image and reference frame information of the 5th frame from the encoded stream (the reference frame of the 5th frame includes four decoded images with POCs of 0, 16, 8 and 4), the POC of the decoded image and the information used as a reference frame (the reference frame information of the 5th frame, the POC of the decoded image and the information used as a reference frame can be included in its compressed image data ), there is no need to update the reference frame marker in the marker information of the first four picture buffers when decoding the 5th frame, and the marker information of the picture buffer remains unchanged. Since there is still an empty picture buffer in the DPB at this time (that is, the 5th picture buffer, whose marker information is empty), the 5th picture buffer can be used to store the decoded picture of the 5th frame. Next, decode the 5th frame, store the decoded image of the 5th frame in the 5th image buffer, and set the mark information of the 5th image buffer to include the display order mark according to the display order of the decoded image of the 5th frame, and add the association between the display order POC 2 of the decoded image of the 5th frame and the 5th image buffer in the status information of the DPB area.

When the 5th frame is decoded, the state information of the DPB area includes the number of image buffers with display order marks, which is 5 (5 is greater than Y (Y=4)). At this time, the decoded image with the smallest POC of the decoded image in the DPB area is determined, that is, the decoded image marked by POC 0. The post-processing circuit 105 obtains the decoded image from the image buffer corresponding to POC 0, processes the decoded image and stores the processed decoded image in the PPC buffer for display. When the post-processing circuit 105 completes reading the decoded image, it sends information (such as interruption) to the processor 104, and the processor 104 responds to the message to clear the display order mark in the mark information of the first image buffer.

At this time, the marker information of the first image buffer only includes the reference frame marker, and the marker information of the second to fourth image buffers remains unchanged.

Decode the 6th frame: Obtain the compressed data of the 6th frame, reference frame information (the reference frame of the 6th frame is the decoded image with POC 0, 16, 8 and 4 respectively), decoded image POC, reference frame information, etc. from the encoded stream (or obtain the reference frame, decoded image POC, reference frame information, etc. from the compressed data), and determine based on the reference frame information that D does not need to be updated The reference frame mark of each image buffer in the PB area. At this time, the marking information of the 5 image buffers in the DPB is not empty, that is, there is no idle image buffer in the DPB, and one new image buffer (i.e., the 6th image buffer) is required to expand the DPB area, and the decoded image of the 6th frame is stored in the newly added image buffer (i.e., the 6th image buffer). Next, the 6th frame is decoded, and the decoded image of the 6th frame is stored in the 6th image buffer. Based on the display order of the decoded image and the fact that it is not used as a reference frame, the marking information of the 6th buffer is set to include a display order mark, and the association between the decoded image display order POC 1 and the 6th image buffer is added to the status information of the DPB area. At this time, the status information of the DPB area includes 5 display order marks, and the decoded image with the smallest POC among the marked decoded images is determined, that is, the decoded image marked by POC 1. The post-processing circuit 105 obtains the decoded image from the 6th image buffer corresponding to POC 1 for processing, and stores the processed decoded image in the PPC buffer for display. When the post-processing circuit 105 completes reading the decoded image, it sends information to the processor 104, and the processor 104 responds to the message to clear the display order mark in the mark information of the 6th image buffer. In this way, the mark information of the 6th image buffer is empty, that is, the 6th image buffer is in an idle state.

Decode the 7th frame: obtain the compressed data, reference frame information, decoded image POC and non-reference frame information of the 7th frame from the encoded stream, and determine that there is no need to update the reference frame mark of each image buffer in the DPB area based on the reference frame information of the 7th frame (the reference frame of the 7th frame includes decoded images with POC of 0, 16, 8 and 4), and determine that the mark information of one (i.e., the 6th) image buffer in the DPB is empty at this time, that is, the 6th image buffer is free. Next, decode the 7th frame, store the decoded image of the 7th frame in the 6th image buffer, set the marking information of the 6th image buffer to include only the display order mark based on the display order of the decoded image and not used as a reference frame, and add the association between the decoded image display order POC 3 and the 6th image buffer in the status information of the DPB area.

When the decoding of the 7th frame is completed, the status information of the DPB area includes the number of image buffers with display order marks of 5, the decoded image with the smallest POC determined (decoded image marked with POC 2), and the post-processing circuit 105 obtains the decoded image from the 5th image buffer corresponding to POC 2 for processing, and stores the processed decoded image in the PPC buffer for display. The processor 104 clears the display order mark in the mark information of the 5th image buffer in response to the message sent by the post-processing circuit 105 that the decoded image has been read. In this way, the mark information of the 5th image buffer is empty, that is, the 5th image buffer is in an idle state.

Decode the 8th frame: The specific decoding process can be found in frame 7. The difference is that the decoded image of the 8th frame is stored in the 5th image buffer, and the marking information of the 5th image buffer is set to include the display order mark and the reference frame mark based on the display order of the decoded image and the use as a reference frame, and the association between the decoded image display order POC 6, the reference frame mark and the 5th image buffer is added to the status information of the DPB area.

When decoding frame 8, the post-processing circuit 105 sequentially processes the decoded image marked with POC3 from the image buffer in the DPB for display. After the post-processing circuit 105 completes the reading, the processor 104 marks the display order in the marking information of the image buffer.

The subsequent 9 frames of the coded stream can be understood according to the decoding process of the 5th to 8th frames mentioned above, and will not be elaborated here.

It can be seen that, combined with Figure 4c, Figure 4d and Figure 4e, the decoding of the coded stream from the 1st frame to the 21st frame has met the coded stream sequence requirements of the maximum number of reference frames of 4 and the maximum number of delayed display frames of 4. In addition, the maximum number of image buffers used by the DPB is 7, which is when decoding the 14th to 17th frames; compared with the initial 5 image buffers of the DPB, 2 more are added, which are when decoding the 6th and 14th frames respectively. Compared with the prior art in which 9 (i.e. X+Y+1) image buffers are initially allocated to the DPB, the image buffer occupied by the DBP can be saved.

FIG5a is a schematic diagram of another coded stream sequence provided by the embodiment of the present application. The coded stream sequence includes 12 frames, and its SPS includes a maximum reference frame number of 4 and a maximum delayed display frame number of 2. FIG5a shows the decoding order, image display order, and reference frame information reference POC (i.e., reference relationship) of each frame. A person with ordinary knowledge in the field can understand the detailed information of each frame in the coded stream in FIG5 from the above description of FIG4a, so it will not be repeated one by one.

Figure 5b shows the reference relationship between frames according to the decoded image POC. A person with ordinary knowledge in the field can know the detailed content of each frame in Figure 5b from the above content of Figure 4b, so it will not be described one by one.

In this embodiment, the processor 104 sets the initial image buffer space of the DPB area to 5 image buffers based on the maximum reference frame number 4. Since the maximum delayed display number is 2, that is, at least 2 decoded images of the coded stream must be decoded before the decoded image is displayed, the decoded image with a POC of 0 in the decoded image is displayed when the 3rd frame in the coded stream is decoded. Figure 5c uses this as the starting point for illustrating the decoding process.

X

16，1

Y

X。 In some embodiments (e.g., for decoding of the HEVC or AVC standards), 1

X

16,1

Y

X.

A person with ordinary knowledge in the field can know the decoding process of each frame in Figures 5c to 5d from the above contents of Figures 4c to 4e, so the decoding process of the frames in the coded stream will not be described one by one.

Compared with the embodiment in FIG4a, the maximum number of reference frames of the coded stream in the embodiment shown in FIG5a is also 4, but the maximum number of delayed display frames is 2. During decoding, the DPB image buffer stores a maximum of 6 frames, which occurs when decoding the 9th frame, that is, the DPB only needs a maximum of (X+2) image buffers, which saves 1 image buffer compared to the prior art in which the DPB is initially set to 7 (X+Y+1) image buffers.

Although the embodiments of this application are described above, the multiple embodiments are not used to limit this application. People with ordinary knowledge in this technical field may change the technical features of this application according to the explicit or implicit content of this application. All these changes may fall within the scope of patent protection sought by this application. In other words, the scope of patent protection of this application shall be subject to the scope of patent application defined in this specification.

101:Decoding device

102: Video decoder

103:Memory controller

104: Processor

105: Post-processing circuit

111:Memory

112: Encoded stream

113: Decoded image buffer

114: Post-processing circuit buffer

115: Video decoder driver

116: Post-processing circuit driver

X,max_ref_num: Maximum number of reference frames

Y, reorder delay: maximum number of delayed display sheets

V: The number of decoded images that need to be processed simultaneously

decode order:decoding order

reference POC: the image display order of the reference frame

NA: No reference frame

I: Intra-frame coded frame

B: Backward reference frame

P: Forward reference frame

201,202,203,204,205: Steps

[Figure 1] is a schematic diagram of the structure of a decoding device provided by an embodiment of the present application; [Figure 2] is a schematic diagram of the process of a video decoding method provided by an embodiment of the present application; [Figure 3] is a schematic diagram of the architecture of a video decoding system provided by an embodiment of the present application; [Figure 4a] is a schematic diagram of the reference relationship of a coded stream sequence provided by an embodiment of the present application; [Figure 4b] is a schematic diagram of the reference relationship of another coded stream sequence provided by an embodiment of the present application; [Figure 4c] is a schematic diagram of buffer management in the decoding process of the 5th to 11th frames of a coded stream sequence provided by an embodiment of the present application; [Figure 4d] is a schematic diagram of the buffer management in the decoding process of the 12th to 11th frames of a coded stream sequence provided by an embodiment of the present application 4e is a schematic diagram of buffer management during the decoding process of the 17th to 21st frames of a coded stream sequence provided by an embodiment of the present application; 5a is a schematic diagram of another reference relationship of a coded stream sequence provided by an embodiment of the present application; 5b is a schematic diagram of another reference relationship of a coded stream sequence provided by an embodiment of the present application; 5c is a schematic diagram of buffer management during the decoding process of the 3rd to 7th frames of a coded stream sequence provided by an embodiment of the present application; and 5d is a schematic diagram of buffer management during the decoding process of the 8th to 12th frames of a coded stream sequence provided by an embodiment of the present application.

201,202,203,204,205: Steps

Claims (16)

A video decoding method is applied to a decoding device to decode a coded stream stored in a memory, the decoding device comprising a video decoder, a memory controller and a processor, the video decoding method comprising: Based on the maximum reference frame number X of the coded stream, a decoded image buffer is set in the memory by the memory controller, the decoded image buffer comprises X+1 image buffers, each image buffer is used to store a decoded image, and X is a positive integer; Based on the status information of the decoded image buffer, an idle image buffer in the decoded image buffer is determined; and Control the video decoder to decode the current frame in the coded stream to obtain a decoded image of the current frame, and store the decoded image of the current frame in the image buffer. The video decoding method of claim 1 further comprises: Based on the status information of the decoded image buffer, determining that there is no free image buffer in the decoded image buffer, adding an image buffer to expand the decoded image buffer, and the added image buffer is used to store the decoded image of the current frame. The video decoding method of claim 1, wherein the status information includes marking information of each of the image buffers, the marking information includes a display order mark and a reference frame mark of the decoded image stored in the decoded image buffer, and the video decoding method further includes: Obtaining reference frame information from the encoded stream; and Updating the reference frame mark of the decoded image stored in the decoded image buffer based on the reference frame information to correspond to updating the marking information of each image buffer in the decoded image buffer. As in the video decoding method of claim 3, the step of determining an empty image buffer of the decoded image buffer based on the status information of the decoded image buffer includes: When the marking information of the image buffer is empty, it is determined that an empty image buffer exists in the decoded image buffer. The video decoding method of claim 3 further includes: Based on at least one of the display order information of the decoded image of the current frame and the reference frame information, at least one of the display order mark and the reference frame mark is generated, and the mark information of the image buffer storing the decoded image of the current frame is updated accordingly. The video decoding method of claim 5 further includes: Displaying the decoded image stored in the decoded image buffer, and updating the display order mark in the mark information of the image buffer storing the decoded image accordingly. The video decoding method of claim 1 further comprises: Based on the maximum delayed display number Y of the encoded stream and the display order mark included in the mark information of each image buffer, reading the current decoded image to be displayed from the decoded image buffer; and Clearing the display order mark of the image buffer storing the current decoded image to be displayed, Y is a positive integer. As in the video decoding method of claim 1, the setting of the decoded image buffer based on the maximum reference frame number X of the coded stream includes: The decoded image buffer is set based on the maximum reference frame number X of the coded stream and the maximum delayed display frame number Y, the number of image buffers included in the decoded image buffer is greater than or equal to X+1 and less than or equal to X+Y, X＞=Y＞1, and Y is a positive integer. A decoding device for decoding a coded stream stored in a memory, wherein the decoding device comprises a video decoder, a memory controller and a processor, and the processor is used to: Based on the maximum number of reference frames X of the coded stream, a decoded image buffer is set in the memory by the memory controller, the decoded image buffer comprises X+1 image buffers, each image buffer is used to store a decoded image, and X is a positive integer; Based on the status information of the decoded image buffer, an empty image buffer in the decoded image buffer is determined; and Control the video decoder to decode the current frame in the coded stream to obtain a decoded image of the current frame, and store the decoded image of the current frame in the image buffer. A decoding device as claimed in claim 9, wherein when it is determined based on the status information of the decoded image buffer that there is no free image buffer in the decoded image buffer, the processor adds a new image buffer to expand the decoded image buffer, and the newly added image buffer is used to store the decoded image of the current frame. A decoding device as claimed in claim 9, wherein the state information includes marking information of each of the image buffers, the marking information includes a display order mark and a reference frame mark of the decoded image stored in the decoded image buffer, and the processor is used to: obtain reference frame information from the encoded stream; and update the reference frame mark of the decoded image stored in the decoded image buffer based on the reference frame information to correspond to updating the marking information of each image buffer in the decoded image buffer. A decoding device as claimed in claim 11, wherein when the marking information of the image buffer is empty, the processor determines that there is an empty image buffer in the decoded image buffer. A decoding device as claimed in claim 11, wherein the processor generates at least one of a display order mark and a reference frame mark based on display order information of the decoded image of the current frame and at least one of the reference frame information, and updates the mark information of the image buffer storing the decoded image of the current frame accordingly. A decoding device as claimed in claim 13, wherein the processor is used to: Display the decoded image stored in the decoded image buffer, and update the display order mark in the mark information of the image buffer storing the decoded image accordingly. The decoding device of claim 9, wherein the processor is used to: read the current decoded image to be displayed from the decoded image buffer based on the maximum delayed display number Y of the encoded stream and the display order mark included in the mark information of each image buffer; and clear the display order mark of the image buffer storing the current decoded image to be displayed, Y is a positive integer. A decoding device as claimed in claim 9, wherein the processor is used to: Set a decoded image buffer based on the maximum number of reference frames X and the maximum number of delayed display frames Y of the coded stream, wherein the number of image buffers included in the decoded image buffer is greater than or equal to X+1 and less than or equal to X+Y, X＞=Y＞1, and Y is a positive integer.

Images