JPH08265751A - Image regenerator based on MPEG system - Google Patents

Abstract

(57) [Abstract] [Purpose] It is possible to alleviate image distortion due to a bitstream error that occurs in a B picture during MPEG image reproduction. [Structure] An error detecting means 16a for detecting an error occurring in a bit stream of encoded data by the MPEG system, an error determining means 16b for determining whether or not the above error occurs in a B picture, and the above error exists. Slice substitute means 16c for performing processing for replacing one slice at a position to be replaced with one slice at the same position in the picture output immediately before, and a work memory for storing a plurality of already decoded pictures. 12 to 1
4 is provided, and when a bitstream error occurs in a B picture, one slice at the error occurrence position is replaced (overwritten) with one slice at the same position in the picture output immediately before. The error occurrence can be repaired in minimum units, and the original B picture can be output almost as it is as a whole.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image reproducing device, and more particularly, it is suitable for use in an apparatus for processing a bit stream error which may occur during image reproduction by the MPEG system.

[0002]

2. Description of the Related Art In recent years, MPEG (Moving Picture Image Coding) has been used as one of compression techniques for digital image signals.
A technique called Experts Group) has been developed. The compression technique of the image signal by this MPEG is ISO
/ IEC (International Organization for Standardization / International Electrotechnical Commission), and is used for storage media such as VideoCD.

For image reproduction by the above-mentioned storage medium, V
As in the case of TR playback, fast forward, rewind, midway playback, reverse playback, etc. are required. In order to realize such a requirement, MPEG1 employs a structure called GOP (Group of Pictures) for a sequence of image data. That is, in MPEG1, the coded image data is created based on the preceding and following image data, so that it is not information that is completed by only one image. For this reason, a video sequence is configured with a GOP, which is a collection of several pieces of image data, as a unit.

That is, as shown in FIG. 5, MPEG1
In the storage medium using, the GO in which a plurality of GOP data are prepared after the sequence start header
The video sequence is formed by sequentially storing the P header (data used for cueing for random access). A sequence end code is recorded at the end of this video sequence.

Three types of image types are defined for the image data in the GOP. That is, I picture,
There are three types, P picture and B picture. Of these image data, the I picture is image data generated by intraframe coding. Therefore, at the time of decoding, one image is reconstructed only by the data of one I picture itself.

The P picture is image data generated by the inter-frame forward predictive coding, and is data obtained by coding the difference from the immediately preceding I picture or P picture. A B picture is image data generated by bidirectional predictive coding, and encodes a difference from an I picture or P picture immediately preceding in time and a difference from an I picture or P picture immediately following in time. It is the converted data.

FIG. 6 is a diagram showing an example of the arrangement of the above-mentioned image types. FIG. 6A shows a sequence of image types in the original image, and FIG. 6B is in the video sequence (bitstream) 1 shown in FIG. 5 obtained when the original image is encoded. FIG. 6C shows the arrangement of image types in one GOP, and FIG. 6C shows the arrangement of image types during image display.

As is apparent from FIG. 6A, the image types in the original image are in the order of B, I, B, P, B, P, .... Further, as is apparent from FIG. 6B, one GOP is composed of m (for example, 15) pictures.

As described above, the B picture is a difference from the I picture or P picture located immediately before in time,
And data obtained by encoding the difference from the I picture or P picture located immediately after in time. Therefore, at the time of decoding a B picture (at the time of image reproduction), it is necessary that the I picture and the P picture located temporally before and after the B picture have already been decoded.

Therefore, in MPEG1, when encoding an original image, the pictures are rearranged in consideration of the processing order at the time of decoding. That is, the arrangement of pictures in each GOP in the bit stream of encoded data is I, B, P, B, P, as shown in FIG.
The order is B, ... Note that, for example, B picture 1
Is coded after I picture 2 and P picture 4 are first coded, and is inserted between them.

When the pictures are rearranged in this way, the image data is decoded in the order of the pictures shown in FIG. 6B. As a result, the I picture and P picture necessary for decoding the B picture are always decoded before the processing of the B picture.

Then, the output order of the image data decoded in the order shown in FIG. 6B is rearranged by using a frame memory or the like not shown. As a result, FIG.
As shown in FIG. 6, the image data is output in the same order as the original image shown in FIG.

In the example of FIG. 6, only one I picture is included in one GOP, but a plurality of I pictures may be included. Also, only one B picture is included between each I picture and P picture, or between each P picture and P picture, but there may be cases where multiple B pictures are included. .

[0014]

As described above, the MP
In EG1, when an image is reproduced from a bit stream based on a video sequence as shown in FIG. 5,
Decoding is performed in the order of pictures shown in FIG. However, an error may occur due to some cause during transmission of the bit stream. When the image is reproduced with this error left as it is, the displayed image is disturbed and becomes very unsightly.

Therefore, when such a bitstream error occurs, it is required to hide the error. However, MPEG1 does not provide any means for hiding the bitstream error. That is, even if a bitstream error occurs during image reproduction, there is a problem in that the disturbance in the reproduced image caused by the error cannot be corrected.

Therefore, as one method for preventing the viewer from feeling the image distortion due to the bitstream error, a method of replacing the picture in which the error has occurred with the picture immediately before the occurrence of the error can be considered. However, simply replacing the picture with the immediately preceding picture is not preferable because it causes discomfort in the reproduced image when the difference between the replaced picture and the original picture is large.

When an error occurs in an I picture or P picture, the error may be propagated to other pictures. However, when an error occurs in a B picture, the error is propagated to other pictures. There is no. This difference is due to the nature of whether the picture is referenced by other pictures during decoding.

The present invention has been made in view of the above circumstances, and an object of the present invention is to make it possible to mitigate image disturbance caused by a bitstream error during image reproduction. In particular, it is an object of the present invention to be able to mitigate image distortion when a bitstream error occurs in a B picture that has no risk of error propagation.

[0019]

An image reproducing device according to the MPEG system of the present invention is an image reproducing device according to the MPEG system having a function of processing an error occurring in a bit stream of encoded data at the time of reproducing an image according to the MPEG system. At the time of reproducing an image based on the MPEG system, an error detecting means for detecting an error occurring in the bitstream and an error detected by the error detecting means are a B picture in the bitstream. Error discriminating means for discriminating whether or not an error has occurred, and, when the error discriminating means judges that an error has occurred in the B picture, one slice at a position where the error detected by the error detecting means exists , Processing to replace with one slice at the same position in the picture output immediately before It is and a slice substitute means for performing.

Another feature of the present invention is that
In an image reproducing device according to the MPEG system having a function of processing an error occurring in a bit stream of encoded data during image reproduction according to the MPEG system, a decoding means for decoding a picture in the bit stream, and a decoding means for decoding by the decoding means. A plurality of work memories storing the stored pictures, a picture selecting means for selecting and outputting any one of the pictures stored in the plurality of work memories, and reproducing an image according to the MPEG system. An error detecting means for detecting an error occurring in the bitstream, and an error determining means for determining whether or not the error detected by the error detecting means has occurred in a B picture in the bitstream. The error determination means determines that an error has occurred in the B picture. When the error occurs, the slice at the same position as the slice at the position where the error has occurred is read from the work memory in which the picture output immediately before is stored, and the slice in the B picture stored in another work memory is read. And a slice substitute means for replacing the slice by overwriting the slice at the error occurrence position with the read slice.

[0021]

Since the present invention comprises the above technical means, an error occurring in a B picture existing in a bit stream is repaired by replacing a normal one slice at the same position in a picture output immediately before. It becomes possible to recover by the smallest unit having the header in the bitstream, and as a whole, the original B picture can be output almost as it is.

[0022]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of an image reproducing device according to the MPEG system of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing a schematic configuration of the image reproducing device according to the present embodiment. The image reproducing device of the present embodiment is provided with a device for processing a bit stream error which may occur during image reproduction by the MPEG method.

That is, in FIG. 1, 11 is a decoding unit for sequentially decoding the image data in each GOP included in the bit stream of the input encoded data (see FIG. 5). The GOP is, for example, as shown in FIG.
It is assumed that the configuration is as shown in (b). in this case,
In the decoding unit 11, following the GOP header, I picture 2 → B picture 1 → P picture 4 → B picture 3 → P
Image data is input and decoded in the order of picture 6 → B picture 5 → ...

Reference numeral 12 is a first work memory for storing the I picture or P picture decoded by the decoding section 11. A second work memory 13 stores the I picture or P picture decoded by the decoding unit 11. 14 is the third
Is a work memory for storing the B picture decoded by the decoding unit 11. Each picture stored in each of the work memories 12 to 14 after being decoded is read by the decoding unit 11 and used for decoding processing of a picture input thereafter.

Reference numeral 15 is an output selection unit, which is the first to third units described above.
Of each of the pictures stored in the work memories 12 to 14 is selected and output. Reference numeral 16 denotes an error control unit, which includes an error detection unit 16a, an error determination unit 16b, a slice substitute unit 16c, and a freeze unit 1.
6d is provided, and when an error occurs in the bit stream input to the decoding unit 11, the decoding unit 11 and the output selecting unit 15 are controlled as described later.

Here, the operation when decoding the pictures arranged as shown in FIG. 6B will be described. As is clear from FIG. 5, in reality, GOPs having a structure as shown in FIG. 6B are sequentially input to the decoding unit 11 and decoded.
Therefore, each of the pictures decoded by the immediately preceding GOP is already stored in the first to third work memories 12 to 14. Here, it is assumed that the last P picture decoded by the immediately preceding GOP (hereinafter referred to as P picture 0 with a code “0”) is stored in, for example, the first work memory 12.

First, the I picture 2 is decoded by the decoding unit 1.
It is input to 1 and decoded. And this decrypted I
The picture 2 is stored in the second work memory 13.
Next, the B picture 1 is input to the decoding unit 11.
Decoding of B picture 1 requires P picture 0, which is located immediately before in time and has already been decoded, and I picture 2 which is located immediately after in time and has already been decoded. And the second work memories 12 and 13, respectively. Therefore, the decoded P picture 0 and I picture 2 are read from the first and second work memories 12 and 13 by the decoding unit 11, and these P picture 0 and I picture 2 are used. The B picture 1 is decoded.

Then, the decoded B picture 1 is
It is stored in the third work memory 14. At this time, the output selection unit 15 selects the output from the third work memory 14. As a result, the B picture 1 is output to the next stage, and the reproduced image is displayed on a display device (not shown) or the like. That is, for B picture 1, the decoding unit 1
What is decoded by 1 is directly selected by the output selection unit 15 and output to the next stage (FIG. 6B).
(C)).

Next, the P picture 4 is input to the decoding unit 11. The decoding of the P picture 4 requires the I picture 2 which is located immediately before in time and has already been decoded. The I picture 2 is stored in the second work memory 13. Therefore, the I picture 2 already decoded from the second work memory 13 is read by the decoding unit 11,
The I picture 2 is used to decode the P picture 4. The decoded P picture 4 is stored in the first work memory 12.

When the P picture 4 is being decoded in this way, the output selection unit 15 selects the output from the second work memory 13. As a result, the I picture 2 is output to the next stage after the B picture 1,
The reproduced image is displayed on a display device (not shown) or the like (see FIGS. 6B and 6C).

Similarly, each picture input in the order of FIG. 6B is sequentially decoded by the decoding unit 11, and each decoded picture is the first to third work memory 1.
It is stored in any of 2 to 14. Then, the output selection unit 1
5, any of the pictures stored in the first to third work memories 12 to 14 is sequentially selected and output to the next stage. As a result, the sequence shown in FIG.
The reproduced images are displayed in the same order as the arrangement of pictures in the original image in (a).

When an error is found in the input bit stream during such operation, the error control section 16 performs the following processing. As shown in Table 1 below, a bitstream error occurs when a value (data in the rightmost column in the fourth row from the top) that does not make sense when interpreted according to the VLC table appears. When such an error occurs, for example, the number of pixels forming one pixel block may become a number other than the original 64 pixels (8 vertical pixels × 8 horizontal pixels). Therefore, this is monitored. If so, it is possible to detect the occurrence of an error.

[0033]

[Table 1]

In this embodiment, the error coping method is divided into two types depending on whether or not an error occurring in a certain picture may be propagated to another picture. That is, as described above, in the MPEG system, I picture,
There are three types of pictures, P-pictures and B-pictures, of which only B-pictures are not referenced by other pictures during decoding. That is, the B picture is not used when decoding a certain picture. Therefore, the error that occurred in the B picture does not propagate to other pictures.

On the other hand, the I picture and P picture are referred to by other pictures at the time of decoding. For example, at the time of decoding the B picture 3 shown in FIG. 6B, the I picture 2 which is located immediately before in time and already decoded and the P picture 4 which is located immediately after in time and already decoded are used. It Therefore, an error that occurs in an I picture or B picture propagates to other pictures.

Therefore, in this embodiment, the method of coping with the error is changed depending on whether an error has occurred in the B picture or in the I picture or P picture. . It should be noted that which picture the error has occurred is, for example, in which part of the bitstream as shown in Table 1 a value which does not make sense when interpreted as the VLC table or a value which does not exist in the VLC table. It can be specified by detecting that.

That is, when the error determining means 16b determines that the error detected by the error detecting means 16a has occurred in the B picture, the slice substituting means 16 is used.
The c controls the decoding means 11 so that one slice at the position where the error has occurred is substituted by the slice at the same position in the picture output (displayed) immediately before.

Here, a slice is a minimum unit having a header in a series of data strings, and an error occurrence position can be specified in units of this slice. As a result of the determination by the error determining means 16b, the error detected by the error detecting means 16a is I
When it occurs in a picture or a P picture, the output selection unit 15 is controlled by the freeze means 16d, and the image immediately before the error is detected is continuously output and frozen.

As for the slice, for example, as shown in FIG. 3, when an error occurs in B picture 3, one slice at the position where the error occurred is output immediately before I.
The slice at the same position in picture 2 is used instead. Specifically, the slice substituting means 16c controls the decoding unit 11 to first generate an error from the I picture 2 stored in either the first work memory 12 or the second work memory 13. Read the slice at the same position as the position. Then, the slice at the error occurrence position of the B picture 3 stored in the third work memory 14 is overwritten with the read slice to replace the slice.

By doing so, the slice in which an error has occurred is replaced by the error-free slice output immediately before. As a result, the image is displayed without any error, so that the disorder of the reproduced image caused by the bit stream error occurring in the B picture can be alleviated.

Moreover, in the case of the present embodiment, the whole picture is not replaced, but the minimum unit called a slice is substituted, so that the distortion of the image is alleviated while the original image is almost maintained. You can Therefore, when the error processing is performed by replacing the slices, it is possible to further reduce the feeling of strangeness given to the viewer of the reproduced image.

On the other hand, when an error occurs in the I picture and P picture, it is not preferable to substitute only the error occurrence slice with the slice in the immediately preceding picture. Because the I picture and the P picture are referred to by other pictures at the time of decoding, unlike the B picture, if only a part of the picture is substituted,
This is because the image displayed after that becomes unnatural.

Therefore, when an error is detected in an I picture or a P picture, it is preferable to substitute the entire immediately preceding picture. In this case, it is easy to keep displaying the picture immediately before the error occurs to the beginning of the next GOP, that is, to freeze, as one method for preventing the viewer from feeling the image distortion. realizable. However, if the displayed image is simply frozen to the beginning of the next GOP, an image disturbed by the next GOP is often displayed.

The reason for this will be described with reference to FIG. As described above, unlike the order of each picture in the GOP and the order of each displayed picture, the B picture 1 at the second position of the GOP is displayed earlier than the I picture 2 at the beginning of the GOP. It Also, the B picture 1
Is the last P picture 0 in the immediately preceding GOP and the GOP
It is created by referring to two pictures, I picture 2 in the picture.

Therefore, when P picture 0 is not correctly decoded as when an error occurs, B picture 1 cannot be correctly decoded. That is, in this case, due to the difference between the display order and the decoding order, the error generated in the previous GOP is propagated to the next GOP, and the display of the B picture 1 of the next GOP is disturbed. become.

Therefore, in the present embodiment, when an error is detected in an I picture and a P picture, FIG.
The freeze means 16d controls the picture output immediately before to output the first I picture in the GOP next to the GOP in which the error has occurred.

That is, in this embodiment, when the display freeze is not released at the beginning of the next GOP (time t1 in FIG. 4), the first I picture 2 of the next GOP is displayed (see FIG. By providing the error control unit 16 with a mechanism for canceling the freeze at time t2) of 4), it is possible to prevent the disordered B picture 1 from being displayed.

FIG. 2 is a flow chart showing the operation of error processing performed by the error control unit 16 shown in FIG. In FIG. 2, first, when the occurrence of a bitstream error is detected in step P1, it is determined in which picture the bitstream error occurred in step P2.

If the error occurs in the B picture, the process proceeds to step P3. In Step P3, as described above, the slice substituting means 16c of the error control unit 16 controls the decoding unit 11, and one slice at the position where the error occurs in the B picture has the same position in the picture output immediately before. Of slices (see FIG. 3). Then, the process goes to step P5 to return to the normal decoding process.

On the other hand, if the error occurs in the I picture or P picture, the process proceeds to step P4. In step P4, as described above, the freeze means 16d
The output selection unit 15 is controlled by the, and the immediately preceding I picture or P picture stored in either the first work memory 12 or the second work memory 13 is selected.

The selection of the immediately preceding picture by the output selection unit 15 is continued until the I picture in the GOP next to the GOP in which the error has occurred is displayed. As a result, the I picture or P picture displayed immediately before is
It is frozen until the first I picture of the next GOP is displayed (see Figure 4). Then, the process goes to step P5 to return to the normal decoding process.

As described above, when a bitstream error occurs in an I picture or a P picture, a normal picture having no error continues to be displayed in place of the picture in which the error has occurred, so an image generated by the bitstream error is displayed. It is possible to reduce the turbulence. At that time, not only the GOP including the picture in which the error occurred,
It is possible not to display a disturbed image even in the next GOP.

[0053]

As described above, according to the present invention, an error occurring in a B picture existing in a bitstream is repaired by replacing a normal one slice at the same position in a picture output immediately before. Therefore, the error can be repaired by the smallest unit having the header in the bitstream, and the normal B picture can be output as it is as a whole. As a result, it is possible to alleviate image distortion without changing the original image, and it is possible to obtain a reproduced image with less discomfort for the viewer.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an image reproducing device according to an MPEG system which is an embodiment of the present invention.

FIG. 2 is a flowchart showing an operation at the time of error processing.

FIG. 3 is a diagram conceptually showing error processing when an error occurs in a B picture.

FIG. 4 is a diagram conceptually showing error processing when an error occurs in an I picture and a P picture.

FIG. 5 is a diagram showing a video sequence of encoded data.

6A and 6B are diagrams showing the arrangement of pictures. FIG. 6A shows the arrangement of pictures in an original image, FIG. 6B shows the arrangement of pictures in a GOP, and FIG. 6C shows the arrangement of pictures when an image is displayed. It is a figure.

[Explanation of symbols]

 11 Decoding section 12, 13, 14 Work memory 15 Output selection section 16 Error control section 16a Error detection means 16b Error determination means 16c Slice substitute means 16d Freeze means

Claims (2)

[Claims] 1. An image reproducing device according to the MPEG system having a function of processing an error occurring in a bit stream of encoded data during image reproduction according to the MPEG system, when the image according to the MPEG system is reproduced. Error detecting means for detecting an error occurring in the bitstream; error determining means for determining whether the error detected by the error detecting means has occurred in a B picture in the bitstream; and the error When the determining means determines that an error has occurred in the B picture, the one slice at the position where the error detected by the error detecting means exists is the one slice at the same position in the picture output immediately before. MP comprising: a slice substitute means for performing a process for replacement. Image regenerator by EG method. 2. An image reproducing device according to the MPEG system having a function of processing an error occurring in a bit stream of encoded data during image reproduction according to the MPEG system, and decoding means for decoding a picture in the bit stream, A plurality of work memories in which the pictures decoded by the decoding means are stored, a picture selection means for selecting and outputting any of the pictures stored in the plurality of work memories, and an image based on the MPEG system. During reproduction, it is determined whether or not the error detection unit that detects an error that occurs in the bitstream and the error that is detected by the error detection unit have occurred in the B picture in the bitstream. An error is detected in the B picture by the error judging means and the error judging means. When it is determined that the error has occurred, the slice at the same position as the slice at the position where the error occurred is read from the work memory in which the picture output immediately before is stored,
A slice substitute means for performing a process of replacing a slice by overwriting the slice at the error occurrence position in the B picture stored in another work memory with the read slice. Image regenerator by method.