JP2000299856A - Device and method for generating stream, device and method for transmitting stream, device and method for encoding and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To transmit history information with a transmission path with small capacity by generating a 2nd encoded stream by encoding an encoded stream and describing combination information representing the combination of an encoding history in encoding processing. SOLUTION: A history information multiplexer 103 multiplexes the current encoding parameter supplied from a decoding device 102 and past encoding parameter outputted by a history coding device 104 into video data of a base band supplied from the device 102 according to a format and outputs them to an history information demultiplexer 105. The demultiplexer 105 extracts the encoding parameters from the video data of a base band supplied from the multiplexer 103, selects an encoding parameter being most suitable to encoding this time among them, and outputs it together with a descriptor to a combined descriptor demultiplexing part 511.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a stream generation apparatus and method, a stream transmission apparatus and method, an encoding apparatus and method, and a recording medium, and more particularly, to a coded bit stream encoded based on the MPEG standard. GO
Stream generating apparatus and method, stream transmitting apparatus and method, encoding apparatus and method suitable for use in transcoding apparatus for changing the structure of P (Group of Pictures) or changing the bit rate of an encoded bit stream And a recording medium.

[0002]

2. Description of the Related Art In recent years, a broadcasting station that produces and broadcasts a television program uses a moving picture expert (MPEG) to compress / encode video data.
roup) technology has become popular. In particular, when video data is recorded on a recording medium material that can be randomly accessed such as a tape, and when video data is transmitted via a cable or a satellite, the MPEG technology is becoming a de facto standard.

[0003] An example of processing in a broadcasting station until a video program produced in the broadcasting station is transmitted to each home will be briefly described. First, a video camera and VTR (Video
The source video data is encoded and recorded on a magnetic tape by an encoder provided in a camcorder integrated with a tape recorder. At this time, the encoder of the camcorder encodes the source video data so as to be suitable for the recording format of the VTR tape. For example, the GOP structure of an MPEG bit stream recorded on the magnetic tape is a structure in which one GOP is composed of two frames (for example, I, B, I, B, I, B,...). Is done. The bit rate of the MPEG bit stream recorded on the magnetic tape is 18 Mbps.

Next, the main broadcast station performs an editing process for editing the video bit stream recorded on the magnetic tape. For this purpose, the GOP structure of the video bit stream recorded on the magnetic tape is converted into a GOP structure suitable for editing processing. The GOP structure suitable for the editing process is a GOP structure in which one GOP is composed of one frame and all pictures are I pictures. This is because an I-picture that is not correlated with other pictures is most suitable for editing on a frame basis. As an actual operation, a video stream recorded on a magnetic tape is temporarily decoded and returned to baseband video data. Then, the baseband video signal is re-encoded so that all pictures become I pictures. By performing the decoding process and the re-encoding process in this manner, a bit stream having a GOP structure suitable for the editing process can be generated.

Next, in order to transmit the edited video program generated by the above-described editing process from the main station to the local station, the bit stream of the edited video program is converted into a GOP structure and a bit rate suitable for the transmission process. I do. The GOP structure suitable for transmission between broadcast stations is, for example, a GOP structure in which one GOP is composed of 15 frames (for example, I, B, B, P, B, B, P...). A bit rate suitable for transmission between broadcast stations is generally a high bit rate of 50 Mbps or more because a dedicated line having a high transmission capacity such as an optical fiber is provided between broadcast stations. It is desirable. Specifically, the bit stream of the edited video program is once decoded and returned to baseband video data. Then, the baseband video data is re-encoded so as to have a GOP structure and a bit rate suitable for the above-described transmission between broadcast stations.

[0006] In the local station, an editing process is performed to insert a local-specific commercial into a video program transmitted from the main station. That is, similarly to the above-described editing processing, the video stream transmitted from the main station is temporarily decoded and returned to the baseband video data. Then, by re-encoding the baseband video signal so that all pictures become I pictures, a GOP suitable for editing processing is obtained.
A bit stream having a structure can be generated.

Subsequently, the video program edited in the local station is converted into a GOP structure and a bit rate suitable for the transmission process in order to transmit the video program to each home via a cable or a satellite. For example, a GOP structure suitable for transmission processing for transmission to each home is such that one GOP is 15
GOP structure composed of frames (for example, I, B,
B, P, B, B, P...), And the bit rate suitable for transmission processing for transmission to each home is a low bit rate of about 5 Mbps. Specifically, the bit stream of the edited video program is once decoded and returned to baseband video data. Then, the baseband video data is transmitted to the GO
Re-encode to have P structure and bit rate.

[0008]

As can be understood from the above description, the decoding process and the encoding process are repeated a plurality of times while the video program is transmitted from the broadcasting station to each home. Actually, processing at a broadcasting station requires various signal processings in addition to the above-described signal processing, and the decoding processing and the encoding processing must be repeated each time.

However, it is well known that the encoding process and the decoding process based on the MPEG standard are not 100% reversible processes. That is, the baseband video data before encoding and the video data after decoding are not 100% the same, and the image quality is degraded by the encoding and decoding processes. That is, as described above, if the decoding process and the encoding process are repeated, there is a problem that the image quality is deteriorated every time the process is performed. In other words, each time decoding / encoding processing is repeated, deterioration in image quality is accumulated.

[0010] The present invention has been made in view of such a situation, and has been developed in order to change the structure of a GOP (Group of Pictures) of an encoded bit stream encoded based on the MPEG standard. This makes it possible to realize a transcoding system in which image quality does not deteriorate even if the image processing is repeated.

[0011]

According to a first aspect of the present invention, there is provided a stream generation apparatus that uses a detection unit that detects an encoding history in a past encoding process of a first encoded stream and uses a detection result of the detection unit. Then, a combination of an encoding unit that encodes the first encoded stream to generate a second encoded stream, and a combination of the encoding history in the encoding process performed so far with respect to the second encoded stream. And description means for describing the combination information to be represented.

[0012] The encoding means can perform encoding by the MPEG system.

[0013] The description means stores the combination information representing the combination of the coding histories in the second format of the MPEG system.
Stream can be described as user_data.

According to a fourth aspect of the present invention, there is provided a stream generation method comprising:
A detection step of detecting an encoding history in a past encoding process of the first encoded stream; and a second encoded stream obtained by encoding the first encoded stream by using a detection result in the detection step. And a description step of writing, for the second coded stream, combination information indicating a combination of coding histories in the previous coding processing.

According to a fifth aspect of the present invention, the recording medium according to the first aspect of the invention generates a second encoded stream from the input first encoded stream and outputs the second encoded stream to a stream generating apparatus which outputs the past encoded stream of the first encoded stream. A detecting step of detecting an encoding history in the encoding process, an encoding step of encoding the first encoded stream to generate a second encoded stream by using a detection result in the detecting step, And a description step of describing combination information indicating a combination of encoding histories in the encoding processing performed up to that time. .

In the stream generating apparatus according to the first aspect, the stream generating method according to the fourth aspect, and the program of the recording medium according to the fifth aspect, the second encoded stream is Combination information indicating a combination of encoding histories in encoding processing is described.

[0017] The stream transmission apparatus according to claim 6 is
Combining means for selectively combining predetermined encoding parameters from a plurality of encoding parameters used in past encoding processing, and generation for generating encoding history information indicating the history of the encoding parameters combined by the combining means Means, and transmission means for superimposing the encoded history information generated by the generating means on the encoded stream and transmitting the information to the medium.

[0018] The coded stream may be a coded stream coded according to the MPEG system.

[0019] The multiplying means can multiply the coding history information as the user_data on the coded stream.

A stream transmission method according to a ninth aspect is characterized in that:
Generating a combination step of selectively combining predetermined coding parameters from a plurality of coding parameters used in the past coding processing, and coding history information indicating a history of the combined coding parameters by the processing of the combination step; And transmitting the encoded history information generated by the process of the generating step to the medium by multiplying the encoded stream by the encoded stream information.

According to a tenth aspect of the present invention, there is provided a program for a recording medium, comprising: a combination step of selectively combining predetermined coding parameters from a plurality of coding parameters used in past coding processing; Generating encoding history information representing the history of the encoded parameters, and transmitting the encoding history information generated by the processing of the generating step to the media by multiplying the encoded stream by the encoding step. It is characterized by including.

A stream transmission device according to claim 6, a stream transmission method according to claim 9, and a stream transmission device according to claim 10.
In the program of the recording medium described in the above, predetermined coding parameters are selectively combined from a plurality of coding parameters used in the past coding process, and coding history information of the combined coding parameters is generated. Is multiplied by the coded stream and transmitted.

According to the present invention, there is provided an encoding apparatus, wherein combination information indicating a combination of encoding parameters described in a first encoded stream in a previous encoding process of the first encoded stream. Detecting means for detecting the combination of the first and second encoding streams, based on the combination information detected by the detecting means,
Encoding means for encoding the first encoded stream into the second encoded stream based on the encoding parameters extracted by the extracting means.

[0024] The encoding means can perform encoding by the MPEG system.

The detecting means can detect the combination information from user_data of the first coded stream.

According to a fourteenth aspect of the present invention, there is provided an encoding method, comprising: combination information indicating a combination of encoding parameters described in a first encoded stream in a previous encoding process of the first encoded stream. , An extraction step of extracting an encoding parameter in the previous encoding processing of the first encoded stream based on the combination information detected by the processing of the detection step, and a processing of the extraction step. Encoding the first encoded stream into a second encoded stream based on the extracted encoding parameters.

[0027] The program of the recording medium according to claim 15 is a combination which is described in a first encoded stream and represents a combination of encoding parameters in the encoding process of the first encoded stream up to then. A detection step of detecting information, an extraction step of extracting an encoding parameter in the previous encoding processing of the first encoded stream based on the combination information detected by the processing of the detection step, and a processing of the extraction step Coding the first coded stream into a second coded stream based on the coding parameters extracted by (1).

[0028] The encoding device according to claim 11, wherein
In the encoding method according to the fourth aspect and the recording medium program according to the fifteenth aspect, the encoding parameters in the encoding process up to that point are extracted based on the combination information, and the extracted encoding parameters are used based on the extracted encoding parameters. Thus, the first encoded stream is encoded into the second encoded stream.

[0029]

DESCRIPTION OF THE PREFERRED EMBODIMENTS A transcoder to which the present invention is applied will be described below. Before that, compression coding of a moving picture signal will be described. In this specification, the term “system” refers to an entire device including a plurality of devices and units.

For example, in a system for transmitting a moving image signal to a remote place, such as a video conference system and a video telephone system, in order to use a transmission path efficiently,
An image signal is compression-encoded using a line correlation or an inter-frame correlation of a video signal.

When the line correlation is used, the image signal can be compressed by, for example, DCT (Discrete Cosine Transform) processing.

The use of the inter-frame correlation makes it possible to further compress and encode the image signal. For example, as shown in FIG. 1, when the frame images PC1 to PC3 are generated from time t1 to time t3, respectively, the difference between the image signals of the frame images PC1 and PC2 is calculated to generate PC12, and Image PC
The difference between PC2 and PC3 is calculated to generate PC23.
Normally, the images of frames temporally adjacent to each other do not have such a large change. Therefore, when the difference between them is calculated, the difference signal has a small value. Therefore, if the difference signal is encoded, the code amount can be compressed.

However, if only the difference signal is transmitted, the original image cannot be restored. Therefore, the image of each frame is set to one of three types of pictures, i.e., I picture, P picture or B picture, and the image signal is compression-coded.

That is, as shown in FIG. 2, for example, image signals of 17 frames from F1 to F17 are set as a group of pictures (GOP), and are set as one unit of processing.
Then, the image signal of the first frame F1 is coded as an I picture, the second frame F2 is processed as a B picture, and the third frame F3 is processed as a P picture. Hereinafter, the fourth and subsequent frames F4 to F17 are alternately processed as B pictures or P pictures.

As an I-picture image signal, an image signal for one frame is transmitted as it is. On the other hand, as an image signal of a P picture, basically, as shown in FIG.
As shown in (1), a difference from an image signal of an I picture or a P picture which is earlier in time is transmitted. Further, as a B picture image signal, basically, as shown in FIG. 3, a difference from an average value of both temporally preceding and succeeding frames is obtained, and the difference is encoded.

FIG. 4 shows the principle of the method for encoding a moving picture signal in this way. As shown in the figure,
Since the first frame F1 is processed as an I picture, it is directly transmitted as transmission data F1X to the transmission path (intra-picture encoding). On the other hand, the second frame F
2 is processed as a B picture, the difference between the temporally preceding frame F1 and the average value of the temporally succeeding frame F3 is calculated, and the difference is calculated as the transmission data F2.
Transmitted as X.

However, the processing as a B picture is as follows.
More specifically, there are four types. The first processing is to transmit the data of the original frame F2 as it is as the transmission data F2X (SP1) (intra coding), and is the same processing as in the case of the I picture. The second process is to calculate the difference from the temporally later frame F3 and transmit the difference (SP2) (backward prediction coding). The third process is to transmit a difference (SP3) from the temporally preceding frame F1 (forward prediction coding). Further, the fourth processing is to generate a difference (SP4) between the average value of the temporally preceding frame F1 and the average value of the following frame F3, and transmit this as transmission data F2X (bidirectional predictive coding). .

In practice, of the four methods described above, the method that minimizes the transmission data is adopted.

When transmitting the difference data, a motion vector x1 (a motion vector between frames F1 and F2) between the image of the frame (reference image) for which the difference is to be calculated (in the case of forward prediction) Or x2 (the motion vector between frames F3 and F2) (for backward prediction),
Alternatively, both x1 and x2 (in the case of bidirectional prediction) are transmitted together with the difference data.

Further, as for the frame F3 of the P picture, a difference signal (SP3) from this frame and a motion vector x3 are calculated using the frame F1 preceding temporally as a reference image, and this is transmitted as transmission data F3X. (Forward prediction coding). Alternatively, the data of the original frame F3 is transmitted as it is as data F3X (SP
1) (Intra coding). Among these methods, a method in which the amount of transmission data is smaller is selected as in the case of the B picture.

FIG. 5 shows an example of the configuration of an apparatus that encodes and transmits a moving image signal based on the principle described above and decodes it. The encoding device 1 encodes an input video signal and transmits the encoded video signal to a recording medium 3 as a transmission path. The decoding device 2 reproduces the signal recorded on the recording medium 3, decodes the signal, and outputs the decoded signal.

In the encoding device 1, the input video signal is input to the pre-processing circuit 11, where the luminance signal and the chrominance signal (color difference signal in the case of the present embodiment) are separated and A / D converted. The signals are converted from analog signals to digital signals by the devices 12 and 13. A video signal converted into a digital signal by the A / D converters 12 and 13 is transmitted to a frame memory 14.
And stored. The frame memory 14 stores the luminance signal in the luminance signal frame memory 15 and stores the color difference signal in the color difference signal frame memory 16, respectively.

The format conversion circuit 17 converts the frame format signal stored in the frame memory 14
Convert to block format signal. That is, as shown in FIG. 6, the video signal stored in the frame memory 14 is data in a frame format as shown in FIG. 6A in which V lines of H dots are collected per line. I have. As shown in FIG. 6B, the format conversion circuit 17 divides the signal of one frame into M slices in units of 16 lines. And
Each slice is divided into M macroblocks. As shown in FIG. 6C, the macro block is configured by luminance signals corresponding to 16 × 16 pixels (dots).
This luminance signal is further divided into blocks Y [1] to Y [4] in units of 8 × 8 dots. The luminance signal of 16 × 16 dots includes Cb of 8 × 8 dots.
The signal corresponds to an 8 × 8 dot Cr signal.

The data converted into the block format is supplied from the format conversion circuit 17 to the encoder 18, where the data is encoded. The details will be described later with reference to FIG.

The signal encoded by the encoder 18 is output to a transmission line as a bit stream. For example, it is supplied to the recording circuit 19 and recorded on the recording medium 3 as a digital signal.

The data reproduced from the recording medium 3 by the reproduction circuit 30 of the decoding device 2 is supplied to a decoder 31.
Decoded. For details of the decoder 31, see FIG.
2 will be described later.

The data decoded by the decoder 31 is input to a format conversion circuit 32 and converted from a block format to a frame format. The luminance signal in the frame format is supplied to and stored in the luminance signal frame memory 34 of the frame memory 33, and the color difference signal is supplied to and stored in the color difference signal frame memory 35. The luminance signal and the color difference signal read from the luminance signal frame memory 34 and the color difference signal frame memory 35 are converted into analog signals by D / A converters 36 and 37, respectively, and supplied to a post-processing circuit 38. The post-processing circuit 38 combines and outputs the luminance signal and the color difference signal.

Next, the configuration of the encoder 18 will be described with reference to FIG. The image data to be encoded is input to the motion vector detection circuit 50 in macroblock units. The motion vector detection circuit 50 processes the image data of each frame as an I picture, a P picture, or a B picture according to a predetermined sequence set in advance. Whether the image of each frame that is sequentially input is processed as one of I, P, and B pictures is determined in advance (for example, as shown in FIGS. 2 and 3, frames F1 to F17). Are grouped by I, B, P, B, P,.
-Processed as B, P).

The image data of a frame (for example, frame F1) processed as an I picture is obtained from the motion vector detecting circuit 50 by using the forward original image section 51 of the frame memory 51.
The image data of a frame (for example, frame F2) transferred and stored in a and processed as a B picture is transferred to the original image unit 51b, stored, and processed as a P picture (for example, frame F3). Data is,
The data is transferred and stored in the rear original image section 51c.

At the next timing, B
When an image of a frame to be processed as a picture (frame F4) or a P picture (frame F5) is input, the image data of the first P picture (frame F3) stored in the rear original image section 51c until then is The image is transferred to the front original image section 51a and the next B picture (frame F
The image data of 4) is stored (overwritten) in the reference original image section 51b, and the image data of the next P picture (frame F5) is stored (overwritten) in the rear original image section 51c. Such operations are sequentially repeated.

The signal of each picture stored in the frame memory 51 is read therefrom, and the prediction mode switching circuit 52 performs frame prediction mode processing or field prediction mode processing.

Further, under the control of the prediction determination circuit 54, the arithmetic unit 53 performs calculations of intra-picture prediction, forward prediction, backward prediction, or bidirectional prediction. Which of these processes is to be performed is determined according to the prediction error signal (the difference between the reference image to be processed and the predicted image corresponding to the reference image). Therefore, the motion vector detection circuit 50 generates a sum of absolute values (or a sum of squares) of the prediction error signal used for this determination.

Here, the frame prediction mode and the field prediction mode in the prediction mode switching circuit 52 will be described.

When the frame prediction mode is set, the prediction mode switching circuit 52 controls the four luminance blocks Y supplied from the motion vector detection circuit 50.
[1] to Y [4] are directly output to the arithmetic unit 53 at the subsequent stage. That is, in this case, as shown in FIG. 8, the data of the odd field lines and the data of the even field lines are mixed in each luminance block. In this frame prediction mode,
Prediction is performed in units of four luminance blocks (macroblocks), and one motion vector corresponds to four luminance blocks.

On the other hand, the prediction mode switching circuit 5
2, in the field prediction mode, the signal input from the motion vector detection circuit 50 having the configuration shown in FIG.
As shown in FIG. 9, of the four luminance blocks, the luminance blocks Y [1] and Y [2] are constituted by, for example, only the dots of the lines of the odd-numbered fields, and the other two luminance blocks Y [3 ] And Y [4] are composed of only the dots of the lines of the even-numbered fields and output to the arithmetic unit 53. In this case, two luminance blocks Y [1] and Y [2]
, One motion vector is associated, and the other two luminance blocks Y [3] and Y [4] are associated with another one motion vector.

The motion vector detecting circuit 50 outputs to the prediction mode switching circuit 52 the absolute value sum of the prediction error in the frame prediction mode and the absolute value sum of the prediction error in the field prediction mode. The prediction mode switching circuit 52 compares the absolute value sums of the prediction errors in the frame prediction mode and the field prediction mode, performs processing corresponding to the prediction mode having a small value, and outputs the data to the arithmetic unit 53
Output to

However, such processing is actually performed by the motion vector detection circuit 50. That is, the motion vector detection circuit 50 outputs a signal having a configuration corresponding to the determined mode to the prediction mode switching circuit 52, and the prediction mode switching circuit 52 outputs the signal as it is to the subsequent computing unit 53.

Note that, in the frame prediction mode, the color difference signal is supplied to the arithmetic unit 53 in a state where the data of the lines of the odd fields and the data of the lines of the even fields are mixed, as shown in FIG. In the case of the field prediction mode, as shown in FIG.
The upper half (4 lines) of b and Cr is the luminance block Y
The color difference signals of the odd fields corresponding to [1] and Y [2] are used, and the lower half (4 lines)
These are color difference signals of even fields corresponding to [3] and Y [4].

Further, the motion vector detecting circuit 50 determines whether to perform intra-picture prediction, forward prediction, backward prediction, or bidirectional prediction in the prediction determination circuit 54 as described below. Generate the sum of absolute values of prediction errors.

That is, as the sum of the absolute values of the prediction errors of the intra-picture prediction, the absolute value | ijAij | of the sum ijAij of the signal Aij of the macroblock of the reference picture and the signal A of the macroblock
The difference between the sum 絶 対 | Aij | of the absolute values | Aij | The sum of absolute values | Aij−Bij | of the difference Aij−Bij | between the signal Aij of the macroblock of the reference image and the signal Bij of the macroblock of the predicted image is used as the absolute value sum of the prediction error of the forward prediction. Bij |. In addition, the absolute value sum of the prediction error between the backward prediction and the bidirectional prediction is obtained in the same manner as in the forward prediction (by changing the predicted image to a different predicted image from that in the forward prediction).

The sum of these absolute values is supplied to the prediction judgment circuit 54. The prediction determination circuit 54 selects the smallest absolute value sum of the prediction errors of the forward prediction, the backward prediction, and the bidirectional prediction as the absolute value sum of the prediction errors of the inter prediction. Furthermore, the absolute value sum of the prediction error of the inter prediction and the absolute value sum of the prediction error of the intra prediction are compared, and the smaller one is selected, and the mode corresponding to the selected absolute value sum is set as the prediction mode. select. That is, if the sum of the absolute values of the prediction errors of the intra prediction is smaller, the intra prediction mode is set. If the sum of the absolute values of the prediction errors in the inter prediction is smaller, the mode in which the corresponding sum of the absolute values is the smallest among the forward prediction, backward prediction, and bidirectional prediction modes is set.

As described above, the motion vector detecting circuit 50
Supplies the macroblock signal of the reference image to the arithmetic unit 53 via the prediction mode switching circuit 52 in a configuration corresponding to the mode selected by the prediction mode switching circuit 52 among the frame or field prediction modes. Prediction prediction circuit 54 of the four prediction modes
, A motion vector between the predicted image corresponding to the prediction mode selected by the above and the reference image is detected, and the variable length coding circuit 5
8 and output to the motion compensation circuit 64. As described above, as the motion vector, the motion vector having the smallest absolute value sum of the corresponding prediction errors is selected.

When the motion vector detection circuit 50 reads out the I-picture image data from the front original image section 51a, the prediction determination circuit 54 sets the intra-frame or field (image) intra prediction mode (performs motion compensation) as the prediction mode. Mode) and switch 53d of arithmetic unit 53
To the contact a side. As a result, the image data of the I picture is input to the DCT mode switching circuit 55.

As shown in FIG. 10 or 11, the DCT mode switching circuit 55 converts the data of the four luminance blocks into a state where the lines of the odd field and the lines of the even field are mixed (frame DCT mode), or The state is set to one of the separated states (field DCT mode) and output to the DCT circuit 56.

That is, the DCT mode switching circuit 55
Compares the coding efficiency when DCT processing is performed by mixing data of odd fields and even fields, and the coding efficiency when DCT processing is performed in a separated state,
Select a mode with good coding efficiency.

For example, as shown in FIG. 10, the input signal has a configuration in which the lines of the odd field and the even field are mixed, and the difference between the signal of the line of the odd field and the signal of the line of the even field adjacent vertically. Is calculated, and the sum (or sum of squares) of the absolute values is calculated.

As shown in FIG. 11, the input signal has a configuration in which the lines of the odd field and the even field are separated from each other. Calculates the difference between the signals of each other and sums (or sums of squares) the absolute values of each
Ask for.

Further, the DCT mode corresponding to the smaller value is set by comparing the two (the sum of absolute values). That is, if the former is smaller, the frame DCT mode is set, and if the latter is smaller, the field DCT mode is set.

Then, the data having the configuration corresponding to the selected DCT mode is output to the DCT circuit 56, and the DCT flag indicating the selected DCT mode is transmitted to the variable-length encoding circuit 5.
8 and the motion compensation circuit 64.

As is clear from the comparison between the prediction mode (FIGS. 8 and 9) in the prediction mode switching circuit 52 and the DCT mode (FIGS. 10 and 11) in the DCT mode switching circuit 55, the luminance block The data structure in each mode is substantially the same.

When the prediction mode switching circuit 52 selects a frame prediction mode (a mode in which odd lines and even lines are mixed), the DCT mode switching circuit 5
5, it is highly possible that the frame DCT mode (a mode in which odd lines and even lines are mixed) is selected,
When the field prediction mode (mode in which the data of the odd field and the data of the even field are separated) is selected in the prediction mode switching circuit 52, the field DCT mode (the data of the odd field and the even field) is (Separated mode) is likely to be selected.

However, the mode is not always selected in this manner. In the prediction mode switching circuit 52, the mode is determined so that the absolute value sum of the prediction error becomes small, and in the DCT mode switching circuit 55, the mode is determined. , The mode is determined so that the coding efficiency is good.

The image data of the I picture output from the DCT mode switching circuit 55 is input to the DCT circuit 56, subjected to DCT processing, and converted into DCT coefficients. The DCT coefficient is input to the quantization circuit 57, quantized by a quantization scale corresponding to the data storage amount (buffer storage amount) of the transmission buffer 59, and then input to the variable length coding circuit 58.

The variable length coding circuit 58 includes a quantization circuit 57
The image data (in this case, I-picture data) supplied from the quantization circuit 57 is converted into a variable-length code such as a Huffman code in accordance with the supplied quantization scale (scale), and transmitted. Output to the buffer 59.

The variable length coding circuit 58 also sets a quantization scale (scale) by the quantization circuit 57 and a prediction mode (intra-picture prediction, forward prediction, backward prediction, or bidirectional prediction) by the prediction determination circuit 54. Mode indicating whether the motion vector has been detected), a motion vector from the motion vector detection circuit 50,
A prediction flag (a flag indicating whether the frame prediction mode or the field prediction mode has been set) from the prediction mode switching circuit 52, and a DCT flag (either the frame DCT mode or the field DCT mode) output from the DCT mode switching circuit 55 Are input, and these are also variable-length coded.

The transmission buffer 59 temporarily stores the input data and converts the data corresponding to the storage amount into a quantization circuit 57.
Output to When the remaining data amount increases to the allowable upper limit, the transmission buffer 59 increases the quantization scale of the quantization circuit 57 by the quantization control signal,
The data amount of the quantized data is reduced. Conversely, when the remaining data amount decreases to the permissible lower limit value, the transmission buffer 59 sends the quantization signal to the quantization circuit 57 by the quantization control signal.
By reducing the quantization scale of, the data amount of the quantized data is increased. In this way, overflow or underflow of the transmission buffer 59 is prevented.

The data stored in the transmission buffer 59 is read at a predetermined timing, output to a transmission path, and recorded on the recording medium 3 via the recording circuit 19, for example.

On the other hand, the I picture data output from the quantization circuit 57 is input to the inverse quantization circuit 60 and inversely quantized according to the quantization scale supplied from the quantization circuit 57. The output of the inverse quantization circuit 60 is input to an IDCT (Inverse Discrete Cosine Transform) circuit 61, subjected to an inverse discrete cosine transform process, supplied to a forward prediction image section 63a of a frame memory 63 via an arithmetic unit 62, and stored. Is done.

The motion vector detection circuit 50 is the first input when processing the sequentially input image data of each frame as, for example, I, B, P, B, P, B... After the image data of the input frame is processed as an I picture, the image data of the next input frame is processed as a P picture before the image of the next input frame is processed as a B picture. This is because a B picture involves backward prediction and cannot be decoded unless a P picture as a backward predicted image is prepared first.

Then, after the processing of the I picture, the motion vector detecting circuit 50 starts the processing of the picture data of the P picture stored in the rear original picture section 51c. Then, as in the case described above, the sum of the absolute values of the inter-frame differences (prediction errors) in macroblock units is supplied from the motion vector detection circuit 50 to the prediction mode switching circuit 52 and the prediction determination circuit 54. Prediction mode switching circuit 52
And the prediction determination circuit 54 sets a frame / field prediction mode or a prediction mode of intra-picture prediction, forward prediction, backward prediction, or bidirectional prediction in accordance with the absolute value sum of the prediction errors of the macroblocks of the P picture. I do.

When the intra-picture prediction mode is set, the arithmetic unit 53 switches the switch 53d to the contact a as described above. Therefore, this data is transmitted to the transmission path via the DCT mode switching circuit 55, the DCT circuit 56, the quantization circuit 57, the variable length coding circuit 58, and the transmission buffer 59, like the I picture data. Also,
This data is supplied to and stored in the backward prediction image section 63b of the frame memory 63 via the inverse quantization circuit 60, the IDCT circuit 61, and the calculator 62.

When the forward prediction mode is set,
The switch 53d is switched to the contact b, and the image (I-picture image in this case) data stored in the forward prediction image section 63a of the frame memory 63 is read out. Motion compensation is performed according to the motion vector output from the circuit 50. That is, the motion compensation circuit 64
4, when the setting of the forward prediction mode is instructed, the read address of the forward prediction image section 63a is set to correspond to the motion vector from the position corresponding to the position of the macroblock currently output by the motion vector detection circuit 50. The data is read out by shifting by an amount and predicted image data is generated.

The predicted image data output from the motion compensation circuit 64 is supplied to a computing unit 53a. Arithmetic unit 53a
Subtracts the predicted image data corresponding to the macroblock supplied from the motion compensation circuit 65 from the macroblock data of the reference image supplied from the prediction mode switching circuit 52, and outputs the difference (prediction error). I do. The difference data is transmitted to the transmission path via the DCT mode switching circuit 55, the DCT circuit 56, the quantization circuit 57, the variable length coding circuit 58, and the transmission buffer 59. Also,
The difference data is locally decoded by the inverse quantization circuit 60 and the IDCT circuit 61, and is input to the arithmetic unit 62.

The arithmetic unit 62 is also supplied with the same data as the predicted image data supplied to the arithmetic unit 53a. The calculator 62 adds the prediction image data output from the motion compensation circuit 64 to the difference data output from the IDCT circuit 61. As a result, image data of the original (decoded) P picture is obtained. The P-picture image data is supplied to and stored in the backward prediction image section 63b of the frame memory 63.

After the data of the I picture and the P picture are stored in the forward predicted image section 63a and the backward predicted image section 63b, respectively, the motion vector detection circuit 50 then executes the processing of the B picture. The prediction mode switching circuit 52 and the prediction determination circuit 54 correspond to the magnitude of the sum of absolute values of the differences between frames in macroblock units,
The frame / field mode is set, and the prediction mode is set to any of the intra prediction mode, the forward prediction mode, the backward prediction mode, and the bidirectional prediction mode.

As described above, in the intra prediction mode or the forward prediction mode, the switch 53d is set to the contact a or b.
Can be switched to At this time, the same processing as in the case of the P picture is performed, and the data is transmitted.

On the other hand, when the backward prediction mode or the bidirectional prediction mode is set, the switch 53d is switched to the contact point c or d.

In the backward prediction mode in which the switch 53d is switched to the contact point c, the image (P-picture in this case) data stored in the backward prediction image section 63b is read out, and the motion compensation circuit 64 As a result, motion compensation is performed corresponding to the motion vector output from the motion vector detection circuit 50. That is, when the setting of the backward prediction mode is instructed by the prediction determination circuit 54,
The motion vector detection circuit 50 shifts the read address of the backward predicted image section 63b from the position corresponding to the position of the currently output macroblock by the amount corresponding to the motion vector, and reads the data to generate predicted image data. I do.

The predicted image data output from the motion compensation circuit 64 is supplied to a calculator 53b. Arithmetic unit 53b
Subtracts the predicted image data supplied from the motion compensation circuit 64 from the macroblock data of the reference image supplied from the prediction mode switching circuit 52, and outputs the difference. This difference data is supplied to the DCT mode switching circuit 5
5, transmitted to the transmission path via the DCT circuit 56, the quantization circuit 57, the variable length coding circuit 58, and the transmission buffer 59.

In the bidirectional prediction mode in which the switch 53d is switched to the contact point d, the image data (in this case, the I-picture image) stored in the forward prediction image section 63a and the image data stored in the backward prediction image section 63b. Data (in this case, an image of a P picture) is read out, and the motion compensation circuit 64 outputs the motion vector detection circuit 5
Motion compensation is performed corresponding to the motion vector output by 0.

That is, when the setting of the bidirectional prediction mode is instructed by the prediction judgment circuit 54, the motion compensation circuit 64 stores the read addresses of the forward predicted image section 63a and the backward predicted image section 63b, and the motion vector detection circuit 50 The data is read out from the position corresponding to the position of the macroblock being output by shifting the data by an amount corresponding to the motion vector (in this case, two motion vectors for the forward predicted image and the backward predicted image). Generate data.

The predicted image data output from the motion compensation circuit 64 is supplied to a calculator 53c. Arithmetic unit 53c
Subtracts the average value of the predicted image data supplied from the motion compensation circuit 64 from the macroblock data of the reference image supplied from the motion vector detection circuit 50, and outputs the difference. The difference data is transmitted to the transmission path via the DCT mode switching circuit 55, the DCT circuit 56, the quantization circuit 57, the variable length coding circuit 58, and the transmission buffer 59.

The B picture is not stored in the frame memory 63 because it is not regarded as a predicted picture of another picture.

In the frame memory 63, the forward prediction image section 63a and the backward prediction image section 63b are switched between banks as necessary, and are stored in one or the other for a predetermined reference image. Can be switched and output as a forward predicted image or a backward predicted image.

In the above description, the luminance block has been mainly described, but the chrominance block is also processed and transmitted in units of the macroblocks shown in FIGS. As a motion vector for processing a chrominance block, a motion vector obtained by halving the motion vector of the corresponding luminance block in the vertical and horizontal directions is used.

FIG. 12 is a block diagram showing a configuration of the decoder 31 of FIG. The encoded image data transmitted via the transmission path (recording medium 3) is received by a receiving circuit (not shown) or reproduced by a reproducing device,
After being temporarily stored in the variable 1, it is supplied to the variable length decoding circuit 82 of the decoding circuit 90. The variable length decoding circuit 82 performs variable length decoding on the data supplied from the reception buffer 81, outputs a motion vector, a prediction mode, a prediction flag, and a DCT flag to the motion compensation circuit 87, and changes the quantization scale to an inverse quantization circuit. 83, and outputs the decoded image data to the inverse quantization circuit 83.

The inverse quantization circuit 83 includes a variable length decoding circuit 82
The image data supplied from the variable length decoding circuit 8
Inverse quantization according to the quantization scale supplied from 2,
Output to the IDCT circuit 84. The data (DCT coefficient) output from the inverse quantization circuit 83 is subjected to inverse discrete cosine transform processing by the IDCT circuit 84 and supplied to the arithmetic unit 85.

When the image data supplied from the IDCT circuit 84 to the computing unit 85 is I-picture data, the data is output from the computing unit 85 and the image data (P or B) which is input to the computing unit 85 later. In order to generate predicted image data (picture data), the data is supplied to and stored in the forward predicted image section 86a of the frame memory 86. This data is output to the format conversion circuit 32 (FIG. 5).

When the image data supplied from the IDCT circuit 84 is P-picture data in which the image data one frame before that is the predicted image data and is data in the forward prediction mode, the forward predicted image in the frame memory 86 Part 8
6a, the image data (I
Picture data) is read out, and the motion compensation circuit 87 performs motion compensation corresponding to the motion vector output from the variable length decoding circuit 82. Then, the arithmetic unit 85 adds the image data (difference data) supplied from the IDCT circuit 84 and outputs the result. The added data, that is, the decoded P-picture data is used to generate predicted image data of image data (B-picture or P-picture data) to be input to the arithmetic unit 85 later.
It is supplied to and stored in the backward prediction image section 86b of the frame memory 86.

Even in the case of P-picture data, the data in the intra-prediction mode is not processed in the computing unit 85 as in the case of the I-picture data, and is stored in the backward prediction image section 86b as it is.

Since this P picture is an image to be displayed next to the next B picture, it is not yet output to the format conversion circuit 32 at this point (as described above, the P picture input after the B picture The picture is processed and transmitted before the B picture).

If the image data supplied from the IDCT circuit 84 is B picture data, the variable length decoding circuit 8
2, the picture data of the I picture stored in the forward predicted image section 86a of the frame memory 86 (in the case of the forward predicted mode) and the P stored in the backward predicted image section 86b. The picture data of the picture (in the case of the backward prediction mode) or both image data (in the case of the bidirectional prediction mode) are read out, and the motion compensation circuit 87 corresponds to the motion vector output from the variable length decoding circuit 82. Is performed, and a predicted image is generated. However, if motion compensation is not required (in the case of the intra-picture prediction mode), no predicted picture is generated.

The data subjected to the motion compensation by the motion compensation circuit 87 in this way is added to the output of the IDCT circuit 84 in the arithmetic unit 85. This addition output is output to the format conversion circuit 32.

However, the added output is data of a B picture and is not used for generating a predicted image of another image, and is not stored in the frame memory 86.

After the picture of the B picture is output, the picture data of the P picture stored in the backward prediction picture section 86b is read out, and the arithmetic unit 85 is output via the motion compensation circuit 87.
Supplied to However, at this time, no motion compensation is performed.

The decoder 31 includes a prediction mode switching circuit 52 in the encoder 18 shown in FIG.
Although circuits corresponding to the mode switching circuit 55 are not shown, processing corresponding to these circuits, that is, processing for returning the configuration in which the signals of the lines of the odd field and the even field are separated to the original configuration as necessary is performed. Is executed by the motion compensation circuit 87.

In the above description, the processing of the luminance signal has been described, but the processing of the chrominance signal is similarly performed. However, the motion vector in this case is obtained by dividing the motion vector for the luminance signal by に in the vertical and horizontal directions.
Is used.

FIG. 13 shows the quality of an encoded image. The image quality (SNR: Signal to Noise Ratio) is controlled according to the picture type, the I and P pictures are of high quality, and the B picture is of poor quality compared to the I and P pictures. Is done. This is a method utilizing human visual characteristics, and the visual quality is better when the quality is vibrated than when all the image qualities are averaged. The control of the image quality corresponding to the picture type is executed by the quantization circuit 57 of FIG.

FIGS. 14 and 15 show a configuration of the transcoder 101 to which the present invention is applied, and FIG. 15 shows a more detailed configuration of FIG. The transcoder 101 performs the GO of the encoded video bit stream (encoded video bit stream) input to the decoding device 102.
The P structure and bit rate are converted into the GOP structure and bit rate desired by the operator. This transcoder 1
Although not shown in FIG. 15 for explaining the function of the transcoder 101, three transcoders having functions substantially similar to those of the transcoder 101 are connected in front of the transcoder 101. I do. That is, in order to variously change the GOP structure and bit rate of the bit stream, the first transcoder, the second transcoder, and the third transcoder are sequentially connected in series, and the third transcoder is connected to the first transcoder. Behind this figure 1
It is assumed that the fourth transcoder shown in FIG. 5 is connected.

In the following description of the present invention, the encoding process performed in the first transcoder is defined as a first generation encoding process, and the second transcoder connected after the first transcoder is defined. An encoding process performed in the transcoder is defined as a second generation encoding process, and an encoding process performed in a third transcoder connected after the second transcoder is defined as a third generation code. And a fourth transcoder connected after the third transcoder (transcoder 10 shown in FIG. 15).
The encoding process performed in 1) is defined as a fourth generation encoding process or a current encoding process.

The encoding parameters generated in the first-generation encoding process are called first-generation encoding parameters, and the encoding parameters generated in the second-generation encoding process are referred to as second-generation encoding parameters. An encoding parameter generated in the third generation encoding process is referred to as a third generation encoding parameter, and an encoding parameter generated in the fourth generation encoding process is referred to as a fourth generation encoding parameter. Will be referred to as the encoding parameter or the current encoding parameter.

First, the coded video stream ST (3r) supplied to the transcoder 101 shown in FIG.
Step d) will be described. ST (3rd) indicates that the stream is a third-generation encoded stream generated in the third-generation encoding process in the third transcoder provided in the preceding stage of the transcoder 101.
The encoded video stream ST (3rd) generated in the third-generation encoding process includes the third-generation encoding parameters generated in the third encoding process in the encoded encoded video stream ST (3rd). The sequence_header () function and sequence_exten are added to the (3rd) sequence layer, GOP layer, picture layer, slice layer, and macroblock layer.
sion () function, group_of_pictures_header () function, pictu
re_header () function, picture_coding_extension () function, p
icture_data () function, slice () function, and macroblock ()
Described as a function. The third encoded stream generated by the third encoding process as described above includes a third encoded stream.
The description of the third encoding parameter used in the encoding process of the above is defined in the MPEG2 standard, and has no novelty.

The unique point of the transcoder 101 of the present invention is that the third coded stream ST (3r
In d), not only the third encoding parameter is described, but also the first-generation and second-generation encoding parameters generated in the first-generation and second-generation encoding processes are described. That is the point.

Specifically, the first generation and second generation coding parameters are stored in the user data area of the picture layer of the third generation coded video stream ST (3rd).
This is described as a history stream history_stream (). In the present invention, the history stream described in the user data area of the picture layer of the third-generation encoded video stream ST (3rd) is referred to as “history information” or “history information”. The coding parameters described are called "history parameters" or "history parameters".

As another method, when the third-generation encoding parameter described in the third-generation encoded stream ST (3rd) is called “the current encoding parameter”, From the viewpoint of the generation encoding process, the first generation and second generation encoding processes are encoding processes performed in the past, and therefore, the third generation encoded stream ST (3rd)
The coding parameter described as the history stream described in the user data area of the picture layer is also referred to as “past coding parameter”.

As described above, not only the third encoding parameter is described in the third encoded stream ST (3rd), but also generated in the first and second generation encoding processes. The reason why the first-generation and second-generation encoding parameters are described is that even if the GOP structure and the bit rate of the encoded stream are repeatedly changed by the transcoding process, it is possible to prevent image quality deterioration.

For example, in order to change a GOP structure of a first generation coded stream, a certain picture is coded as a P picture in a first generation coding process, and the picture is converted to a B picture in a second generation coding process. In order to further change the GOP structure of the encoded stream of the second generation, the encoding is performed as a picture.
It is conceivable that the picture is encoded as a P picture again. Since the encoding process and the decoding process based on the MPEG standard are not 100% reversible processes, it is known that the image quality is deteriorated every time the encoding process and the decoding process are repeated.

In such a case, in the third generation encoding process, instead of calculating the encoding parameters such as the quantization scale, the motion vector, and the prediction mode again, the first generation encoding process is performed. Reuse the generated coding parameters such as the generated quantization scale, motion vector, and prediction mode. Newly generated quantization scale, motion vector, prediction mode by the first generation encoding process, rather than encoding parameters such as quantization scale, motion vector, and prediction mode newly generated by the third generation encoding process. Since the encoding parameters such as are clearly higher in accuracy, by reusing the first generation parameters, even if the encoding and decoding processes are repeated, the image quality degradation can be reduced.

In order to explain the processing according to the present invention, the fourth-generation transcoder 10 shown in FIG.
The process 1 will be described in more detail by way of example.

[0120] The decoding apparatus 102 decodes the encoded video included in the third-generation encoded bit stream ST (3rd) using the third-generation encoding parameters, and decodes the decoded baseband digital signal. This is a device for generating video data. Further, the decoding device 102 decodes the first-generation and second-generation encoding parameters described as a history stream in the user data area of the picture layer of the third-generation encoded bit stream ST (3rd). It is also a device.

More specifically, as shown in FIG. 16, the decoding device 102 is the decoder 3 of the decoding device 2 shown in FIG.
1 (FIG. 12), a reception buffer 81 for buffering a supplied bit stream, a variable length decoding circuit 112 for variable length decoding of an encoded bit stream, and a variable length decoding. An inverse quantization circuit 83 that inversely quantizes the obtained data according to the quantization scale supplied from the variable length decoding circuit 112, an IDCT circuit 84 that performs an inverse discrete cosine transform of the inversely quantized DCT coefficient, and performs motion compensation processing. 85, a frame memory 86, and a motion compensation circuit 87.

The variable length decoding circuit 112 decodes the third generation coded bit stream ST (3rd) to decode the third generation coded bit stream ST (3rd).
d) Third-generation coding parameters described in the picture layer, slice layer, and macroblock layer are extracted. For example, the third-generation encoding parameters extracted by the variable-length decoding circuit 112 include picture_coding_type indicating a picture type, quantizer_scale_code indicating a quantization scale step size, and a prediction mode.
macroblock_type, motion_vector indicating motion vector,
Frame / fiel indicating Frame prediction mode or Field prediction mode
d_motion_type and Frame DCT mode or Field DCT
Dct_type indicating the mode. The quatntiser_scale_code extracted in the variable length decoding circuit 112 is
The picture_coding_type, q
uatntiser_scale_code, macroblock_type, motion_vect
Parameters such as or, frame / field_motion_type, and dct_type are supplied to the motion compensation circuit 87.

The variable-length decoding circuit 112 outputs not only the encoding parameters necessary for decoding the encoded bit stream ST (3rd) of the third generation, but also the third-generation The coding parameters to be transmitted as the history information of the generation are stored in the sequence layer of the encoded bit stream ST (3rd) of the third generation, GOP
Extract from the layer, picture layer, slice layer, and macroblock layer. Of course, picture_coding_type, quatntiser_scale_code, and macr used in the third generation decoding process
oblock_type, motion_vector, frame / field_motion_typ
Third generation encoding parameters such as e and dct_type are included in the third generation history information. The type of encoding parameter to be extracted as the history information is preset by the operator or the host computer according to the transmission capacity or the like.

Further, the variable length decoding circuit 112 extracts the user data described in the user data area of the picture layer of the encoded bit stream ST (3rd) of the third generation, and extracts the user data from the history decoding device. 104.

The history decoding device 104
Are generated from the user data described in the picture layer of the third generation coded bit stream ST (3rd), using the first generation coding parameters and the second generation coding parameters This is a circuit for extracting the encoding parameters of the generations earlier than the generation (2). Specifically, the history decoding device 1
04 detects the unique History_Data_Id described in the user data by analyzing the syntax of the received user data.
rted_history_stream () can be extracted. Further, the history decoding apparatus 104 converts the converted_
1 inserted at a certain interval in history_stream ()
Get the history_stream () by removing the marker bit (marker_bit) of the bit and its history
By analyzing the syntax of y_stream (), hi
The first-generation and second-generation encoding parameters described in story_stream () can be obtained. For detailed operation of the history decoding device 104,
It will be described later.

The history information multiplexing device 103 converts the first-generation, second-generation, and third-generation coding parameters into fourth-generation coding parameters.
These first-generation, second-generation, and third-generation encoding parameters are multiplexed on the baseband video data decoded by the decoding device 102 so as to be supplied to the encoding device 106 that performs the generation encoding process. It is a circuit for making the circuit. Specifically, the history information multiplexing device 103
Are the baseband video data output from the arithmetic unit 85 of the decoding device 102, the third-generation encoding parameters output from the variable-length decoding device 112 of the decoding device 102,
Also, the first-generation coding parameter and the second-generation coding parameter output from the history decoding device 104 are received, and the first-generation, second-generation and third-generation video data are added to the baseband video data. The generation coding parameters are multiplexed. The baseband video data in which the first generation, second generation, and third generation coding parameters are multiplexed is supplied to the history information separation device 105 via a transmission cable.

Next, a method of multiplexing the first generation, second generation, and third generation coding parameters into baseband video data will be described with reference to FIGS. FIG. 17 shows one macro block of 16 pixels × 16 pixels defined in the MPEG standard. The macroblock of 16 pixels × 16 pixels is composed of four 8
A sub-block consisting of pixels x 8 pixels (Y [0],
[1], [2] and Y [3]) and four 8
Sub-block consisting of pixels x 8 pixels (Cr
[0], r [1], b [0], and Cb [1]).

FIG. 18 shows a format of video data. This format conforms to ITU Recommendation-RDT6
01, which is the so-called “D1 format” used in the broadcasting industry. This D1 format is 10
Since one bit of video data is standardized as a format for transmitting video data of one bit, one pixel of video data is
It can be represented by 0 bits.

Since the baseband video data decoded according to the MPEG standard is 8 bits, the transcoder according to the present invention employs D1 as shown in FIG.
The upper 8 bits (D9 to D2) of the 10 bits of the format are used to transmit baseband video data decoded based on the MPEG standard. Thus, when the decoded 8-bit video data is written in the D1 format, the lower 2 bits (D1
And D0) are unallocated bits.
In the transcoder of the present invention, this empty area (unalloca
ted area) to transmit history information.

The data blocks shown in FIG. 18 are composed of sub blocks (Y [0], Y [1], Y [2], Y [3], C
r [0], Cr [1], Cb [0], and Cb [1]) are data blocks for transmitting one pixel. Therefore, in order to transmit data of one macroblock, FIG. The 64 data blocks shown are transmitted. If the lower two bits (D1 and D0) are used, a total of 1024 (= 16 ×
64) bit of history information can be transmitted. Therefore, 1
Since the history information for the generation is generated to have 256 bits, the history information for the past 4 (= 1024/256) generations can be superimposed on the video data of one macroblock. In the example shown in FIG. 18, the first generation history information, the second generation history information, and the third generation history information are superimposed.

[0131] The history information separating apparatus 105 calculates the upper 8 bits of the data transmitted as the D1 format.
This is a circuit for extracting baseband video data and extracting history information from lower two bits. In the example shown in FIG. 15, the history information separating device 105 extracts baseband video data from the transmission data, supplies the video data to the encoding device 106, and outputs the first generation and second generation data from the transmission data. Generation and third generation history information is extracted and supplied to the encoding device 106 and the history encoding device 107, respectively.

The encoding device 106 encodes the baseband video data supplied from the history information separating device 105 into a bit stream having a GOP structure and a bit rate specified by an operator or a host computer. Device. GOP
Changing the structure includes, for example, the number of pictures included in a GOP, the number of P pictures existing between I pictures and I pictures, and the number of B pictures existing between I pictures and P pictures (or I pictures). It means changing the number.

In the example shown in FIG. 15, first-, second-, and third-generation history information is superimposed on the supplied baseband video data. The fourth generation encoding process is performed by selectively reusing these pieces of history information so that image quality deterioration due to the re-encoding process is reduced.

FIG. 19 is a diagram showing a specific configuration of the encoder 121 provided in the encoding device 106. This encoder 121 basically has the configuration shown in FIG.
, A motion vector detecting circuit 50, a frame / field prediction mode switching circuit 52, a computing unit 53, a DCT mode switching circuit 55, a DCT circuit 56, a quantization circuit 57, a variable length encoding circuit 58, transmission buffer 59, inverse quantization circuit 60, inverse D
It includes a CT circuit 61, a computing unit 62, a frame memory 63, and a motion compensation circuit 64. The functions of these circuits are almost the same as the functions of the encoder 18 described with reference to FIG. 7, and thus description thereof is omitted. The following mainly describes differences between the encoder 121 and the encoder 18 described with reference to FIG.

The encoder 121 has a controller 70 for controlling the operation and function of each circuit described above. The controller 70 receives an instruction regarding the GOP structure from the operator or the host computer, and determines a picture type of each picture so as to correspond to the GOP structure. The controller 70 receives information on the target bit rate from the operator or the host computer, and controls the quantization circuit 57 so that the bit rate output from the encoder 121 becomes the specified target bit rate. I do.

Further, the controller 70 receives history information of a plurality of generations output from the history information separating device 105, and performs a reference picture encoding process by reusing the history information. This will be described in detail below.

First, the controller 70 determines whether or not the picture type of the reference picture determined from the GOP structure specified by the operator matches the picture type included in the history information. That is, it is determined whether or not this reference picture has been encoded in the past with the same picture type as the designated picture type.

To explain this more clearly with reference to the example shown in FIG. 15, the controller 70 determines that the picture type assigned to this reference picture as the fourth generation encoding process is the first generation Whether the picture type of the reference picture in the encoding process, the picture type of the reference picture in the second generation encoding process, or the picture type of the reference picture in the third generation encoding process is matched. to decide.

If the picture type designated as the reference picture as the fourth-generation encoding process does not match any of the picture types in the past encoding process, the controller 70 executes the “normal encoding process”. "
I do. That is, in this case, in any of the first generation, second generation, and third generation encoding processes, the fourth generation
This means that the reference picture has not been subjected to the encoding process with the picture type assigned as the encoding process of the generation. On the other hand, if the picture type specified as the fourth-generation encoding process for this reference picture is
If the picture type matches one of the picture types in the past encoding processing, the controller 70 performs the “parameter reuse encoding processing”. That is, in this case, in the encoding processing of any of the first generation, the second generation, and the third generation, the reference picture is encoded using the picture type assigned as the fourth generation encoding processing. It has been processed.

First, the normal encoding process of the controller 70 will be described.

The motion vector detection circuit 50 detects a prediction error in the frame prediction mode and a prediction error in the field prediction mode to determine whether the frame prediction mode or the field prediction mode is to be selected. Error value of controller 70
To supply. The controller 70 compares the values of the prediction errors and selects the prediction mode with the smaller value of the prediction error. The prediction mode switching circuit 52 performs signal processing so as to correspond to the prediction mode selected by the controller 70, and supplies it to the arithmetic unit 53.

More specifically, the prediction mode switching circuit 52
When the frame prediction mode is selected, as described with reference to FIG. 8, the luminance signal is subjected to signal processing so as to be output to the computing unit 53 as it is input, and the chrominance signal With regard to, the signal processing is performed so that the odd field lines and the even field lines are mixed.
On the other hand, when the field prediction mode is selected, as described with reference to FIG. 9, for the luminance signal, the luminance blocks Y [1] and Y [2] are configured by odd field lines, and Signal processing is performed so that Y [3] and Y [4] are composed of even-numbered field lines. Regarding the color difference signal, the upper four lines are composed of odd-numbered field lines, and the lower four lines are composed of even-numbered field lines. Perform signal processing.

Further, the motion vector detecting circuit 50 determines whether to select any of the intra prediction mode, the forward prediction mode, the backward prediction mode, and the bidirectional prediction mode in each prediction mode. A prediction error is generated, and the prediction error in each prediction mode is supplied to the controller 70. The controller 70
The smallest one of the prediction errors of the forward prediction, the backward prediction, and the bidirectional prediction is selected as the prediction error of the inter prediction. Further, the prediction error of the inter prediction is compared with the prediction error of the intra prediction, and the smaller one is selected, and the mode corresponding to the selected prediction error is selected as the prediction mode. That is, if the prediction error of the intra prediction is smaller, the intra prediction mode is set. If the prediction error of the inter prediction is smaller, the mode with the smallest corresponding prediction error among the forward prediction, backward prediction, and bidirectional prediction modes is set. The controller 70 controls the arithmetic unit 53 and the motion compensation circuit 64 so as to correspond to the selected prediction mode.

The DCT mode switching circuit 55 converts the data of the four luminance blocks into a signal form in which an odd field line and an even field line are mixed in order to select either the frame DCT mode or the field DCT mode. In addition to the conversion into the frame DCT mode, the signal is converted into a signal form in which the odd field lines and the even field lines are separated (field DCT mode), and the respective signals are supplied to the DCT circuit 56. DC
The T circuit 56 calculates the coding efficiency when the odd field and the even field are mixed and performs the DCT processing, and calculates the coding efficiency when the odd field and the even field are separated and performs the DCT processing. 70. The controller 70 compares the respective coding efficiencies supplied from the DCT circuit 56, selects the DCT mode with the higher coding efficiency, and controls the DCT mode switching circuit 55 to be the selected DCT mode. .

The controller 70 receives the target bit rate indicating the target bit rate supplied from the operator or the host computer, and a signal indicating the amount of bits buffered in the transmission buffer 59, that is, a signal indicating the remaining buffer amount. , A feed buffer for controlling the quantization step size of the quantization circuit 57 based on the target bit rate and the remaining buffer capacity.
Generate ck_q_scale_code. This feedback_q_scale_c
ode is a control signal generated according to the remaining buffer capacity of the transmission buffer 59 so that the transmission buffer 59 does not overflow or underflow, and the bit rate of the bit stream output from the transmission buffer 59 Is a signal for controlling the target bit rate.

More specifically, for example, when the amount of bits buffered in the transmission buffer 59 is reduced, the quantization step size is set so that the amount of generated bits of the next picture to be encoded increases. If the amount of bits buffered in the transmission buffer 59 increases, the quantization step size is increased so that the number of bits generated in the next picture to be encoded decreases. . Feedback_q_scale_cod
e and the quantization step size are proportional, and feedback_q_scale_
The larger the code, the larger the quantization step size, and the smaller the feedback_q_scale_code, the smaller the quantization step size.

Next, the parameter reuse encoding process, which is one of the features of the transcoder 101, will be described. To better explain this process,
The reference picture is coded as a P picture in the first generation coding process, is coded as an I picture in the second generation coding process, and is coded as a B picture in the third generation coding process. It is assumed that this reference picture must be coded as a P picture in the current fourth generation coding process.

In this case, since the reference picture has the same picture type (I picture) as the picture type assigned as the fourth generation picture type and has been encoded in the first generation encoding process, the controller 70 performs an encoding process using the first-generation encoding parameters, instead of newly creating encoding parameters from the supplied video data. The encoding parameters to be reused in the fourth encoding process include, as representative parameters, quantizer_scale_code indicating a quantization scale step size and macr indicating a prediction direction mode.
oblock_type, motion_vector indicating motion vector, Fram
frame / field_mo indicating e prediction mode or Field prediction mode
and a dct_type indicating whether the DCT mode is the Frame DCT mode or the Field DCT mode.

The controller 70 does not reuse all the coding parameters transmitted as history information, but reuses the above-mentioned coding parameters for which it is assumed that it is desirable to reuse them. Encoding parameters that are considered to be desirably not used are newly generated.

Next, the encoding parameter reuse encoding process will be described focusing on the differences from the above-described normal encoding process.

The motion vector detection circuit 50 detects the motion vector of the reference picture in the above-described normal encoding process, but does not perform the motion vector motion_vector detection process in the parameter reuse encoding process. Then, the motion vector motion_vector supplied as the first generation history information is reused. The reason will be described.

Since the baseband video data obtained by decoding the third generation coded stream has been decoded and coded at least three times, the image quality is clearly degraded as compared with the original video data. I have. Even if a motion vector is detected from video data having deteriorated image quality, an accurate motion vector cannot be detected. That is, the motion vector supplied as the first generation history information is clearly a higher-precision motion vector than the motion vector detected in the fourth generation encoding process. That is, by reusing the motion vector transmitted as the first generation encoding parameter, the image quality does not deteriorate even when the fourth generation encoding process is performed. The controller 70 converts the motion vector motion_vector supplied as the first generation history information into
The motion vector information is supplied to the motion compensation circuit 64 and the variable length coding circuit 58 as motion vector information of the reference picture to be coded in the fourth generation coding process.

Further, the motion vector detection circuit 50 detects the prediction error in the frame prediction mode and the prediction error in the field prediction mode in order to determine whether the frame prediction mode or the field prediction mode is selected. In the parameter reuse coding process, the process of detecting the prediction error in the frame prediction mode and the prediction error in the field prediction mode is not performed, and the frame prediction mode supplied as the first generation history information or the field prediction mode is not performed. Mode / frame /
Reuse field_motion_type. The reason is that the prediction error in each prediction mode detected in the first generation is higher in accuracy than the prediction error in each prediction mode detected in the fourth generation encoding process, so that the prediction error is determined by the highly accurate prediction error. This is because the more optimal encoding process can be performed by selecting the predicted mode.

More specifically, the controller 70 transmits the frame / fiel supplied as the first generation history information.
The control signal corresponding to d_motion_type is supplied to the prediction mode switching circuit 52, and the prediction mode switching circuit 52
The signal processing corresponding to the reused frame / field_motion_type is performed.

Further, the motion vector detection circuit 50
In the normal encoding process, any one of the intra prediction mode, the forward prediction mode, the backward prediction mode, and the bidirectional prediction mode (hereinafter, this prediction mode is referred to as
In order to determine whether to select the prediction direction mode), the prediction error in each prediction direction mode is calculated. In this parameter reuse encoding process, the prediction error in each prediction direction mode is calculated. No, 1st
Macroblock_type supplied as generation history information
The prediction direction mode is determined based on. Because the fourth
Since the prediction error in each prediction direction mode in the first generation encoding process is more accurate than the prediction error in each prediction direction mode in the generation encoding process,
This is because the more efficient encoding process can be performed by selecting the prediction direction mode determined by the more accurate prediction error. Specifically, the controller 70
The prediction direction mode indicated by the macroblock_type included in the history information of the generation is selected, and the arithmetic unit 53 and the motion compensation circuit 64 are controlled so as to correspond to the selected prediction direction mode.

In the normal encoding process, the DCT mode switching circuit 55 compares a signal converted into a signal format in the frame DCT mode with a signal converted in the frame DCT mode in order to compare the encoding efficiency in the frame DCT mode with the encoding efficiency in the field DCT mode. ,
Although both the signal converted into the field DCT mode signal form are supplied to the DCT circuit 56, in this parameter reuse encoding processing, the signal converted into the frame DCT mode signal form and the field DCT mode signal form are used. No processing to generate both of the signals converted to
Only the processing corresponding to the DCT mode indicated by dct_type included in the history information of the generation is performed. Specifically, the controller 70 reuses the dct_type included in the first-generation history information, and performs the DCT so that the DCT mode switching circuit 55 performs signal processing corresponding to the DCT mode indicated by the dct_type.
The mode switching circuit 55 is controlled.

In the normal encoding process, the controller 70 controls the quantization step size of the quantization circuit 57 based on the target bit rate and the remaining amount of the transmission buffer specified by the operator. In the encoding process, the quantization step size of the quantization circuit 57 is controlled based on the target bit rate, the remaining amount of the transmission buffer, and the past quantization scale included in the history information. In the following description, the past quantization scale included in the history information will be described as history_q_scale_code. In the history stream described later, this quantization scale is described as quantizer_scale_code.

First, the controller 70 controls the current quantization scale feedback_q_sca in the same manner as in the normal encoding process.
Generate le_code. This feedback_q_scale_code is
This value is determined according to the remaining amount of the transmission buffer 59 so that the transmission buffer 59 does not overflow or underflow. Next, the past quantization scale included in the first generation history stream
The value of history_q_scale_code is compared with the value of the current quantization scale feedback_q_scale_code to determine which quantization scale is larger. A large quantization scale means that the quantization step is large. If the current quantization scale feedback_q_scale_cod
If e is larger than the past quantization scale history_q_scale_code, the controller 70 converts the current quantization scale feedback_q_scale_code into the quantization circuit 5
7 On the other hand, the past quantization scale history_q_
scale_code is the current quantization scale feedback_q_scale
If it is larger than _code, the controller 70
The past quantization scale history_q_scale_code is supplied to the quantization circuit 57.

That is, the controller 70 sets the largest quantization scale code among the plurality of past quantization scales included in the history information and the current quantization scale calculated from the remaining amount of the transmission buffer. select. In other words, the controller 70 is used in a quantization step in a past (first, second, and third generation) encoding process or in a current (fourth generation) encoding process. The quantization circuit 57 is controlled so as to perform quantization using the largest quantization step among the quantization steps. The reason will be described below.

For example, if the bit rate of the stream generated in the third generation encoding process is 4 [Mbps], and the encoder 121 performs the fourth generation encoding process,
Target bit rate set to 15 [Mbp
s]. At this time, since the target bit rate is increased, it is not actually the case that the quantization step should be simply reduced. Even if a picture coded in a large quantization step in a past coding process is coded with a small quantization step in a current coding process, the picture quality of the picture is improved. No. In other words, encoding with a quantization step smaller than the quantization step in the past encoding process merely increases the bit amount, but does not improve the image quality. Accordingly, the largest quantization step among the quantization steps in the past (first, second, and third generation) encoding processes or the quantization steps used in the current (fourth generation) encoding process is defined as When the quantization is performed using this, the most efficient encoding process can be performed.

Next, the history decoding device 104 and the history encoding device 107 in FIG.
Will be further described. As shown in the figure, the history decoding device 104 includes a user data decoder 201 for decoding user data supplied from the decoding device 102, a converter 202 for converting the output of the user data decoder 201, and a history from the output of the converter 202. It comprises a history VLD 203 for reproducing information.

The history encoding device 10
Reference numeral 7 denotes a history for formatting the encoding parameters for three generations supplied from the history information separating device 105.
It comprises a VLC 211, a converter 212 for converting the output of the history VLC 211, and a user data formatter 213 for formatting the output of the converter 212 into a user data format.

[0163] The user data decoder 201 decodes the user data supplied from the decoding device 102 and outputs it to the converter 202. Although details will be described later with reference to FIG. 51, the user data (user_data ())
_start_code and user_data. In the MPEG standard,
In user_data, continuous 23-bit “0” (start
(the same code as _code) is prohibited. This is to prevent that data from being erroneously detected as start_code. History information (history_stre
am ()) is the user data area (user_dat
a), and there may be such continuous “0” s of 23 bits or more, so that such “0” s of 23 bits or more are not generated. At this time, “1” is inserted at a predetermined timing, and converted_history_stream () (FIG. 38 described later)
Need to be converted to It is the converter 212 of the history encoding device 107 that performs this conversion. Converter 2 of the history decoding device 104
Numeral 02 performs a conversion process reverse to that of the converter 212 (removes “1” inserted in order not to generate consecutive “0” of 23 bits or more).

The history VLD 203 includes a converter 202
, The history information (in this case, the first-generation coding parameter and the second-generation coding parameter) is generated and output to the history information multiplexing device 103.

On the other hand, the history encoding device 10
7, the history VLC 211 corresponds to three generations supplied from the history information separation device 105 (first generation, second generation).
The encoding parameters of the generation and the third generation are converted into the format of the history information. This format includes a fixed-length format (FIGS. 40 to 46 described later) and a variable-length format (FIG. 47 described later). Details of these will be described later.

The history information formatted by the history VLC 211 is converted by the converter 212 into a conv
Converted to erted_history_stream (). This is a process for preventing the start_code of user_data () from being erroneously detected, as described above. That is, there are consecutive “0” s of 23 bits or more in the history information,
Since it is not possible to arrange consecutive “0” s of 23 bits or more in _data, the data is converted by the converter 212 (“1” is inserted at a predetermined timing) so as not to touch this prohibited item. .

The user data formatter 213 converts the converted_history_stream supplied from the converter 212.
(), History_Data_ID is added based on FIG. 38 described later, and further, user_data_stream_code is added, and v
It generates MPEG_standard user_data that can be inserted into the ideo stream and outputs it to the encoding device 106.

FIG. 20 shows an example of the configuration of the history VLC 211. The code word converter 301 and the code length converter 3
05, an encoding parameter (an encoding parameter to be transmitted as history information this time) (item data) and information for specifying a stream in which the encoding parameter is arranged (for example, syntax name (for example, sequence
_header name)) (item No.) is supplied from the history information separation device 105. The codeword converter 301
The input encoding parameter is converted into a codeword corresponding to the specified syntax, and output to the barrel shifter 302. The barrel shifter 302 shifts the codeword input from the codeword converter 301 by an amount corresponding to the shift amount supplied from the address generation circuit 306, and outputs the result to the switch 303 as a codeword in byte units. A switch 303 that is switched by a bit select signal output from the address generation circuit 306 is provided for each bit, and converts a code word supplied from the barrel shifter 302 into RA.
It is supplied to M304 and stored. The write address at this time is specified by the address generation circuit 306. When a read address is specified from the address generation circuit 306, data (codeword) stored in the RAM 304 is read and supplied to the converter 212 at the subsequent stage. hand
The data is again supplied to the RAM 304 and stored.

The code length converter 305 determines the code length of the coding parameter from the input syntax and the coding parameter, and outputs it to the address generation circuit 306. The address generation circuit 306 generates the above-described shift amount, bit select signal, write address, or read address in accordance with the input code length, and transfers them to the barrel shifter 302, the switch 303,
Alternatively, the data is supplied to the RAM 304.

As described above, the history VLC 211 is configured as a so-called variable-length coder, and performs variable-length coding on input coding parameters and outputs the result.

FIG. 21 shows a history VLD2 for decoding the history-formatted data as described above.
03 shows a configuration example. In the history VLD 203, the data of the encoding parameter supplied from the converter 202 is supplied to the RAM 311 and stored. The write address at this time is supplied from the address generation circuit 315. The address generation circuit 315 also generates a read address at a predetermined timing and supplies it to the RAM 311. At this time, the RAM 311 reads the data stored at the read address, and
Output to 2. The barrel shifter 312 shifts the input data by an amount corresponding to the shift amount output from the address generation circuit 315, and outputs the data to the inverse code length converter 313 and the inverse code word converter 314.

The inverse code length converter 313 is also supplied from the converter 202 with the name of the syntax (item No.) of the stream in which the encoding parameters are arranged. The inverse code length converter 313 calculates the code length from the input data (code word) based on the syntax, and outputs the calculated code length to the address generation circuit 315.

The inverse codeword converter 314 decodes the data supplied from the barrel shifter 312 based on the syntax (converts the data into an inverse codeword) and outputs the decoded data to the history information multiplexing device 103.

The inverse codeword converter 314 extracts information necessary for specifying what codeword is included (information necessary for determining a codeword delimiter). Output to the address generation circuit 315. The address generation circuit 315 generates a write address and a read address based on the information and the code length input from the inverse code length converter 313, outputs the write address and the read address to the RAM 311, generates a shift amount, and outputs the shift amount to the barrel shifter 312. I do.

FIG. 22 shows a configuration example of the converter 212. In this example, the buffer memory 3 arranged between the history VLC 211 and the converter 212
The 8-bit data is read from the 20 read addresses output by the controller 326, supplied to a D-type flip-flop (D-FF) 321, and held therein. Then, the D-type flip-flop 321
The read data is supplied to the stuff circuit 323 and the D-type flip-flop 32
2 is also supplied and held. D-type flip-flop 3
The 8-bit data read from 22 is combined with the 8-bit data read from the D-type flip-flop 321 and supplied to the stuff circuit 323 as 16-bit parallel data.

The stuff circuit 323 includes the controller 32
Signal indicating staff position (stuff posi
1) is inserted (stuffing) into the position of the barrel shifter 3 as a total of 17 bits.
24.

The barrel shifter 324 includes the controller 32
The input data is shifted based on a signal (shift) indicating the shift amount supplied from 6 to extract 8-bit data, and output to the 8-bit D-type flip-flop 325. The data held in the D-type flip-flop 325 is read therefrom and supplied to the subsequent user data formatter 213 via the buffer memory 327. At this time, the controller 326 generates a write address together with the output data,
2 and a user data formatter 213.

FIG. 23 shows a configuration example of the stuff circuit 323. The 16-bit data input from the D-type flip-flops 322 and 321
It is input to the contacts a of 31-16 to 331-1.
The contact c of the switch 331-i (i = 0 to 15)
The data of the adjacent switch is supplied to the MSB side (upper side in the figure). For example, to the contact c of the switch 331-12, the 13th data from the LSB supplied to the contact a of the switch 331-13 adjacent to the MSB side is supplied, and to the contact c of the switch 331-13. , MSB side, is supplied to the contact a of the switch 331-14 adjacent thereto.
The fourteenth data is supplied from the LSB side.

However, the switch 331-1 corresponding to the LSB
The contact a of the switch 331-0 on the lower side is open. Also, the switch 331-1 corresponding to the MSB
The contact c of No. 6 is open because there is no higher-order switch.

Data "1" is supplied to a contact point b of each of the switches 331-0 to 331-16.

The decoder 332 outputs a signal st indicating the position at which the data "1" supplied from the controller 326 is to be inserted.
Switches 331-0 through 33 corresponding to the uff position
Of the switches 1-16, one switch is switched to the contact b side, the switch on the LSB side is switched to the contact c side, and the switch on the MSB side is switched to the contact a side.

FIG. 23 shows the thirteenth data from the LSB side.
1 "is inserted. In this case, the switches 331-0 to 331-12 are
Both are switched to the contact c side, and the switch 331-1
3 is switched to the contact b side, and the switch 331-14
The switches 331-16 are switched to the contact a side.

The converter 212 shown in FIG. 22 converts a 22-bit code into 23 bits and outputs the converted code according to the above configuration.

FIG. 24 shows the timing of the output data of each section of the converter 212 shown in FIG. When the controller 326 of the converter 212 generates a read address (FIG. 24A) in synchronization with a clock in units of bytes, data corresponding to the read address is read from the buffer memory 320 in units of bytes, and the D-type flip-flop is read. 321 is temporarily held. The data (FIG. 24B) read from the D-type flip-flop 321 is
The signal is supplied to the stuff circuit 323 and supplied to and held in the D-type flip-flop 322. The data held in the D-type flip-flop 322 is further read therefrom (FIG. 24C) and supplied to the stuff circuit 323.

Therefore, the input of the stuff circuit 323 (FIG. 2)
4 (D)) is the first 1-byte data D0 at the timing of the read address A1, and 2 bytes composed of 1-byte data D0 and 1-byte data D1 at the next read address A2 timing. At the timing of the read address A3, and becomes 2-byte data composed of data D1 and data D2.

The stuff circuit 323 has a signal stuff position (FIG. 24) indicating the position where data "1" is to be inserted.
(E)) is supplied from the controller 326. The decoder 332 of the stuff circuit 323 includes a switch 331-1.
Among 6 to 331-0, the switch corresponding to the signal stuff position is switched to the contact b, the switch on the LSB side is switched to the contact c side, and the switch on the MSB side is further switched to the contact a side. As a result, since the data “1” is inserted, the stuff circuit 323 outputs data (FIG. 24F) with the data “1” inserted at the position indicated by the signal stuff position.

The barrel shifter 324 barrel-shifts the input data by the amount indicated by the signal shift (FIG. 24 (G)) supplied from the controller 326 and outputs (FIG. 24 (H)). This output is once held by the D-type flip-flop 325 and then output to the subsequent stage (FIG. 24 (I)).

The data output from the D-type flip-flop 325 includes data “22” next to data “22”.
Therefore, the number of consecutive 0s is 22 between the data "1" and the next data "1", even if all the bits between them are "0". .

FIG. 25 shows a configuration example of the converter 202. The configuration of the converter 202 including the D flip-flop 341 to the controller 346 is the same as that shown in FIG.
D-type flip-flop 3 of converter 212 shown in FIG.
21 to the controller 326, except that the converter 212 has a delay circuit 343 in place of the stuff circuit 323. Other configurations are the same as those of converter 212 in FIG.

In other words, in converter 202, according to signal delete position indicating the position of the bit to be deleted output from controller 346, delete circuit 343 replaces the bit (stuff circuit 323 in FIG. 22).
The data "1") inserted in is deleted.

Other operations are similar to those of converter 21 shown in FIG.
2 is the same as in the case of FIG.

FIG. 26 shows a configuration example of the delay circuit 343. In this configuration example, of the 16-bit data input from the D-type flip-flops 342 and 341, 15 bits on the LSB side are supplied to the contacts a of the corresponding switches 351-0 to 351-14. . Only one bit of data on the MSB side is supplied to the contact b of each switch. The decoder 352 deletes the bit specified by the signal delete position supplied from the controller 346 and outputs it as 15-bit data.

FIG. 26 shows a state where the 13th bit from the LSB is deleted. Therefore, in this case, the switches 351-0 to 351-11 are switched to the contact a side, and the 12th from the LSB to the twelfth is switched.
Bits are selected and output as they are. The switches 351-12 to 351-14 are respectively connected to the contact b
Side, the 14th to 16th data are selected and output as the 13th to 15th bit data.

The stuff circuit 323 in FIG. 23 and FIG.
The input of the stuffing circuit 323 of the converter 212 in FIG.
The 16 bits are supplied from the converter 21 and the converter 202 in FIG.
This is because the input of 43 is made 16 bits by the D-type flip-flops 342 and 341. In FIG. 22, 17 bits output from the stuff circuit 323 are barrel-shifted by the barrel shifter 324, for example, to 8 bits.
As with the final selection and output of the bits, FIG.
In the converter 202 of FIG.
3 is output to the barrel shifter 34.
The data is barrel-shifted by a predetermined amount at 4 to obtain 8-bit data.

FIG. 27 shows another configuration example of converter 212. In this configuration example, the counter 361
Counts the number of consecutive 0 bits in the input data, and outputs the count result to the controller 326. The controller 326 outputs a signal stuff position to the stuff circuit 323 when, for example, the counter 361 counts 22 consecutive 0 bits. At this time, the controller 326 resets the counter 361 and causes the counter 361 to count the number of consecutive 0 bits again.

Other structures and operations are the same as those in FIG.

FIG. 28 shows another configuration example of converter 202. In this configuration example, the counter 371 counts the number of consecutive 0s in the input data, and outputs the count result to the controller 346. When the count value of the counter 371 reaches 22, the controller 346 sends the signal delete positio
n is output to the delete circuit 343, the counter 371 is reset, and the counter 371 again counts the number of new continuous 0 bits. Other configurations are the same as those in FIG.

As described above, in this configuration example, based on a predetermined pattern (a continuous number of data “0”),
Data "1" as a marker bit is inserted and deleted.

The configuration shown in FIGS. 27 and 28 enables more efficient processing than the configurations shown in FIGS. 22 and 25. However, the length after conversion depends on the original history information.

FIG. 29 shows the user data formatter 21
3 shows a configuration example. In this example, when the controller 383 outputs a read address to a buffer memory (not shown) arranged between the converter 212 and the user data formatter 213, the data read therefrom is output to the user data formatter 213.
Is supplied to the contact a side of the switch 382 of FIG. ROM381
Contains user data start code, data ID, etc.
Data necessary to generate user_data () is stored. At a predetermined timing, the controller 313 switches the switch 382 to the contact a side or the contact b side, and appropriately selects and outputs data stored in the ROM 381 or data supplied from the converter 212. As a result, data in the format of user_data () is output to the encoding device 106.

Although not shown, the user data decoder 201 is read from the ROM 381 in FIG.
This can be realized by outputting input data via a switch for deleting inserted data.

FIG. 30 shows a state in which a plurality of transcoders 101-1 to 101-N are connected in series and used, for example, in a video editing studio. The history information multiplexing device 103-i of each transcoder 101-i (i = 1 to N) uses the history information multiplexing device 103-i in a section where the oldest coding parameter in the above-described coding parameter area is recorded. Overwrite the latest encoding parameters. As a result, the baseband image data includes
The encoding parameters (generation history information) for the last four generations corresponding to the same macroblock are recorded (FIG. 18).

The encoder 12 of each encoding device 106-i
1-i (FIG. 19) in the variable-length coding circuit 58 based on the currently used coding parameters supplied from the history information separation device 105-i.
Encoding the supplied video data. The bit stream generated in this way (for example, picture_he
During ader ()), the current coding parameters are multiplexed.

The variable length coding circuit 58 also multiplexes user data (including generation history information) supplied from the history encoding device 107-i into a bit stream to be output (embedding as shown in FIG. 18). Multiplexing in the bitstream, not processing). Then, the bit stream output from the encoding device 106-i is converted into a serial data transfer interface (SDTI) 108.
-I, the subsequent transcoder 101- (i +
Input to 1).

The transcoder 101-i and the transcoder 101- (i + 1) are each configured as shown in FIG. Therefore, the processing is the same as that described with reference to FIG.

When it is desired to change the encoding currently performed as an I picture to a P or B picture as the encoding using the actual encoding parameter history, the past encoding parameter history is checked. P or B
A search is made for a picture, and if these histories exist, the picture type is changed using parameters such as the motion vector. Conversely, if there is no history in the past, the change of the picture type for which no motion detection is performed is abandoned. Of course, even if there is no history, the picture type can be changed by performing motion detection.

In the case of the format shown in FIG. 18, encoding parameters for four generations are embedded.
It is also possible to embed parameters of each picture type of P and B. FIG. 31 shows an example of the format in this case. In this example, one generation of encoding parameters (picture history information) are recorded for each picture type when the same macroblock was previously encoded with a change in picture type. Therefore, the decoder 111 shown in FIG. 16 and the encoder 121 shown in FIG.
Instead of the second-generation and first-generation coding parameters, one-generation coding parameters corresponding to I-pictures, P-pictures, and B-pictures are input and output.

In the case of this example, Cb [1] [x] and Cr [1] [x]
The present invention can be applied to image data of 4: 2: 0 format which does not have the areas of Cb [1] [x] and Cr [1] [x].

In the case of this example, the decoding device 102 takes out the coding parameters at the same time as decoding, determines the picture type, writes the coding parameters in a location corresponding to the picture type of the image signal (multiplexes). Output to the history information separation device 105. The history information separation device 105 can separate the coding parameters and perform re-encoding while changing the picture type in consideration of a picture type to be coded and a past coding parameter input.

Next, in each transcoder 101,
Regarding the process of determining a picture type that can be changed,
This will be described with reference to the flowchart in FIG. The change of the picture type in the transcoder 101 is as follows.
Since a past motion vector is used, this process is assumed to be executed without performing motion detection. The processing described below is performed by the history information separating apparatus 10.
5 is performed.

In step S 1, one generation of coding parameters (picture history information) for each picture type is input to the history information separating device 105.

[0212] In step S2, the history information separating apparatus 105 determines whether or not the picture history information includes an encoding parameter when the picture is changed to a B picture. If it is determined in the picture history information that there is an encoding parameter when the picture is changed to the B picture, the process proceeds to step S3.

[0213] In step S3, the history information separating apparatus 105 determines whether or not the picture history information includes an encoding parameter when changed to a P picture. If it is determined that the picture history information includes the encoding parameter used when the picture is changed to the P picture, the process proceeds to step S4.

In step S4, history information separating apparatus 105 determines that the changeable picture types are I picture, P picture, and B picture.

[0215] If it is determined in step S3 that the picture history information does not include the encoding parameter used when the picture is changed to the P picture, the flow advances to step S5.

In step S5, the history information separating device 105 determines that the changeable picture types are an I picture and a B picture. Furthermore, the history information separation device 105 can pseudo-change to a P picture by performing special processing (using only forward prediction vectors without using backward prediction vectors included in B picture history information). to decide.

In step S2, when it is determined that the picture history information does not include the encoding parameter when the picture is changed to the B picture, the process proceeds to step S6.

In step S6, the history information separating apparatus 105 determines whether or not the picture history information includes an encoding parameter when the picture is changed to the P picture. If it is determined in the picture history information that there is an encoding parameter for changing to the P picture, the process proceeds to step S7.

[0219] In step S7, the history information separating apparatus 105 determines that the changeable picture types are the I picture and the P picture. Further, the history information separating apparatus 105 determines that the picture can be changed to a B picture by performing a special process (using only the forward prediction vector included in the history information for the P picture).

[0220] If it is determined in step S6 that the picture history information does not include the encoding parameter used when the picture is changed to the P picture, the flow advances to step S8. In step S8, the history information separating apparatus 105 determines that the changeable picture type is only the I picture since there is no motion vector.

After the processing in steps S4, S5, S7, and S8, in step S9, the history information separating apparatus 10
5 displays a changeable picture type on a display device (not shown) to notify the user.

FIG. 33 shows an example of changing the picture type. When changing the picture type, the number of frames forming the GOP is changed. That is, in this case,
From a 4 Mbps Long GOP (first generation) composed of N = 15 (the number of frames in a GOP N = 15) and M = 3 (the appearance cycle of an I or P picture in a GOP M = 3), N = 1, M = 1 is converted to a 50 Mbps Short GOP (2nd generation) composed of frames, and again 4M composed of N = 15, M = 3 frames
It is converted to bps Long GOP (3rd generation). In addition,
In the figure, broken lines indicate the boundaries of GOPs.

In the case where the picture type is changed from the first generation to the second generation, as is clear from the description of the changeable picture type determination processing, the picture type of all frames is changed to an I picture. Is possible. At the time of this picture type change, all motion vectors calculated when the moving image (0th generation) is converted to the 1st generation are stored (remaining) in the picture history information. Next, when converted to a Long GOP again (the picture type is changed from the second generation to the third generation), the motion vector for each picture type when converted from the 0th generation to the first generation is stored. So
By reusing this, it is possible to suppress the image quality degradation and convert it to a Long GOP again.

FIG. 34 shows another example of changing the picture type. In this example, N = 14, M = 2, 4Mbps L
18 Mbps S with N = 2 and M = 2 from ong GOP (1st generation)
The GOP is converted to a short GOP (2nd generation), and further converted to a 50 Mbps Short GOP (3rd generation) with N = 1 and M = 1 and the number of frames is 1; (4th generation).

Also in this example, the motion vector for each picture type when converted from the 0th generation to the 1st generation is stored until the conversion from the 3rd generation to the 4th generation. Therefore, as shown in FIG. 34, even if the picture type is changed in a complicated manner, the stored encoding parameters are reused, so that the image quality deterioration can be suppressed to a small value. Furthermore, if the stored quantization scale of the encoding parameter is effectively used, encoding with less image quality degradation can be realized.

Reuse of the quantization scale will be described with reference to FIG. FIG. 35 shows that a predetermined frame is always converted to an I picture from the first generation to the fourth generation, and only the bit rate is 4 Mbps, 18 Mbps,
Or, it has been changed to 50 Mbps.

For example, from the first generation (4 Mbps) to the second generation
At the time of conversion to (18 Mbps), the image quality does not improve even if re-encoding is performed with a fine quantization scale as the bit rate increases. This is because data that has been quantized in the past in a coarse quantization step is not restored. Therefore, as shown in FIG. 35, even if the bit rate is increased on the way, quantizing with a fine quantization step only increases the amount of information and does not lead to improvement in image quality. Therefore, if the control is performed so as to maintain the past coarsest (larger) quantization scale, the most efficient coding can be performed with the least waste.

At the time of changing from the third generation to the fourth generation, the bit rate is reduced from 50 Mbps to 4 Mbps.
The quantization scale is maintained.

As described above, when the bit rate is changed, it is very effective to perform encoding using the history of the past quantization scale.

This quantization control processing will be described with reference to the flowchart in FIG. In step S11, the history information separation device 105 determines whether or not the input picture history information includes an encoding parameter of a picture type to be converted. If it is determined that there is a coding parameter of the picture type to be converted, the process proceeds to step S12.

[0231] In step S12, the history information separation device 105 extracts history_q_scale_code from the coding parameter to be the target of the picture history information.

In step S13, the history information separation device 105 sends the quantization information from the transmission buffer 59 to the quantization circuit 57.
Feed based on the remaining buffer
Calculate back_q_scale_code.

[0233] In step S14, the history information separating apparatus 105 sets the history_q_scale_code to feedback_q_s
It is determined whether it is larger (coarse) than cale_code. his
When it is determined that tory_q_scale_code is larger than feedback_q_scale_code, the process proceeds to step S15.

[0234] In step S15, the history information separating apparatus 105 sets history_q_scal as the quantization scale.
e_code is output to the quantization circuit 57. Quantization circuit 57
Performs quantization using history_q_scale_code.

In step S16, it is determined whether or not all the macro blocks included in the frame have been quantized. If it is determined that all the macroblocks have not been quantized yet, the process returns to step S12, and the processes of steps S12 to S16 are repeated until all the macroblocks are quantized.

At step S14, history_q_scal
If it is determined that e_code is not larger (fine) than feedback_q_scale_code, the process proceeds to step S17.

[0237] In step S17, the history information separating device 105 sets feedback_q_sca as the quantization scale.
The le_code is output to the quantization circuit 57. Quantization circuit 57
Performs quantization using feedback_q_scale_code.

If it is determined in step S11 that the encoding parameter of the picture type to be converted does not exist in the history information, the process proceeds to step S18.

[0239] In step S18, the history information separation device 105 sends the
Feed based on the remaining buffer
Calculate back_q_scale_code.

In step S19, the quantization circuit 57
Performs quantization using Feedback_q_scale_code.

In step S20, it is determined whether or not all the macro blocks included in the frame have been quantized. When it is determined that all the macroblocks have not been quantized, the process returns to step S18, and the processes of steps S18 to S20 are repeated until all the macroblocks are quantized.

Note that, as described above, the decoding side and the code side are roughly coupled inside the transcoder 101 in the present embodiment, and the coding parameters are multiplexed with the image data and transmitted. As shown in FIG. 37, the decoding device 102 and the encoding device 106 may be directly connected (closely coupled).

Transcoder 1 described with reference to FIG.
No. 01 multiplexes the past encoding parameters to the baseband video data and transmits the same to supply the encoding apparatus 106 with the first to third generation past encoding parameters. However, in the present invention, the technique of multiplexing the past coding parameters to the baseband video data is not essential, and as shown in FIG. 37, a transmission path different from the baseband video data (for example, a data transfer bus). ) May be used to transmit the past coding parameters.

That is, the decoding apparatus 10 shown in FIG.
2. History decoding device 104, coding device 1
06 and the history encoding device 107 are shown in FIG.
5 has exactly the same functions and configurations as the decoding device 102, the history decoding device 104, the encoding device 106, and the history encoding device 107 described in FIG.

Variable Length Decoding Circuit 112 of Decoding Device 102
Extracts the third-generation coding parameters from the sequence layer, GOP layer, picture layer, slice layer, and macroblock layer of the third-generation encoded stream ST (3rd), and extracts them from the history encoding device 107 and It is supplied to the controller 70 of the encoding device 106, respectively. The history encoding device 107 receives the third
The coding parameter of the generation is converted into converted_history_stream () so that it can be described in the user data area of the picture layer, and the converted_history_stream () is supplied to the variable length coding circuit 58 of the coding device 106 as user data.

Further, the variable length decoding circuit 112 extracts user data user_data including the first generation coding parameter and the second coding parameter from the user data area of the picture layer of the third generation coded stream. Then, the data is supplied to the history decoding device 104 and the variable length coding circuit 58 of the coding device 106. The history decoding device 104 stores conv in the user data area.
First-generation coding parameters and second-generation coding parameters are extracted from the history stream described as erted_history_stream (), and are extracted by the coding apparatus 106.
To the controller.

The controller 70 of the encoding device 106
The first received from the history decoding device 104
Generation and second generation coding parameters and the coding device 1
Based on the third-generation encoding parameters received from the H.02, the encoding processing of the encoding device 106 is controlled.

Variable length coding circuit 58 of coding apparatus 106
Is the user data us containing the first-generation encoding parameter and the second encoding parameter from the decoding device 102.
er_data as well as user data user_data including third-generation encoding parameters from the history encoding device 107, and using the user data as history information, a user data area in a picture layer of a fourth-generation encoded stream. Describe in.

FIG. 38 is a diagram showing the syntax for decoding an MPEG video stream. The decoder extracts a plurality of meaningful data items (data elements) from the bit stream by decoding the MPEG bit stream according to this syntax. The syntax described below is
The functions and conditional statements are shown in fine print, and the data elements are shown in bold print. Each data item is described by a mnemonic (Mnemonic) indicating its name, bit length, and its type and transmission order.

First, the function used in the syntax shown in FIG. 38 will be described.

The next_start_code () function is a function for searching for a start code described in a bit stream. In the syntax shown in FIG. 38, next to this next_start_code () function, sequence_head
Since the er () function and the sequence_extension () function are arranged in order, this bit stream includes the sequence
A data element defined by the _header () function and the sequence_extension () function is described. Therefore,
When decoding the bit stream, this next_start_c
sequence_header () function and sequence
The start code (a kind of data element) described at the beginning of the _extension () function is found in the bit stream, and based on that, the sequence_header () function and the sequence_extension () function are further found and defined by them. Decode each data element.

The sequence_header () function is a function for defining the header data of the sequence layer of the MPEG bit stream, and the sequence_extension () function is
This is a function for defining extension data of the sequence layer of the EG bitstream.

The do {} while syntax located next to the sequence_extension () function is a data described based on the function in {} of the do statement while the condition defined by the while statement is true. This is a syntax for extracting an element from a data stream. That is, while the condition defined by the while statement is true, the decoding process of extracting the data element described based on the function in the do statement from the bit stream is performed by the do {} while syntax.

Nextbits () used in this while statement
The function is a function for comparing a bit or a bit string appearing in a bit stream with a data element to be decoded next. In the example of the syntax shown in FIG. 38, the nextbits () function includes a bit string in the bit stream and a sequence_end_code indicating the end of the video sequence.
Is compared with the bit sequence in the bit stream and sequence
When the _end_code does not match, the condition of this while statement is true. Therefore, the do {} while syntax located next to the sequence_extension () function uses sequence_end_cod indicating the end of the video sequence in the bit stream.
While e does not appear, it indicates that the data element defined by the function in the do statement is described in the bit stream.

[0255] In the bit stream, sequence_exten
Following each data element defined by the sion () function, a data element defined by the extension_and_user_data (0) function is described. This extensio
The n_and_user_data (0) function is a function for defining extended data of the sequence layer of the MPEG bit stream and user data.

The do {} while syntax placed next to the extension_and_user_data (0) function is described based on the function in the {} of the do statement while the condition defined by the while statement is true. This is a function for extracting the data element from the bit stream. The nextbits () function used in this while statement is used to determine the bits or bit strings appearing in the bit stream and the picture_start_
A function for determining a match with code or group_start_code. If a bit or a bit string appearing in a bit stream matches picture_start_code or group_start_code, the condition defined by the while statement is true. Therefore, this do {} while syntax is used in the bit stream for picture_start_code or group_st
If art_code appears, the code of the data element defined by the function in the do statement is described after the start code, so this picture_start_co
By finding the start code indicated by de or group_start_code, do
Data elements defined in the sentence can be extracted.

The if statement described at the beginning of the do statement is
If group_start_code appears in the bitstream,
The condition is shown. If the condition by this if statement is true, the group_start_
After code, group_of_picture_header (1) function and extens
Data elements defined by the ion_and_user_data (1) function are described in order.

The group_of_picture_header (1) function is
Extension_and_user_data (1) is a function for defining the header data of the GOP layer of the MPEG bit stream.
The function is the GOP layer extension data (e
xtension_data) and user data (user_data).

Further, in this bit stream, gr
oup_of_picture_header (1) function and extension_and_user
After the data element defined by the _data (1) function, the picture_header () function and picture_coding_exten
Describes the data element defined by the sion () function. Of course, if the condition of the if statement described above is not true, the data element defined by the group_of_picture_header (1) function and the extension_and_user_data (1) function is not described.
After the data element defined by the extension_and_user_data (0) function, the picture_header () function and pic
A data element defined by the ture_coding_extension () function is described.

[0260] The picture_header () function is for defining the header data of the picture layer of the MPEG bit stream, and the picture_coding_extension () function is for defining the first extension data of the picture layer of the MPEG bit stream. Function.

The next while statement is a function for determining the condition of the next if statement while the condition defined by the while statement is true. The nextbits () function used in this while statement includes a bit string appearing in the bit stream and extension_start_code or user_data_start_
A function to determine the match with code, a bit string appearing in the bit stream, and extension_start_cod
If e or user_data_start_code matches, the condition defined by this while statement is true.

The first if sentence is a function for judging whether a bit string appearing in the bit stream matches extension_start_code. If the bit string appearing in the bit stream matches the 32-bit extension_start_code, the extension_st
Following the art_code, a data element defined by the extension_data (2) function is described.

The second if sentence is a syntax for judging a match between a bit string appearing in the bit stream and user_data_start_code. When the bit string appearing in the bit stream matches a 32-bit user_data_start_code, The condition determination of the third if statement is performed. This user
_data_start_code is a start code indicating the start of the user data area in the picture layer of the MPEG bit stream.

[0264] The third if sentence is a syntax for judging whether a bit string appearing in the bit stream matches History_Data_ID. When the bit string appearing in the bit stream matches the 32-bit History_Data_ID, the 32-bit History_Data_I is stored in the user data area of the picture layer of the MPEG bit stream.
Following the code indicated by D, converted_history_s
A data element defined by the tream () function is described.

The converted_history_stream () function is
It is a function for describing history information and history data for transmitting all encoding parameters used at the time of encoding. Details of the data element defined by the converted_history_stream () function will be described later as history_stream () with reference to FIGS.
The History_Data_ID is a start code indicating the head of the history information and the history data described in the user data area of the picture layer of the MPEG bit stream.

The else statement is a syntax for indicating that the condition is non-true in the third if statement. So this MP
If the data element defined by the converted_history_stream () function is not described in the user data area of the picture layer of the EG bit stream, the data element defined by the user_data () function is described.

In FIG. 38, the history information is converted_converted_
Although described in history_stream () and not in user_data (), this converted_history_stream () is
It is described as a kind of user_data of the MPEG standard. Therefore, in this specification, the history information may
It is explained that it is described in user_data, but it is MPEG
It means that it is described as a kind of standard user_data.

[0268] The picture_data () function is a function for describing data elements related to the slice layer and the macroblock layer after the user data of the picture layer of the MPEG bit stream. Normally, the data element indicated by the picture_data () function is the conv. Described in the user data area of the picture layer of the bit stream.
Although described next to the data element defined by the erted_history_stream () function or the data element defined by the user_data () function, the extens
If there is no ion_start_code or user_data_start_code, the data element indicated by the picture_data () function is described next to the data element defined by the picture_coding_extension () function.

The data element indicated by the picture_data () function is followed by the sequence_header () function and s
Data elements defined by the equence_extension () function are arranged in order. This sequence_heade
The data element described by the r () function and the sequence_extension () function is composed of the sequence_header () function and the sequence_
Exactly the same as the data element described by the extension () function. The reason for describing the same data in the stream in this way is that when the bit stream receiving apparatus starts receiving data in the middle of the data stream (for example, the bit stream portion corresponding to the picture layer), the data in the sequence layer is received. This is to prevent a situation in which the stream cannot be decoded and the stream cannot be decoded.

This last sequence_header () function and sequence
After the data element defined by the nce_extension () function, that is, at the end of the data stream, a 32-bit sequence_end_cod indicating the end of the sequence
e is described.

The basic structure of the above syntax is schematically shown in FIG.

Next, a history stream defined by the converted_history_stream () function will be described.

This converted_history_stream () is an MPEG
Is a function for inserting a history stream indicating history information into the user data area of the picture layer. still,
“Converted” means a stream that has undergone a conversion process of inserting a marker bit (1 bit) at least every 22 bits of a history stream composed of history data to be inserted into the user area in order to prevent start emulation. It means that

This converted_history_stream () is a fixed-length history stream described below (FIGS. 40 to 4).
6) or a variable-length history stream (FIG. 47). When a fixed-length history stream is selected on the encoder side, there is an advantage that a circuit and software for decoding each data element from the history stream on the decoder side are simplified. On the other hand, when a variable-length history stream is selected on the encoder side, the history information (data element) described in the user area of the picture layer can be arbitrarily selected as needed by the encoder. Can be reduced, and as a result, the data rate of the entire coded bit stream can be reduced.

In the present invention, “history stream”, “history stream”, “history information”, “history information”, “history data”, “history data”,
The “history parameters” and “history parameters” mean the coding parameters (or data elements) used in the past coding process, and mean the coding parameters used in the current (last stage) coding process. It does not do. For example, in a first generation encoding process, a certain picture is encoded and transmitted by an I picture, and in a subsequent second generation encoding process, this picture is encoded and transmitted as a P picture, and further, Third
In the generation encoding process, an example will be described in which this picture is encoded with a B picture and transmitted.

The encoding parameters used in the third generation encoding process are the sequence layer of the encoded bit stream generated in the third generation encoding process, the GOP
Layer, picture layer, slice layer, and macroblock layer are described at predetermined positions. On the other hand, the encoding parameters used in the first-generation and second-generation encoding processes, which are the past encoding processes, are a sequence layer or GO that describes the encoding parameters used in the third-generation encoding process.
Instead of being described in the P layer, it is described in the user data area of the picture layer as coding parameter history information in accordance with the syntax described above.

First, the fixed-length history stream syntax will be described with reference to FIGS.

First, in the user data area of the picture layer of the bit stream generated in the encoding process of the final stage (for example, the third generation), the past (for example, the first generation and the second generation) encoding processes are performed. The coding parameter included in the sequence header of the sequence layer used in is inserted as a history stream. Note that history information such as the sequence header of the sequence layer of the bit stream generated in the past encoding process is not inserted into the sequence header of the sequence layer of the bit stream generated in the last-stage encoding process. It should be noted that.

The data elements included in the sequence header (sequence_header) used in the past encoding process are sequence_header_code, sequence_header_presen
t_flag, horizontal_size_value, marker_bit, vertica
l_size_value, aspect_ratio_information, frame_rate
_code, bit_rate_value, VBV_buffer_size_value, cons
trained_parameter_flag, load_intra_quantiser_matri
x, load_non_intra_quantiser_matrix, intra_quantise
r_matrix, non_intra_quantiser_matrix, etc.

[0280] sequence_header_code is data representing a start synchronization code of the sequence layer. sequence_hea
der_present_flag is data indicating whether data in sequence_header is valid or invalid. horizontal_size_val
ue is data consisting of the lower 12 bits of the number of pixels in the horizontal direction of the image. marker_bit is bit data inserted to prevent start code emulation. vertical_size_value is data composed of lower 12 bits of the number of vertical lines of the image. aspect_ratio_infor
“mation” is data representing an aspect ratio (aspect ratio) or a display screen aspect ratio of a pixel. frame_rate_code
Is data representing a display cycle of an image.

Bit_rate_value is lower 18 bits (rounded up in units of 400 bsp) of the bit rate for limiting the amount of generated bits. VBV_buffer_size_value
Is the lower 10-bit data of the value that determines the size of the virtual buffer (video buffer verifier) for controlling the generated code amount. constrained_parameter_flag is data indicating that each parameter is within the limit. load_i
ntra_quantiser_matrix is data indicating the existence of intra MB quantization matrix data. load_non_i
ntra_quantiser_matrix is data indicating the existence of non-intra MB quantization matrix data. intra_
quantiser_matrix is data indicating the value of the intra MB quantization matrix. non_intra_quantiser_matr
ix is data representing the value of the non-intra MB quantization matrix.

[0282] In the user data area of the picture layer of the bit stream generated in the encoding process at the last stage, a data element representing the sequence extension of the sequence layer used in the past encoding process is described as a history stream. You.

The data elements representing the sequence extension (sequence_extension) used in the past encoding process are extension_start_code, extension
_start_code_identifier, sequence_extension_present
_flag, profile_and_level_indication, progressive_s
equence, chroma_format, horizontal_size_extensio
n, vertical_size_extension, bit_rate_extension, vb
v_buffer_size_extension, low_delay, frame_rate_ext
data elements such as extension_n and frame_rate_extension_d.

[0284] extension_start_code is data representing a start synchronization code of extension data. ex
tension_start_code_identifier is data indicating which extension data is sent. sequence_extension_p
resent_flag is data indicating whether the data in the sequence extension is valid or invalid. pr
ofile_and_level_indication is data for specifying the profile and level of video data. progre
ssive_sequence is data indicating that video data is sequentially scanned. chroma_format is data for specifying a color difference format of video data.

[0285] horizontal_size_extension is high-order 2 bits of data to be added to horizontal_size_value of the sequence header. vertical_size_extension is upper two bits of data added to vertical_size_value of the sequence header. bit_rate_extension is upper 12 bits of data to be added to bit_rate_value of the sequence header. vbv_buffer_size_extension is upper 8 bits of data added to vbv_buffer_size_value of the sequence header. low_delay is data indicating that a B picture is not included. frame_rate_extension_n is
This is data for obtaining a frame rate in combination with frame_rate_code of the sequence header. frame_rate_e
xtension_d is data for obtaining a frame rate in combination with frame_rate_code of the sequence header.

Subsequently, in the user area of the picture layer of the bit stream, a data element representing the sequence display extension of the sequence layer used in the past encoding processing is described as a history stream.

The data elements described as the sequence display extension (sequence_display_extension) are extension_start_code, exte
nsion_start_code_identifier, sequence_display_exte
nsion_present_flag, video_format, colour_descripti
on, colour_primaries, transfer_characteristics, ma
trix_coeffients, display_horizontal_size, and disp
It consists of lay_vertical_size.

[0288] The extension_start_code is data representing the start synchronization code of the extension data. ex
tension_start_code_identifier is a code indicating which extension data is sent. sequence_display_ext
The extension_present_flag is data indicating whether a data element in the sequence display extension is valid or invalid. video_format is data representing the video format of the original signal. color_description is data indicating that there is detailed data of a color space. colo
r_primaries is data indicating details of the color characteristics of the original signal. transfer_characteristics is data indicating details of how the photoelectric conversion was performed. matrix_c
The oeffients are data showing details of how the original signal was converted from the three primary colors of light. display_horizont
al_size is data representing the intended active area (horizontal size) of the display. display_vertical_size
Is data representing the intended active area (vertical size) of the display.

Subsequently, macroblock assignment data (macroblock_assignment_in_user_data) indicating the phase information of the macroblock generated in the past encoding process is provided in the user area of the picture layer of the bit stream generated in the final stage encoding process. ) Is described as a history stream.

Macr indicating phase information of this macroblock
oblock_assignment_in_user_data is macroblock_assig
It is composed of data elements such as nment_present_flag, v_phase, and h_phase.

This macroblock_assignment_present_flag
Is data indicating whether a data element in macroblock_assignment_in_user_data is valid or invalid. v_phase
Is data indicating vertical phase information when a macroblock is cut out from image data. h_phase is data indicating horizontal phase information when a macroblock is cut out from image data.

Subsequently, in the user area of the picture layer of the bit stream generated by the encoding process at the last stage, a data element indicating the GOP header of the GOP layer used in the past encoding process is stored as a history stream. It has been described.

The GOP header (group_of_picture_heade)
The data element representing r) is group_start_code, g
roup_of_picture_header_present_flag, time_code, cl
osed_gop and broken_link.

[0294] group_start_code is data indicating the start synchronization code of the GOP layer. group_of_picture_header_p
resent_flag is data indicating whether a data element in group_of_picture_header is valid or invalid. time_code is a time code indicating the time from the beginning of the sequence of the first picture of the GOP. closed_g
op is flag data indicating that an image in a GOP can be reproduced independently from another GOP. broken_link is flag data indicating that the first B picture in the GOP cannot be accurately reproduced due to editing or the like.

Subsequently, in the user area of the picture layer of the bit stream generated by the encoding process at the last stage, a data element representing the picture header of the picture layer used in the past encoding process is stored as a history stream. It has been described.

Data elements related to the picture header (picture_header) are picture_start_code, temp
oral_reference, picture_coding_type, vbv_delay, fu
ll_pel_forward_vector, forward_f_code, full_pel_ba
It consists of ckward_vector and backward_f_code.

[0297] Specifically, picture_start_code is data representing the start synchronization code of the picture layer. temporal
_reference is data indicating the display order of pictures and is reset at the beginning of the GOP. picture_coding_typ
e is data indicating a picture type. vbv_delay
Is data indicating the initial state of the virtual buffer at the time of random access. full_pel_forward_vector is data indicating whether the accuracy of the forward motion vector is an integer unit or a half pixel unit. forward_f_code is data representing a forward motion vector search range. full_pel_backward_vector
Is data indicating whether the accuracy of the backward motion vector is an integer unit or a half pixel unit. backward_f_code is data representing a backward motion vector search range.

Subsequently, in the user area of the picture layer of the bit stream generated by the encoding process at the last stage, the picture coding extension of the picture layer used in the past encoding process is described as a history stream. I have.

The data elements related to the picture coding extension (picture_coding_extension) are extension_start_code, extension_start_code
_identifier, f_code [0] [0], f_code [0] [1], f_code [1]
[0], f_code [1] [1], intra_dc_precision, picture_str
ucture, top_field_first, frame_predictive_frame_dc
t, concealment_motion_vectors, q_scale_type, intra
_vlc_format, alternate_scan, repeat_firt_field, ch
roma_420_type, progressive_frame, composite_displa
y_flag, v_axis, field_sequence, sub_carrier, burst
_amplitude and sub_carrier_phase.

[0300] The extension_start_code is a start code indicating the start of extension data in the picture layer. extension_start_code_identifier is a code indicating which extension data is sent. f_code [0] [0]
Is data indicating a horizontal motion vector search range in the forward direction. f_code [0] [1] is data representing the vertical motion vector search range in the forward direction. f_code [1]
[0] is data representing a horizontal motion vector search range in the backward direction. f_code [1] [1] is data representing the vertical motion vector search range in the backward direction.

[0301] intra_dc_precision is data representing the precision of the DC coefficient. picture_structure is data indicating a frame structure or a field structure. In the case of the field structure, it is data indicating the upper field or the lower field. top_
field_first is data indicating whether the first field is higher or lower in the case of a frame structure. frame_
predictive_frame_dct is data indicating that, in the case of a frame structure, the prediction of the frame mode DCT is only the frame mode. concealment_motion
_vectors is data indicating that a motion vector for concealing a transmission error is attached to an intra macroblock.

[0302] q_scale_type is data indicating whether to use a linear quantization scale or a non-linear quantization scale. intra_vlc_format is data indicating whether another two-dimensional VLC is used for an intra macroblock. alternate_scan uses zigzag scan,
This is data indicating whether to use an alternate scan. repeat_firt_field is data used at the time of 2: 3 pull-down. chroma_420_type is the next progressive_frame when the signal format is 4: 2: 0.
Is the same value as, otherwise 0. pr
ogressive_frame is data indicating whether or not this picture has been sequentially scanned. composite_display_
The flag is data indicating whether the source signal is a composite signal.

[0303] v_axis is data used when the source signal is PAL. field_sequence is data used when the source signal is PAL. sub_carrier
Is data used when the source signal is PAL. burst_amplitude is data used when the source signal is PAL. sub_carrier_phase is data used when the source signal is PAL.

Subsequently, in the user area of the picture layer of the bit stream generated by the encoding process at the last stage, the quantization matrix extension used in the past encoding process is described as a history stream.

Data elements related to the quantization matrix extension (quant_matrix_extension) are extension_start_code and extension_start_code_ide.
ntifier, quant_matrix_extension_present_flag, load
_intra_quantiser_matrix, intra_quantiser_matrix [6
4], load_non_intra_quantiser_matrix, non_intra_qua
ntiser_matrix [64], load_chroma_intra_quantiser_mat
rix, chroma_intra_quantiser_matrix [64], load_chrom
a_non_intra_quantiser_matrix and chroma_non_intra
_quantiser_matrix [64].

Extension_start_code is a start code indicating the start of this quantization matrix extension. extension_start_code_identifier is a code indicating which extension data is sent. quant_mat
rix_extension_present_flag is data for indicating whether a data element in the quantization matrix extension is valid or invalid. load_intra_quantiser_m
atrix is data indicating the existence of quantization matrix data for an intra macroblock. intra_quantise
r_matrix is data indicating the value of a quantization matrix for an intra macroblock.

[0307] load_non_intra_quantiser_matrix is data indicating the existence of quantization matrix data for a non-intra macroblock. non_intra_quantiser_matr
ix is data representing a value of a quantization matrix for a non-intra macroblock. load_chroma_intra_quanti
ser_matrix is data indicating the existence of quantization matrix data for a chrominance intra macroblock. chro
ma_intra_quantiser_matrix is data indicating a value of a quantization matrix for a chroma intra macroblock. load_chroma_non_intra_quantiser_matrix is data indicating the existence of quantization matrix data for a chrominance non-intra macroblock. chroma_non_intra_qua
ntiser_matrix is data indicating a value of a quantization matrix for a color difference non-intra macroblock.

[0308] Subsequently, in the user area of the picture layer of the bit stream generated by the encoding process at the last stage, the copyright extension used in the past encoding process is described as a history stream.

The copyright extension (copy
data element for right_extension) is exten
sion_start_code, extension_start_code_itentifier,
copyright_extension_present_flag, copyright_flag,
copyright_identifier, original_or_copy, copyright_
number_1, copyright_number_2, and copyright_numbe
Consists of r_3.

[0310] The extension_start_code is a start code indicating the start of the copyright extension. exte
This code indicates which extension data of nsion_start_code_itentifier is sent. copyright_e
xtension_present_flag is data for indicating whether a data element in the copyright extension is valid or invalid. copyright_flag indicates whether or not a copy right has been granted to the encoded video data until the next copyright extension or sequence end.

[0311] copyright_identifier is ISO / IEC JTC / SC
This is data for identifying the registration authority of the copy right specified by 29. original_or_copy is data indicating whether the data in the bit stream is original data or copy data. copyright_number_1
Is data representing bits 44 to 63 of the copyright number. copyright_number_2 is data representing bits 22 to 43 of the copyright number.
copyright_number_3 is data representing bits 0 to 21 of the copyright number.

[0312] Subsequently, the user area of the picture layer of the bit stream generated by the encoding process at the last stage includes the picture display extension (picture_display_extensi) used in the past encoding process.
on) is described as a history stream.

The data element representing this picture display extension is extension_start_cod
e, extension_start_code_identifier, picture_displa
y_extension_present_flag, frame_center_horizontal_
offset_1, frame_center_vertical_offset_1, frame_ce
nter_horizontal_offset_2, frame_center_vertical_of
fset_2, frame_center_horizontal_offset_3, and fram
It consists of e_center_vertical_offset_3.

[0314] The extension_start_code is a start code for indicating the start of the picture display extension. extension_start_code_identifier is a code indicating which extension data is sent. pictur
e_display_extension_present_flag is data indicating whether the data element in the picture display extension is valid or invalid. frame_center_horizontal_
The offset is data indicating a horizontal offset of the display area, and can be defined up to three offset values. frame_center_vertical_offset is data indicating a vertical offset of the display area, and can define up to three offset values.

In the user area of the picture layer of the bit stream generated in the encoding process of the final stage, the user information (user data used in the past encoding process) user_data) is described as a history stream.

Following the user data, information on the macroblock layer used in the past encoding processing is described as a history stream.

The information on the macroblock layer is ma
croblock_address_h, macroblock_address_v, slice_he
ader_present_flag, skipped_macroblock_flag, and other data elements related to the position of the macroblock, and macroblock_quant, macroblock_motion_forwar
d, macroblock_motion_backward, macroblock_patter
n, macroblock_intra, spatial_temporal_weight_code_
Data elements related to macroblock modes (macroblock_modes []) such as flag, frame_motion_type, and dct_type, data elements related to quantization step control such as quantizer_scale_code, and PMV [0] [0] [0], PMV
[0] [0] [1], motion_vertical_field_select [0] [0], PMV
[0] [1] [0], PMV [0] [1] [1], motion_vertical_field_sel
ect [0] [1], PMV [1] [0] [0], PMV [1] [0] [1], motion_vert
ical_field_select [1] [0], PMV [1] [1] [0], PMV [1] [1]
Data elements related to motion compensation such as [1], motion_vertical_field_select [1] [1], and coded_block_pattern
Data elements related to macroblock patterns such as num_mv_bits, num_coef_bits, and num_other_bits
And so on.

The data elements relating to the macroblock layer will be described in detail below.

[0319] macroblock_address_h is data for defining the absolute position of the current macroblock in the horizontal direction. macroblock_address_v is data for defining the vertical absolute position of the current macroblock.
slice_header_present_flag is data indicating whether or not this macroblock is the head of a slice layer and has a slice header. skipped_macroblock_flag is
This data indicates whether to skip this macroblock in the decoding process.

The macroblock_quant is a macroblock type (macroblock_ty) shown in FIGS.
pe) data, quantizer_scale_c
This is data indicating whether or not ode appears in the bit stream. macroblock_motion_forward is shown in FIG. 63 and FIG.
4 is data derived from the macroblock type shown in FIG. 4 and used in the decoding process. macrob
lock_motion_backward is data derived from the macroblock type shown in FIGS. 63 and 64, and is data used in the decoding process. mocroblock_pattern
Is data derived from the macroblock types shown in FIGS. 63 and 64, and is data indicating whether or not coded_block_pattern appears in the bit stream.

The macroblock_intra is data derived from the macroblock type shown in FIGS. 63 and 64, and is data used in the decoding process. spatial_temp
oral_weight_code_flag is data derived from the macroblock types shown in FIGS. 63 and 64, and spatial_temporal_weight_code indicating a method of upsampling a lower layer image with temporal scalability is data indicating whether or not the bitstream exists. It is.

[0324] frame_motion_type is a 2-bit code indicating the prediction type of the macroblock of the frame.
If the number of prediction vectors is two and the field-based prediction type is “00”, if the number of prediction vectors is one and the field-based prediction type is “01”, the number of prediction vectors is one and the frame is base Is "10" for the prediction type of "1", and "11" for the prediction type of one prediction vector and dual prime. field_mo
tion_type is a 2-bit code indicating motion prediction of a macroblock in the field. If the number of prediction vectors is one and the field-based prediction type is “01”, if the number of prediction vectors is two and the prediction type is a 18 × 8 macroblock-based “10”, the prediction vector is one If it is a forecast type of dual prime, "1
1 ". dct_type indicates whether the DCT is in frame DCT mode
This is data indicating whether the mode is the field DCT mode. quantiser
_scale_code is data indicating the quantization step size of the macroblock.

Next, a data element relating to a motion vector will be described. The motion vector is coded as a difference with respect to the previously coded vector in order to reduce the motion vector required for decoding. In order to perform motion vector decoding, the decoder must maintain four motion vector predictions, each with a horizontal and vertical component. This predicted motion vector is represented as PMV [r] [s] [v]. [r] is a flag indicating whether the motion vector in the macroblock is the first vector or the second vector, and is “0” when the vector in the macroblock is the first vector. Becomes “1” when the vector in the macro block is the second vector. [s] is a flag indicating whether the direction of the motion vector in the macroblock is forward or backward. In the case of a forward motion vector, the flag is set to “0”, and the backward motion vector In this case, it becomes "1". [v] is a flag indicating whether the vector component in the macroblock is horizontal or vertical, and is “0” in the case of a horizontal component, and is “0” in the case of a vertical component. Becomes “1”.

Therefore, PMV [0] [0] [0] represents the data of the horizontal component of the forward motion vector of the first vector, and PMV [0] [0] [1] represents the first component. Represents the vertical component data of the motion vector in the forward direction of the vector of PMV [0] [1] [0]
Represents the data of the horizontal component of the backward motion vector of the first vector, and PMV [0] [1] [1] represents the data of the vertical component of the backward motion vector of the first vector. PMV [1] [0] [0] represents the horizontal component data of the forward motion vector of the second vector, and PMV [1]
[0] [1] represents the data of the vertical component of the forward motion vector of the second vector, and PMV [1] [1] [0] represents the second component.
Represents the horizontal component data of the backward motion vector of the second vector, and PMV [1] [1] [1] represents the vertical component data of the second motion vector of the second vector.

[0325] motion_vertical_field_select [r] [s] is
This is data indicating which reference field is used for the prediction format. This motion_vertical_field_select
When [r] [s] is “0”, it indicates that the top reference field is used, and when “1”, the bottom reference field is used.

Therefore, motion_vertical_field_select
[0] [0] indicates a reference field when a forward motion vector of the first vector is generated, and motion_vertical_fi
eld_select [0] [1] indicates a reference field when a backward motion vector of the first vector is generated, and motion_v
ertical_field_select [1] [0] indicates a reference field when a forward motion vector of the second vector is generated, and motion_vertical_field_select [1] [1] generates a backward motion vector of the second vector. Reference field at the time is shown.

[0327] The coded_block_pattern is variable-length data indicating which DCT block has a significant coefficient (non-zero coefficient) among a plurality of DCT blocks storing DCT coefficients. num_mv_bits is data indicating the code amount of a motion vector in a macroblock. num_coef_bits is data indicating the code amount of the DCT coefficient in the macro block. n
um_other_bits is the code amount of the macroblock and is data indicating the code amount other than the motion vector and the DCT coefficient.

Next, the syntax for decoding each data element from the variable-length history stream will be described with reference to FIGS.

The variable-length history stream is next_sta
rt_code () function, sequence_header () function, sequence_ext
extension () function, extension_and_user_data (0) function, grou
p_of_picture_header () function, extension_and_user_data
(1) function, picture_header () function, picture_coding_exte
nsion () function, re_coding_stream_info () function, extensio
It is composed of data elements defined by n_and_user_data (2) function and picture_data () function.

Since the next_start_code () function is a function for searching for a start code existing in the bit stream, the next_start_code () function is used in the past encoding process as shown in FIG. A data element, which is a data element defined by the sequence_header () function, is described.

The data elements defined by the sequence_header () function are sequence_header_code, sequence
ce_header_present_flag, horizontal_size_value, ver
tical_size_value, aspect_ratio_information, frame_
rate_code, bit_rate_value, marker_bit, VBV_buffer_
size_value, constrained_parameter_flag, load_intra
_quantiser_matrix, intra_quantiser_matrix, load_no
n_intra_quantiser_matrix and non_intra_quantiser_
matrix.

[0332] sequence_header_code is data representing a start synchronization code of the sequence layer. sequence_hea
der_present_flag is data indicating whether data in sequence_header is valid or invalid. horizontal_size_val
ue is data consisting of the lower 12 bits of the number of pixels in the horizontal direction of the image. vertical_size_value is data composed of lower 12 bits of the number of vertical lines of the image. aspect_r
atio_information is data representing the pixel aspect ratio (aspect ratio) or the display screen aspect ratio. frame_
rate_code is data representing a display cycle of an image. bit
_rate_value is the lower 18 bits of the bit rate for limiting the amount of generated bits (rounded up to the nearest 400 bsp)
Data.

Marker_bit is bit data inserted to prevent start code emulation. VBV_buffer_size_value is lower 10-bit data of a value that determines the size of a virtual buffer (video buffer verifier) for controlling the generated code amount. constrained_
parameter_flag is data indicating that each parameter is within the limit. load_intra_quantiser_matrix
Is data indicating the presence of intra MB quantization matrix data. intra_quantiser_matrix is data indicating the value of the quantization matrix for intra MB.
load_non_intra_quantiser_matrix is a non-intra MB
This is data indicating the existence of quantization matrix data for use. non_intra_quantiser_matrix is data representing the value of the non-intra MB quantization matrix.

Next to the data element defined by the sequence_header () function, as shown in FIG.
The data element defined by the quence_extension () function is described as a history stream.

The data elements defined by the sequence_extension () function are extension_start_code, ex
tension_start_code_identifier, sequence_extension_
present_flag, profile_and_level_indication, progre
ssive_sequence, chroma_format, horizontal_size_ext
tension, vertical_size_extension, bit_rate_extensio
n, vbv_buffer_size_extension, low_delay, frame_rat
Data elements such as e_extension_n and frame_rate_extension_d.

[0336] The extension_start_code is data representing a start synchronization code of extension data. ex
tension_start_code_identifier is data indicating which extension data is sent. sequence_extension_p
resent_flag is data indicating whether the data in the sequence extension is valid or invalid.
profile_and_level_indication is data for specifying the profile and level of video data. prog
ressive_sequence is data indicating that video data is sequentially scanned. chroma_format is data for specifying a color difference format of video data.
horizontal_size_extension is hor in the sequence header
Upper 2 bits of data to be added to izntal_size_value. vertical_size_extension is the v in the sequence header
Upper 2 bits of data to be added to ertical_size_value. bit_rate_extension is bit_ra of the sequence header
Upper 12 bits of data to be added to te_value. vbv_
buffer_size_extension is the sequence header vbv_buf
Upper 8 bits of data to be added to fer_size_value.

[0337] low_delay is data indicating that a B picture is not included. frame_rate_extension_n is data for obtaining a frame rate in combination with frame_rate_code of the sequence header. frame_rate_exten
sion_d is data for obtaining a frame rate in combination with frame_rate_code of the sequence header.

The data element defined by the sequence_extension () function is followed by ex as shown in FIG.
The data element defined by the tension_and_user_data (0) function is described as a history stream. When "i" is other than 1, the extension_and_user_data (i) function describes only the data element defined by the user_data () function as a history stream without describing the data element defined by the extension_data () function. . Therefore, extension_and_user_data
The (0) function describes only a data element defined by the user_data () function as a history stream.

The user_data () function describes user data as a history stream based on the syntax as shown in FIG.

Following the data element defined by the extension_and_user_data (0) function, the data element defined by the group_of_picture_header () function as shown in FIG. 52, and the extension_and_user_data (1)
The data element defined by the function is described as a history stream. However, only when group_start_code indicating the start code of the GOP layer is described in the history stream, group_of_picture_header
Data elements defined by the () function, and exte
A data element defined by the nsion_and_user_data (1) function is described.

The data elements defined by the group_of_picture_header () function are group_start_code, gr
oup_of_picture_header_present_flag, time_code, clo
It consists of sed_gop and broken_link.

Group_start_code is data indicating the start synchronization code of the GOP layer. group_of_picture_header_p
resent_flag is data indicating whether a data element in group_of_picture_header is valid or invalid. time_code is a time code indicating the time from the beginning of the sequence of the first picture of the GOP. closed_g
op is flag data indicating that an image in a GOP can be reproduced independently from another GOP. broken_link is flag data indicating that the first B picture in the GOP cannot be accurately reproduced due to editing or the like.

The extension_and_user_data (1) function is called ext
As in the case of the extension_and_user_data (0) function, user_data
Describe only the data elements defined by the () function as a history stream.

If there is no group_start_code indicating the start code of the GOP layer in the history stream, the group_of_picture_header () function and the exten
The data elements defined by the sion_and_user_data (1) function are not described in the history stream. In that case, after the data element defined by the extension_and_user_data (0) function,
The data element defined by the headr () function is described as a history stream.

The data element defined by the picture_headr () function is, as shown in FIG.
art_code, temporal_reference, picture_coding_typ
e, vbv_delay, full_pel_forward_vector, forward_f_c
ode, full_pel_backward_vector, backward_f_code, ex
tra_bit_picture and extra_information_picture.

[0346] Specifically, picture_start_code is data representing the start synchronization code of the picture layer. temporal
_reference is data indicating the display order of pictures and is reset at the beginning of the GOP. picture_coding_typ
e is data indicating a picture type. vbv_delay
Is data indicating the initial state of the virtual buffer at the time of random access. full_pel_forward_vector is data indicating whether the accuracy of the forward motion vector is an integer unit or a half pixel unit. forward_f_code is data representing a forward motion vector search range. full_pel_backward_vector
Is data indicating whether the accuracy of the backward motion vector is an integer unit or a half pixel unit. backward_f_code is data representing a backward motion vector search range. extra_bit_p
“icture” is a flag indicating the presence of the following additional information. If this extra_bit_picture is "1", then e
xtra_information_picture exists and extra_bit_pictur
When e is “0”, it indicates that there is no data following this. extra_information_picture is information reserved in the standard.

The data element defined by the picture_headr () function is followed by a pictur as shown in FIG.
The data element defined by the e_coding_extension () function is described as a history stream.

The data element defined by the picture_coding_extension () function is extension_star
t_code, extension_start_code_identifier, f_code [0]
[0], f_code [0] [1], f_code [1] [0], f_code [1] [1], int
ra_dc_precision, picture_structure, top_field_firs
t, frame_predictive_frame_dct, concealment_motion_
vectors, q_scale_type, intra_vlc_format, alternate
_scan, repeat_firt_field, chroma_420_type, progres
sive_frame, composite_display_flag, v_axis, field_
sequence, sub_carrier, burst_amplitude, and sub_ca
It is composed of rrier_phase.

[0349] extension_start_code is a start code indicating the start of extension data in the picture layer. extension_start_code_identifier is a code indicating which extension data is sent. f_code [0] [0]
Is data indicating a horizontal motion vector search range in the forward direction. f_code [0] [1] is data representing the vertical motion vector search range in the forward direction. f_code [1]
[0] is data representing a horizontal motion vector search range in the backward direction. f_code [1] [1] is data representing the vertical motion vector search range in the backward direction. in
tra_dc_precision is data representing the accuracy of the DC coefficient.

[0350] The picture_structure is data indicating a frame structure or a field structure.
In the case of the field structure, it is data indicating the upper field or the lower field. top_fiel
In the case of a frame structure, d_first is data indicating whether the first field is higher or lower. frame_pred
ictive_frame_dct is the frame structure
This is data indicating that the prediction of the frame mode DCT is only the frame mode. concealment_motion_vec
tors is data indicating that a motion vector for concealing a transmission error is attached to an intra macroblock. q_scale_type is data indicating whether to use a linear quantization scale or a non-linear quantization scale. intra_vlc_format is data indicating whether another two-dimensional VLC is used for an intra macroblock.

[0351] "alternate_scan" is data representing a choice between using a zigzag scan or an alternate scan. repeat_firt_field is data used at the time of 2: 3 pull-down. chroma_420_type is the next progressive_type when the signal format is 4: 2: 0.
The same value as frame, otherwise it represents 0. progressive_frame is data indicating whether or not this picture has been sequentially scanned. composite_di
splay_flag is data indicating whether or not the source signal was a composite signal. v_axis is data used when the source signal is PAL. field_sequenc
e is data used when the source signal is PAL. sub_carrier is data used when the source signal is PAL. burst_amplitude indicates that the source signal is P
Data used in the case of AL. sub_carrier_phase
Is data used when the source signal is PAL.

[0352] After the data element defined by the picture_coding_extension () function, re_coding_stre
The data element defined by the am_info () function is described as a history stream. This re_coding_
The stream_info () function is mainly used when describing a combination of history information, and details thereof will be described later with reference to FIG.

After the data element defined by the re_coding_stream_info () function, extensions_and_u
The data element defined by ser_data (2)
It is described as a history stream. This extension_
The and_user_data (2) function includes, as shown in FIG. 50, an extension start code (ex
tension_start_code) if present
Describes a data element defined by the data () function. Following this data element, the user data start code (user_data_
If start_code) exists, a data element defined by the user_data () function is described.
However, when the extension start code and the user data start code do not exist in the bit stream, the data elements defined by the extension_data () function and the user_data () function are not described in the bit stream.

As shown in FIG. 55, the extension_data () function includes a data element indicating extension_start_code, a quant_matrix_extension () function, and a copyright_extens
This is a function for describing a data element defined by an ion () function and a picture_display_extension () function as a history stream in a bit stream.

The data element defined by the quant_matrix_extension () function is, as shown in FIG.
tension_start_code, extension_start_code_identifie
r, quant_matrix_extension_present_flag, load_intra
_quantiser_matrix, intra_quantiser_matrix [64], loa
d_non_intra_quantiser_matrix, non_intra_quantiser_
matrix [64], load_chroma_intra_quantiser_matrix, ch
roma_intra_quantiser_matrix [64], load_chroma_non_i
ntra_quantiser_matrix and chroma_non_intra_quanti
ser_matrix [64].

The extension_start_code is a start code indicating the start of the quantization matrix extension. extension_start_code_identifier is a code indicating which extension data is sent. quant_mat
rix_extension_present_flag is data for indicating whether a data element in the quantization matrix extension is valid or invalid. load_intra_quantiser_m
atrix is data indicating the existence of quantization matrix data for an intra macroblock. intra_quantise
r_matrix is data indicating the value of a quantization matrix for an intra macroblock.

[0357] load_non_intra_quantiser_matrix is data indicating the existence of quantization matrix data for a non-intra macroblock. non_intra_quantiser_matr
ix is data representing a value of a quantization matrix for a non-intra macroblock. load_chroma_intra_quanti
ser_matrix is data indicating the existence of quantization matrix data for a chrominance intra macroblock. chro
ma_intra_quantiser_matrix is data indicating a value of a quantization matrix for a chroma intra macroblock. load_chroma_non_intra_quantiser_matrix is data indicating the existence of quantization matrix data for a chrominance non-intra macroblock. chroma_non_intra_qua
ntiser_matrix is data indicating a value of a quantization matrix for a color difference non-intra macroblock.

The data element defined by the copyright_extension () function is, as shown in FIG.
extension_start_code, extension_start_code_itentifie
r, copyright_extension_present_flag, copyright_fla
g, copyright_identifier, original_or_copy, copyrig
ht_number_1, copyright_number_2, and copyright_nu
Consists of mber_3.

[0359] extension_start_code is a start code indicating the start of the copyright extension. exte
nsion_start_code_itentifier This is a code indicating which extension data is sent. copyright_ext
The extension_present_flag is data for indicating whether the data element in the copyright extension is valid or invalid.

[0360] copyright_flag indicates whether or not the copy right has been granted to the encoded video data until the next copyright extension or sequence end. copyright_identifier is data for identifying the registration authority of the copy right specified by ISO / IEC JTC / SC29. original_or_copy is data indicating whether the data in the bit stream is original data or copy data. copyright_number_1
Data representing bits 44 to 63 of the copyright number. copyright_number_2 is data representing bits 22 to 43 of the copyright number. copy
right_number_3 is data representing bits 0 to 21 of the copyright number.

As shown in FIG. 58, the data elements defined by the picture_display_extension () function are extension_start_code_identifier, frame_center_
horizontal_offset, frame_center_vertical_offset, etc.

[0362] The extension_start_code_identifier is a code indicating which extension data is to be sent. frame
_center_horizontal_offset is data indicating the horizontal offset of the display area, and number_of_frame_
You can define a number of offset values defined by center_offsets. frame_center_vertical_offset
Is data indicating the vertical offset of the display area, and can define the number of offset values defined by number_of_frame_center_offsets.

Returning to FIG. 47 again, extension_and_user
Following the data element defined by the _data (2) function, the data element defined by the picture_data () function is described as a history stream. However, this picture_data () function exists when red_bw_flag is not 1 or red_bw_indicator is 2 or less. This red_bw_flag and red_bw_indicator are r
These are described in the e_coding_stream_info () function, which will be described later with reference to FIGS. 71 and 72.

The data element defined by the picture_data () function is a data element defined by the slice () function, as shown in FIG. This slic
At least one data element defined by the e () function is described in the bit stream.

[0365] As shown in Fig. 60, the slice () function includes slice_start_code, slice_quantiser_scale_code,
intra_slice_flag, intra_slice, reserved_bits, extr
a_bit_slice, extra_information_slice, and extra_bi
This is a function for describing a data element such as t_slice and a data element defined by the macroblock () function as a history stream.

[0366] slice_start_code is a start code indicating the start of a data element defined by the slice () function. slice_quantiser_scale_code is data indicating a quantization step size set for a macroblock existing in this slice layer. But,
When quantizer_scale_code is set for each macroblock, the macroblock_quantiser_scale_code data set for each macroblock is used with priority.

[0367] The intra_slice_flag is a flag indicating whether or not intra_slice and reserved_bits exist in the bit stream. intra_slice is data indicating whether or not a non-intra macroblock exists in the slice layer. If any of the macroblocks in the slice layer is a non-intra macroblock, intra_
slice is “0”, and when all of the macroblocks in the slice layer are non-intra macroblocks, intra_slice is “1”. reserved_bits is 7
It is bit data and takes a value of "0". extra_bit_
slice is a flag indicating that additional information exists as a history stream, and then extra_information_sl
If ice exists, it is set to “1”. If there is no additional information, it is set to “0”.

Following these data elements, macr
The data element defined by the oblock () function
It is described as a history stream.

As shown in FIG. 61, the macroblock () function includes macroblock_escape and macroblock_address_incremen.
t, macroblock_quantiser_scale_code, and marker
data elements such as _bit, macroblock_modes () function, motion_vectors (s) function, and code_block_patter
This is a function for describing the data element defined by the n () function.

[0370] macroblock_escape is a fixed bit string indicating whether or not the horizontal difference between the reference macroblock and the previous macroblock is 34 or more. If the horizontal difference between the reference macroblock and the previous macroblock is 34 or more, the macroblock_address_increment value is set to 33.
Plus macroblock_address_increment is data indicating a horizontal difference between the reference macroblock and the previous macroblock. If this macroblock_address_i
If there is one macroblock_escape before ncrement, the value of this macroblock_address_increment is 33
Is the data indicating the horizontal difference between the actual reference macroblock and the previous macroblock.

[0371] macroblock_quantiser_scale_code is a quantization step size set for each macroblock, and exists only when macroblock_quant is "1".
In each slice layer, slice_quantiser_scale_code indicating the quantization step size of the slice layer is set, but macroblock_quantiser
If _scale_code is set, this quantization step size is selected.

After macroblock_address_increment,
Describes a data element defined by the macroblock_modes () function. macroblock_modes () function
As shown in FIG. 62, macroblock_type, frame_motion_
This is a function for describing data elements such as type, field_motion_type, and dct_type as a history stream.

[0373] macroblock_type is data indicating the coding type of the macroblock. The details are shown in FIG.
It will be described later with reference to FIG.

If macroblock_motion_forward or mac
When roblock_motion_backward is “1”, the picture structure is a frame, and frame_pred_frame_dct is “0”, a data element representing frame_motion_type is described next to a data element representing macroblock_type. This frame_pred_fra
me_dct is a flag indicating whether or not frame_motion_type exists in the bit stream.

[0375] frame_motion_type is a 2-bit code indicating the prediction type of the macroblock of the frame.
If the number of prediction vectors is two and the field-based prediction type is “00”, if the number of prediction vectors is one and the field-based prediction type is “01”, the number of prediction vectors is one and the frame is base Is "10" for the prediction type of "1", and "11" for the prediction type of one prediction vector and dual prime.

If the condition for describing frame_motion_type is not satisfied, a data element representing field_motion_type is described next to a data element representing macroblock_type.

[0377] field_motion_type is a 2-bit code indicating motion prediction of a field macroblock.
If the number of prediction vectors is one and the field-based prediction type is “01”, the prediction vector is 18 ×
"10" for prediction type based on 8 macroblocks
And if the number of prediction vectors is one and the prediction type is dual prime, it is “11”.

If the picture structure is a frame,
_pred_frame_dct indicates that frame_motion_type exists in the bit stream, and frame_pred_frame_d
When ct indicates that dct_type exists in the bit stream, a data element representing dct_type is described next to a data element representing macroblock_type. Note that dct_type is data indicating whether the DCT is a frame DCT mode or a field DCT mode.

Referring back to FIG. 61, if the reference macroblock is a forward prediction macroblock, or if the reference macroblock is an intra macroblock and any of the concealed macroblocks,
A data element defined by the motion_vectors (0) function is described. If the reference macroblock is a backward prediction macroblock, motion_vectors
(1) A data element defined by a function is described. The motion_vectors (0) function is a function for describing a data element relating to a first motion vector, and the motion_vectors (1) function is a function for describing a data element relating to a second motion vector. It is.

The motion_vectors (s) function is a function for describing a data element relating to a motion vector, as shown in FIG.

If the number of motion vectors is one and the dual prime prediction mode is not used, motion_ver
A data element defined by tical_field_select [0] [s] and motion_vector (0, s) is described.

[0382] This motion_vertical_field_select [r] [s]
Indicates whether the first motion vector (whichever vector is forward or backward) is a vector created with reference to the bottom field or a vector created with reference to the top field Is a flag that indicates The index “r” is an index indicating which vector is the first vector or the second vector, and “s” is whether the prediction direction is forward or backward prediction. It is an index that indicates

As shown in FIG. 64, the motion_vector (r, s) function includes a data string relating to motion_code [r] [s] [t], a data string relating to motion_residual [r] [s] [t], dm
This is a function for describing data representing vector [t].

[0384] motion_code [r] [s] [t] is variable-length data representing the magnitude of a motion vector in the range of -16 to +16. motion_residual [r] [s] [t] is variable-length data representing a residual of a motion vector. So this mo
A detailed motion vector can be described by the values of tion_code [r] [s] [t] and motion_residual [r] [s] [t]. dmvector [t] is used in dual prime prediction mode to generate a motion vector in one field (for example, the top field is one field with respect to the bottom field). Are scaled, and are corrected in the vertical direction to reflect the vertical shift between the lines of the top field and the bottom field. The index “r” is an index indicating which vector is the first vector or the second vector, and “s” is whether the prediction direction is forward or backward prediction. It is an index that indicates “S” is data indicating whether the motion vector is a vertical component or a horizontal component.

First, a data string representing motion_coder [r] [s] [0] in the horizontal direction is described as a history stream by the motion_vector (r, s) function shown in FIG. mo
tion_residual [0] [s] [t] and motion_residual [1] [s] [t]
Since the number of bits of both is indicated by f_code [s] [t], f
motion_residual [r] if _code [s] [t] is not 1
This indicates that [s] [t] is present in the bitstream. Motion_residual [r] [s] [0] of the horizontal component is not “1” and motion_code [r] [s] [0] of the horizontal component
Is not "0" means that mo
data_residual [r] [s] [0] exists, which means that the horizontal component of the motion vector exists.
A data element representing motion_residual [r] [s] [0] is described.

Subsequently, motion_coder [r] [s] in the vertical direction
A data string representing [1] is described as a history stream. Similarly, motion_residual [0] [s] [t] and motion
The number of bits of both _residual [1] [s] [t] is f_code [s] [t]
In the case that f_code [s] [t] is not 1,
This indicates that motion_residual [r] [s] [t] is present in the bitstream. motion_residual [r]
That [s] [1] is not “1” and motion_code [r] [s] [1] is not “0” means that the moti
Since there is a data element representing on_residual [r] [s] [1], which means that the vertical component of the motion vector exists, in that case, the mo component of the vertical component
A data element representing tion_residual [r] [s] [1] is described.

Next, referring to FIG. 65 to FIG.
The oblock_type will be described. macroblock_type is ma
croblock_quant, dct_type_flag, macroblock_motion_f
Variable length data generated from flags such as orward and macroblock_motion_backward. macroblock_quan
t is a flag indicating whether or not macroblock_quantiser_scale_code for setting a quantization step size for a macroblock is set. If macroblock_quantiser_scale_code is present in the bit stream, macroblock_quant is a value of “1”. I take the.

[0388] dct_type_flag is a flag indicating whether there is a dct_type indicating whether the reference macroblock is encoded by the frame DCT or the field DCT (in other words, a flag indicating whether the reference macroblock is DCT). If dct_type exists in the bit stream, dct_type_flag takes a value of “1”. macroblo
ck_motion_forward is a flag indicating whether or not the reference macroblock is predicted forward, and takes a value of “1” when the reference macroblock is predicted forward. macroblock_motion_
backward is a flag indicating whether or not the reference macroblock is backward predicted, and takes a value of “1” when backward predicted.

In the variable-length format, history information can be reduced in order to reduce the bit rate to be transmitted.

That is, the macroblock_type and motion_vect
When ors () is transferred but quantizer_scale_code is not transferred, slice_quantiser_scale_code is set to “0000”.
By setting it to 0 ", the bit rate can be reduced.

Also, only macroblock_type is transferred, and motio
n_vectors (), quantiser_scale_code, and dct_type
Is not transferred, macroblock_type is set to "not
By using "coded", the bit rate can be reduced.

Further, when only picture_coding_type is transferred and all information after slice () is not transferred, s
By using picture_data () without lice_start_code, the bit rate can be reduced.

In the above description, in order to prevent consecutive 23-bit "0" in user_data from being output, "1" is inserted every 22 bits, but it is not limited to every 22 bits. . Also, instead of counting the number of consecutive “0” and inserting “1”, Byte_allign can be checked and inserted.

Furthermore, in MPEG, the generation of consecutive 23-bit "0" is prohibited. However, in practice, only the case where 23 bits continue from the head of a byte is considered as a problem. If 0 bits continue for 23 bits from the middle, no problem occurs. Therefore, for example, "1" may be inserted at a position other than the LSB every 24 bits.

In the above description, the history information is stored in vide
o Format is close to elementary stream, but packetized
The format may be close to the elementary stream or the transport stream. Although the location of the user_data of the Elementary Stream is before the picture_data, it can be another location.

In the transcoder 101 shown in FIG. 15, the encoding parameters for four generations are output as history information at the subsequent stage. However, not all of the history information is actually required. The required history information will be different. In addition, the capacity of an actual transmission path or recording medium (transmission medium) is limited. Even if the history information is compressed, transmitting all the history information imposes a load on the capacity. The bit rate of the image bit stream is suppressed, and the effectiveness of history information transmission is impaired.

[0397] Therefore, a descriptor describing a combination of items to be transmitted as history information is incorporated in the history information and transmitted at a later stage, so that information corresponding to various applications is transmitted instead of transmitting all the history information. It can be transmitted. FIG. 68 illustrates a configuration example of the transcoder 101 in such a case.

In FIG. 68, portions corresponding to those in FIG. 15 are denoted by the same reference numerals, and description thereof will be omitted as appropriate. In the configuration example of FIG. 68, an encoding parameter selection circuit 501 is inserted between the history information separating device 105 and the encoding device 106 and between the history encoding device 107 and the encoding device 106.

An encoding parameter selection circuit 501 calculates an encoding parameter from a baseband video signal output from the history information separating device 105, and an encoding parameter calculating section 512. The transcoder 101 outputs from the history information separating device 105. , The coding parameters and the descriptors (red_bw_flag, red_bw_indicator) are obtained from information on the coding parameters determined to be optimal for coding (for example, second-generation coding parameters).
(To be described later with reference to FIG. 72), a combination descriptor separation unit 511, a coding parameter output from the coding parameter calculation unit 512, and a combination descriptor separation unit 51.
1 is selected according to the descriptor separated by the combination descriptor separation unit 511, and the switch 5 is output to the coding device 106.
13. Other configurations are the same as those in FIG.

[0400] Here, the combination of items transmitted as history information will be described. Historical information can be categorized as
The information can be divided into picture unit information and macroblock unit information. The information of the slice unit is the mac included in it.
The information can be obtained by collecting the information of the roblock, and the information of the GOP unit can be obtained by collecting the information of the picture unit included therein.

[0401] The information in picture units is one for each frame.
Since the data is transmitted only once, the bit rate occupying the information transmission is not so large. In contrast, macroblo
Since information in units of ck is transmitted for each macroblock, for example, in a video system in which the number of scanning lines in one frame is 525 and the field rate is 60 fields / sec, 1
If the number of pixels in the frame is 720 × 480, macroblo
The information in units of ck is 1350 (= (72
0/16) × (480/16)) transmissions. Therefore, a considerable part of the history information is occupied by information for each macroblock. Therefore, as the history information, at least the information in picture units is always transmitted, but the information in macroblock units is selected and transmitted according to the application, so that the amount of information to be transmitted can be suppressed.

[0402] Information in macroblock units transferred as history information includes, for example, num_coef_bits, num_mv_bits, num_
other_bits, q_scale_code, q_scale_type, motion_typ
e, mv_vert_field_sel [] [], mv [] [] [], mb_mfwd, mb_mb
wd, mb_pattern, coded_block_pattern, mb_intra, sli
There are ce_start, dct_type, mb_quant, skipped_mb, etc. These are expressed using elements of macroblock rate information.

Num_coef_bits represents the code amount required for the DCT coefficient among the code amounts of the macroblock. num_mv_bits is
Indicates the amount of code required for the motion vector among the amounts of code of the macroblock. num_other_bits represents a code amount other than num_coef_bits and num_mv_bits in the code amount of the macroblock.

[0404] The q_scale_code is applied to the macroblock.
Represents q_scale_code. motion_type represents the type of the motion vector applied to the macroblock. mv_vert_field_se
l [] [] is the field of the motion vector applied to the macroblock
Represents select.

[0405] mv [] [] [] represents a motion vector applied to a macroblock. mb_mfwd is a flag indicating that the prediction mode of the macroblock is forward prediction. mb_mbwd
Is a flag indicating that the prediction mode of the macroblock is backward prediction. mb_pattern is a flag indicating the presence or absence of a non-zero DCT coefficient of the macroblock.

The coded_block_pattern is the DCT of the macroblock.
This flag indicates whether or not there is a non-zero coefficient for each DCT block. mb_intra is a flag indicating whether the macroblock is intra_macro or not. slice_start is macrobl
This is a flag indicating whether or not ock is the head of slice. dc
t_type is a flag indicating whether the macroblock is field_dct or flame_dct.

[0407] For mb_quant, the macroblock is quantizer_scal.
This is a flag indicating whether or not to transmit e_code. skipped_
mb is a flag indicating whether the macroblock is a skipped macroblock.

[0408] All these items are not always necessary, and the necessary items change depending on the application. For example, items such as num_coef_bits and slice_start are necessary for an application having a request of “transparent” to restore a bit stream after re-encoding as much as possible. In other words, these items are not necessary in an application that changes the bit rate. Also, there are applications in which when the transmission path is extremely restricted, it is only necessary to know the coding type of each picture. From such a situation, for example, a combination as shown in FIG. 69 can be considered as an example of a combination of items for transmitting history information.

In FIG. 69, a value “2” corresponding to an item in each combination means that the information exists and can be used, and “0” means that the information does not exist. I do. “1” indicates that the information itself has no meaning, such as for the purpose of assisting the existence of other information or in the syntax, but having no relation to the original bit stream information. . For example, slice_start is
In the macroblock at the head of the slice when transmitting the history information, it becomes “1”. However, if the slice is not necessarily in the same positional relationship with the original bit stream, it is meaningless as the history information. become.

In the example of FIG. 69, (num_coef_bit
s, num_mv_bits, num_other_bits), (q_scale_code,
q_scale_type), (motion_type, mv_vert_field_sel []
[], Mv [] [] []), (mb_mfwd, mb_mbwd), (mb_patter
n), (coded_block_pattern), (mb_intra), (slic
e_start), (dct_type), (mb_quant), (skipped_m)
Five combinations of combinations 1 to 5 are prepared depending on the presence or absence of each item of b).

[0411] The combination 1 is a combination for reconstructing a completely transparent bit stream. According to this combination, highly accurate transcoding by using the generated code amount information can be realized. The combination 2 is also a combination for reconstructing a completely transparent bit stream. Combination 3 is a combination that cannot completely reconstruct a completely transparent bit stream, but allows a visually almost transparent bit stream to be reconstructed. Combination 4 is tran
Inferior to combination 3 in terms of sparent,
This is a combination that can reconstruct the bit stream without any visual problem. The combination 5 is inferior to the combination 4 from the viewpoint of transparency, but is a combination that allows incomplete reconstruction of the bit stream with a small amount of history information.

[0412] Among these combinations, the smaller the number of the combination number is, the higher in function, but the larger the capacity required to transfer the history becomes. Therefore, it is necessary to determine the combination to be transmitted by considering the assumed application and the capacity available for the history.

Next, the operation of the transcoder 101 shown in FIG. 68 will be described with reference to the flowchart shown in FIG. In step S41, the transcoder 101
Decoding apparatus 102 decodes an input bit stream, extracts an encoding parameter (4th) used when encoding the bit stream, and uses the encoding parameter (4th) as a history information multiplexing apparatus. 103, and also outputs the decoded video data to the history information multiplexing device 103. In step S42, the decoding device 102 also extracts user_data from the input bit stream and outputs the user_data to the history decoding device 104. History decoding device 1
04 is the user_dat input in step S43.
Extract the combination information (descriptor) from a, and further use it to encode the coding parameter (1s
t, 2nd, 3rd) to extract the history information multiplexing device 1
03 is output.

[0414] In step S44, the history information multiplexing apparatus 103 outputs the current encoding parameter (4) supplied from the decoding apparatus 102 extracted in step S41.
th) and the past encoding parameters (1st, 2n) output by the history decoding device 104 in step S43.
d, 3rd) are multiplexed with the baseband video data supplied from the decoding device 102 according to the format shown in FIG. 18 or FIG. 31 and output to the history information separating device 105.

[0415] In step S45, the history information separation device 105 extracts coding parameters from the baseband video data supplied from the history information multiplexing device 103, and from there, encodes a code most suitable for the current coding. An encoding parameter (for example, a second-generation encoding parameter) is selected and output to the combination descriptor separating unit 511 together with the descriptor. Also, the history information separation device 105
Are the encoding parameters other than the encoding parameters determined to be optimal for the current encoding (for example, if the optimal encoding parameter is determined to be the second-generation encoding parameter, the other The first-generation, third-generation, and fourth-generation encoding parameters are output to the history encoding device 107. History encoding device 1
07, the encoding parameter input from the history information separating device 105 is stored in the user_data
And user_data (converted_history_strea
m ()) is output to the encoding device 106.

[0416] The combination descriptor separation unit 511 of the coding parameter selection circuit 501 separates the coding parameter and the descriptor from the data supplied from the history information separation device 105, and sets the coding parameter (2nd) of the switch 513. Supply to one contact. To the other contact of the switch 513, an encoding parameter calculation unit 512 calculates and supplies an encoding parameter from baseband video data output by the history information separation device 105. The switch 513 determines in step S48 the encoding parameter output from the combination descriptor separating unit 511 or the encoding parameter output from the encoding parameter calculating unit 512, corresponding to the descriptor output from the combination descriptor separating unit 511. And outputs it to the encoding device 106. That is, in the switch 513, the combination descriptor separating unit 51
If the coding parameter supplied from 1 is valid, the coding parameter output from the combination descriptor separation unit 511 is selected, but the coding parameter output from the combination descriptor separation unit 511 is invalid. Is determined, the encoding parameter calculated by the encoding parameter calculation unit 512 processing the baseband video is selected. This selection is made according to the capacity of the transmission medium.

[0417] In step S49, the encoding device 106 encodes the baseband video signal supplied from the history information separating device 105 based on the encoding parameter supplied from the switch 513. In addition, in step S50, the encoding device 106 adds the history encoding device 10 to the encoded bit stream.
7 is multiplexed and output from the user_data supplied from 7.

[0418] In this way, even when the combinations of the encoding parameters obtained by the respective histories are different, transcoding can be performed without any trouble.

[0419] As described above, the history information is, as shown in Fig. 38, history_stream () as a kind of user_data () function of the video stream (more precisely, converted_hi
transmitted by story_stream ()). That history_stream ()
Is as shown in FIG. Descriptors (red_bw_flag, red
_bw_indicator) and items that are not supported in MPEG stream (num_other_bits, num_mv_bits, n
um_coef_bits) is re_coding_stream in FIG.
It is transmitted by the _info () function.

[0420] The re_coding_stream_info () function is shown in FIG.
User_data_start_code, re_coding_stre
am_info_ID, red_bw_flag, red_bw_indicator, marker_
It is composed of data elements such as bit, num_other_bits, num_mv_bits, and num_coef_bits.

[0421] user_data_start_code is a start code indicating that user_data starts. re_coding_st
ream_info_ID is a 16-bit integer.
Used to identify the stream_info () function. Specifically, the value is set to “1001 00011110 1100” (0x91ec).

[0422] red_bw_flag is a 1-bit flag, and is set to 0 when the history information transmits all items. When the value of this flag is 1, the flag follows this flag.
By examining red_bw_indicator, it is possible to determine which of the five combinations shown in FIG. 69 is sending an item.

[0423] red_bw_indicator is a 2-bit integer, and describes a combination of items as shown in FIG.

That is, in the case of combination 1 among the five combinations shown in FIG. 69, red_bw_flag is set to 0,
For combinations 2 to 5, red_bw_flag is set to 1. On the other hand, red_bw_indicator is set to 0 for combination 2, 1 for combination 3, 2 for combination 4, and 3 for combination 5.

[0425] Therefore, red_bw_indicator is equivalent to red_bw_fla
It is defined when g is 1 (for combinations 2 to 5).

Further, as shown in FIG. 71, red_bw_fla
When g is 0 (in the case of combination 1), marker_bit, num_other_bits, num_mv_bit
s and num_coef_bits are described. These four data elements are used in combination 2 to combination 5 (re
Not specified if d_bw_flag is 1).

As shown in FIG. 59, the picture_data () function is composed of one or more slice () functions. However, in the case of the combination 5, syntax elements lower than that including the picture_data () function are not transmitted (FIG. 69). In this case, the history information is pi such as picture_type.
This is intended to transmit information in units of cture.

In the case of combinations 1 to 4, the slice () function shown in FIG. 60 exists. However,
The slice position information determined by the slice () function and the slice position information of the original bit stream depend on a combination of items of history information. In the case of combination 1 or combination 2, the position information of the slice of the bit stream that is the source of the history information and the position information of the slice determined by the slice () function need to be the same.

The syntax element of the macroblock () function shown in FIG. 61 depends on the combination of items of history information.
macroblock_escape, macroblock_address_increment, m
The acroblock_modes () function always exists. However, macroblock_escape and macroblock_address_incremen
The validity of t as information is determined by the combination. When the combination of items of history information is combination 1 or combination 2, skip of the original bit stream
The same ped_mb information needs to be transmitted.

In the case of the combination 4, there is no motion_vectors () function. In the case of the combinations 1 to 3, the existence of the motion_vectors () function is determined by the macroblock_type of the macroblock_modes () function. In the case of combination 3 or combination 4, coded_block_pa
There is no ttern () function. In case of combination 1 and combination 2, macroblock_type of macroblock_modes () function
Determines the existence of the coded_block_pattern () function.

[0431] The syntax element of the macroblock_modes () function shown in Fig. 62 depends on the combination of items of history information. macroblock_type always exists. If the combination is combination 4, flame_motion_type, fie
ld_motion_type and dct_type do not exist.

The validity of the parameter obtained from macroblock_type as information is determined by the combination of items of history information.

If the combination of the items of the history information is combination 1 or combination 2, macroblock_quant
Must be the same as the original bitstream. Macroblock_qua for combination 3 or combination 4
nt indicates the presence of quantizer_scale_code in the macroblock () function, and need not be the same as the original bitstream.

If the combination is combination 1 to combination 3, macroblock_motion_forward and macrobloc
k_motion_backward needs to be the same as the original bitstream. If the combination is combination 4 or combination 5, it is not necessary.

When the combination is combination 1 or combination 2, macroblock_pattern needs to be the same as the original bit stream. In the case of combination 3, macroblock_pattern is used to indicate the presence of dct_type. When the combination is combination 4, the relationship as in the case of combinations 1 to 3 is not established.

When the combination of items of the history information is combination 1 to combination 3, macroblock_intra needs to be the same as the original bit stream. In the case of combination 4, this is not always the case.

[0437] The history_stream () in Fig. 47 is the syntax when the history information is of variable length. As shown in Figs. 40 to 46, when the syntax is of fixed length, the history information is of a fixed length. In the descriptor (red_bw_flag) as information indicating which of the transmitted items is valid
And red_bw_indicator) are superimposed on the baseband image and transmitted. As a result, by examining this descriptor, it is possible to determine that the field exists as a field but its contents are invalid.

Therefore, as shown in FIG. 44, re_codin
As g_stream_information, user_data_start_code,
re_coding_stream_info_ID, red_bw_flag, red_bw_indi
cator and marker_bit are arranged. Each meaning is the same as in FIG. 71.

As described above, by transmitting the elements of the encoding parameter transmitted as the history in a combination corresponding to the application, the history corresponding to the application can be transmitted with an appropriate data amount.

As described above, when transmitting the history information as a variable-length code, the re_coding_stream_info () function is configured as shown in FIG. 71, and as shown in FIG.
Transmitted as part of the ory_stream () function. On the other hand, when the history information is transmitted as a fixed-length code, re_coding_stream_information () is transmitted as a part of the history_stream () function, as shown in FIG. In the example of FIG. 44, as re_coding_stream_information,
user_data_start_code, re_coding_stream_info_ID, re
d_bw_flag and red_bw_indicator are transmitted.

Also, the history information multiplexing device 1 shown in FIG.
03 for transmitting the history information in the baseband signal output by Re_Coding inform shown in FIG.
ation Bus macroblock format is specified. This macro block is composed of 16 × 16 (= 256) bits. Then, in FIG. 73, 32 bits shown in the third and fourth rows from the top are set as picrate_element. The picture rate elements shown in FIGS. 74 to 76 are described in the picrate_element. A 1-bit red_bw_flag is defined in the second row from the top in FIG.
In the third row, a 3-bit red_bw_indicator is defined. That is, these flags red_bw_flag, red_bw_in
The indicator is transmitted as picrate_element in FIG.

Referring to the other data shown in FIG. 73, SRIB_sync_code is a code indicating that the first row of a macro block in this format is left-aligned, and specifically, is set to “11111”. Is done. fr_fl_SRIB is set to 1 when picture_structure is a frame picture structure (when its value is “11”), and Re_Coding Information Bus macr
Indicates that oblock is transmitted over 16 lines, and pi
If the structure_structure is not a frame structure, it is set to 0, which means that the Re_Coding Information Bus is transmitted over 16 lines. By this mechanism, Re_C
The oding information bus is locked to the corresponding pixels of the video frame or field that have been decoded spatially and temporally.

[0443] SRIB_top_field_first is set to the same value as top_field_first held in the original bit stream, and the Re_Coding Information Bus of the related video is set.
Is shown together with repeat_first_field. SRIB_repeat_first_field is set to the same value as repeat_first_field held in the original bit stream. Re_Coding Information B of first field
The contents of us need to be repeated as indicated by this flag.

[0444] 422_420_chroma indicates whether the original bit stream is 4: 2: 2 or 4: 2: 0. A value of 0 indicates that the bit stream is 4: 2: 0 and that the up-sampling of the chrominance signal has been performed such that a 4: 2: 2 video is output. The value 0 indicates that the filtering process of the color difference signal has not been executed.

[0445] rolling_SRIB_mb_ref represents a 16-bit modulo 65521, and this value is incremented for each macroblock. This value needs to be continuous over the frames of the frame picture structure. Otherwise, this value must be contiguous across fields. This value is initialized to a predetermined value between 0 and 65520. This allows a unique Re_Coding Information Bus identifier to be incorporated into the recorder system.

[0446] The meanings of the other data of the Re_Coding Information Bus macroblock are as described above, and are therefore omitted here.

As shown in FIG. 77, the data of the 256-bit Re_Coding Information Bus of FIG. 73 is Cb [0] [0], Cr [0] [0], and Cb, which are LSBs of color difference data one bit at a time.
[1] [0], Cr [1] [0]. Since 4-bit data can be transmitted by the format shown in FIG. 77, the 256-bit data shown in FIG. 73 can be transmitted by transmitting 64 (= 256/4) data in the format shown in FIG.

According to the transcoder of the present invention, since the encoding parameters generated in the past encoding process are reused in the current encoding process, the decoding process and the encoding process are repeated. No image quality degradation occurs. That is, it is possible to reduce accumulation of image quality deterioration due to repetition of decoding processing and encoding processing.

According to the transcoder of the present invention, the coding parameters generated in the past coding process are described in the user data area of the coded stream generated in the current coding process. Since the bit stream is a coded stream conforming to the MPEG standard, any existing decoder can perform decoding processing. Furthermore, according to the transcoder of the present invention, it is not necessary to provide a kind of dedicated line for transmitting the coding parameters in the past coding process, so that the conventional data stream transmission environment can be used as it is. , Past coding parameters can be transmitted.

According to the transcoder of the present invention, the coding parameters generated in the past coding process are selectively described in the coded stream generated in the current coding process. The past encoding parameters can be transmitted without extremely increasing the bit rate of the output bit stream.

According to the transcoder of the present invention, an encoding parameter optimum for the current encoding process is selected from the past encoding parameters and the current encoding parameters, and the encoding process is performed. Therefore, even if the decoding process and the encoding process are repeated, the image quality degradation is not accumulated.

According to the transcoder of the present invention, the encoding process is performed by selecting the most suitable encoding parameter for the current encoding process from the past encoding parameters according to the picture type. Therefore, even if the decoding process and the encoding process are repeated, the image quality does not deteriorate.

According to the transcoder of the present invention, whether to reuse past coding parameters is determined based on the picture type included in past coding parameters. It can be performed.

The computer program for performing each of the above processes is provided by being recorded on a recording medium such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and transmitted via a network such as the Internet or a digital satellite. It can be provided by recording on a user's recording medium.

[0455]

As described above, according to the stream generating apparatus of claim 1, the stream generating method of claim 4, and the program of the recording medium of claim 5,
Since the combination information indicating the combination of the coding histories in the previous coding processing is described in the second coded stream, the stream including the history information that can be transmitted through the transmission path with a small capacity is used. Can be generated.

[0456] The stream transmitting apparatus according to claim 6, the stream transmitting method according to claim 9, and the stream transmitting apparatus according to claim 9.
According to the program of the recording medium described in the above, a plurality of encoding parameters used in the past encoding process are selectively combined, to generate the encoding history information, and to multiply the encoded stream Therefore, it is possible to transmit a stream that can suppress the deterioration of an image due to re-encoding via a small-capacity medium.

[0457] The encoding apparatus according to the eleventh aspect, the first aspect.
According to the encoding method of the fourth aspect and the program of the recording medium of the fifteenth aspect, an encoding parameter is extracted based on the combination information, and the first encoding is performed based on the extracted encoding parameter. Since the stream is encoded into the second encoded stream, it is possible to encode a stream that can be transmitted to a transmission medium with a small transmission capacity while suppressing image degradation due to re-encoding.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating the principle of high-efficiency coding.

FIG. 2 is a diagram illustrating a picture type when image data is compressed.

FIG. 3 is a diagram illustrating a picture type when image data is compressed.

FIG. 4 is a diagram illustrating the principle of encoding a moving image signal.

FIG. 5 is a block diagram illustrating a configuration of an apparatus that encodes and decodes a moving image signal.

FIG. 6 is a diagram illustrating a configuration of image data.

FIG. 7 is a block diagram showing a configuration of an encoder 18 of FIG.

8 is a diagram for explaining the operation of the prediction mode switching circuit 52 of FIG.

FIG. 9 is a diagram illustrating the operation of the prediction mode switching circuit 52 of FIG.

FIG. 10 is a diagram illustrating the operation of the prediction mode switching circuit 52 of FIG.

11 is a diagram illustrating the operation of the prediction mode switching circuit 52 of FIG.

FIG. 12 is a block diagram showing a configuration of a decoder 31 of FIG.

FIG. 13 is a diagram illustrating SNR control corresponding to a picture type.

FIG. 14 is a block diagram illustrating a configuration of a transcoder 101 to which the present invention has been applied.

FIG. 15 is a block diagram showing a more detailed configuration of the transcoder 101 of FIG.

16 is a block diagram illustrating a configuration of a decoder 111 built in the decoding device 102 in FIG.

FIG. 17 is a diagram illustrating pixels of a macroblock.

FIG. 18 is a diagram illustrating an area where an encoding parameter is recorded.

19 is a block diagram showing a configuration of an encoder 121 built in the encoding device 106 in FIG.

20 is a block diagram illustrating a configuration example of a history VLC 211 in FIG.

21 is a block diagram illustrating a configuration example of a history VLD 203 in FIG.

FIG. 22 is a block diagram illustrating a configuration example of a converter 212 in FIG.

23 is a block diagram illustrating a configuration example of a stuff circuit 323 in FIG.

24 is a timing chart illustrating the operation of converter 212 in FIG.

25 is a block diagram illustrating a configuration example of a converter 202 in FIG.

26 is a block diagram illustrating a configuration example of a delay circuit 343 of FIG.

FIG. 27 is a block diagram showing another configuration example of converter 212 in FIG.

FIG. 28 is a block diagram showing another configuration example of converter 202 in FIG.

FIG. 29 is a block diagram illustrating a configuration example of a user data formatter 213 in FIG.

FIG. 30 is a diagram showing a state where the transcoder 101 of FIG. 14 is actually used.

FIG. 31 is a diagram illustrating an area where an encoding parameter is recorded.

32 is a flowchart illustrating a changeable picture type determination process of the encoding device 106 in FIG.

FIG. 33 is a diagram illustrating an example in which a picture type is changed.

FIG. 34 is a diagram illustrating another example in which the picture type is changed.

FIG. 35 is a diagram illustrating quantization control processing of the encoding device 106 in FIG. 14;

FIG. 36 is a flowchart illustrating a quantization control process of the encoding device 106 in FIG. 14;

FIG. 37 is a block diagram illustrating a configuration of a transcoder 101 that is tightly coupled.

FIG. 38 is a diagram illustrating the syntax of a stream of a video sequence.

FIG. 39 is a diagram illustrating the configuration of the syntax in FIG. 38.

FIG. 40 history_stream for recording fixed-length history information
It is a figure explaining the syntax of ().

FIG. 41 history_stream for recording fixed-length history information
It is a figure explaining the syntax of ().

FIG. 42 history_stream for recording fixed-length history information
It is a figure explaining the syntax of ().

FIG. 43: history_stream for recording fixed-length history information
It is a figure explaining the syntax of ().

FIG. 44 history_stream for recording fixed-length history information
It is a figure explaining the syntax of ().

FIG. 45: history_stream for recording fixed-length history information
It is a figure explaining the syntax of ().

FIG. 46 shows history_stream for recording fixed-length history information.
It is a figure explaining the syntax of ().

FIG. 47 history_stream for recording variable-length history information
It is a figure explaining the syntax of ().

[Fig. 48] Fig. 48 is a diagram for describing the syntax of sequence_header ().

Fig. 49 is a diagram for describing the syntax of sequence_extension ().

[Fig. 50] Fig. 50 is a diagram for describing the syntax of extension_and_user_data ().

Fig. 51 is a diagram for describing the syntax of user_data ().

Fig. 52 is a diagram for describing the syntax of group_of_pictures_header ().

Fig. 53 is a diagram for describing the syntax of picture_header ().

Fig. 54 is a diagram for describing the syntax of picture_coding_extension ().

[Fig. 55] Fig. 55 is a diagram for describing the syntax of extension_data ().

Fig. 56 is a diagram for describing the syntax of quant_matrix_extension ().

[Fig. 57] Fig. 57 is a diagram for describing the syntax of copyright_extension ().

Fig. 58 is a diagram for describing the syntax of picture_display_extension ().

Fig. 59 is a diagram for describing the syntax of picture_data ().

Fig. 60 is a diagram for describing the syntax of slice ().

Fig. 61 is a diagram for describing the syntax of macroblock ().

Fig. 62 is a diagram for describing the syntax of macroblock_modes ().

Fig. 63 is a diagram for describing the syntax of motion_vectors (s).

Fig. 64 is a diagram for describing the syntax of motion_vector (r, s).

[Fig. 65] Fig. 65 is a diagram for describing a macroblock_type variable-length code for an I picture.

[Fig. 66] Fig. 66 is a diagram for describing a macroblock_type variable-length code for a P picture.

[Fig. 67] Fig. 67 is a diagram illustrating a macroblock_type variable length code for a B picture.

FIG. 68 is a block diagram showing another configuration of the transcoder 101 to which the present invention is applied.

FIG. 69 is a diagram illustrating a combination of items of history information.

70 is a flowchart illustrating the operation of the transcoder 101 in FIG. 68.

Fig. 71 is a diagram for describing the syntax of re_coding_stream_info ().

FIG. 72 is a view for explaining red_bw_flag and red_bw_indicator.

FIG. 73: Re_Coding Information Bus macroblock for
It is a figure explaining mation.

FIG. 74 is a diagram illustrating Picture rate elements.

Fig. 75 is a diagram for describing Picture rate elements.

FIG. 76 is a view for explaining Picture rate elements.

Fig. 77 is a diagram illustrating an area where a Re_Coding Information Bus is recorded.

[Explanation of symbols]

1 encoding device, 2 decoding device, 3 recording medium,
12, 13 A / D converter, 14 frame memory,
15 luminance signal frame memory, 16 color difference signal frame memory, 17 format conversion circuit, 18
Encoder, 31 decoder, 32 format conversion circuit, 33 frame memory, 34 luminance signal frame memory, 35 color difference signal frame memory, 3
6, 37 D / A converter, 50 motion vector detection circuit, 51 frame memory, 52 prediction mode switching circuit, 53 arithmetic unit, 54 prediction judgment circuit, 5
5 DCT mode switching circuit, 56 DCT circuit, 57
Quantization circuit, 58 variable length coding circuit, 59 transmission buffer, 60 inverse quantization circuit, 61 IDCT circuit,
62 arithmetic unit, 63 frame memory, 64 motion compensation circuit, 81 reception buffer, 82 variable length decoding circuit, 83 inverse quantization circuit, 84 IDCT circuit, 85
Arithmetic unit, 86 frame memory, 87 motion compensation circuit, 101 transcoder, 102 decoding device, 103 history information multiplexing device, 105 history information separating device, 106 encoding device, 11
1 decoder, 112 variable length decoding circuit, 121
Encoder,

────────────────────────────────────────────────── ───

[Procedure amendment]

[Submission date] December 10, 1999 (1999.12.
10)

[Procedure amendment 1]

[Document name to be amended] Drawing

[Correction target item name] Figure 3

[Correction method] Change

[Correction contents]

FIG. 3

[Procedure amendment 2]

[Document name to be amended] Drawing

[Correction target item name] FIG.

[Correction method] Change

[Correction contents]

FIG.

[Procedure amendment 3]

[Document name to be amended] Drawing

[Correction target item name] FIG.

[Correction method] Change

[Correction contents]

FIG.

[Procedure amendment 4]

[Document name to be amended] Drawing

[Correction target item name] FIG.

[Correction method] Change

[Correction contents]

FIG. 39

Claims (15)

[Claims] 1. A stream generating apparatus for generating and outputting a second encoded stream from an input first encoded stream, wherein the encoding history in a past encoding process of the first encoded stream is Detecting means for detecting; coding means for coding the first coded stream to generate the second coded stream by using the detection result of the detecting means; and the second coded stream And a description unit for describing combination information indicating a combination of encoding histories in the encoding process up to that time. 2. The stream generating apparatus according to claim 1, wherein said encoding means performs encoding by an MPEG system. 3. The stream generation apparatus according to claim 2, wherein the description unit describes combination information indicating a combination of the encoding histories as user_data of the second stream of the MPEG system. . 4. A stream generation method of a stream generation device for generating and outputting a second coded stream from an input first coded stream, wherein the stream generation method includes the steps of: A detecting step of detecting an encoding history; an encoding step of encoding the first encoded stream to generate the second encoded stream by using a detection result in the detecting step; A description step of describing combination information indicating a combination of encoding histories in the encoding process up to that time for the two encoded streams. 5. A program for controlling a stream generation device that generates and outputs a second coded stream from an input first coded stream, the program comprising: A detecting step of detecting an encoding history; an encoding step of encoding the first encoded stream to generate the second encoded stream by using a detection result in the detecting step; A description step of describing combination information indicating a combination of encoding histories in the encoding process up to that time with respect to the second encoded stream. Medium. 6. A stream transmitting apparatus for transmitting an encoded stream via a predetermined medium, comprising: a combination means for selectively combining predetermined encoding parameters from a plurality of encoding parameters used in past encoding processing; Generating means for generating encoding history information indicating the history of the encoding parameters combined by the combination means, the encoding history information generated by the generating means,
A transmission means for multiplying the coded stream and transmitting the multiplexed stream to the medium. 7. The stream transmission device according to claim 6, wherein the encoded stream is encoded by an MPEG method. 8. The stream generation apparatus according to claim 6, wherein the multiplication unit multiplies the encoding history information as the user_data on the encoded stream. 9. A stream transmission method of a stream transmission apparatus for transmitting an encoded stream via a predetermined medium, wherein the predetermined encoding parameter is selectively selected from a plurality of encoding parameters used in a past encoding process. A combining step of combining, a generating step of generating coding history information representing a history of the coding parameters combined by the processing of the combining step, and the coding history information generated by the processing of the generating step, Transmitting the encoded stream to the medium by multiplying the encoded stream. 10. A program for controlling a stream transmission apparatus for transmitting an encoded stream via a predetermined medium, wherein the program selectively selects a predetermined encoding parameter from a plurality of encoding parameters used in a past encoding process. A combining step of combining, a generating step of generating coding history information representing a history of the coding parameters combined by the processing of the combining step, and the coding history information generated by the processing of the generating step, And transmitting the encoded stream to the medium by superimposing the encoded stream on the encoded stream. 11. An encoding device that encodes an input first coded stream to generate a second coded stream, wherein the first code described in the first coded stream is Detecting means for detecting combination information representing a combination of encoding parameters in the encoding process up to that of the first encoded stream, based on the combination information detected by the detecting means, Extracting means for extracting an encoding parameter in the encoding processing of the above, and encoding means for encoding the first encoded stream into a second encoded stream based on the encoding parameter extracted by the extracting means. An encoding device comprising: 12. The encoding apparatus according to claim 11, wherein said encoding means performs encoding by an MPEG method. 13. The encoding apparatus according to claim 11, wherein the detection unit detects the combination information from user_data of the first encoded stream. 14. An encoding method of an encoding device for encoding an input first encoded stream to generate a second encoded stream, wherein the encoding method is described in the first encoded stream. A detecting step of detecting combination information indicating a combination of encoding parameters in the previous encoding processing of the first encoded stream; and the first step based on the combination information detected by the processing of the detecting step An extraction step of extracting an encoding parameter in a previous encoding process of the encoded stream, and a second encoding process of the first encoded stream based on the encoding parameter extracted by the extraction step. Encoding step of encoding the stream. 15. A program for controlling an encoding device that encodes an input first encoded stream to generate a second encoded stream, the program including a description of the first encoded stream. A detecting step of detecting combination information indicating a combination of encoding parameters in the previous encoding processing of the first encoded stream; and the first step based on the combination information detected by the processing of the detecting step An extraction step of extracting an encoding parameter in a previous encoding process of the encoded stream, and a second encoding process of the first encoded stream based on the encoding parameter extracted by the extraction step. Encoding the stream into a stream. A recording medium on which a program is recorded.