JP2001169278A - Device and method for generating stream, device and method for transmitting stream, device and method for coding and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a trans coding system for not causing image quality degradation. SOLUTION: A history information separating device 115 detects a coding history in coding processing in the past and a comparison device 114 detects the non-continuity of images. A coder 116 generates coded streams, based on the coding history in the coding processing in the past and the non-continuity of the images.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a stream generation apparatus and method, a stream transmission apparatus and method, an encoding apparatus and method, and a recording medium, and more particularly, to a coded bit stream encoded based on the MPEG standard. GO
Stream generating apparatus and method, stream transmitting apparatus and method, encoding apparatus and method suitable for use in transcoding apparatus for changing the structure of P (Group of Pictures) or changing the bit rate of an encoded bit stream And a recording medium.

[0002]

2. Description of the Related Art In recent years, a broadcasting station that produces and broadcasts a television program uses a moving picture expert (MPEG) to compress / encode video data.
roup) technology has become popular. In particular, when video data is recorded on a recording medium material that can be randomly accessed such as a tape, and when video data is transmitted via a cable or a satellite, the MPEG technology is becoming a de facto standard.

[0003] An example of processing in a broadcasting station until a video program produced in the broadcasting station is transmitted to each home will be briefly described. First, a video camera and VTR (Video
The source video data is encoded and recorded on a magnetic tape by an encoder provided in a camcorder integrated with a tape recorder. At this time, the encoder of the camcorder encodes the source video data so as to be suitable for the recording format of the VTR tape. For example, the GOP structure of an MPEG bit stream recorded on the magnetic tape is a structure in which one GOP is composed of two frames (for example, I, B, I, B, I, B,...). Is done. The bit rate of the MPEG bit stream recorded on the magnetic tape is 18 Mbps.

Next, the main broadcast station performs an editing process for editing the video bit stream recorded on the magnetic tape. For this purpose, the GOP structure of the video bit stream recorded on the magnetic tape is converted into a GOP structure suitable for editing processing. The GOP structure suitable for the editing process is a GOP structure in which one GOP is composed of one frame and all pictures are I pictures. This is because an I-picture that is not correlated with other pictures is most suitable for editing on a frame basis. As an actual operation, a video stream recorded on a magnetic tape is temporarily decoded and returned to baseband video data. Then, the baseband video signal is re-encoded so that all pictures become I pictures. By performing the decoding process and the re-encoding process in this manner, a bit stream having a GOP structure suitable for the editing process can be generated.

Next, in order to transmit the edited video program generated by the above-described editing process from the main station to the local station, the bit stream of the edited video program is converted into a GOP structure and a bit rate suitable for the transmission process. I do. The GOP structure suitable for transmission between broadcast stations is, for example, a GOP structure in which one GOP is composed of 15 frames (for example, I, B, B, P, B, B, P...). In addition, the bit rate suitable for transmission between broadcasting stations should be a high bit rate of 50 Mbps or more because dedicated lines with high transmission capacity such as optical fibers are generally provided between broadcasting stations. Is desirable. In particular,
The bit stream of the edited video program is temporarily decoded and returned to baseband video data.
Then, the baseband video data is re-encoded so as to have a GOP structure and a bit rate suitable for the above-described transmission between broadcast stations.

[0006] In the local station, an editing process is performed to insert a local-specific commercial into a video program transmitted from the main station. That is, similarly to the above-described editing processing, the video stream transmitted from the main station is temporarily decoded and returned to the baseband video data. Then, by re-encoding the baseband video signal so that all pictures become I pictures, a GOP suitable for editing processing is obtained.
A bit stream having a structure can be generated.

Subsequently, the video program edited in the local station is converted into a GOP structure and a bit rate suitable for the transmission process in order to transmit the video program to each home via a cable or a satellite. For example, a GOP structure suitable for transmission processing for transmission to each home is such that one GOP is 15
GOP structure composed of frames (for example, I, B,
B, P, B, B, P...), And the bit rate suitable for transmission processing for transmission to each home is a low bit rate of about 5 Mbps. Specifically, the bit stream of the edited video program is once decoded and returned to baseband video data. Then, the baseband video data is transmitted to the GO
Re-encode to have P structure and bit rate.

As described above, while the video program is transmitted from the broadcasting station to each home, the decoding, encoding, and editing processes are repeated a plurality of times. actually,
The processing at the broadcasting station requires various signal processings in addition to the signal processing described above, and each time the decoding processing and the encoding processing are repeated.

It is well known that the encoding process and the decoding process based on the MPEG standard are not 100% reversible processes. That is, the baseband video data before encoding and the video data after decoding are 100
%, The image quality is degraded by the encoding process and the decoding process. That is, as described above, if the decoding process and the encoding process are repeated, there is a problem that the image quality is deteriorated every time the process is performed. In other words, each time decoding / encoding processing is repeated, deterioration in image quality is accumulated.

Therefore, in order to prevent the image quality from deteriorating due to the decoding / encoding process, the encoding parameters used in the previous encoding process are transmitted in association with the image and used in the previous encoding process. A system that performs encoding using the obtained encoding parameters is used.

[0011]

However, it is attempted to encode an image on which editing processing such as deletion of an image or insertion of an image has been performed by using the encoding parameters used in the previous encoding processing. In such a case, the image quality may be greatly deteriorated.

FIG. 1 shows an example of a picture displayed from the left side to the right side in FIG. 1 over time. B in FIG. 1 indicates a picture encoded as a B picture in the previous encoding process, I in FIG. 1 indicates a picture encoded as an I picture in the previous encoding process, P in FIG. 1 indicates a picture coded as a P picture in the previous coding process.

For example, four pictures on the left side in FIG. 1A shown in FIG. 1A are added to the four pictures shown in FIG.
The five pictures on the right in the middle are stitched together,
When the image is edited so as to be generated as a new image shown in FIG.
If the image included in (A) and corresponding to the fourth image from the left in FIG. 1A) is a B picture in the previous encoding, the code used in the previous encoding is used. When encoding is to be performed using the encoding parameter, encoding is performed by referring to a P picture different from that before editing, and the image quality is greatly deteriorated.

Similarly, the first image at the joint of the editing (the image corresponding to the fifth image from the left side in FIG. 1B, which was included in FIG. 1B before editing) is replaced with the previous image. In the case of a B picture in encoding, when encoding is to be performed using the encoding parameters used in the previous encoding, the encoding is performed with reference to a different I picture than before editing, The image quality is greatly deteriorated.

When such editing is performed, VB
Since the bit stream rate control based on the V (Video Buffering Verifier) Buffer is inconsistent, overflow or underflow may occur.

For example, FIG. 2A shows the amount of data stored in the VBV Buffer when the image shown in FIG. 1A is encoded in the previous encoding process.
When (B) shows the amount of data stored in the VBV Buffer when encoding the image shown in FIG. 1B in the previous encoding process, the image shown in FIG. , Encoding using the previous encoding parameter as it is,
As shown in FIG. 2C, the data stored in the VBV Buffer overflows.

Further, the frame synchronizer absorbs the shift of the frame period, so that the temporal continuity of the image is broken even when the image frame is thinned out or the same frame is displayed twice. Therefore, the same problem as in the above-described editing process occurs.

T is a 10-bit counter that counts up for each input image in the MPEG2 bit stream.
Since the emporal_reference is inserted in the picture_header () layer, the discontinuity of such an image can be detected using the temporal_reference.

However, this temporal_reference is
It must be reset after p_of_pictures_header (). The group_of_pictures_header () has no particular definition for the cycle, but is usually inserted at the GOP cycle.

Therefore, when one GOP has a GOP structure composed of 15 frames, the period of temporal_reference is 15, and even if this image is edited as described above, there is a high possibility that temporal_reference will be continuous. Therefore, there is a high possibility that discontinuity of an image cannot be detected.
When one GOP has a GOP structure composed of one frame, temporal_reference is always 0, so that discontinuity of an image cannot be detected.

[0021] The present invention has been made in view of such a situation, and a decoding process for changing the structure of a GOP (Group of Pictures) of an encoded bit stream encoded based on the MPEG standard is performed. Even if the encoding process and the editing process are repeated, the image quality does not deteriorate.

[0022]

According to a first aspect of the present invention, there is provided a stream generation apparatus comprising: a first detection unit configured to detect an encoding history in a past encoding process of a first stream;
A second detecting means for detecting discontinuity of an image of the stream of the second stream, and a second stream based on the first stream by using a detection result of the first detecting means and a detection result of the second detecting means. And generating means for generating.

The generating means encodes the data by the MPEG method,
Can be generated.

[0024] The stream generating apparatus may further include first description means for describing the coding history in the coding process up to that time for the second stream.

The first description means writes the information of the encoding history into
It can be described as user_data of the second stream of the MPEG system.

[0026] The stream generating apparatus may further include second description means for describing continuity information indicating continuation of images for the second stream.

[0027] The second description means can describe continuous information indicating a continuous image as user_data of a second stream of the MPEG system.

The second description means can generate and describe continuous information by addition or subtraction for each access unit.

The second description means can describe continuous information in a blanking portion of a predetermined bit of a luminance signal or a color difference signal of the decoded image signal.

[0030] The second description means can describe continuous information in a blanking portion of the decoded image signal.

[0031] The stream generator can packetize the continuous information.

The second description means multiplexes the continuous information with the information of the coding history multiplexed on the blanking portion of the predetermined bit of the luminance signal or the color difference signal of the decoded image signal. can do.

[0033] The second description means can multiplex the continuous information with the information of the coding history multiplexed on the blanking portion of the decoded image signal.

According to a thirteenth aspect of the present invention, in the stream generation method, a first detection step of detecting an encoding history in a past encoding process of the first stream, and a discontinuity of an image of the first stream are detected. A second detection step;
Generating a second stream based on the first stream using the detection result in the processing of the first detection step and the detection result in the processing of the second detection step. And

According to a fourteenth aspect of the present invention, there is provided a recording medium program comprising: a first detection step of detecting an encoding history in a past encoding process of a first stream; and a discontinuity of an image of the first stream. A second detection step of:
Generating a second stream based on the first stream using the detection result in the processing of the first detection step and the detection result in the processing of the second detection step. And

[0036] A stream generating apparatus according to claim 1, a stream generating method according to claim 13, and a stream generating apparatus according to claim 13.
In the recording medium described in (1), the encoding history in the past encoding process of the first stream is detected, the discontinuity of the image of the first stream is detected, the detection result of the encoding history and the discontinuity of the image are detected. Is used to generate a second stream based on the first stream.

[0037]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a transcoding system to which the present invention is applied will be described. In this specification, the term “system” refers to an entire device including a plurality of devices and units.

FIGS. 3 and 4 show the configuration of the transcoding system 1 to which the present invention is applied.
Shows a more detailed configuration of FIG. This transcoding system 1 includes a video decoding system 11
Encoded video bit stream (encoded
The GOP structure and bit rate of the video bit stream are converted to the GOP structure and bit rate desired by the operator.

The transcoding system 1 includes a video decoding system 11, a video encoding system 12, a VT
It comprises an R (Video Tape Recorder) 13, a switch 14, and a switch 15.

The video decoding system 11 generates a baseband digital video signal based on the input coded video bit stream. Video coding system 12
Outputs an encoded video bit stream having a GOP structure and a bit rate desired by the operator based on the baseband video signal.

The VTR 13 records the baseband digital video signal supplied via the switch 14 and supplies the recorded baseband digital video signal to the video encoding system 12 via the switch 15.

The switch 14 is connected to the video decoding system 11
Is switched to the VTR 13 or the switch 15 for supplying the baseband digital video signal output from the. The switch 15 selects either the baseband digital video signal output from the video decoding system 11 or the baseband digital video signal output from the VTR 13 and converts the selected baseband digital video signal into the video encoding system 12. Output to

Although not shown in FIG. 4 to explain the function of the transcoding system 1, three stages having almost the same functions as the transcoding system 1 are provided before the transcoding system 1. It is assumed that a transcoding system is connected. That is, in order to variously change the GOP structure and bit rate of the bit stream, the first transcoding system, the second transcoding system, and the third transcoding system are sequentially connected in series, and the third transcoding system is connected in series. It is assumed that the fourth transcoding system shown in FIG. 4 is connected behind the transcoding system.

In the following description of the present invention, the encoding process performed in the first transcoding system will be defined as a first generation encoding process, and the encoding process performed after the first transcoding system will be described. 2 is defined as a second-generation encoding process, and the encoding process performed in a third transcoding system connected after the second transcoding system is defined as a second-generation encoding process. The encoding process performed in a fourth transcoding system (transcoding system 1 shown in FIG. 4) connected behind the third transcoding system is defined as a third generation encoding process. It is defined as a generation encoding process or a current encoding process.

The encoding parameters generated in the first generation encoding process are called first generation encoding parameters, and the encoding parameters generated in the second generation encoding process are referred to as second generation encoding parameters. An encoding parameter generated in the third generation encoding process is referred to as a third generation encoding parameter, and an encoding parameter generated in the fourth generation encoding process is referred to as a fourth generation encoding parameter. Will be referred to as the encoding parameter or the current encoding parameter.

First, the coded video stream ST (3rd) supplied to the transcoding system 1 shown in FIG. 4 will be described. ST (3rd) indicates that the stream is a third-generation encoded stream generated in the third-generation encoding process in the third transcoding system provided in the preceding stage of the transcoding system 1. . The encoded video stream ST generated in the third generation encoding process
In (3rd), the third-generation encoding parameters generated in the third encoding process include a sequence layer, a GOP layer, a picture layer, a slice layer, and a sequence layer of the encoded video stream ST (3rd). Sequence in the macroblock layer
_header () function, sequence_extension () function, group_o
f_pictures_header () function, picture_header () function, pic
ture_coding_extension () function, picture_data () function, s
Described as the lice () function and macroblock () function. Writing the third encoding parameter used in the third encoding process in the third encoded stream generated by the third encoding process as described above is performed by MPEG.
It is defined in the two standards and has no novelty.

Transcoding system 1 of the present invention
Is that not only the third encoding parameter is described in the third encoded stream ST (3rd), but also the first encoded stream generated in the first and second generation encoding processes. Generation and second-generation coding parameters are also described, and a sufficiently long counter value that is counted up for each frame or field as an access unit is associated with an image.

Specifically, the first-generation and second-generation encoding parameters are stored in the user data area of the picture layer of the third-generation encoded video stream ST (3rd).
This is described as a history stream history_stream (). In the present invention, the history stream described in the user data area of the picture layer of the third-generation encoded video stream ST (3rd) is referred to as “history information” or “history information”. The coding parameters described are called "history parameters" or "history parameters".

As another method, when the third-generation encoding parameter described in the third-generation encoded stream ST (3rd) is called “current encoding parameter”, From the viewpoint of the generation encoding process, the first generation and second generation encoding processes are encoding processes performed in the past, and therefore, the third generation encoded stream ST (3rd)
The coding parameter described as the history stream described in the user data area of the picture layer is also referred to as “past coding parameter”.

As described above, not only the third encoding parameter is described in the third encoded stream ST (3rd), but also the third encoded stream ST (3rd) generated in the first and second generation encoding processes. The reason why the first-generation and second-generation encoding parameters are described is that even if the GOP structure and the bit rate of the encoded stream are repeatedly changed by the transcoding process, it is possible to prevent image quality deterioration.

For example, in order to change a GOP structure of a first generation coded stream, a picture is coded as a P picture in a first generation coding process, and is converted to a B picture in a second generation coding process. In order to further change the GOP structure of the encoded stream of the second generation, the encoding is performed as a picture.
It is conceivable that the picture is encoded as a P picture again. Since the encoding process and the decoding process based on the MPEG standard are not 100% reversible processes, it is known that the image quality is deteriorated every time the encoding process and the decoding process are repeated.

In such a case, in the third generation encoding process, instead of calculating the encoding parameters such as the quantization scale, the motion vector, and the prediction mode again, the first generation encoding process is performed. Reuse the generated coding parameters such as the generated quantization scale, motion vector, and prediction mode. The quantization scale, the motion vector, and the prediction mode newly generated by the first-generation encoding process are smaller than the encoding parameters such as the quantization scale, the motion vector, and the prediction mode newly generated by the third-generation encoding process. Since the encoding parameters such as are clearly higher in accuracy, by reusing the first generation parameters, even if the encoding and decoding processes are repeated, the image quality degradation can be reduced.

The images included in the baseband digital video signal output from the video decoding system 11 include:
A counter value counted up for each frame or field that is an access unit is associated with the access unit. The counter value has, for example, a period of 65,536, becomes 0 after the maximum value is set, and is counted up from 0.

The video encoding system 12 uses the counter values corresponding to the images included in the baseband digital video signal to control the discontinuity of the images included in the input baseband digital video signal (for example, by connecting the images). Matching points, inserting images, decimating images, etc.)
Can be detected.

For example, a picture is encoded as a P picture in the first generation encoding process, and the picture is converted to a B picture in the second generation encoding process in order to change the GOP structure of the first generation encoded stream. In order to further change the GOP structure of the encoded stream of the second generation, the encoding is performed as a picture.
When the picture is encoded again as a P picture based on the encoding parameters in the first generation encoding process,
When the image is not edited such that the picture before it is deleted (such that the I-picture or P-picture referenced by the picture is deleted), the video encoding system 12 may generate a first-generation code. The picture is encoded as a P picture using the encoding parameters generated in the encoding process, and the picture before the picture is deleted (such that the I picture or P picture referred to by the picture is deleted). 3.) When an image is being edited, generate encoding parameters and encode the picture as a P picture.

Thus, the video encoding system 12
Detects a discontinuous point of an image based on a counter value counted up for each frame or field as an access unit, and generates a quantization scale, a motion vector, and a prediction mode generated in a first generation encoding process. Since encoding is performed by using an encoding parameter such as, image quality degradation can be prevented.

In order to explain the above-mentioned processing according to the present invention, the processing of the fourth generation transcoding system 1 shown in FIG. 4 will be described in more detail as an example.

The counter 101 has a frame / fidelity synchronized with a frame or a field supplied from the decoding device 102.
This is a 16-bit counter that counts up (adds 1) based on the eld synchronization signal. The counter 101 is set to 0
To the counter value multiplexing device 105.

When the counter 101 has a counter value of 65,535, the counter value is set to 0 when a frame / field synchronization signal synchronized with a frame or a field is supplied from the decoding apparatus 102, and thereafter, the counter 101 supplies the frame / field synchronization signal. The count-up is continued based on the Frame / Field synchronization signal.

It should be noted that the counter 101 has a
Fr synchronized with frame or field supplied from
Countdown (subtraction of 1) may be performed based on the ame / Field synchronization signal.

The decoding device 102 decodes the encoded video included in the third-generation encoded bit stream ST (3rd) using the third-generation encoding parameters, and decodes the decoded baseband digital signal. This is a device for generating video data. Further, the decoding device 102 decodes the first-generation and second-generation encoding parameters described as a history stream in the user data area of the picture layer of the third-generation encoded bit stream ST (3rd). It is also a device.

Specifically, as shown in FIG.
The decoder 251 of the decoding device 102 receives the buffer 2 for buffering the supplied bit stream.
61, a variable length decoding circuit 262 for performing variable length decoding on an encoded bit stream, an inverse quantization circuit 263 for performing inverse quantization on the variable length decoded data in accordance with the quantization scale supplied from the variable length decoding circuit 262, Quantized D
IDCT circuit 264 for performing an inverse discrete cosine transform of a CT (discrete cosine transform) coefficient, computing unit 265 for performing motion compensation processing, motion compensation circuit 266 and frame memory 26
7 is provided.

The encoded image data transmitted via the transmission path (or a predetermined recording medium) is received by a receiving circuit (not shown), reproduced by a reproducing device, and temporarily stored in a receiving buffer 261. Thereafter, it is supplied to the variable length decoding circuit 262. The variable length decoding circuit 262
1 is variable-length decoded, the motion vector, the prediction mode, the prediction flag, and the DCT flag are output to the motion compensation circuit 266, and the quantization scale is output to the inverse quantization circuit 263. The image data is output to the inverse quantization circuit 263.

The inverse quantization circuit 263 is connected to the variable length decoding circuit 2
The image data supplied from 62 is inversely quantized according to the quantization scale also supplied from the variable length decoding circuit 262 and output to the IDCT circuit 264. Inverse quantization circuit 263
Data (DCT coefficient) output from the IDCT circuit 264
, An inverse discrete cosine transform process is performed.
5 is supplied.

When the image data supplied from the IDCT circuit 264 to the arithmetic unit 265 is I-picture data, the data is output from the arithmetic unit 265 and the image data (P or B) which is input to the arithmetic unit 265 later. Frame data for generating predicted image data
The data is supplied to and stored in the forward prediction image section 267a of the image processing section 67. This data is stored in the history information multiplexing device 10.
3 (FIG. 3).

If the image data supplied from the IDCT circuit 264 is P-picture data in which the image data one frame before that is the predicted image data, and is data in the forward prediction mode, the forward predicted image in the frame memory 267 is stored. The image data (I picture data) one frame before stored in the section 267a is read out, and the motion compensation circuit 2
At 66, motion compensation corresponding to the motion vector output from the variable length decoding circuit 262 is performed. And the arithmetic unit 26
At 5, the image data is added to the image data (difference data) supplied from the IDCT circuit 264 and output. The added data, that is, the decoded P-picture data is transmitted to the rear of the frame memory 267 in order to generate predicted image data of image data (B-picture or P-picture data) to be input to the arithmetic unit 265 later. Prediction image section 26
7b and stored.

Even in the case of P picture data, the data in the intra prediction mode is not processed in the arithmetic unit 265 like the I picture data, and is stored in the backward prediction image section 267b as it is.

Since this P picture is an image to be displayed next to the next B picture, it is not yet output to the history information multiplexing device 103 at this time (the P picture input after the B picture is Processed and transmitted before the B picture).

If the image data supplied from the IDCT circuit 264 is B picture data, the variable length decoding circuit 2
In accordance with the prediction mode supplied from 62, the image data of the I picture stored in the forward predicted image section 267a of the frame memory 267 (in the case of the forward predicted mode) and the P data stored in the backward predicted image section 267b. The picture data of the picture (in the case of the backward prediction mode) or both image data (in the case of the bidirectional prediction mode) are read out.
The motion compensation corresponding to the motion vector output from 2 is performed, and a predicted image is generated. However, if motion compensation is not required (in the case of the intra-picture prediction mode), no predicted picture is generated.

The data subjected to the motion compensation by the motion compensation circuit 266 in this way is output to the arithmetic unit 265 by the ID
It is added to the output of the CT circuit 264. This addition output is output to the history information multiplexing device 103.

However, this added output is B picture data, and is not used for generating a predicted image of another image, and is not stored in the frame memory 267.

After the picture of the B picture is output, the picture data of the P picture stored in the backward prediction picture section 267b is read and supplied to the arithmetic unit 265 via the motion compensation circuit 266. However, at this time, no motion compensation is performed.

In the decoder 251, the process of returning the configuration in which the signals of the lines of the odd field and the even field are separated to the original configuration as necessary is executed by the motion compensation circuit 266.

In the above description, the processing of the luminance signal has been described, but the processing of the chrominance signal is similarly performed. However, the motion vector in this case is obtained by dividing the motion vector for the luminance signal by に in the vertical and horizontal directions.
Is used.

The variable length decoding circuit 262 decodes the third generation coded bit stream ST (3rd) to decode the third generation coded bit stream ST (3rd).
d) Third-generation coding parameters described in the picture layer, slice layer, and macroblock layer are extracted. For example, the third-generation encoding parameters extracted by the variable-length decoding circuit 262 include picture_coding_type indicating a picture type, quantizer_scale_code indicating a quantization scale step size, and a prediction mode.
macroblock_type, motion_vector indicating motion vector,
Frame / fiel indicating Frame prediction mode or Field prediction mode
d_motion_type and Frame DCT mode or Field DCT
Dct_type indicating the mode. The quatntiser_scale_code extracted in the variable length decoding circuit 262 is
Supplied to the inverse quantization circuit 263,
e, quatntiser_scale_code, macroblock_type, motion_
Parameters such as vector, frame / field_motion_type, and dct_type are supplied to the motion compensation circuit 266.

The variable length decoding circuit 262 includes not only these encoding parameters necessary for decoding the third generation coded bit stream ST (3rd), but also the following fifth generation transcoding system. The encoding parameters to be transmitted as the third generation history information are extracted from the sequence layer, GOP layer, picture layer, slice layer, and macroblock layer of the third generation encoded bit stream ST (3rd). Of course, picture_coding_type and quatntise used in the third generation decoding process
r_scale_code, macroblock_type, motion_vector, fram
Third-generation encoding parameters such as e / field_motion_type and dct_type are included in the third-generation history information. The type of encoding parameter to be extracted as the history information is preset by the operator or the host computer according to the transmission capacity or the like.

The variable length decoding circuit 262 supplies a frame / field synchronization signal corresponding to switching of a frame or a field as an access unit to the counter 101.

Further, the variable length decoding circuit 262 extracts the user data described in the user data area of the picture layer of the encoded bit stream ST (3rd) of the third generation, and converts the user data into a history decoding device. 104.

This history decoding device 104
Are generated from the user data described in the picture layer of the third generation coded bit stream ST (3rd), using the first generation coding parameters and the second generation coding parameters This is a circuit for extracting the encoding parameters of the generations earlier than the generation (2). Specifically, the history decoding device 1
04 detects the unique History_Data_Id described in the user data by analyzing the syntax of the received user data.
rted_history_stream () can be extracted. Further, the history decoding apparatus 104 converts the converted_
1 inserted at a certain interval in history_stream ()
Get the history_stream () by removing the marker bit (marker_bit) of the bit and its history
By analyzing the syntax of y_stream (), hi
The first-generation and second-generation encoding parameters described in story_stream () can be obtained. For detailed operation of the history decoding device 104,
It will be described later.

The history information multiplexing device 103 converts the first generation, second generation and third generation
These first-generation, second-generation, and third-generation encoding parameters are multiplexed on the baseband video data decoded by the decoding device 102 so as to be supplied to the encoding device 116 that performs the encoding process of the generation. It is a circuit for making the circuit. Specifically, the history information multiplexing device 103
Are the baseband video data output from the arithmetic unit 265 of the decoding device 102, the third-generation encoding parameters output from the variable-length decoding device 112 of the decoding device 102, and the output from the history decoding device 104. The first-generation coding parameter and the second-generation coding parameter are received, and the first-generation, second-generation, and third-generation coding parameters are multiplexed on the baseband video data. The baseband video data in which the first generation, second generation, and third generation coding parameters are multiplexed is supplied to the counter value multiplexing device 105.

The counter value multiplexing device 105 receives the first generation and the second generation data supplied from the history information multiplexing device 103.
The counter value supplied from the counter 101 is further multiplexed on the baseband video data on which the generation and third generation coding parameters are multiplexed.

Next, a method of multiplexing the first generation, second generation and third generation coding parameters into baseband video data will be described with reference to FIGS. 6 and 7. FIG. FIG. 6 is a diagram illustrating 1 and 2 defined in the MPEG standard.
One macroblock consisting of 6 pixels × 16 pixels is shown. The macroblock of 16 pixels × 16 pixels is a subblock (Y [0], [1], [2]) composed of four 8 pixels × 8 pixels with respect to the luminance signal.
And Y [3]) and four sub-blocks (Cr [0], r [1],
b [0] and Cb [1]).

FIG. 7 shows a format of video data. This format conforms to ITU Recommendation-RDT60
1. The so-called "D1 format", which is defined in I.1, and is used in the broadcasting industry
Is represented. Since the D1 format is standardized as a format for transmitting 10-bit video data, one pixel of the video data can be represented by 10 bits.

Since the baseband video data decoded according to the MPEG standard is 8 bits, in the transcoding system of the present invention, as shown in FIG. 7, the upper 8 bits (D9) among the 10 bits of the D1 format are used. To D2), the baseband video data decoded based on the MPEG standard is transmitted. Thus, when the decoded 8-bit video data is written in the D1 format,
Bits (D1 and D0) are unallocated bits
s). Transcoding system 1 of the present invention
Then, using this unallocated area,
The history information is transmitted together with the counter value.

The data block described in FIG.
Each sub-block (Y [0], Y [1], Y [2], Y [3], Cr
[0], Cr [1], Cb [0], Cb [1]), the data block is for transmitting one pixel. Therefore, in order to transmit the data of one macroblock, the data block shown in FIG. 64 data blocks are transmitted. If the lower two bits (D1 and D0) are used, a total of 1024 (= 16 × 6) video data of one macroblock is used.
4) A bit counter value and history information can be transmitted. Therefore, since the history information for one generation is generated to have 256 bits, the past 4 (= 102
4/256) generations of history information can be superimposed on one macroblock of video data. In the example shown in FIG. 7, the first generation history information, the second generation history information, the third generation history information, and the counter value are superimposed.

The first generation history information, the second generation history information, and the third generation history information are superimposed on the sub-block corresponding to the luminance signal, and the counter value is superimposed on the sub-block corresponding to the color difference signal. May be superimposed.

Further, the 16-bit counter value is SMPTE
(The Society of Motion Picture and Television Eng
ineers) Ancillary Data Packet specified by 291M
And may be transmitted. FIG.
It is a figure showing the example of llary Data Packet.

The ADF (Ancillary Data Flag) contains Ancill
Predetermined data ("000", "3F
F "," 3FF ") is stored.DID (Data Identification W)
ord) includes a value that specifies the format of data stored in the Ancillary Data Packet, for example, a value (C0h to C0h) corresponding to the User Application defined by SMPTE 291M
Fh) is stored.

Since the word storing the 16-bit counter value is divided into three words and stored in the Ancillary Data Packet, the DC (D) following the SDID (Secondary Data ID) is used.
Ata Count Number Word) is set to “3”. U
The ser Data Words include a word (CC0) for storing the lower 6 bits of the counter value shown in FIG. 9 and a word (CC0) for storing 7 to 12 bits of the counter value shown in FIG.
1) and a word (CC2) for storing the upper 4 bits of the counter value shown in FIG. 11 are stored in order.

FIG. 9 is a view for explaining an example of a word (CC0) for storing the lower 6 bits of the counter value. The lower two bits (B0 and B1) are set to "0".
The lower 6 bits (counter value [0] to counter value [5]) of the counter value are set in the 3 bits (B2) to 8 bits (B8), respectively. The even parity of the lower 8 bits (B0 to B7) is set in 9 bits, and 10 bits.
In the bit, a negative value of the even parity set to 9 bits is set.

FIG. 10 shows the case where the counter value is from 7 bits to 12 bits.
FIG. 4 is a diagram illustrating an example of a word (CC1) that stores bits. The lower two bits (B0 and B1) are set to "0". The 3 bits (B2) to 8 bits (B8) are respectively set to 7 bits to 12 bits of the counter value (counter values [6] to [11]). Nine bits are set to the even parity of the lower eight bits (B0 to B7), and ten bits are set to the negative value of the even parity set to nine bits.

FIG. 11 is a view for explaining an example of a word (CC2) for storing the upper 4 bits of the counter value. Lower 2
Bits (B0 and B1) are each set to “0”. The upper 4 bits (counter value [12] to counter value [15]) of the counter value are set in the 3 bits (B2) to 6 bits (B5), respectively. 7 bits (B
6) and 8 bits (B7) are each set to "0". 9 bits, lower 8 bits (B0 to B7)
Is set, and the negated value of the even parity set to 9 bits is set to 10 bits.

At the end of the Ancillary Data Packet, CS (C
hecksum Word) is stored.

As described above, the transcoding system 1 converts the 16-bit counter value into the Ancillary Data Pac
It can be stored and transmitted in ket.

[0095] The counter value can also be multiplexed with the encoding parameters multiplexed in the LSB of the baseband digital video signal. FIG. 12 shows a configuration of functions corresponding to the history information multiplexing device 103 and the counter value multiplexing device 105 when multiplexing the counter value to the encoding parameter multiplexed in the LSB of the baseband digital video signal. FIG. The timing signal generator 271 generates the LSB of the baseband digital video signal based on the input baseband digital video signal.
A chroma timing pulse synchronized with the above is generated and supplied to the switch 275.

Counter value format converter 272
Converts the counter value supplied from the counter 101 into a user data format, and outputs it to the encoding parameter format converter 273. The encoding parameter format conversion device 273 includes an encoding parameter (3RD) supplied from the decoding device 102 and an encoding parameter (1S) supplied from the history decoding device 104.
T, 2ND), multiplexes the counter value of the user data format supplied from the counter value format converter 272 and outputs the multiplexed value to the serial-parallel converter 274.

The serial-to-parallel converter 274 converts the encoding parameter with the multiplexed counter value from serial to parallel, and supplies it to the switch 275.
The switch 275 multiplexes, based on the chroma timing pulse supplied from the timing signal generation circuit 271, an encoding parameter in which the counter value is multiplexed on the LSB of the baseband digital video signal.

As described above, the counter value is multiplexed with the coding parameter multiplexed on the blanking portion of the baseband digital video signal.

The counter value can be multiplexed with the coding parameter multiplexed in the blanking portion of the luminance or color difference of the baseband digital video signal. FIG. 13 shows the history information multiplexing device 103 and the counter value multiplexing device 105 when multiplexing a counter value to an encoding parameter multiplexed in a blanking portion of luminance or color difference of a baseband digital video signal. FIG. 3 is a diagram illustrating a configuration of a corresponding function. The timing signal generation circuit 281 generates a blanking timing pulse synchronized with a blanking portion of the luminance or color difference of the baseband digital video signal based on the input baseband digital video signal, and supplies the blanking timing pulse to the switch 282.

Counter value format converter 272
Converts the counter value supplied from the counter 101 into a user data format, and outputs it to the encoding parameter format converter 273. The encoding parameter format conversion device 273 includes an encoding parameter (3RD) supplied from the decoding device 102 and an encoding parameter (1S) supplied from the history decoding device 104.
T, 2ND), multiplexes the counter value of the user data format supplied from the counter value format converter 272 and outputs the multiplexed value to the switch 282.

The switch 282 multiplexes, based on the blanking timing pulse supplied from the timing signal generation circuit 281, an encoding parameter in which a counter value is multiplexed into a blanking portion of the luminance or color difference of the baseband digital video signal. Become

As described above, the counter value is multiplexed with the coding parameter multiplexed in the blanking portion of the luminance or color difference of the baseband digital video signal.

The counter value separating device 111 is a circuit for extracting a counter value from the lower two bits of data transmitted in the D1 format. The counter value separating device 111 supplies the data from which the counter value has been extracted and transmitted as the separated D1 format to the history information separating device 115.

The counter value separating device 111 supplies the counter value separated from the data transmitted as the D1 format to the format converting device 112,
Counter 1
13.

The counter 113 is a counter value separating device 1
16 is incremented (added by 1) based on a signal synchronized with a frame or a field supplied from 11.
This is a bit counter. The counter 113 has 0 to 65,
One of the counter values 535 is output to the comparing device 114.

When the counter 113 has a counter value of 65,535, the counter value is set to 0 when a signal synchronized with the frame or the field is supplied from the counter value separation device 111, and thereafter, the counter value separation device 111
Counting up based on the signal synchronized with the frame or the field supplied from.

When the counter 101 counts down (subtracts 1), the counter 113 also counts down.
It is configured to count down based on a signal synchronized with a frame or a field supplied from the counter value separation device 111.

The format converter 112 converts the format of the counter value (for example, the word format described with reference to FIGS. 9 to 11) supplied from the counter value separator 111 and separated from the data transmitted in the D1 format. It is converted into a 16-bit counter value (for example, the same method as the counter value output from the counter 113), and
14 is output.

The comparison device 114 compares the counter value supplied from the format conversion device 112 with the counter value supplied from the counter 113, and when the values are the same, encodes a discontinuous parameter having a predetermined value. Device 116
And when the values are different, a discontinuous parameter of another value is output to the encoding device 116.

The format converter 1 is added to the comparator 114.
When the counter value supplied from the counter 12 and the counter value supplied from the counter 113 take different values, the counter 113 loads the counter value output from the format converter 112 and sets it as its own counter value. By doing so, the comparison device 114 once
Even after detecting a discontinuous point in an image, if a discontinuous point is included in the image again, the discontinuous point in the image can be detected.

When images included in a baseband digital video signal are joined by editing or the like, a predetermined frame is inserted, or a frame is deleted, a counter stored in the baseband digital video signal corresponding to the image. Since the value takes a discontinuous value, the counter value supplied from the format converter 112 and the counter 11
The value becomes different from the counter value supplied from 3, and the comparing device 114 supplies the discontinuous parameter of another value to the encoding device 116. If the image included in the baseband digital video signal is not edited or the like, the counter value stored in the baseband digital video signal corresponding to the image takes a continuous value.
The counter value supplied from the counter 12 and the counter value supplied from the counter 113 have the same value, and
Supplies a discrete parameter of a predetermined value to the encoding device 116.

As described above, the encoding device 116 can determine whether or not the image has been edited based on the signal supplied from the comparison device 114.

[0113] The history information separation device 115 calculates the upper 8 bits of the data transmitted as the D1 format.
This is a circuit for extracting baseband video data and extracting history information from lower two bits. In the example shown in FIG. 4, the history information separating device 115 extracts baseband video data from the transmission data, supplies the video data to the encoding device 116, and also extracts the first generation and second generation data from the transmission data. Generation and third generation history information is extracted and supplied to the encoding device 116 and the history encoding device 117, respectively.

The encoding device 116 encodes the baseband video data supplied from the history information separating device 115 into a bit stream having a GOP structure and a bit rate specified by an operator or a host computer. Device. GOP
Changing the structure includes, for example, the number of pictures included in a GOP, the number of P pictures existing between I pictures and I pictures, and the number of B pictures existing between I pictures and P pictures (or I pictures). It means changing the number.

In the example shown in FIG. 4, the first-, second-, and third-generation history information is superimposed on the supplied baseband video data. The fourth generation encoding process is performed by selectively reusing these pieces of history information so that image quality deterioration due to the re-encoding process is reduced.

FIG. 14 is a diagram showing a specific configuration of the encoder 301 provided in the encoding device 116. The encoder 301 includes a motion vector detection circuit 310, a frame memory 311, a frame / field prediction mode switching circuit 312, a computing unit 313,
CT mode switching circuit 315, DCT circuit 316, quantization circuit 317, variable length encoding circuit 318, transmission buffer 319, inverse quantization circuit 320, inverse DCT circuit 321,
An arithmetic unit 322, a frame memory 323, and a motion compensation circuit 324 are provided.

First, the encoding processing of the reference picture by the encoder 301 when there is no history information will be described.

The image data to be encoded is input to the motion vector detection circuit 310 on a macroblock basis. The motion vector detection circuit 310 converts the image data of each frame into I
Process as a picture, P picture, or B picture. Images of each frame input sequentially are
Which of I, P and B is processed as a picture is predetermined (for example, frames F1 to F
17, the group of pictures I,
B, P, B, P,..., B, P).

Image data of a frame (for example, frame F1) processed as an I picture is transferred from the motion vector detection circuit 310 to the front original image section 311a of the frame memory 311 and stored therein, and is processed as a B picture ( For example, the image data of the frame F2) is
Image data of a frame (for example, frame F3) transferred and stored in the reference original image unit 311b and processed as a P picture is transferred and stored in the rear original image unit 311c.

At the next timing, B
When an image of a frame to be processed as a picture (frame F4) or a P picture (frame F5) is input, the image data of the first P picture (frame F3) stored in the rear original image unit 311c until then is The image data of the next B picture (frame F4) is transferred to the front original image section 311a, and the image data of the next P picture (frame F5) is stored (overwritten) in the reference original image section 311b. The information is stored (overwritten) in the unit 311c. Such operations are sequentially repeated.

The signal of each picture stored in the frame memory 311 is read therefrom, and the frame / field prediction mode switching circuit 312 performs frame prediction mode processing or field prediction mode processing.

Further, under the control of the controller 330, the arithmetic unit 313 performs intra prediction, forward prediction,
Calculation of backward prediction or bidirectional prediction is performed. Which of these processes is to be performed is determined according to the prediction error signal (the difference between the reference image to be processed and the predicted image corresponding to the reference image). For this reason, the motion vector detection circuit 310 generates a sum of absolute values (or a sum of squares) of the prediction error signal used for this determination.

Here, the frame prediction mode and the field prediction mode in the frame / field prediction mode switching circuit 312 will be described.

When the frame prediction mode is set, the frame / field prediction mode switching circuit 312
Directly outputs the four luminance blocks Y [1] to Y [4] supplied from the motion vector detection circuit 310 to the subsequent computing unit 313. That is, in this case, in each luminance block, the data of the line of the odd field and the data of the line of the even field are mixed. In this frame prediction mode, 4
Prediction is performed in units of luminance blocks (macroblocks), and one motion vector corresponds to four luminance blocks.

On the other hand, in the field prediction mode, the frame / field prediction mode switching circuit 312 converts the signal input from the motion vector detection circuit 310 into the luminance block Y [1] out of the four luminance blocks.
And Y [2] are composed of, for example, only the dots of the lines in the odd field, and the other two luminance blocks Y [3] and Y [2]
[4] is composed of only the dots of the lines of the even-numbered fields, and is output to the arithmetic unit 313. In this case, for two luminance blocks Y [1] and Y [2],
One motion vector corresponds, and another one motion vector corresponds to the other two luminance blocks Y [3] and Y [4].

The motion vector detecting circuit 310 calculates the sum of the absolute value of the prediction error in the frame prediction mode and the sum of the absolute value of the prediction error in the field prediction mode as Frame / Fi
Output to the eld prediction mode switching circuit 312. Frame / F
The ield prediction mode switching circuit 312 compares the sum of absolute values of prediction errors between the frame prediction mode and the field prediction mode, performs processing corresponding to the prediction mode having a small value, and outputs data to the arithmetic unit 313.

However, such processing is actually performed by the motion vector detection circuit 310. That is, the motion vector detection circuit 310 converts the signal having the configuration corresponding to the determined mode into a frame / field prediction mode switching circuit 31.
2 and a frame / field prediction mode switching circuit 31
2 outputs the signal as it is to the subsequent operation unit 313.

In the frame prediction mode, the chrominance signal is supplied to the arithmetic unit 313 in a state where the data of the odd field lines and the data of the even field lines are mixed.
Supplied to In the field prediction mode, the upper half (4 lines) of each of the color difference blocks Cb and Cr is used as a color difference signal of an odd field corresponding to the luminance block Y [1], Y [2], and the lower half (4 lines). ) Are color difference signals of even fields corresponding to the luminance blocks Y [3] and Y [4].

The motion vector detecting circuit 310 determines a prediction error for determining whether to perform intra-picture prediction, forward prediction, backward prediction, or bidirectional prediction in the controller 330 as described below. Generates the sum of absolute values of.

That is, as the sum of the absolute values of the prediction errors of the intra-picture prediction, the absolute value | ΣAij | of the sum ΣAij of the macroblock signals Aij of the reference picture and the macroblock signal A
The difference between the sum 絶 対 | Aij | of the absolute values | Aij | The sum of absolute values | Aij−Bij | of the difference Aij−Bij | between the signal Aij of the macroblock of the reference image and the signal Bij of the macroblock of the predicted image is used as the absolute value sum of the prediction error of the forward prediction. Bij |. In addition, the absolute value sum of the prediction error between the backward prediction and the bidirectional prediction is obtained in the same manner as in the forward prediction (by changing the predicted image to a different predicted image from that in the forward prediction).

The sum of these absolute values is calculated by the controller 330
Supplied to The controller 330 selects the smallest absolute value sum of the prediction errors of the forward prediction, the backward prediction, and the bidirectional prediction as the absolute value sum of the prediction errors of the inter prediction. Furthermore, the absolute value sum of the prediction error of the inter prediction and the absolute value sum of the prediction error of the intra prediction are compared,
The smaller one is selected, and the mode corresponding to the selected sum of absolute values is selected as the prediction mode. That is, if the sum of the absolute values of the prediction errors of the intra prediction is smaller, the intra prediction mode is set. If the sum of the absolute values of the prediction errors in the inter prediction is smaller, the mode in which the corresponding sum of the absolute values is the smallest among the forward prediction, backward prediction, and bidirectional prediction modes is set.

As described above, the motion vector detecting circuit 310
The frame / field prediction mode switching circuit 312 converts a macroblock signal of a reference image into a frame / field prediction mode corresponding to the mode selected by the frame / field prediction mode switching circuit 312.
And a motion vector between the predicted image and the reference image corresponding to the prediction mode selected by the controller 330 among the four prediction modes, and the variable-length encoding circuit 318 Output to the motion compensation circuit 324. As described above, as the motion vector, the motion vector having the smallest absolute value sum of the corresponding prediction errors is selected.

When the motion vector detection circuit 310 reads out the I-picture image data from the forward original image section 311a, the controller 330 sets the prediction mode as the intra-frame or field (image) prediction mode (mode in which motion compensation is not performed). ) Is set, and the switch 313d of the arithmetic unit 313 is switched to the contact a side. Thus, the image data of the I picture is input to the Frame / FieldDCT mode switching circuit 315.

Frame / Field DCT mode switching circuit 315
Sets the data of the four luminance blocks to one of a state in which the lines of the odd field and the lines of the even field are mixed (frame DCT mode) or a state in which the lines are separated (field DCT mode), DCT circuit 31
6 is output.

That is, the Frame / Field DCT mode switching circuit 315 determines the coding efficiency when the data of the odd field and the even field are mixed and subjected to the DCT processing,
The coding efficiency when DCT processing is performed in the separated state is compared, and a mode having good coding efficiency is selected.

For example, the input signal has a configuration in which odd field and even field lines are mixed, and the difference between the signal of the vertically adjacent odd field line and the signal of the even field line is calculated. Find the sum (or sum of squares) of the values.

Further, the input signal is configured such that the lines of the odd field and the even field are separated, and the difference between the signal of the odd field adjacent vertically and the difference of the signal of the lines of the even field are calculated. Then, the sum (or sum of squares) of the respective absolute values is obtained.

Further, the two (the sum of absolute values) are compared, and the DCT mode corresponding to the smaller value is set. That is, if the former is smaller, the frame DCT mode is set, and if the latter is smaller, the field DCT mode is set.

Then, the data having the configuration corresponding to the selected DCT mode is output to the DCT circuit 316, and the DCT flag indicating the selected DCT mode is transmitted to the variable length coding circuit 3.
18 and the motion compensation circuit 324.

Frame / Field prediction mode switching circuit 31
As is clear from the comparison between the prediction mode in the frame No. 2 and the DCT mode in the frame / field DCT mode switching circuit 315, the luminance block has substantially the same data structure in each mode.

Frame / Field prediction mode switching circuit 31
When the frame prediction mode (mode in which odd lines and even lines are mixed) is selected in 2, the frame / fiel
In the dDCT mode switching circuit 315, the frame DC
T mode (mode where odd and even lines are mixed)
Is highly likely to be selected, and when the frame / field prediction mode switching circuit 312 selects the field prediction mode (a mode in which the data of the odd field and the data of the even field are separated), the frame / field DCT mode switching circuit 315 , It is highly possible that the field DCT mode (mode in which the data of the odd field and the data of the even field are separated) is selected.

However, the mode is not always selected in this way. In the frame / field prediction mode switching circuit 312, the mode is determined so that the absolute value sum of the prediction error is small, and the frame / field DCT is determined.
In the mode switching circuit 315, the mode is determined so that the encoding efficiency becomes good.

Frame / Field DCT mode switching circuit 315
The I-picture image data output from the DCT circuit 3
The DCT coefficients are input to DCT 16 and converted into DCT coefficients.
The DCT coefficient is input to the quantization circuit 317, quantized at a quantization scale corresponding to the data storage amount (buffer storage amount) of the transmission buffer 319, and then input to the variable length coding circuit 318.

The variable length encoding circuit 318 is
The image data (in this case, I-picture data) supplied from the quantization circuit 317 is converted into a variable-length code such as a Huffman code in accordance with the quantization scale (scale) supplied from Output to the transmission buffer 319.

The variable length coding circuit 318 also sets a quantization scale (scale) by the quantization circuit 317 and a prediction mode (intra-picture prediction, forward prediction, backward prediction, or bidirectional prediction) by the controller 330. , A motion vector from the motion vector detection circuit 310, a prediction flag from the Frame / Field prediction mode switching circuit 312 (a flag indicating whether the frame prediction mode or the field prediction mode is set), and
DC output by the Frame / FieldDCT mode switching circuit 315
A T flag (a flag indicating whether the frame DCT mode or the field DCT mode is set) is input, and these are also subjected to variable-length coding.

The transmission buffer 319 temporarily stores the input data, and converts the data corresponding to the storage amount into the quantization circuit 3.
17 is output. When the remaining data amount increases to the allowable upper limit value, the transmission buffer 319 reduces the data amount of the quantized data by increasing the quantization scale of the quantization circuit 317 by the quantization control signal. Also,
Conversely, when the remaining data amount decreases to the allowable lower limit, the transmission buffer 319 increases the data amount of the quantized data by reducing the quantization scale of the quantization circuit 317 by the quantization control signal. . In this way, overflow or underflow of the transmission buffer 319 is prevented.

The data stored in the transmission buffer 319 is read at a predetermined timing and output to the transmission path.

On the other hand, the I output from the quantization circuit 317
The picture data is input to the inverse quantization circuit 320,
The inverse quantization is performed according to the quantization scale supplied from the quantization circuit 317. The output of the inverse quantization circuit 320 is ID
After being input to a CT (Inverse Discrete Cosine Transform) circuit 321 and subjected to inverse discrete cosine transform processing, it is supplied to a forward prediction image section 323 a of a frame memory 323 via an arithmetic unit 322 and stored.

The motion vector detecting circuit 310 receives the image data of each sequentially input frame, for example, as I, B, P, B, P, B... After the image data of the input frame is processed as an I picture, the image data of the next input frame is processed as a P picture before the image of the next input frame is processed as a B picture. This is because a B picture involves backward prediction and cannot be decoded unless a P picture as a backward predicted image is prepared first.

Therefore, the motion vector detection circuit 310
After the processing of the picture, the processing of the image data of the P picture stored in the rear original image section 311c is started. Then, as in the case described above, the absolute value sum of the inter-frame difference (prediction error) in macroblock units is supplied from the motion vector detection circuit 310 to the Frame / Field prediction mode switching circuit 312 and the controller 330. Frame /
Field prediction mode switching circuit 312 and controller 3
Reference numeral 30 sets a frame / field prediction mode or a prediction mode of intra-picture prediction, forward prediction, backward prediction, or bidirectional prediction in accordance with the absolute value sum of the prediction errors of the macroblocks of the P picture.

When the intra prediction mode is set, the arithmetic unit 313 sets the switch 313d to the contact a as described above.
Switch to the side. Therefore, this data is transmitted to the transmission path via the Frame / Field DCT mode switching circuit 315, DCT circuit 316, quantization circuit 317, variable length coding circuit 318, and transmission buffer 319, like the I picture data. You. The data is supplied to the backward prediction image section 323b of the frame memory 323 via the inverse quantization circuit 320, the IDCT circuit 321 and the arithmetic unit 322, and is stored.

If the forward prediction mode is set,
While the switch 313d is switched to the contact b,
The image data (in this case, the I-picture image) stored in the forward prediction image unit 323a of the frame memory 323 is read out, and the motion compensation circuit 324 corresponds to the motion vector output from the motion vector detection circuit 310. Motion compensation. That is, when the setting of the forward prediction mode is instructed by the controller 330, the motion compensation circuit 324 sets the read address of the forward prediction image unit 323a to the position of the macroblock currently output by the motion vector detection circuit 310. Data is read out from the corresponding position by shifting by an amount corresponding to the motion vector, and predicted image data is generated.

The prediction image data output from the motion compensation circuit 324 is supplied to a calculator 313a. Arithmetic unit 31
3a is based on the macroblock data of the reference image supplied from the frame / field prediction mode switching circuit 312,
The prediction image data corresponding to this macro block supplied from the motion compensation circuit 324 is subtracted, and the difference (prediction error) is output. The difference data is Frame / FieldDCT
The signal is transmitted to the transmission path via the mode switching circuit 315, the DCT circuit 316, the quantization circuit 317, the variable length coding circuit 318, and the transmission buffer 319. The difference data is supplied to the inverse quantization circuit 320 and the IDCT circuit 321.
, And is input to the arithmetic unit 322.

The operation unit 322 also includes an operation unit 313
The same data as the predicted image data supplied to a is supplied. The arithmetic unit 322 adds the prediction image data output from the motion compensation circuit 324 to the difference data output from the IDCT circuit 321. This allows the original (decrypted)
Image data of a P picture is obtained. The image data of the P picture is supplied to and stored in the backward prediction image section 323b of the frame memory 323.

After the data of the I picture and the P picture are stored in the forward prediction image section 323a and the backward prediction image section 323b, the motion vector detection circuit 310 next executes the processing of the B picture. Frame / Field
Prediction mode switching circuit 312 and controller 330
Sets the frame / field mode in accordance with the magnitude of the sum of absolute values of the inter-frame differences in macroblock units, and sets the prediction mode to an intra-picture prediction mode, a forward prediction mode, a backward prediction mode, or a bidirectional prediction mode. Set to one of the prediction modes.

As described above, in the intra prediction mode or the forward prediction mode, the switch 313d is switched to the contact point a or b. At this time, the same processing as in the case of the P picture is performed, and the data is transmitted.

On the other hand, when the backward prediction mode or the bidirectional prediction mode is set, the switch 313d is switched to the contact point c or d.

In the backward prediction mode in which the switch 313d is switched to the contact point c, the image stored in the backward prediction image section 323b (P picture in this case).
The data is read, and the motion compensation circuit 324 performs motion compensation corresponding to the motion vector output from the motion vector detection circuit 310. That is, the motion compensation circuit 324
When the setting of the backward prediction mode is instructed by the controller 330, the read address of the backward prediction image unit 323b is changed by the motion vector detection circuit 310 from the position corresponding to the position of the macroblock currently output. The data is read out by shifting by an amount corresponding to (i), and predicted image data is generated.

The prediction image data output from the motion compensation circuit 324 is supplied to a calculator 313b. Arithmetic unit 31
3b is based on the macroblock data of the reference image supplied from the frame / field prediction mode switching circuit 312,
The prediction image data supplied from the motion compensation circuit 324 is subtracted, and the difference is output. This difference data is
FieldDCT mode switching circuit 315, DCT circuit 316,
The data is transmitted to the transmission path via the quantization circuit 317, the variable length coding circuit 318, and the transmission buffer 319.

In the bidirectional prediction mode in which the switch 313d is switched to the contact point d, the forward prediction image section 323a
(In this case, an I-picture image) data and an image (in this case, a P-picture image) data stored in the backward prediction image unit 323b are read out, and the motion compensation circuit 324 As a result, motion compensation is performed corresponding to the motion vector output from the motion vector detection circuit 310.

That is, when the setting of the bidirectional prediction mode is instructed by the controller 330, the motion compensation circuit 324 outputs the forward predicted image portion 323a and the backward predicted image portion 323.
b, the read address of the motion vector detection circuit 310
The data is read out from the position corresponding to the position of the macro block currently being output by shifting the motion vector (in this case, two motion vectors for the forward predicted image and the backward predicted image) by the amount corresponding to the motion vector, Generate predicted image data.

The predicted image data output from the motion compensation circuit 324 is supplied to a calculator 313c. Arithmetic unit 31
Reference numeral 3c denotes a motion compensation circuit 3c based on the macroblock data of the reference image supplied from the motion vector detection circuit 310.
Subtracting the average value of the predicted image data supplied from 24,
Output the difference. This difference data is Frame / FieldD
The signal is transmitted to the transmission path via the CT mode switching circuit 315, the DCT circuit 316, the quantization circuit 317, the variable length coding circuit 318, and the transmission buffer 319.

The picture of the B picture is not stored in the frame memory 323 because it is not regarded as a predicted picture of another picture.

In the frame memory 323, the forward prediction image section 323a and the backward prediction image section 323b are switched between banks as necessary, and are stored in one or the other for a predetermined reference image. Can be switched and output as a forward predicted image or a backward predicted image.

In the above description, the luminance block is mainly described, but the chrominance block is also processed and transmitted in units of macro blocks. As a motion vector for processing a chrominance block, a motion vector obtained by halving the motion vector of the corresponding luminance block in the vertical and horizontal directions is used.

Further, the controller 330 receives instructions regarding the GOP structure from the operator or the host computer, and determines the picture type of each picture so as to correspond to the GOP structure. Also,
The controller 330 receives information on the target bit rate from the operator or the host computer, and controls the quantization circuit 317 so that the bit rate output from the encoder 301 becomes the specified target bit rate.

Further, the controller 330 receives history information of a plurality of generations output from the history information separating device 115, and encodes a reference picture by reusing the history information. This will be described in detail below.

First, the controller 330 determines whether or not the picture type of the reference picture determined from the GOP structure specified by the operator matches the picture type included in the history information. That is, it is determined whether or not this reference picture has been encoded in the past with the same picture type as the designated picture type.

[0169] To explain this more clearly with the example shown in FIG. 4, the controller 330 determines that the picture type assigned to this reference picture as the fourth generation encoding process is the first generation. Whether the picture type of the reference picture in the encoding process, the picture type of the reference picture in the second generation encoding process, or the picture type of the reference picture in the third generation encoding process is matched. to decide.

If the picture type designated as the reference picture as the fourth-generation encoding process does not match any of the picture types in the past encoding process, the controller 330 executes the above-described “normal encoding”. Process ". That is, in this case, the first generation,
In any of the second-generation and third-generation encoding processes, it means that this reference picture has not been encoded with the picture type assigned as the fourth-generation encoding process. On the other hand, if the picture type designated as the reference picture as the fourth-generation encoding process matches any of the picture types in the past encoding process, the controller 33
0 performs “parameter reuse encoding processing”. That is, in this case, in the encoding processing of any of the first generation, the second generation, and the third generation, the reference picture is encoded using the picture type assigned as the fourth generation encoding processing. It has been processed.

Even when the “parameter reuse encoding process” is being executed, the discontinuity parameter supplied from the comparison device 114 indicates that the image is discontinuous (the baseband digital video signal When the included image has a value indicating that the images included are connected by editing or the like, a predetermined frame has been inserted, or the frame has been deleted, etc. Since the images before and after the continuous point are greatly deteriorated, the controller 330 performs the “normal encoding process”.

Encoder 30 when there is no history information
Although the description partially overlaps with the coding process of the reference picture of No. 1, the normal coding process of the controller 330 will be described first.

The motion vector detection circuit 310 detects a prediction error in the frame prediction mode and a prediction error in the field prediction mode to determine whether the frame prediction mode or the field prediction mode should be selected. Error value of controller 3
30. The controller 330 compares the values of the prediction errors and selects the prediction mode with the smaller value of the prediction error. The Frame / Field prediction mode switching circuit 312 performs signal processing so as to correspond to the prediction mode selected by the controller 330, and supplies it to the arithmetic unit 313.

Specifically, when the frame prediction mode is selected, the frame / field prediction mode switching circuit 312 performs signal processing so that the luminance signal is output to the arithmetic unit 313 as it is input. Is performed on the color difference signal so that the odd field lines and the even field lines are mixed. On the other hand, when the field prediction mode is selected, for the luminance signal, the luminance blocks Y [1] and Y [2] are constituted by odd field lines, and the luminance blocks Y [3] and Y [4] are even. Signal processing is performed so as to be composed of field lines. Regarding color difference signals, the upper four lines are composed of odd field lines,
Signal processing is performed so that the line is composed of even field lines.

Further, the motion vector detection circuit 310
Intra-picture prediction mode, forward prediction mode, backward prediction mode,
Alternatively, in order to determine which of the bidirectional prediction modes to select, a prediction error in each prediction mode is generated, and the prediction error in each prediction mode is supplied to the controller 330. Controller 33
0 selects the smallest prediction error among the forward prediction, backward prediction and bidirectional prediction as the prediction error of inter prediction. Further, the prediction error of the inter prediction is compared with the prediction error of the intra prediction, and the smaller one is selected, and the mode corresponding to the selected prediction error is selected as the prediction mode. That is, if the prediction error of the intra prediction is smaller, the intra prediction mode is set. If the prediction error of the inter prediction is smaller, the mode with the smallest corresponding prediction error among the forward prediction, backward prediction, and bidirectional prediction modes is set. Controller 33
0 corresponds to the arithmetic unit 3 so as to correspond to the selected prediction mode.
13 and the motion compensation circuit 324.

The DCT mode switching circuit 315 converts the data of the four luminance blocks into a signal form in which odd field lines and even field lines are mixed in order to select either the frame DCT mode or the field DCT mode. In addition to the conversion into the frame DCT mode, the signal is converted into a signal form in which the odd field lines and the even field lines are separated (field DCT mode), and the respective signals are supplied to the DCT circuit 316. D
The CT circuit 316 determines the coding efficiency when the odd field and the even field are mixed and performs the DCT processing, and the DC efficiency when the odd field and the even field are separated.
The coding efficiency in the case of performing the T processing is calculated, and the result is supplied to the controller 330. The controller 330
The respective coding efficiencies supplied from the CT circuit 316 are compared, and the DCT mode having the higher coding efficiency is selected.
The DCT mode switching circuit 315 is controlled so as to be in the selected DCT mode.

The controller 330 receives the target bit rate indicating the target bit rate supplied from the operator or the host computer, and the signal indicating the amount of bits buffered in the transmission buffer 319, that is, the signal indicating the remaining buffer amount. , Based on the target bit rate and the remaining buffer amount, generates feedback_q_scale_code for controlling the quantization step size of the quantization circuit 317. This feedback_q
_scale_code is a control signal generated according to the remaining buffer amount of the transmission buffer 319 so that the transmission buffer 319 does not overflow or underflow, and the bit rate of the bit stream output from the transmission buffer 319 Is a signal for controlling the target bit rate.

More specifically, for example, the transmission buffer 319
If the amount of bits buffered in the buffer has decreased, the quantization step size is reduced so that the amount of bits generated in the next picture to be encoded increases.
On the other hand, if the amount of bits buffered in the transmission buffer 319 has increased, the quantization step size is increased so that the amount of generated bits of the next picture to be encoded decreases. Feedback_q_scale
_code and quantization step size are proportional, feedback_q_sc
Increasing ale_code increases the quantization step size, and decreasing feedback_q_scale_code decreases the quantization step size.

Next, the parameter reuse encoding process, which is one of the features of the transcoding system 1, will be described. In order to explain this process more clearly, a reference picture is coded as a P-picture in the first-generation coding process, is coded as an I-picture in the second-generation coding process, It is assumed that the encoding process has been encoded as a B picture, and that the reference picture must be encoded as a P picture in the current fourth generation encoding process.

In this case, since the reference picture is of the same picture type (I picture) as the picture type assigned as the fourth generation picture type and has been encoded in the first generation encoding process, The 330 performs an encoding process using the first-generation encoding parameters, instead of newly creating encoding parameters from the supplied video data. Typical encoding parameters reused in the fourth encoding process are quantizer_scale_code indicating a quantization scale step size, and ma indicating a prediction direction mode.
croblock_type, motion_vector indicating motion vector, Fr
frame / field_ indicating whether the mode is ame prediction mode or field prediction mode
motion_type, dct_type indicating Frame DCT mode or Field DCT mode, and the like.

The controller 330 does not reuse all of the coding parameters transmitted as history information, but reuses the above-mentioned coding parameters for which it is considered desirable to reuse them. Encoding parameters that are considered to be desirably not used are newly generated.

Next, as a fourth generation encoding process, the picture type specified for this reference picture matches any of the picture types in the past encoding process,
Also, the discontinuous parameter supplied from the comparison device 114 is executed when the image has a value indicating that the image is continuous. The encoding parameter reuse encoding process is different from the above-described normal encoding process in that I will explain mainly.

The motion vector detection circuit 310 detects the motion vector of the reference picture in the above-described normal encoding process, but does not perform the motion vector motion_vector detection process in the parameter reuse encoding process. Then, the motion vector motion_vector supplied as the first generation history information is reused. The reason will be described.

Since the baseband video data obtained by decoding the third generation coded stream has been decoded and coded at least three times, the image quality is clearly deteriorated as compared with the original video data. I have. Even if a motion vector is detected from video data having deteriorated image quality, an accurate motion vector cannot be detected. That is, the motion vector supplied as the first generation history information is clearly a higher-precision motion vector than the motion vector detected in the fourth generation encoding process. That is, by reusing the motion vector transmitted as the first generation encoding parameter, the image quality does not deteriorate even when the fourth generation encoding process is performed. The controller 330 controls the motion vector motion_vector supplied as the first generation history information.
As motion vector information of the reference picture to be encoded in the fourth generation encoding process,
24 and the variable length coding circuit 318.

Furthermore, the motion vector detection circuit 310
In order to determine which of the frame prediction mode and the field prediction mode is selected, the prediction error in the frame prediction mode and the prediction error in the field prediction mode are detected, respectively. The fram indicating whether the frame prediction mode or the field prediction mode is supplied as the first generation history information without performing the process of detecting the prediction error in the prediction mode and the prediction error in the field prediction mode.
Reuse e / field_motion_type. The reason is that the prediction error in each prediction mode detected in the first generation is higher in accuracy than the prediction error in each prediction mode detected in the fourth generation encoding process, so that the prediction error is determined by the highly accurate prediction error. This is because the more optimal encoding process can be performed by selecting the predicted mode.

More specifically, the controller 330 transmits the frame / fi supplied as the first generation history information.
The control signal corresponding to eld_motion_type is supplied to the Frame / Field prediction mode switching circuit 312, and the Frame / Field prediction mode switching circuit 312
Performs signal processing corresponding to field_motion_type.

Further, the motion vector detecting circuit 310
Selects any one of an intra-picture prediction mode, a forward prediction mode, a backward prediction mode, and a bidirectional prediction mode (hereinafter, this prediction mode is also referred to as a prediction direction mode) in the normal encoding process. The prediction error in each prediction direction mode was calculated in order to determine whether or not the prediction error in each prediction direction mode was calculated in this parameter reuse encoding process. Macroblo supplied as
The prediction direction mode is determined based on ck_type. This is because the prediction error in each prediction direction mode in the first-generation encoding process has higher accuracy than the prediction error in each prediction direction mode in the fourth-generation encoding process. This is because, when the prediction direction mode determined by the above is selected, more efficient encoding processing can be performed. Specifically, the controller 33
0 is the macroblo included in the first generation history information.
The prediction direction mode indicated by the ck_type is selected, and the arithmetic unit 3 is set so as to correspond to the selected prediction direction mode.
13 and the motion compensation circuit 324.

In the normal coding process, the DCT mode switching circuit 315 compares the signal converted into the signal format of the frame DCT mode with the signal converted in the frame DCT mode in order to compare the coding efficiency of the frame DCT mode with the coding efficiency of the field DCT mode. , The signal converted into the field DCT mode signal form is supplied to the DCT circuit 316. In this parameter reuse coding process, the signal converted into the frame DCT mode signal form and the field DCT mode signal are The processing for generating both of the signals converted into the form is not performed, and the dct_type included in the first generation history information is not included.
Only the processing corresponding to the DCT mode indicated by is performed. Specifically, the controller 330 reuses the dct_type included in the first generation history information,
The DCT mode switching circuit 315 is controlled so that the DCT mode switching circuit 315 performs signal processing corresponding to the DCT mode indicated by the dct_type.

In the normal encoding process, the controller 330 uses the quantization circuit 317 based on the target bit rate specified by the operator and the remaining amount of the transmission buffer.
In the parameter reuse encoding process, the quantization step size is controlled based on the target bit rate, the remaining amount of the transmission buffer, and the past quantization scale included in the history information.
7 is controlled. In addition,
In the following description, the past quantization scale included in the history information will be described as history_q_scale_code. In the history stream described later, this quantization scale is described as quantizer_scale_code.

First, the controller 330 controls the current quantization scale feedback_q_s in the same manner as in the normal encoding process.
Generate cale_code. This feedback_q_scale_code
Is a value determined according to the remaining buffer capacity of the transmission buffer 319 so that the transmission buffer 319 does not overflow or underflow. Then, the first
The value of the past quantization scale history_q_scale_code included in the history stream of the generation is compared with the value of the current quantization scale feedback_q_scale_code, and it is determined which quantization scale is larger. A large quantization scale means that the quantization step is large. If the current quantization scale feedback_q_sca
le_code is the past quantization scale history_q_scale_cod
If it is larger than e, the controller 330 supplies the current quantization scale feedback_q_scale_code to the quantization circuit 317. On the other hand, past quantization scale
history_q_scale_code is the current quantization scale feedba
If it is larger than ck_q_scale_code, the controller 330 returns this past quantization scale history_q_scal
The e_code is supplied to the quantization circuit 317.

That is, the controller 330 determines the largest quantization scale code among a plurality of past quantization scales included in the history information and the current quantization scale calculated from the remaining amount of the transmission buffer. select. In other words, the controller 330 is used in a quantization step in a past (first, second, and third generation) encoding process or in a current (fourth generation) encoding process. In the quantization step
The quantization circuit 317 is controlled so as to perform quantization using the largest quantization step. The reason will be described below.

For example, if the bit rate of the stream generated in the third-generation encoding process is 4 [Mbps], and the encoder 301 performs the fourth-generation encoding process,
Target bit rate set to 15 [Mbp
s]. At this time, since the target bit rate is increased, it is not actually the case that the quantization step should be simply reduced. Even if a picture coded in a large quantization step in a past coding process is coded with a small quantization step in a current coding process, the picture quality of the picture is improved. There is no. In other words, encoding with a quantization step smaller than the quantization step in the past encoding process merely increases the bit amount, but does not improve the image quality. Accordingly, the largest quantization step among the quantization steps in the past (first, second, and third generation) encoding processes or the quantization steps used in the current (fourth generation) encoding process is defined as When the quantization is performed using this, the most efficient encoding process can be performed.

Further, if the encoder 301 is executing the “parameter reuse encoding process”, the comparison device 11
Since the “normal encoding process” is performed before and after the discontinuous image based on the discontinuous parameters supplied from the step 4, deterioration of the image before and after the discontinuous point can be prevented.

Next, the history decoding device 104 and the history encoding device 117 in FIG. 4 will be further described. As shown in the figure, the history decoding device 104 includes a user data decoder 201 for decoding user data supplied from the decoding device 102, a converter 202 for converting the output of the user data decoder 201, and a history from the output of the converter 202. It comprises a history VLD 203 for reproducing information.

The history encoding device 11
Reference numeral 7 denotes a history for formatting the encoding parameters for three generations supplied from the history information separating device 115.
It comprises a VLC 211, a converter 212 for converting the output of the history VLC 211, and a user data formatter 213 for formatting the output of the converter 212 into a user data format.

[0196] User data decoder 201 decodes user data supplied from decoding apparatus 102 and outputs the result to converter 202. Although details will be described later with reference to FIG. 31, the user data (user_data ()) is
_start_code and user_data. In the MPEG standard,
In user_data, continuous 23-bit “0” (start
(the same code as _code) is prohibited. This is to prevent that data from being erroneously detected as start_code. History information (history_stre
am ()) is the user data area (user_dat
a), and there may be such continuous “0” s of 23 bits or more, so that such “0” s of 23 bits or more are not generated. At this time, "1" is inserted at a predetermined timing, and converted_history_stream () (FIG. 18 described later)
Need to be converted to It is the converter 212 of the history encoding device 117 that performs this conversion. Converter 2 of the history decoding device 104
Numeral 02 performs a conversion process reverse to that of the converter 212 (removes “1” inserted in order not to generate consecutive “0” of 23 bits or more).

The history VLD 203 includes a converter 202
, The history information (in this case, the first-generation coding parameter and the second-generation coding parameter) is generated and output to the history information multiplexing device 103.

On the other hand, the history encoding device 11
7, the history VLC 211 corresponds to three generations (first generation, second generation) supplied from the history information separation device 115.
The encoding parameters of the generation and the third generation are converted into the format of the history information. This format includes a fixed-length format (FIGS. 20 to 26 described later) and a variable-length format (FIG. 27 described later). Details of these will be described later.

The history information formatted by the history VLC 211 is converted by the converter 212 into a conv
Converted to erted_history_stream (). This is a process for preventing the start_code of user_data () from being erroneously detected, as described above. That is, there are consecutive “0” s of 23 bits or more in the history information,
Since it is not possible to arrange consecutive “0” s of 23 bits or more in _data, the data is converted by the converter 212 (“1” is inserted at a predetermined timing) so as not to touch this prohibited item. .

[0200] The user data formatter 213 converts the converted_history_stream supplied from the converter 212.
(), History_Data_ID is added based on FIG. 18 described later, and further, user_data_stream_code is added, and v
MPEG-format user_data that can be inserted into the ideo stream is generated and output to the encoding device 116.

FIG. 15 shows a plurality of transcoding systems 1-1 to 1 in a video editing studio, for example.
−N is used in a state of being connected in series.
The history information multiplexing device 103-i of each transcoding system 1-i (i = 1 to N) stores its own information in the section where the oldest encoding parameter in the above-described encoding parameter area is recorded. Overwrites the latest encoding parameters that were used. As a result, the encoding parameters (generation history information) for the four most recent generations corresponding to the same macroblock are recorded in the baseband image data (FIG. 7).

The encoder 30 of each encoder 116-i
1-i (FIG. 14) is generated by the variable-length coding circuit 318 based on the currently used coding parameter supplied from the history information separation device 115-i.
The video data supplied from 17 is encoded. The bit stream generated in this way (for example, pictur
During e_header ()), the current coding parameters are multiplexed.

The variable length coding circuit 318 also multiplexes user data (including generation history information) supplied from the history encoding device 117-i into a bit stream to be output (embedding as shown in FIG. 7). Multiplexing in the bitstream, not processing). The bit stream output from the encoding device 116-i is an SDTI (Serial Data Transfer Interface) 351.
-I, the subsequent transcoding system 1
− (I + 1) is input.

Each of the transcoding system 1-i and the transcoding system 1- (i + 1) is configured as shown in FIG. Therefore, the process is the same as the case described with reference to FIG.

When it is desired to change the coding currently used as an I picture to a P or B picture as the coding using the actual coding parameter history, look at the past coding parameter history and check the past coding parameter history. P or B
A search is made for a picture, and if these histories exist, the picture type is changed using parameters such as the motion vector. Conversely, if there is no history in the past, the change of the picture type for which no motion detection is performed is abandoned. Of course, even if there is no history, the picture type can be changed by performing motion detection.

Note that, as described above, the decoding side and the code side are roughly coupled inside the transcoding system 1 in the present embodiment, and the coding parameters are multiplexed with the image data and transmitted. FIG.
, The decoding device 102 and the encoding device 116 may be directly connected (tightly coupled).

The transcoding system 1 described with reference to FIG. 4 multiplexes the past coding parameters on the baseband video data in order to supply the first to third generation past coding parameters to the coding device 116. And transmitted it. However, in the present invention, a technique of multiplexing past coding parameters to baseband video data is not essential, and as shown in FIG. 16, a transmission path different from baseband video data (for example, a data transfer bus). ) May be used to transmit the past coding parameters.

That is, the decoding device 10 shown in FIG.
2. History decoding device 104, coding device 1
16 and the history encoding device 117 are shown in FIG.
Has exactly the same functions and configurations as the decoding device 102, the history decoding device 104, the encoding device 116, and the history encoding device 117 described in.

Variable length decoding circuit 262 of decoding apparatus 102
Extracts the third-generation coding parameters from the sequence layer, GOP layer, picture layer, slice layer, and macroblock layer of the third-generation encoded stream ST (3rd), and extracts them from the history encoding device 117 and It is supplied to the controller 330 of the encoding device 116, respectively.

Variable Length Decoding Circuit 262 of Decoding Device 102
Supplies the counter value separated from the third-generation encoded stream ST to the format converter 361, and also outputs a Frame / Fiel synchronized with the frame or field.
The d synchronization signal is supplied to the counter 362.

The counter 362 includes a variable length decoding circuit 262
Fr synchronized with frame or field supplied from
16 which is counted up based on the ame / Field synchronization signal
This is a bit counter. The counter 362 indicates 0 to 65,
One of the counter values 535 is output to the comparing device 363.

When the counter 362 has a counter value of 65,535, when the frame / field synchronization signal synchronized with the frame or field is supplied from the variable length decoding circuit 262, the counter value is set to 0, and thereafter, the variable length decoding circuit 262 sets the counter value to 0. Based on the Frame / Field synchronization signal supplied from the circuit 262,
Continue counting up.

The counter 362 counts down (1) based on the Frame / Field synchronization signal synchronized with the frame or field supplied from the variable length decoding circuit 262.
May be subtracted).

FIG. 17 is a diagram showing a configuration example of the counter 362. The counter 381 is a 16-bit binary counter that is counted up by a Clock signal. When all the outputs of the counter 381 become "1" (that is, when the output becomes 65,535), the output of the AND circuit 382 becomes "1", so that the counter 381 is reset (that is, the counter 381 is reset). Value becomes 0).

Note that the counter 101, the counter 113,
A counter 364 described later can be configured similarly to the counter 362.

The format conversion device 361 converts the format of the counter value separated from the third-generation encoded stream ST supplied from the variable length decoding circuit 262 into a 16-bit counter value (for example, a counter output by the counter 362). (The same method as the value) and outputs the result to the comparing device 363.

The comparison device 363 compares the counter value supplied from the format conversion device 361 with the counter value supplied from the counter 362, and when the values are the same, encodes a discontinuous parameter having a predetermined value. Device 116
And when the values are different, a discontinuous parameter of another value is output to the encoding device 116.

The format converter 3 is added to the comparator 363.
When the counter value supplied from 61 and the counter value supplied from the counter 362 take different values, the counter 362 loads the counter value output by the format conversion device 361 and sets it as its own counter value. By doing so, the comparison device 363 once
Even after detecting a discontinuous point in the image, if the discontinuous point is included in the image again, the discontinuous parameter of another value can be output to the encoding device 116.

The history encoding device 117 is
Converted_hist so that the received third-generation coding parameters can be described in the user data area of the picture layer
The data is converted into ory_stream (), and the converted_history_stream () is supplied as user data to the variable length coding circuit 318 of the coding device 116.

Further, the variable length decoding circuit 262 extracts user data user_data including the first generation coding parameter and the second coding parameter from the user data area of the picture layer of the third generation coded stream. Then, the data is supplied to the history decoding device 104 and the variable length coding circuit 318 of the coding device 116. The history decoding device 104 stores co in the user data area.
A first-generation encoding parameter and a second-generation encoding parameter are extracted from a history stream described as nverted_history_stream (), and are extracted by the encoding device 11.
6 controller.

The controller 330 of the encoding device 116
Is based on the first and second generation coding parameters received from the history decoding device 104 and the third generation coding parameters received from the coding device 102. Control.

Variable length coding circuit 31 of coding apparatus 116
Reference numeral 8 denotes user data including a first-generation encoding parameter and a second encoding parameter from the decoding device 102.
In addition to receiving the user_data, the user data user_data including the third-generation encoding parameter is received from the history encoding device 117, and the user data is used as history information, and the user data area of the picture layer of the fourth-generation encoded stream is used as the user information Describe in.

The encoding device 116 has a counter 36
4. Frame / Field synchronized with frame or field
Provides a synchronization signal.

[0224] The counter 364 has a frame / frame synchronized with the frame or field supplied from the encoding device 116.
It is a 16-bit counter that counts up based on the Field synchronization signal. The counter 364 is 0 to 65,535
The format conversion device 365
Output to

When the counter 364 has a counter value of 65,535, the counter value is set to 0 when a frame / field synchronization signal synchronized with a frame or a field is supplied from the encoding device 116, and thereafter, the counter device 364 sets the counter value to 0.
Count up is continued based on the Frame / Field synchronization signal supplied from.

Note that the counter 364 indicates that the encoding device 11
Synchronized with frame or field supplied from 6
Countdown (subtraction of 1) may be performed based on the Frame / Field synchronization signal.

The format conversion device 365 converts the 16-bit counter value supplied from the counter 364 into a format that can be multiplexed into the coded stream ST, and outputs it to the coding device 116.

[0228] The coding device 116 stores the counter value supplied from the format conversion device 365 in the fourth generation coded stream.

FIG. 18 is a diagram showing a syntax for decoding an MPEG video stream. The decoder extracts a plurality of meaningful data items (data elements) from the bit stream by decoding the MPEG bit stream according to this syntax. The syntax described below is
The functions and conditional statements are shown in fine print, and the data elements are shown in bold print. Each data item is described by a mnemonic (Mnemonic) indicating its name, bit length, and its type and transmission order.

First, the functions used in the syntax shown in FIG. 18 will be described.

[0231] The next_start_code () function is a function for searching for a start code described in the bit stream. In the syntax shown in FIG. 18, next to this next_start_code () function, sequence_head
Since the er () function and the sequence_extension () function are arranged in order, this bit stream includes the sequence
A data element defined by the _header () function and the sequence_extension () function is described. Therefore,
When decoding the bit stream, this next_start_c
sequence_header () function and sequence
The start code (a kind of data element) described at the beginning of the _extension () function is found in the bit stream, and based on it, the sequence_header () function and the sequence_extension () function are further found and defined by them. Decode each data element.

The sequence_header () function is a function for defining the header data of the sequence layer of the MPEG bit stream, and the sequence_extension () function is
This is a function for defining extension data of the sequence layer of the EG bitstream.

[0233] The do {} while syntax placed next to the sequence_extension () function is a data described based on the function in {} of the do statement while the condition defined by the while statement is true. This is a syntax for extracting an element from a data stream. That is, while the condition defined by the while statement is true, the decoding process of extracting the data element described based on the function in the do statement from the bit stream is performed by the do {} while syntax.

The nextbits () used in this while statement
The function is a function for comparing a bit or a bit string appearing in a bit stream with a data element to be decoded next. In the example of the syntax illustrated in FIG. 18, the nextbits () function includes a bit string in the bit stream and a sequence_end_code indicating the end of the video sequence.
Is compared with the bit sequence in the bit stream and sequence
When the _end_code does not match, the condition of this while statement is true. Therefore, the do {} while syntax located next to the sequence_extension () function uses sequence_end_cod indicating the end of the video sequence in the bit stream.
While e does not appear, it indicates that the data element defined by the function in the do statement is described in the bit stream.

[0235] In the bit stream, sequence_exten
Following each data element defined by the sion () function, a data element defined by the extension_and_user_data (0) function is described. This extensio
The n_and_user_data (0) function is a function for defining extended data of the sequence layer of the MPEG bit stream and user data.

The do {} while syntax located next to the extension_and_user_data (0) function is described based on the function in the {} of the do statement while the condition defined by the while statement is true. This is a function for extracting the data element from the bit stream. The nextbits () function used in this while statement is used to determine the bits or bit strings appearing in the bit stream and the picture_start_
A function for determining a match with code or group_start_code. If a bit or a bit string appearing in a bit stream matches picture_start_code or group_start_code, the condition defined by the while statement is true. Therefore, this do {} while syntax is used in the bit stream for picture_start_code or group_st
If art_code appears, the code of the data element defined by the function in the do statement is described after the start code, so this picture_start_co
By finding the start code indicated by de or group_start_code, do
Data elements defined in the sentence can be extracted.

The if statement described at the beginning of the do statement is
If group_start_code appears in the bitstream,
The condition is shown. If the condition by this if statement is true, the group_start_
After code, group_of_picture_header (1) function and extens
Data elements defined by the ion_and_user_data (1) function are described in order.

The group_of_picture_header (1) function is
Extension_and_user_data (1) is a function for defining the header data of the GOP layer of the MPEG bit stream.
The function is the GOP layer extension data (e
xtension_data) and user data (user_data).

Further, in this bit stream, gr
oup_of_picture_header (1) function and extension_and_user
After the data element defined by the _data (1) function, the picture_header () function and picture_coding_exten
Describes the data element defined by the sion () function. Of course, if the condition of the if statement described above is not true, the data element defined by the group_of_picture_header (1) function and the extension_and_user_data (1) function is not described.
After the data element defined by the extension_and_user_data (0) function, the picture_header () function and pic
A data element defined by the ture_coding_extension () function is described.

[0240] The picture_header () function is for defining the header data of the picture layer of the MPEG bit stream, and the picture_coding_extension () function is for defining the first extension data of the picture layer of the MPEG bit stream. Function.

The next while statement is a function for determining the condition of the next if statement while the condition defined by the while statement is true. The nextbits () function used in this while statement includes a bit string appearing in the bit stream and extension_start_code or user_data_start_
A function to determine the match with code, a bit string appearing in the bit stream, and extension_start_cod
If e or user_data_start_code matches, the condition defined by this while statement is true.

The first if sentence is a function for determining whether a bit string appearing in a bit stream matches extension_start_code. If the bit string appearing in the bit stream matches the 32-bit extension_start_code, the extension_st
Following the art_code, a data element defined by the extension_data (2) function is described.

The second if sentence is a syntax for judging the match between the bit string appearing in the bit stream and the user_data_start_code. When the bit string appearing in the bit stream matches the 32-bit user_data_start_code, The condition determination of the third if statement is performed. This user
_data_start_code is a start code indicating the start of the user data area in the picture layer of the MPEG bit stream.

[0244] The video_continuity_counter_ID is user_da
It is used to identify that ta () is video_continuity_counter (). marker_bit_1 is set to “1” to prevent emulation of the start code. vi
A counter value is set in deo_continuity_counter. marker_bit_2 is set to “1” to prevent emulation of the start code.

[0245] video_continuity_counter contains video_co
CRC (Cyclic Redundancy Checker) for checking errors corresponding to ntinuity_counter_ID to marker_bit_2
k) is set.

The third if sentence is a syntax for judging whether a bit string appearing in a bit stream matches History_Data_ID. If the bit string appearing in the bit stream matches the 32-bit History_Data_ID, the 32-bit History_Data_I is stored in the user data area of the picture layer of the MPEG bit stream.
Following the code indicated by D, converted_history_s
A data element defined by the tream () function is described.

The converted_history_stream () function is used for MPEG
It is a function for describing history information and history data for transmitting all encoding parameters used at the time of encoding. Details of the data element defined by the converted_history_stream () function will be described later as history_stream () with reference to FIGS.
The History_Data_ID is a start code indicating the head of the history information and the history data described in the user data area of the picture layer of the MPEG bit stream.

The else sentence is a syntax for indicating that the condition is non-true in the third if sentence. So this MP
If the data element defined by the converted_history_stream () function is not described in the user data area of the picture layer of the EG bit stream, the data element defined by the user_data () function is described.

In FIG. 18, the history information is converted_converted_
Although described in history_stream () and not in user_data (), this converted_history_stream () is
It is described as a kind of user_data of the MPEG standard. Therefore, in this specification, the history information may
It is explained that it is described in user_data, but it is MPEG
It means that it is described as a kind of standard user_data.

[0250] The picture_data () function is a function for describing data elements related to the slice layer and the macroblock layer after the user data of the picture layer of the MPEG bit stream. Normally, the data element indicated by the picture_data () function is the conv. Described in the user data area of the picture layer of the bit stream.
Although described next to the data element defined by the erted_history_stream () function or the data element defined by the user_data () function, the extens
If there is no ion_start_code or user_data_start_code, the data element indicated by the picture_data () function is described next to the data element defined by the picture_coding_extension () function.

The data element indicated by the picture_data () function is followed by the sequence_header () function and s
Data elements defined by the equence_extension () function are arranged in order. This sequence_heade
The data element described by the r () function and the sequence_extension () function is composed of the sequence_header () function and the sequence_
Exactly the same as the data element described by the extension () function. The reason for describing the same data in the stream in this way is that when the bit stream receiving apparatus starts receiving data from the middle of the data stream (for example, the bit stream corresponding to the picture layer), the data in the sequence layer is received. This is to prevent the stream from being unable to be decoded.

This last sequence_header () function and sequence
After the data element defined by the nce_extension () function, that is, at the end of the data stream, a 32-bit sequence_end_cod indicating the end of the sequence
e is described.

FIG. 19 shows an outline of the basic structure of the syntax described above.

Next, a history stream defined by the converted_history_stream () function will be described.

This converted_history_stream () is an MPEG
Is a function for inserting a history stream indicating history information into the user data area of the picture layer. still,
“Converted” means a stream that has undergone a conversion process of inserting a marker bit (1 bit) at least every 22 bits of a history stream composed of history data to be inserted into the user area in order to prevent start emulation. It means that

This converted_history_stream () is a fixed-length history stream described below (see FIGS. 20 to 2).
6) or a variable-length history stream (FIG. 27). When a fixed-length history stream is selected on the encoder side, there is an advantage that a circuit and software for decoding each data element from the history stream on the decoder side are simplified. On the other hand, when a variable-length history stream is selected on the encoder side, the history information (data element) described in the user area of the picture layer can be arbitrarily selected as needed by the encoder. Can be reduced, and as a result, the data rate of the entire coded bit stream can be reduced.

The “history stream”, “history stream”, “history information”, “history information”, “history data”, “history data”,
The “history parameters” and “history parameters” mean the coding parameters (or data elements) used in the past coding process, and mean the coding parameters used in the current (last stage) coding process. It does not do. For example, in a first generation encoding process, a certain picture is encoded and transmitted by an I picture, and in a subsequent second generation encoding process, this picture is encoded and transmitted as a P picture, and further, Third
In the generation encoding process, an example will be described in which this picture is encoded with a B picture and transmitted.

The encoding parameters used in the third generation encoding process are the sequence layer of the encoded bit stream generated in the third generation encoding process, the GOP
Layer, picture layer, slice layer, and macroblock layer are described at predetermined positions. On the other hand, the encoding parameters used in the first-generation and second-generation encoding processes, which are the past encoding processes, are a sequence layer or GO that describes the encoding parameters used in the third-generation encoding process.
Instead of being described in the P layer, it is described in the user data area of the picture layer as coding parameter history information in accordance with the syntax described above.

First, the fixed-length history stream syntax will be described with reference to FIGS.

In the user data area of the picture layer of the bit stream generated in the encoding process of the last stage (for example, the third generation), first, the past (for example, the first generation and the second generation) encoding process is performed. The coding parameter included in the sequence header of the sequence layer used in is inserted as a history stream. Note that history information such as the sequence header of the sequence layer of the bit stream generated in the past encoding process is not inserted into the sequence header of the sequence layer of the bit stream generated in the last-stage encoding process. It should be noted that.

The data elements included in the sequence header (sequence_header) used in the past encoding process are sequence_header_code, sequence_header_presen
t_flag, horizontal_size_value, marker_bit, vertica
l_size_value, aspect_ratio_information, frame_rate
_code, bit_rate_value, VBV_buffer_size_value, cons
trained_parameter_flag, load_intra_quantiser_matri
x, load_non_intra_quantiser_matrix, intra_quantise
r_matrix, non_intra_quantiser_matrix, etc.

[0262] sequence_header_code is data representing a start synchronization code of the sequence layer. sequence_hea
der_present_flag is data indicating whether data in sequence_header is valid or invalid. horizontal_size_val
ue is data composed of lower 12 bits of the number of pixels in the horizontal direction of the image. marker_bit is bit data inserted to prevent start code emulation. vertical_size_value is data composed of lower 12 bits of the number of vertical lines of the image. aspect_ratio_infor
“mation” is data representing an aspect ratio (aspect ratio) or a display screen aspect ratio of a pixel. frame_rate_code
Is data representing a display cycle of an image.

Bit_rate_value is the lower 18 bits (rounded up in units of 400 bsp) of the bit rate for limiting the amount of generated bits. VBV_buffer_size_value
Is the lower 10-bit data of the value that determines the size of the virtual buffer (video buffer verifier) for controlling the generated code amount. constrained_parameter_flag is data indicating that each parameter is within the limit. load_i
ntra_quantiser_matrix is data indicating the existence of intra MB quantization matrix data. load_non_i
ntra_quantiser_matrix is data indicating the existence of non-intra MB quantization matrix data. intra_
quantiser_matrix is data indicating the value of the intra MB quantization matrix. non_intra_quantiser_matr
ix is data representing the value of the non-intra MB quantization matrix.

[0264] In the user data area of the picture layer of the bit stream generated in the encoding process of the last stage, a data element representing the sequence extension of the sequence layer used in the past encoding process is described as a history stream. You.

The data elements representing the sequence extension (sequence_extension) used in the past encoding process are extension_start_code, extension
_start_code_identifier, sequence_extension_present
_flag, profile_and_level_indication, progressive_s
equence, chroma_format, horizontal_size_extensio
n, vertical_size_extension, bit_rate_extension, vb
v_buffer_size_extension, low_delay, frame_rate_ext
data elements such as extension_n and frame_rate_extension_d.

[0266] extension_start_code is data representing a start synchronization code of extension data. ex
tension_start_code_identifier is data indicating which extension data is sent. sequence_extension_p
resent_flag is data indicating whether the data in the sequence extension is valid or invalid. pr
ofile_and_level_indication is data for specifying the profile and level of video data. progre
ssive_sequence is data indicating that video data is sequentially scanned. chroma_format is data for specifying a color difference format of video data.

[0267] horizontal_size_extension is high-order 2 bits of data to be added to horizontal_size_value of the sequence header. vertical_size_extension is upper two bits of data added to vertical_size_value of the sequence header. bit_rate_extension is upper 12 bits of data to be added to bit_rate_value of the sequence header. vbv_buffer_size_extension is upper 8 bits of data added to vbv_buffer_size_value of the sequence header. low_delay is data indicating that a B picture is not included. frame_rate_extension_n is
This is data for obtaining a frame rate in combination with frame_rate_code of the sequence header. frame_rate_e
xtension_d is data for obtaining a frame rate in combination with frame_rate_code of the sequence header.

Subsequently, in the user area of the picture layer of the bit stream, a data element representing the sequence display extension of the sequence layer used in the past encoding processing is described as a history stream.

The data elements described as the sequence display extension (sequence_display_extension) are extension_start_code, exte
nsion_start_code_identifier, sequence_display_exte
nsion_present_flag, video_format, colour_descripti
on, colour_primaries, transfer_characteristics, ma
trix_coeffients, display_horizontal_size, and disp
It consists of lay_vertical_size.

[0270] extension_start_code is data representing a start synchronization code of extension data. ex
tension_start_code_identifier is a code indicating which extension data is sent. sequence_display_ext
The extension_present_flag is data indicating whether a data element in the sequence display extension is valid or invalid. video_format is data representing the video format of the original signal. color_description is data indicating that there is detailed data of a color space. colo
r_primaries is data indicating details of the color characteristics of the original signal. transfer_characteristics is data indicating details of how the photoelectric conversion was performed. matrix_c
The oeffients are data showing details of how the original signal was converted from the three primary colors of light. display_horizont
al_size is data representing the intended active area (horizontal size) of the display. display_vertical_size
Is data representing the intended active area (vertical size) of the display.

Subsequently, macroblock assignment data (macroblock_assignment_in_user_data) indicating the phase information of the macroblock generated in the past encoding process is provided in the user area of the picture layer of the bit stream generated in the last-stage encoding process. ) Is described as a history stream.

Macr indicating phase information of this macroblock
oblock_assignment_in_user_data is macroblock_assig
It is composed of data elements such as nment_present_flag, v_phase, and h_phase.

This macroblock_assignment_present_flag
Is data indicating whether a data element in macroblock_assignment_in_user_data is valid or invalid. v_phase
Is data indicating vertical phase information when a macroblock is cut out from image data. h_phase is data indicating horizontal phase information when a macroblock is cut out from image data.

Subsequently, in the user area of the picture layer of the bit stream generated by the encoding process at the last stage, a data element representing the GOP header of the GOP layer used in the past encoding process is stored as a history stream. It has been described.

The GOP header (group_of_picture_heade)
The data element representing r) is group_start_code, g
roup_of_picture_header_present_flag, time_code, cl
osed_gop and broken_link.

[0276] group_start_code is data indicating the start synchronization code of the GOP layer. group_of_picture_header_pr
esent_flag is data indicating whether the data element in group_of_picture_header is valid or invalid. time_code is a time code indicating the time from the beginning of the sequence of the first picture of the GOP. closed_gop
Is flag data indicating that an image in a GOP can be reproduced independently from another GOP. broken_link is flag data indicating that the first B picture in the GOP cannot be accurately reproduced due to editing or the like.

Subsequently, in the user area of the picture layer of the bit stream generated by the encoding process at the last stage, a data element representing the picture header of the picture layer used in the past encoding process is stored as a history stream. It has been described.

Data elements related to the picture header (picture_header) are picture_start_code, temp
oral_reference, picture_coding_type, vbv_delay, fu
ll_pel_forward_vector, forward_f_code, full_pel_ba
It consists of ckward_vector and backward_f_code.

[0279] Specifically, picture_start_code is data representing the start synchronization code of the picture layer. temporal
_reference is data indicating the display order of pictures and is reset at the beginning of the GOP. picture_coding_typ
e is data indicating a picture type. vbv_delay
Is data indicating the initial state of the virtual buffer at the time of random access. full_pel_forward_vector is data indicating whether the accuracy of the forward motion vector is an integer unit or a half pixel unit. forward_f_code is data representing a forward motion vector search range. full_pel_backward_vector
Is data indicating whether the accuracy of the backward motion vector is an integer unit or a half pixel unit. backward_f_code is data representing a backward motion vector search range.

Subsequently, in the user area of the picture layer of the bit stream generated by the encoding process at the last stage, the picture coding extension of the picture layer used in the past encoding process is described as a history stream. I have.

Data elements related to the picture coding extension (picture_coding_extension) are extension_start_code and extension_start_code.
_identifier, f_code [0] [0], f_code [0] [1], f_code [1]
[0], f_code [1] [1], intra_dc_precision, picture_str
ucture, top_field_first, frame_predictive_frame_dc
t, concealment_motion_vectors, q_scale_type, intra
_vlc_format, alternate_scan, repeat_firt_field, ch
roma_420_type, progressive_frame, composite_displa
y_flag, v_axis, field_sequence, sub_carrier, burst
_amplitude and sub_carrier_phase.

[0282] The extension_start_code is a start code indicating the start of extension data in the picture layer. extension_start_code_identifier is a code indicating which extension data is sent. f_code [0] [0]
Is data indicating a horizontal motion vector search range in the forward direction. f_code [0] [1] is data representing the vertical motion vector search range in the forward direction. f_code [1]
[0] is data representing a horizontal motion vector search range in the backward direction. f_code [1] [1] is data representing the vertical motion vector search range in the backward direction.

[0283] intra_dc_precision is data representing the accuracy of the DC coefficient. picture_structure is data indicating a frame structure or a field structure. In the case of the field structure, it is data indicating the upper field or the lower field. top_
field_first is data indicating whether the first field is higher or lower in the case of a frame structure. frame_
predictive_frame_dct is data indicating that, in the case of a frame structure, the prediction of the frame mode DCT is only the frame mode. concealment_motion
_vectors is data indicating that a motion vector for concealing a transmission error is attached to an intra macroblock.

[0284] q_scale_type is data indicating whether to use a linear quantization scale or a non-linear quantization scale. intra_vlc_format is data indicating whether another two-dimensional VLC is used for an intra macroblock. alternate_scan uses zigzag scan,
This is data indicating whether to use an alternate scan. repeat_firt_field is data used at the time of 2: 3 pull-down. chroma_420_type is the next progressive_frame when the signal format is 4: 2: 0.
Is the same value as, otherwise 0. pr
ogressive_frame is data indicating whether or not this picture has been sequentially scanned. composite_display_
The flag is data indicating whether the source signal is a composite signal.

V_axis is data used when the source signal is PAL. field_sequence is data used when the source signal is PAL. sub_carrier
Is data used when the source signal is PAL. burst_amplitude is data used when the source signal is PAL. sub_carrier_phase is data used when the source signal is PAL.

Subsequently, in the user area of the picture layer of the bit stream generated by the encoding process at the last stage, the quantization matrix extension used in the past encoding process is described as a history stream.

The data elements relating to the quantization matrix extension (quant_matrix_extension) are extension_start_code and extension_start_code_ide.
ntifier, quant_matrix_extension_present_flag, load
_intra_quantiser_matrix, intra_quantiser_matrix [6
4], load_non_intra_quantiser_matrix, non_intra_qua
ntiser_matrix [64], load_chroma_intra_quantiser_mat
rix, chroma_intra_quantiser_matrix [64], load_chrom
a_non_intra_quantiser_matrix and chroma_non_intra
_quantiser_matrix [64].

Extension_start_code is a start code indicating the start of this quantization matrix extension. extension_start_code_identifier is a code indicating which extension data is sent. quant_matr
ix_extension_present_flag is data for indicating whether a data element in the quantization matrix extension is valid or invalid. load_intra_quantiser_m
atrix is data indicating the existence of quantization matrix data for an intra macroblock. intra_quantise
r_matrix is data indicating the value of a quantization matrix for an intra macroblock.

Load_non_intra_quantiser_matrix is data indicating the existence of quantization matrix data for a non-intra macro block. non_intra_quantiser_matr
ix is data representing a value of a quantization matrix for a non-intra macroblock. load_chroma_intra_quanti
ser_matrix is data indicating the existence of quantization matrix data for a chrominance intra macroblock. chro
ma_intra_quantiser_matrix is data indicating a value of a quantization matrix for a chroma intra macroblock. load_chroma_non_intra_quantiser_matrix is data indicating the existence of quantization matrix data for a chrominance non-intra macroblock. chroma_non_intra_qua
ntiser_matrix is data indicating a value of a quantization matrix for a color difference non-intra macroblock.

Subsequently, in the user area of the picture layer of the bit stream generated by the encoding process at the last stage, the copyright extension used in the past encoding process is described as a history stream.

The copyright extension (copy
data element for right_extension) is exten
sion_start_code, extension_start_code_itentifier,
copyright_extension_present_flag, copyright_flag,
copyright_identifier, original_or_copy, copyright_
number_1, copyright_number_2, and copyright_number
_3.

[0292] The extension_start_code is a start code indicating the start of the copyright extension. exte
This code indicates which extension data of nsion_start_code_itentifier is sent. copyright_ex
tension_present_flag is data for indicating whether a data element in the copyright extension is valid or invalid. copyright_flag indicates whether or not a copy right has been granted to the encoded video data until the next copyright extension or sequence end.

The copyright_identifier is ISO / IEC JTC / SC
This is data for identifying the registration authority of the copy right specified by 29. original_or_copy is data indicating whether the data in the bit stream is original data or copy data. copyright_number_1
Is data representing bits 44 to 63 of the copyright number. copyright_number_2 is data representing bits 22 to 43 of the copyright number.
copyright_number_3 is data representing bits 0 to 21 of the copyright number.

Subsequently, in the user area of the picture layer of the bit stream generated by the encoding process at the final stage, the picture display extension (picture_display_extensio) used in the past encoding process is stored.
n) is described as a history stream.

A data element representing this picture display extension is extension_start_cod
e, extension_start_code_identifier, picture_displa
y_extension_present_flag, frame_center_horizontal_
offset_1, frame_center_vertical_offset_1, frame_ce
nter_horizontal_offset_2, frame_center_vertical_of
fset_2, frame_center_horizontal_offset_3, and fram
It consists of e_center_vertical_offset_3.

[0296] extension_start_code is a start code for indicating the start of the picture display extension. extension_start_code_identifier is a code indicating which extension data is sent. pictur
e_display_extension_present_flag is data indicating whether the data element in the picture display extension is valid or invalid. frame_center_horizontal_
The offset is data indicating a horizontal offset of the display area, and can be defined up to three offset values. frame_center_vertical_offset is data indicating a vertical offset of the display area, and can define up to three offset values.

[0297] In the user area of the picture layer of the bit stream generated in the encoding process of the final stage, the user information (user information) used in the past encoding process is stored next to the history information indicating the picture display extension described above. user_data) is described as a history stream.

[0298] Following this user data, information on the macroblock layer used in the past encoding process is described as a history stream.

The information on this macroblock layer is
croblock_address_h, macroblock_address_v, slice_he
ader_present_flag, skipped_macroblock_flag, and other data elements related to the position of the macroblock, and macroblock_quant, macroblock_motion_forwar
d, macroblock_motion_backward, macroblock_patter
n, macroblock_intra, spatial_temporal_weight_code_
Data elements related to macroblock modes (macroblock_modes []) such as flag, frame_motion_type, and dct_type, data elements related to quantization step control such as quantizer_scale_code, and PMV [0] [0] [0], PMV
[0] [0] [1], motion_vertical_field_select [0] [0], PMV
[0] [1] [0], PMV [0] [1] [1], motion_vertical_field_sel
ect [0] [1], PMV [1] [0] [0], PMV [1] [0] [1], motion_vert
ical_field_select [1] [0], PMV [1] [1] [0], PMV [1] [1]
Data elements related to motion compensation such as [1], motion_vertical_field_select [1] [1], and coded_block_pattern
Data elements related to macroblock patterns such as num_mv_bits, num_coef_bits, and num_other_bits
And so on.

The data elements related to the macroblock layer will be described below in detail.

[0301] macroblock_address_h is data for defining the horizontal absolute position of the current macroblock. macroblock_address_v is data for defining the vertical absolute position of the current macroblock.
slice_header_present_flag is data indicating whether or not this macroblock is the head of a slice layer and has a slice header. skipped_macroblock_flag is
This data indicates whether to skip this macroblock in the decoding process.

The macroblock_quant is a macroblock type (macroblock_ty) shown in FIGS. 43 and 44 described later.
pe) data, quantizer_scale_c
This is data indicating whether or not ode appears in the bit stream. macroblock_motion_forward is shown in FIG. 43 and FIG.
4 is data derived from the macroblock type shown in FIG. 4 and used in the decoding process. macrob
lock_motion_backward is data derived from the macroblock type shown in FIGS. 43 and 44, and is data used in the decoding process. mocroblock_pattern
Is data derived from the macroblock types shown in FIGS. 43 and 44, and is data indicating whether or not coded_block_pattern appears in the bit stream.

The macroblock_intra is data derived from the macroblock type shown in FIGS. 43 and 44, and is used in the decoding process. spatial_temp
oral_weight_code_flag is data derived from the macroblock types shown in FIGS. 43 and 44, and spatial_temporal_weight_code indicating a method of upsampling a lower layer image with time scalability is data indicating whether or not the bitstream exists. It is.

[0304] frame_motion_type is a 2-bit code indicating the prediction type of the macroblock of the frame.
If the number of prediction vectors is two and the field-based prediction type is “00”, if the number of prediction vectors is one and the field-based prediction type is “01”, the number of prediction vectors is one and the frame is base Is "10" for the prediction type of "1", and "11" for the prediction type of one prediction vector and dual prime. field_mo
tion_type is a 2-bit code indicating motion prediction of a macroblock in the field. If the number of prediction vectors is one and the field-based prediction type is “01”, if the number of prediction vectors is two and the prediction type is a 18 × 8 macroblock-based “10”, the prediction vector is one If it is a forecast type of dual prime, "1
1 ". dct_type indicates whether the DCT is in frame DCT mode
This is data indicating whether the mode is the field DCT mode. quantiser
_scale_code is data indicating the quantization step size of the macroblock.

Next, a data element relating to a motion vector will be described. The motion vector is coded as a difference with respect to the previously coded vector in order to reduce the motion vector required for decoding. In order to perform motion vector decoding, the decoder must maintain four motion vector predictions, each with a horizontal and vertical component. This predicted motion vector is represented as PMV [r] [s] [v]. [r] is a flag indicating whether the motion vector in the macroblock is the first vector or the second vector, and is “0” when the vector in the macroblock is the first vector. Becomes “1” when the vector in the macro block is the second vector. [s] is a flag indicating whether the direction of the motion vector in the macroblock is forward or backward. In the case of a forward motion vector, the flag is set to “0”, and the backward motion vector In this case, it becomes "1". [v] is a flag indicating whether the vector component in the macroblock is horizontal or vertical, and is “0” in the case of a horizontal component, and is “0” in the case of a vertical component. Becomes “1”.

Accordingly, PMV [0] [0] [0] represents the data of the horizontal component of the forward motion vector of the first vector, and PMV [0] [0] [1] represents the first component. Represents the vertical component data of the motion vector in the forward direction of the vector of PMV [0] [1] [0]
Represents the data of the horizontal component of the backward motion vector of the first vector, and PMV [0] [1] [1] represents the data of the vertical component of the backward motion vector of the first vector. PMV [1] [0] [0] represents the horizontal component data of the forward motion vector of the second vector, and PMV [1]
[0] [1] represents the data of the vertical component of the forward motion vector of the second vector, and PMV [1] [1] [0] represents the second component.
Represents the horizontal component data of the backward motion vector of the second vector, and PMV [1] [1] [1] represents the vertical component data of the second motion vector of the second vector.

[0307] motion_vertical_field_select [r] [s] is
This is data indicating which reference field is used for the prediction format. This motion_vertical_field_select
When [r] [s] is “0”, it indicates that the top reference field is used, and when “1”, the bottom reference field is used.

[0308] Therefore, motion_vertical_field_select
[0] [0] indicates a reference field when a forward motion vector of the first vector is generated, and motion_vertical_fi
eld_select [0] [1] indicates a reference field when a backward motion vector of the first vector is generated, and motion_v
ertical_field_select [1] [0] indicates a reference field when a forward motion vector of the second vector is generated, and motion_vertical_field_select [1] [1] generates a backward motion vector of the second vector. Reference field at the time is shown.

[0309] The coded_block_pattern is variable-length data indicating which DCT block has a significant coefficient (non-zero coefficient) among a plurality of DCT blocks storing DCT coefficients. num_mv_bits is data indicating the code amount of a motion vector in a macroblock. num_coef_bits is data indicating the code amount of the DCT coefficient in the macro block. n
um_other_bits is the code amount of the macroblock and is data indicating the code amount other than the motion vector and the DCT coefficient.

Next, the syntax for decoding each data element from the variable-length history stream will be described with reference to FIGS.

[0311] The variable-length history stream is next_sta.
rt_code () function, sequence_header () function, sequence_ext
extension () function, extension_and_user_data (0) function, grou
p_of_picture_header () function, extension_and_user_data
(1) function, picture_header () function, picture_coding_exte
nsion () function, re_coding_stream_info () function, extensio
It is composed of data elements defined by n_and_user_data (2) function and picture_data () function.

[0312] The next_start_code () function is a function for searching for a start code existing in the bit stream. Therefore, the next_start_code () function is used in the past encoding process as shown in FIG. A data element, which is a data element defined by the sequence_header () function, is described.

The data elements defined by the sequence_header () function are sequence_header_code, sequence
ce_header_present_flag, horizontal_size_value, ver
tical_size_value, aspect_ratio_information, frame_
rate_code, bit_rate_value, marker_bit, VBV_buffer_
size_value, constrained_parameter_flag, load_intra
_quantiser_matrix, intra_quantiser_matrix, load_no
n_intra_quantiser_matrix and non_intra_quantiser_
matrix.

[0314] sequence_header_code is data representing a start synchronization code of the sequence layer. sequence_hea
der_present_flag is data indicating whether data in sequence_header is valid or invalid. horizontal_size_val
ue is data consisting of the lower 12 bits of the number of pixels in the horizontal direction of the image. vertical_size_value is data composed of lower 12 bits of the number of vertical lines of the image. aspect_r
atio_information is data representing the pixel aspect ratio (aspect ratio) or the display screen aspect ratio. frame_
rate_code is data representing a display cycle of an image. bit
_rate_value is the lower 18 bits of the bit rate for limiting the amount of generated bits (rounded up to the nearest 400 bsp)
Data.

[0315] Marker_bit is bit data inserted to prevent start code emulation. VBV_buffer_size_value is lower 10-bit data of a value that determines the size of a virtual buffer (video buffer verifier) for controlling the generated code amount. constrained_
parameter_flag is data indicating that each parameter is within the limit. load_intra_quantiser_matrix
Is data indicating the presence of intra MB quantization matrix data. intra_quantiser_matrix is data indicating the value of the quantization matrix for intra MB.
load_non_intra_quantiser_matrix is a non-intra MB
This is data indicating the existence of quantization matrix data for use. non_intra_quantiser_matrix is data representing the value of the non-intra MB quantization matrix.

Next to the data element defined by the sequence_header () function, as shown in FIG.
The data element defined by the quence_extension () function is described as a history stream.

[0317] The data elements defined by the sequence_extension () function are extension_start_code, ex
tension_start_code_identifier, sequence_extension_
present_flag, profile_and_level_indication, progre
ssive_sequence, chroma_format, horizontal_size_ext
tension, vertical_size_extension, bit_rate_extensio
n, vbv_buffer_size_extension, low_delay, frame_rat
Data elements such as e_extension_n and frame_rate_extension_d.

[0318] The extension_start_code is data representing a start synchronization code of the extension data. ex
tension_start_code_identifier is data indicating which extension data is sent. sequence_extension_p
resent_flag is data indicating whether the data in the sequence extension is valid or invalid.
profile_and_level_indication is data for specifying the profile and level of video data. prog
ressive_sequence is data indicating that video data is sequentially scanned. chroma_format is data for specifying a color difference format of video data.
horizontal_size_extension is hor in the sequence header
Upper 2 bits of data to be added to izntal_size_value. vertical_size_extension is the v in the sequence header
Upper 2 bits of data to be added to ertical_size_value. bit_rate_extension is bit_ra of the sequence header
Upper 12 bits of data to be added to te_value. vbv_
buffer_size_extension is the sequence header vbv_buf
Upper 8 bits of data to be added to fer_size_value.

[0319] low_delay is data indicating that a B picture is not included. frame_rate_extension_n is data for obtaining a frame rate in combination with frame_rate_code of the sequence header. frame_rate_exten
sion_d is data for obtaining a frame rate in combination with frame_rate_code of the sequence header.

The data element defined by the sequence_extension () function is followed by ex as shown in FIG.
The data element defined by the tension_and_user_data (0) function is described as a history stream. When "i" is other than 1, the extension_and_user_data (i) function describes only the data element defined by the user_data () function as a history stream without describing the data element defined by the extension_data () function. . Therefore, extension_and_user_data
The (0) function describes only a data element defined by the user_data () function as a history stream.

[0321] The user_data () function describes user data as a history stream based on the syntax as shown in FIG.

The data element defined by the extension_and_user_data (0) function is followed by the data element defined by the group_of_picture_header () function as shown in FIG. 32, and extension_and_user_data (1).
The data element defined by the function is described as a history stream. However, only when group_start_code indicating the start code of the GOP layer is described in the history stream, group_of_picture_header
Data elements defined by the () function, and exte
A data element defined by the nsion_and_user_data (1) function is described.

The data element defined by the group_of_picture_header () function is group_start_code, gr
oup_of_picture_header_present_flag, time_code, clo
It consists of sed_gop and broken_link.

[0324] group_start_code is data indicating the start synchronization code of the GOP layer. group_of_picture_header_pr
esent_flag is data indicating whether the data element in group_of_picture_header is valid or invalid. time_code is a time code indicating the time from the beginning of the sequence of the first picture of the GOP. closed_gop
Is flag data indicating that an image in a GOP can be reproduced independently from another GOP. broken_link is flag data indicating that the first B picture in the GOP cannot be accurately reproduced due to editing or the like.

[0325] The extension_and_user_data (1) function is
As with the nsion_and_user_data (0) function, user_dat
Describe only the data elements defined by the a () function as a history stream.

If there is no group_start_code indicating the start code of the GOP layer in the history stream, the group_of_picture_header () function and the exten
The data elements defined by the sion_and_user_data (1) function are not described in the history stream. In that case, after the data element defined by the extension_and_user_data (0) function,
The data element defined by the headr () function is described as a history stream.

[0327] The data element defined by the picture_headr () function, as shown in FIG.
art_code, temporal_reference, picture_coding_typ
e, vbv_delay, full_pel_forward_vector, forward_f_c
ode, full_pel_backward_vector, backward_f_code, ex
tra_bit_picture and extra_information_picture.

[0328] Specifically, picture_start_code is data representing the start synchronization code of the picture layer. temporal
_reference is data indicating the display order of pictures and is reset at the beginning of the GOP. picture_coding_typ
e is data indicating a picture type. vbv_delay
Is data indicating the initial state of the virtual buffer at the time of random access. full_pel_forward_vector is data indicating whether the accuracy of the forward motion vector is an integer unit or a half pixel unit. forward_f_code is data representing a forward motion vector search range. full_pel_backward_vector
Is data indicating whether the accuracy of the backward motion vector is an integer unit or a half pixel unit. backward_f_code is data representing a backward motion vector search range. extra_bit_pi
“cture” is a flag indicating the presence of subsequent additional information. If this extra_bit_picture is "1", then e
xtra_information_picture exists and extra_bit_pictur
When e is “0”, it indicates that there is no data following this. extra_information_picture is information reserved in the standard.

Next to the data element defined by the picture_headr () function, a pictur as shown in FIG.
The data element defined by the e_coding_extension () function is described as a history stream.

The data element defined by the picture_coding_extension () function is extension_star
t_code, extension_start_code_identifier, f_code [0]
[0], f_code [0] [1], f_code [1] [0], f_code [1] [1], int
ra_dc_precision, picture_structure, top_field_firs
t, frame_predictive_frame_dct, concealment_motion_
vectors, q_scale_type, intra_vlc_format, alternate
_scan, repeat_firt_field, chroma_420_type, progres
sive_frame, composite_display_flag, v_axis, field_
sequence, sub_carrier, burst_amplitude, and sub_ca
It is composed of rrier_phase.

[0331] The extension_start_code is a start code indicating the start of extension data in the picture layer. extension_start_code_identifier is a code indicating which extension data is sent. f_code [0] [0]
Is data indicating a horizontal motion vector search range in the forward direction. f_code [0] [1] is data representing the vertical motion vector search range in the forward direction. f_code [1]
[0] is data representing a horizontal motion vector search range in the backward direction. f_code [1] [1] is data representing the vertical motion vector search range in the backward direction. in
tra_dc_precision is data representing the accuracy of the DC coefficient.

[0332] The picture_structure is data indicating a frame structure or a field structure.
In the case of the field structure, it is data indicating the upper field or the lower field. top_fiel
In the case of a frame structure, d_first is data indicating whether the first field is higher or lower. frame_pred
ictive_frame_dct is the frame structure
This is data indicating that the prediction of the frame mode DCT is only the frame mode. concealment_motion_vec
tors is data indicating that a motion vector for concealing a transmission error is attached to an intra macroblock. q_scale_type is data indicating whether to use a linear quantization scale or a non-linear quantization scale. intra_vlc_format is data indicating whether another two-dimensional VLC is used for an intra macroblock.

[0333] "alternate_scan" is data representing a selection of whether to use a zigzag scan or an alternate scan. repeat_firt_field is data used at the time of 2: 3 pull-down. chroma_420_type is the next progressive_type when the signal format is 4: 2: 0.
The same value as frame, otherwise it represents 0. progressive_frame is data indicating whether or not this picture has been sequentially scanned. composite_di
splay_flag is data indicating whether or not the source signal was a composite signal. v_axis is data used when the source signal is PAL. field_sequenc
e is data used when the source signal is PAL. sub_carrier is data used when the source signal is PAL. burst_amplitude indicates that the source signal is P
Data used in the case of AL. sub_carrier_phase
Is data used when the source signal is PAL.

Following the data element defined by the picture_coding_extension () function, re_coding_stre
The data element defined by the am_info () function is described as a history stream. This re_coding_
The stream_info () function is mainly used when describing a combination of history information, and details thereof will be described later with reference to FIG.

[0335] Following the data element defined by the re_coding_stream_info () function, extensions_and_u
The data element defined by ser_data (2)
It is described as a history stream. This extension_
The and_user_data (2) function includes an extension start code (ex
tension_start_code) if present
Describes a data element defined by the data () function. Following this data element, the user data start code (user_data_
If start_code) exists, a data element defined by the user_data () function is described.
However, when the extension start code and the user data start code do not exist in the bit stream, the data elements defined by the extension_data () function and the user_data () function are not described in the bit stream.

As shown in FIG. 35, the extension_data () function includes a data element indicating the extension_start_code, a quant_matrix_extension () function, and a copyright_extens
This is a function for describing a data element defined by an ion () function and a picture_display_extension () function as a history stream in a bit stream.

The data element defined by the quant_matrix_extension () function is, as shown in FIG.
tension_start_code, extension_start_code_identifie
r, quant_matrix_extension_present_flag, load_intra
_quantiser_matrix, intra_quantiser_matrix [64], loa
d_non_intra_quantiser_matrix, non_intra_quantiser_
matrix [64], load_chroma_intra_quantiser_matrix, ch
roma_intra_quantiser_matrix [64], load_chroma_non_i
ntra_quantiser_matrix and chroma_non_intra_quanti
ser_matrix [64].

[0338] extension_start_code is a start code indicating the start of this quantization matrix extension. extension_start_code_identifier is a code indicating which extension data is sent. quant_mat
rix_extension_present_flag is data for indicating whether a data element in the quantization matrix extension is valid or invalid. load_intra_quantiser_m
atrix is data indicating the existence of quantization matrix data for an intra macroblock. intra_quantise
r_matrix is data indicating the value of a quantization matrix for an intra macroblock.

[0339] load_non_intra_quantiser_matrix is data indicating the existence of quantization matrix data for a non-intra macroblock. non_intra_quantiser_matr
ix is data representing a value of a quantization matrix for a non-intra macroblock. load_chroma_intra_quanti
ser_matrix is data indicating the existence of quantization matrix data for a chrominance intra macroblock. chro
ma_intra_quantiser_matrix is data indicating a value of a quantization matrix for a chroma intra macroblock. load_chroma_non_intra_quantiser_matrix is data indicating the existence of quantization matrix data for a chrominance non-intra macroblock. chroma_non_intra_qua
ntiser_matrix is data indicating a value of a quantization matrix for a color difference non-intra macroblock.

The data element defined by the copyright_extension () function is, as shown in FIG.
extension_start_code, extension_start_code_itentifie
r, copyright_extension_present_flag, copyright_fla
g, copyright_identifier, original_or_copy, copyrig
ht_number_1, copyright_number_2, and copyright_nu
Consists of mber_3.

Extension_start_code is a start code indicating the start of the copyright extension. exte
nsion_start_code_itentifier This is a code indicating which extension data is sent. copyright_ext
The extension_present_flag is data for indicating whether the data element in the copyright extension is valid or invalid.

[0342] copyright_flag indicates whether or not a copy right has been granted to the encoded video data until the next copyright extension or sequence end. copyright_identifier is data for identifying the registration authority of the copy right specified by ISO / IEC JTC / SC29. original_or_copy is data indicating whether the data in the bit stream is original data or copy data. copyright_number_1
Data representing bits 44 to 63 of the copyright number. copyright_number_2 is data representing bits 22 to 43 of the copyright number. copy
right_number_3 is data representing bits 0 to 21 of the copyright number.

As shown in FIG. 38, the data elements defined by the picture_display_extension () function are extension_start_code_identifier, frame_center_
horizontal_offset, frame_center_vertical_offset, etc.

[0344] The extension_start_code_identifier is a code indicating which extension data is to be sent. frame
_center_horizontal_offset is data indicating the horizontal offset of the display area, and number_of_frame_
You can define a number of offset values defined by center_offsets. frame_center_vertical_offset
Is data indicating the vertical offset of the display area, and can define the number of offset values defined by number_of_frame_center_offsets.

Returning to FIG. 27 again, extension_and_user
Following the data element defined by the _data (2) function, the data element defined by the picture_data () function is described as a history stream. However, this picture_data () function exists when red_bw_flag is not 1 or red_bw_indicator is 2 or less. This red_bw_flag and red_bw_indicator are r
These are described in the e_coding_stream_info () function, which will be described later with reference to FIGS. 51 and 52.

The data element defined by the picture_data () function is a data element defined by the slice () function, as shown in FIG. This slic
At least one data element defined by the e () function is described in the bit stream.

[0347] As shown in Fig. 40, the slice () function includes slice_start_code, slice_quantiser_scale_code,
intra_slice_flag, intra_slice, reserved_bits, extr
a_bit_slice, extra_information_slice, and extra_bi
This is a function for describing a data element such as t_slice and a data element defined by the macroblock () function as a history stream.

[0348] slice_start_code is a start code indicating the start of a data element defined by the slice () function. slice_quantiser_scale_code is data indicating a quantization step size set for a macroblock existing in this slice layer. But,
When quantizer_scale_code is set for each macroblock, the macroblock_quantiser_scale_code data set for each macroblock is used with priority.

[0349] Intra_slice_flag is a flag indicating whether intra_slice and reserved_bits are present in the bit stream. intra_slice is data indicating whether or not a non-intra macroblock exists in the slice layer. If any of the macroblocks in the slice layer is a non-intra macroblock, intra_
slice is “0”, and when all of the macroblocks in the slice layer are non-intra macroblocks, intra_slice is “1”. reserved_bits is 7
It is bit data and takes a value of "0". extra_bit_
slice is a flag indicating that additional information exists as a history stream, and then extra_information_sl
If ice exists, it is set to “1”. If there is no additional information, it is set to “0”.

Next to these data elements, macr
The data element defined by the oblock () function
It is described as a history stream.

As shown in FIG. 41, the macroblock () function includes macroblock_escape and macroblock_address_incremen.
t, macroblock_quantiser_scale_code, and marker
data elements such as _bit, macroblock_modes () function, motion_vectors (s) function, and code_block_patter
This is a function for describing the data element defined by the n () function.

Macroblock_escape is a fixed bit string indicating whether or not the horizontal difference between the reference macroblock and the previous macroblock is 34 or more. If the horizontal difference between the reference macroblock and the previous macroblock is 34 or more, the macroblock_address_increment value is set to 33.
Plus macroblock_address_increment is data indicating a horizontal difference between the reference macroblock and the previous macroblock. If this macroblock_address_i
If there is one macroblock_escape before ncrement, the value of this macroblock_address_increment is 33
Is the data indicating the horizontal difference between the actual reference macroblock and the previous macroblock.

[0353] macroblock_quantiser_scale_code is a quantization step size set for each macroblock, and exists only when macroblock_quant is "1".
In each slice layer, slice_quantiser_scale_code indicating the quantization step size of the slice layer is set, but macroblock_quantiser
If _scale_code is set, this quantization step size is selected.

Next to macroblock_address_increment,
Describes a data element defined by the macroblock_modes () function. macroblock_modes () function
As shown in FIG. 42, macroblock_type, frame_motion_
This is a function for describing data elements such as type, field_motion_type, and dct_type as a history stream.

[0355] macroblock_type is data indicating the coding type of the macroblock. The details are shown in FIG.
47 will be described later with reference to FIG.

If macroblock_motion_forward or mac
When roblock_motion_backward is “1”, the picture structure is a frame, and frame_pred_frame_dct is “0”, a data element representing frame_motion_type is described next to a data element representing macroblock_type. This frame_pred_fra
me_dct is a flag indicating whether or not frame_motion_type exists in the bit stream.

[0357] frame_motion_type is a 2-bit code indicating the prediction type of the macroblock of the frame.
If the number of prediction vectors is two and the field-based prediction type is “00”, if the number of prediction vectors is one and the field-based prediction type is “01”, the number of prediction vectors is one and the frame is base Is "10" for the prediction type of "1", and "11" for the prediction type of one prediction vector and dual prime.

When the condition for describing frame_motion_type is not satisfied, a data element representing field_motion_type is described next to a data element representing macroblock_type.

[0359] field_motion_type is a 2-bit code indicating motion prediction of the macroblock in the field.
If the number of prediction vectors is one and the field-based prediction type is “01”, the prediction vector is 18 ×
"10" for prediction type based on 8 macroblocks
And if the number of prediction vectors is one and the prediction type is dual prime, it is “11”.

If the picture structure is a frame,
_pred_frame_dct indicates that frame_motion_type exists in the bit stream, and frame_pred_frame_d
When ct indicates that dct_type exists in the bit stream, a data element representing dct_type is described next to a data element representing macroblock_type. Note that dct_type is data indicating whether the DCT is a frame DCT mode or a field DCT mode.

Returning to FIG. 41 again, if the reference macroblock is a forward prediction macroblock, or if the reference macroblock is an intra macroblock and any of the concealed macroblocks,
A data element defined by the motion_vectors (0) function is described. If the reference macroblock is a backward prediction macroblock, motion_vectors
(1) A data element defined by a function is described. The motion_vectors (0) function is a function for describing a data element relating to a first motion vector, and the motion_vectors (1) function is a function for describing a data element relating to a second motion vector. It is.

The motion_vectors (s) function is a function for describing a data element relating to a motion vector, as shown in FIG.

If the number of motion vectors is one and the dual prime prediction mode is not used, motion_ver
A data element defined by tical_field_select [0] [s] and motion_vector (0, s) is described.

The motion_vertical_field_select [r] [s]
Indicates whether the first motion vector (whichever vector is forward or backward) is a vector created with reference to the bottom field or a vector created with reference to the top field Is a flag that indicates The index “r” is an index indicating which vector is the first vector or the second vector, and “s” is whether the prediction direction is forward or backward prediction. It is an index that indicates

As shown in FIG. 44, the motion_vector (r, s) function includes a data string related to motion_code [r] [s] [t], a data string related to motion_residual [r] [s] [t], dm
This is a function for describing data representing vector [t].

[0366] motion_code [r] [s] [t] is variable-length data representing the magnitude of a motion vector in the range of -16 to +16. motion_residual [r] [s] [t] is variable-length data representing a residual of a motion vector. So this mo
A detailed motion vector can be described by the values of tion_code [r] [s] [t] and motion_residual [r] [s] [t]. dmvector [t] is used in dual prime prediction mode to generate a motion vector in one field (for example, the top field is one field with respect to the bottom field). Are scaled, and are corrected in the vertical direction to reflect the vertical shift between the lines of the top field and the bottom field. The index “r” is an index indicating which vector is the first vector or the second vector, and “s” is whether the prediction direction is forward or backward prediction. It is an index that indicates “S” is data indicating whether the motion vector is a vertical component or a horizontal component.

First, a data string representing motion_coder [r] [s] [0] in the horizontal direction is described as a history stream by the motion_vector (r, s) function shown in FIG. mo
tion_residual [0] [s] [t] and motion_residual [1] [s] [t]
Since the number of bits of both is indicated by f_code [s] [t], f
motion_residual [r] if _code [s] [t] is not 1
This indicates that [s] [t] is present in the bitstream. Motion_residual [r] [s] [0] of the horizontal component is not “1” and motion_code [r] [s] [0] of the horizontal component
Is not "0" means that mo
data_residual [r] [s] [0] exists, which means that the horizontal component of the motion vector exists.
A data element representing motion_residual [r] [s] [0] is described.

Subsequently, motion_coder [r] [s] in the vertical direction
A data string representing [1] is described as a history stream. Similarly, motion_residual [0] [s] [t] and motion
The number of bits of both _residual [1] [s] [t] is f_code [s] [t]
In the case that f_code [s] [t] is not 1,
This indicates that motion_residual [r] [s] [t] is present in the bitstream. motion_residual [r]
That [s] [1] is not “1” and motion_code [r] [s] [1] is not “0” means that the moti
Since there is a data element representing on_residual [r] [s] [1], which means that the vertical component of the motion vector exists, in that case, the mo component of the vertical component
A data element representing tion_residual [r] [s] [1] is described.

Next, referring to FIG. 45 to FIG.
The oblock_type will be described. macroblock_type is ma
croblock_quant, dct_type_flag, macroblock_motion_f
Variable length data generated from flags such as orward and macroblock_motion_backward. macroblock_quan
t is a flag indicating whether or not macroblock_quantiser_scale_code for setting a quantization step size for a macroblock is set. If macroblock_quantiser_scale_code is present in the bit stream, macroblock_quant is a value of “1”. I take the.

[0370] dct_type_flag is a flag indicating whether there is a dct_type indicating whether or not the reference macroblock is encoded by the frame DCT or the field DCT (in other words, a flag indicating whether the reference macroblock is DCT). If dct_type exists in the bit stream, dct_type_flag takes a value of “1”. macroblo
ck_motion_forward is a flag indicating whether or not the reference macroblock is predicted forward, and takes a value of “1” when the reference macroblock is predicted forward. macroblock_motion_
backward is a flag indicating whether or not the reference macroblock is backward predicted, and takes a value of “1” when backward predicted.

In the variable length format, history information can be reduced in order to reduce the bit rate to be transmitted.

That is, the macroblock_type and motion_vect
When ors () is transferred but quantizer_scale_code is not transferred, slice_quantiser_scale_code is set to “0000”.
By setting it to 0 ", the bit rate can be reduced.

Also, only macroblock_type is transferred and motio
n_vectors (), quantiser_scale_code, and dct_type
Is not transferred, macroblock_type is set to "not
By using "coded", the bit rate can be reduced.

If only picture_coding_type is transferred and all information after slice () is not transferred, s
By using picture_data () without lice_start_code, the bit rate can be reduced.

In the above description, in order to prevent consecutive “0” of 23 bits in user_data from being output, “1” is inserted every 22 bits. However, it is not limited to every 22 bits. . Also, instead of counting the number of consecutive “0” and inserting “1”, Byte_allign can be checked and inserted.

[0376] Further, in MPEG, generation of 23-bit continuous "0" is prohibited. However, actually, only the case where 23 bits continue from the head of a byte is considered as a problem. If 0 bits continue for 23 bits from the middle, no problem occurs. Therefore, for example, "1" may be inserted at a position other than the LSB every 24 bits.

In the above description, the history information is
o Format is close to elementary stream, but packetized
The format may be close to the elementary stream or the transport stream. Although the location of the user_data of the Elementary Stream is before the picture_data, it can be another location.

In the transcoding system 1 of FIG. 4, the encoding parameters for four generations are output as history information at the subsequent stage. However, not all of the history information is actually required. The required history information differs for each. In addition, the capacity of an actual transmission path or recording medium (transmission medium) is limited. Even if the history information is compressed, transmitting all the history information imposes a load on the capacity. The bit rate of the image bit stream is suppressed, and the effectiveness of history information transmission is impaired.

Therefore, a descriptor describing a combination of items to be transmitted as history information is incorporated in the history information and transmitted at a later stage, and information corresponding to various applications is transmitted instead of transmitting all the history information. It can be transmitted. FIG. 48 illustrates a configuration example of the transcoding system 1 in such a case.

In FIG. 48, portions corresponding to those in FIG. 4 are denoted by the same reference numerals, and description thereof will be omitted as appropriate. In the configuration example of FIG. 48, between the history information separating device 115 and the encoding device 116, and between the history encoding device 117 and the encoding device 116,
An encoding parameter selection circuit 501 is inserted.

An encoding parameter selection circuit 501 calculates an encoding parameter from a baseband video signal output from the history information separating device 115, an encoding parameter calculating section 512, and the transcoding system output from the history information separating device 115. 1, information on the encoding parameters determined to be optimal for encoding (for example, second-generation encoding parameters)
Encoding parameters and descriptors (red_bw_flag, red_bw_indi
cator) (described later with reference to FIG. 52), a combination descriptor separation unit 511 for separating the coding parameters output by the coding parameter calculation unit 512, and a coding parameter output by the combination descriptor separation unit 511. A switch 513 for selecting one of them according to the descriptor separated by the combination descriptor separating unit 511 and outputting the selected one to the encoding device 116 is provided. Other configurations are the same as those in FIG.

Here, the combination of items transmitted as history information will be described. Historical information can be categorized as
The information can be divided into picture unit information and macroblock unit information. The information of the slice unit is the mac included in it.
The information can be obtained by collecting the information of the roblock, and the information of the GOP unit can be obtained by collecting the information of the picture unit included therein.

[0383] Information in picture units is one for each frame.
Since the data is transmitted only once, the bit rate occupying the information transmission is not so large. In contrast, macroblo
Since information in units of ck is transmitted for each macroblock, for example, in a video system in which the number of scanning lines in one frame is 525 and the field rate is 60 fields / sec, 1
If the number of pixels in the frame is 720 × 480, macroblo
The information in units of ck is 1350 (= (72
0/16) × (480/16)) transmissions. For this reason, a considerable portion of the history information is occupied by information for each macroblock. Thus, as the history information, at least the information in picture units is always transmitted, but the information in macroblock units is selected and transmitted according to the application, so that the amount of information to be transmitted can be suppressed.

The information in units of macroblocks transferred as history information includes, for example, num_coef_bits, num_mv_bits, num_
other_bits, q_scale_code, q_scale_type, motion_typ
e, mv_vert_field_sel [] [], mv [] [] [], mb_mfwd, mb_mb
wd, mb_pattern, coded_block_pattern, mb_intra, sli
There are ce_start, dct_type, mb_quant, skipped_mb, etc. These are expressed using elements of macroblock rate information.

[0385] num_coef_bits represents the code amount required for the DCT coefficient among the code amounts of the macroblock. num_mv_bits is
Indicates the amount of code required for the motion vector among the amounts of code of the macroblock. num_other_bits represents a code amount other than num_coef_bits and num_mv_bits in the code amount of the macroblock.

The q_scale_code is applied to the macroblock
Represents q_scale_code. motion_type represents the type of the motion vector applied to the macroblock. mv_vert_field_se
l [] [] is the field of the motion vector applied to the macroblock
Represents select.

[0387] mv [] [] [] represents a motion vector applied to a macroblock. mb_mfwd is a flag indicating that the prediction mode of the macroblock is forward prediction. mb_mbwd
Is a flag indicating that the prediction mode of the macroblock is backward prediction. mb_pattern is a flag indicating the presence or absence of a non-zero DCT coefficient of the macroblock.

[0388] The coded_block_pattern is the DCT of the macroblock.
This flag indicates whether or not there is a non-zero coefficient for each DCT block. mb_intra is a flag indicating whether the macroblock is intra_macro or not. slice_start is macrobl
This is a flag indicating whether or not ock is the head of slice. dc
t_type is a flag indicating whether the macroblock is field_dct or flame_dct.

Mb_quant is a macroblock whose quantizer_scal
This is a flag indicating whether or not to transmit e_code. skipped_
mb is a flag indicating whether the macroblock is a skipped macroblock.

[0390] These items are not always necessary, and necessary items change depending on the application. For example, items such as num_coef_bits and slice_start are necessary for an application having a request of “transparent” to restore a bit stream after re-encoding as much as possible. In other words, these items are not necessary in an application that changes the bit rate. Also, there are applications in which when the transmission path is extremely restricted, it is only necessary to know the coding type of each picture. From such a situation, for example, a combination as shown in FIG. 49 can be considered as an example of a combination of items for transmitting history information.

In FIG. 49, a value “2” corresponding to an item in each combination means that the information exists and can be used, and “0” means that the information does not exist. I do. “1” indicates that the information itself has no meaning, such as for the purpose of assisting the existence of other information or in the syntax, but having no relation to the original bit stream information. . For example, slice_start is
In the macroblock at the head of the slice when transmitting the history information, it becomes “1”. However, if the slice is not necessarily in the same positional relationship with the original bit stream, it is meaningless as the history information. become.

In the example of FIG. 49, (num_coef_bit
s, num_mv_bits, num_other_bits), (q_scale_code,
q_scale_type), (motion_type, mv_vert_field_sel []
[], Mv [] [] []), (mb_mfwd, mb_mbwd), (mb_patter
n), (coded_block_pattern), (mb_intra), (slic
e_start), (dct_type), (mb_quant), (skipped_m)
Five combinations of combinations 1 to 5 are prepared depending on the presence or absence of each item of b).

The combination 1 is a combination for reconstructing a completely transparent bit stream. According to this combination, highly accurate transcoding by using the generated code amount information can be realized. The combination 2 is also a combination for reconstructing a completely transparent bit stream. Combination 3 is a combination that cannot completely reconstruct a completely transparent bit stream, but allows a visually almost transparent bit stream to be reconstructed. Combination 4 is tran
Inferior to combination 3 in terms of sparent,
This is a combination that can reconstruct the bit stream without any visual problem. The combination 5 is inferior to the combination 4 from the viewpoint of transparency, but is a combination that allows incomplete reconstruction of the bit stream with a small amount of history information.

Of these combinations, the smaller the number of the combination is, the higher the function is, but the larger the capacity required to transfer the history is. Therefore, it is necessary to determine the combination to be transmitted by considering the assumed application and the capacity available for the history.

Next, the operation of the transcoding system 1 of FIG. 48 will be described with reference to the flowchart of FIG. In step S41, the decoding device 102 of the transcoding system 1 decodes the input bit stream, extracts an encoding parameter (4th) used when encoding the bit stream, and extracts the encoding parameter (4th). 4th) is output to the history information multiplexing device 103, and the decoded video data is also output to the history information multiplexing device 103. In step S42, the decoding device 102 also
Extract user_data from the input bit stream,
Output to the history decoding device 104. In step S43, the history decoding device 104 extracts combination information (descriptor) from the input user_data, and further uses the combination information (descriptor) to extract encoding parameters (1st, 2nd, 3rd) as history information. , To the history information multiplexer 103.

[0396] In step S44, the history information multiplexing apparatus 103 outputs the current encoding parameter (4) supplied from the decoding apparatus 102 extracted in step S41.
th) and the past encoding parameters (1st, 2n) output by the history decoding device 104 in step S43.
d, 3rd) are multiplexed with baseband video data supplied from the decoding device 102 according to a format as shown in FIG. 7 or FIG. 35, and output to the history information separating device 115.

[0397] In step S45, the history information separation device 115 extracts the coding parameters from the baseband video data supplied from the history information multiplexing device 103, and from these, extracts the code most suitable for the current coding. An encoding parameter (for example, a second-generation encoding parameter) is selected and output to the combination descriptor separating unit 511 together with the descriptor. Also, the history information separating device 115
Are the encoding parameters other than the encoding parameters determined to be optimal for the current encoding (for example, if the optimal encoding parameter is determined to be the second-generation encoding parameter, the other The first, third, and fourth generation encoding parameters) are output to the history encoding device 117. History encoding device 1
In step S46, user_data 17 encodes the encoding parameter input from the history information separation device 115.
And user_data (converted_history_strea
m ()) is output to the encoding device 116.

The combination descriptor separation unit 511 of the coding parameter selection circuit 501 separates the coding parameter and the descriptor from the data supplied from the history information separation device 115, and sets the coding parameter (2nd) of the switch 513 Supply to one contact. To the other contact of the switch 513, an encoding parameter calculation unit 512 calculates and supplies an encoding parameter from baseband video data output by the history information separation device 115. The switch 513 determines in step S48 the encoding parameter output from the combination descriptor separating unit 511 or the encoding parameter output from the encoding parameter calculating unit 512, corresponding to the descriptor output from the combination descriptor separating unit 511. And outputs it to the encoding device 116. That is, in the switch 513, the combination descriptor separating unit 51
If the encoding parameter supplied from 1 is valid, the encoding parameter output by the combination descriptor separation unit 511 is selected, but the encoding parameter output by the combination descriptor separation unit 511 is invalid. Is determined, the encoding parameter calculated by the encoding parameter calculation unit 512 processing the baseband video is selected. This selection is made according to the capacity of the transmission medium.

[0399] In step S49, the encoding device 116 encodes the baseband video signal supplied from the history information separating device 115 based on the encoding parameter supplied from the switch 513. In step S50, the encoding device 116 adds the history encoding device 11 to the encoded bit stream.
7 is multiplexed and output from the user_data supplied from 7.

[0400] In this way, transcoding can be performed without any problem even when the combinations of the encoding parameters obtained for each history are different.

As described above, the history information is, as shown in FIG. 18, history_stream () (more precisely, converted_hi
transmitted by story_stream ()). That history_stream ()
Is as shown in FIG. Descriptors (red_bw_flag, red
_bw_indicator) and items that are not supported in MPEG stream (num_other_bits, num_mv_bits, n
um_coef_bits) is re_coding_stream in FIG.
It is transmitted by the _info () function.

[0402] The re_coding_stream_info () function is shown in FIG.
User_data_start_code, re_coding_stre
am_info_ID, red_bw_flag, red_bw_indicator, marker_
It is composed of data elements such as bit, num_other_bits, num_mv_bits, and num_coef_bits.

[0404] user_data_start_code is a start code indicating that user_data starts. re_coding_st
ream_info_ID is a 16-bit integer.
Used to identify the stream_info () function. Specifically, the value is set to “1001 00011110 1100” (0x91ec).

[0404] red_bw_flag is a 1-bit flag, which is set to 0 when the history information transmits all items, and is followed by this flag when the value of the flag is 1.
By examining the red_bw_indicator, it is possible to determine which of the five combinations shown in FIG. 49 the item is sent.

[0405] red_bw_indicator is a 2-bit integer, and describes a combination of items as shown in FIG.

That is, in the case of combination 1 among the five combinations shown in FIG. 49, red_bw_flag is set to 0,
For combinations 2 to 5, red_bw_flag is set to 1. On the other hand, red_bw_indicator is set to 0 for combination 2, 1 for combination 3, 2 for combination 4, and 3 for combination 5.

[0407] Therefore, red_bw_indicator is equivalent to red_bw_fla
It is defined when g is 1 (for combinations 2 to 5).

Further, as shown in FIG. 51, red_bw_fla
When g is 0 (in the case of combination 1), marker_bit, num_other_bits, num_mv_bit
s and num_coef_bits are described. These four data elements are used in combination 2 to combination 5 (re
Not specified if d_bw_flag is 1).

[0409] As shown in Fig. 39, the picture_data () function is composed of one or more slice () functions. However, in the case of the combination 5, syntax elements lower than that including the picture_data () function are not transmitted (FIG. 49). In this case, the history information is pi such as picture_type.
This is intended to transmit information in units of cture.

In the case of combinations 1 to 4, the slice () function shown in FIG. 40 exists. However,
The slice position information determined by the slice () function and the slice position information of the original bit stream depend on a combination of items of history information. In the case of combination 1 or combination 2, the position information of the slice of the bit stream that is the source of the history information and the position information of the slice determined by the slice () function need to be the same.

[0411] The syntax element of the macroblock () function shown in Fig. 41 depends on the combination of items of history information.
macroblock_escape, macroblock_address_increment, m
The acroblock_modes () function always exists. However, macroblock_escape and macroblock_address_incremen
The validity of t as information is determined by the combination. When the combination of items of history information is combination 1 or combination 2, skip of the original bit stream
The same ped_mb information needs to be transmitted.

In the case of the combination 4, the motion_vectors () function does not exist. In the case of the combinations 1 to 3, the existence of the motion_vectors () function is determined by the macroblock_type of the macroblock_modes () function. In the case of combination 3 or combination 4, coded_block_pa
There is no ttern () function. In case of combination 1 and combination 2, macroblock_type of macroblock_modes () function
Determines the existence of the coded_block_pattern () function.

[0413] The syntax element of the macroblock_modes () function shown in Fig. 42 depends on the combination of items of history information. macroblock_type always exists. If the combination is combination 4, flame_motion_type, fie
ld_motion_type and dct_type do not exist.

The validity of the parameter obtained from the macroblock_type as information is determined by a combination of items of history information.

When the combination of the items of the history information is combination 1 or combination 2, macroblock_quant
Must be the same as the original bitstream. Macroblock_qua for combination 3 or combination 4
nt indicates the presence of quantizer_scale_code in the macroblock () function, and need not be the same as the original bitstream.

When the combination is combination 1 to combination 3, macroblock_motion_forward and macrobloc
k_motion_backward needs to be the same as the original bitstream. If the combination is combination 4 or combination 5, it is not necessary.

When the combination is combination 1 or combination 2, macroblock_pattern needs to be the same as the original bit stream. In the case of combination 3, macroblock_pattern is used to indicate the presence of dct_type. When the combination is combination 4, the relationship as in the case of combinations 1 to 3 is not established.

When the combinations of items of the history information are combination 1 to combination 3, macroblock_intra needs to be the same as the original bit stream. In the case of combination 4, this is not always the case.

[0419] The history_stream () in Fig. 27 is a syntax when the history information is of variable length. As shown in Figs. 20 to 26, when the syntax is of fixed length, the history_stream () is of a fixed length. In the descriptor (red_bw_flag) as information indicating which of the transmitted items is valid
And red_bw_indicator) are superimposed on the baseband image and transmitted. As a result, by examining this descriptor, it is possible to determine that the field exists as a field but its contents are invalid.

For this reason, as shown in FIG.
As g_stream_information, user_data_start_code,
re_coding_stream_info_ID, red_bw_flag, red_bw_indi
cator and marker_bit are arranged. Each meaning is the same as in FIG.

As described above, by transmitting the elements of the encoding parameter transmitted as the history in a combination corresponding to the application, the history corresponding to the application can be transmitted with an appropriate data amount.

As described above, when transmitting the history information as a variable length code, the re_coding_stream_info () function is configured as shown in FIG. 51, and as shown in FIG.
Transmitted as part of the ory_stream () function. On the other hand, when the history information is transmitted as a fixed-length code, as shown in FIG. 24, re_coding_stream_information () is transmitted as a part of the history_stream () function. In the example of FIG. 24, as re_coding_stream_information,
user_data_start_code, re_coding_stream_info_ID, re
d_bw_flag and red_bw_indicator are transmitted.

The history information multiplexing device 1 shown in FIG.
In order to transmit the history information in the baseband signal output from the base station 03, Re_Coding inform as shown in FIG.
ation Bus macroblock format is specified. This macro block is composed of 16 × 16 (= 256) bits. In FIG. 53, 32 bits shown in the third and fourth rows from the top are set as picrate_element. The picture rate elements shown in FIGS. 54 to 56 are described in the picrate_element. In the second row from the top in FIG. 54, 1-bit red_bw_flag is defined.
In the third row, a 3-bit red_bw_indicator is defined. That is, these flags red_bw_flag, red_bw_in
The indicator is transmitted as picrate_element in FIG.

Explaining the other data in FIG. 53, SRIB_sync_code is a code indicating that the first row of a macroblock in this format is left-justified, and specifically, is set to “11111”. Is done. fr_fl_SRIB is set to 1 when picture_structure is a frame picture structure (when its value is “11”), and Re_Coding Information Bus macr
Indicates that oblock is transmitted over 16 lines, and pi
If the structure_structure is not a frame structure, it is set to 0, which means that the Re_Coding Information Bus is transmitted over 16 lines. By this mechanism, Re_C
The oding information bus is locked to the corresponding pixels of the video frame or field that have been decoded spatially and temporally.

[0425] SRIB_top_field_first is set to the same value as top_field_first held in the original bit stream, and the Re_Coding Information Bus of the related video is set.
Is shown together with repeat_first_field. SRIB_repeat_first_field is set to the same value as repeat_first_field held in the original bit stream. Re_Coding Information B of first field
The contents of us need to be repeated as indicated by this flag.

[0426] 422_420_chroma indicates whether the original bit stream is 4: 2: 2 or 4: 2: 0. A value of 0 indicates that the bit stream is 4: 2: 0 and that the up-sampling of the chrominance signal has been performed such that a 4: 2: 2 video is output. The value 0 indicates that the filtering process of the color difference signal has not been executed.

[0447] rolling_SRIB_mb_ref represents a 16-bit modulo 65521, and this value is incremented for each macroblock. This value needs to be continuous over the frames of the frame picture structure. Otherwise, this value must be contiguous across fields. This value is initialized to a predetermined value between 0 and 65520. This allows a unique Re_Coding Information Bus identifier to be incorporated into the recorder system.

[0428] The meanings of the other data of the Re_Coding Information Bus macroblock are as described above, and are therefore omitted here.

As shown in FIG. 57, the data of the 256-bit Re_Coding Information Bus in FIG. 53 is Cb [0] [0], Cr [0] [0], and Cb, which are LSBs of color difference data one bit at a time.
[1] [0], Cr [1] [0]. Since 4-bit data can be transmitted by the format shown in FIG. 57, the 256-bit data of FIG. 53 can be transmitted by transmitting 64 (= 256/4) data of the format of FIG.

According to the transcoding system of the present invention, since the encoding parameters generated in the past encoding process are reused in the current encoding process, the decoding process and the encoding process are repeatedly performed. Even if it does, the image quality does not deteriorate. That is, it is possible to reduce accumulation of image quality deterioration due to repetition of decoding processing and encoding processing.

[0431] According to the transcoding system of the present invention, the coding parameters generated in the past coding processing are described in the user data area of the coded stream generated in the current coding processing. Since the bit stream thus obtained is an encoded stream conforming to the MPEG standard, any existing decoder can perform decoding processing. Furthermore, according to the transcoding system of the present invention, there is no need to provide a dedicated line for transmitting the encoding parameters in the past encoding process, so that the conventional data stream transmission environment can be used as it is. Thus, the past coding parameters can be transmitted.

According to the transcoding system of the present invention, the encoding parameters generated in the past encoding processing are selectively described in the encoded stream generated in the current encoding processing. Therefore, past encoding parameters can be transmitted without extremely increasing the bit rate of the output bit stream.

According to the transcoding system of the present invention, an encoding parameter optimal for the current encoding process is selected from the past encoding parameters and the current encoding parameters to perform the encoding process. Therefore, even if the decoding process and the encoding process are repeated, the deterioration of the image quality is not accumulated.

According to the transcoding system of the present invention, an encoding parameter optimal for the current encoding processing is selected from the past encoding parameters according to the picture type, and the encoding processing is performed. Therefore, even if the decoding process and the encoding process are repeated, the image quality degradation is not accumulated.

According to the transcoding system of the present invention, whether to reuse the past coding parameters is determined based on the picture type included in the past coding parameters. Processing can be performed.

The counter 101, the counter 113,
Although the counter 362 and the counter 364 have been described as being binary counters, they may be gray code (cyclic binary code) counters.

[0437] Also, the video decoding system 11 or the decoding device 102 outputs a baseband digital video signal, and outputs the baseband digital video signal.
Has been described as inputting a baseband digital video signal, but the video decoding system 11 or the decoding device 1
02 outputs an analog video signal, and the video encoding system 12 or the encoding device 116 may receive the analog video signal.

[0438] Although the counter value has been described as being multiplexed on the image, it may be multiplexed on a signal associated with the image, for example, an audio signal.

[0439] The computer program for performing each of the above processes is provided by being recorded on a recording medium such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory. It can be provided by recording on a user's recording medium.

[0440]

The stream generation device according to claim 1,
A stream generation method according to claim 13, and claim 1
According to the recording medium described in No. 4, the encoding history in the past encoding process of the first stream is detected, the discontinuity of the image of the first stream is detected, the detection result of the encoding history and the image Since the second stream is generated based on the first stream using the detection result of discontinuity, even if the decoding process, the encoding process, and the editing process are repeated, the image quality does not deteriorate. .

[Brief description of the drawings]

FIG. 1 is a diagram illustrating editing of an image.

FIG. 2 is a diagram illustrating overflow of a VBV buffer.

FIG. 3 is a block diagram showing a configuration of a transcoding system 1 to which the present invention has been applied.

FIG. 4 is a block diagram showing a more detailed configuration of the transcoding system 1 of FIG.

FIG. 5 is a decoder 2 built in the decoding device 102 in FIG. 3;
It is a block diagram which shows the structure of 51.

FIG. 6 is a diagram illustrating pixels of a macroblock.

FIG. 7 is a diagram illustrating an area where an encoding parameter is recorded.

FIG. 8 is a diagram illustrating an example of an ancillary data packet.

FIG. 9 is a diagram illustrating data stored by dividing a counter value.

FIG. 10 is a diagram illustrating data stored by dividing a counter value.

FIG. 11 is a diagram for explaining data stored by dividing a counter value.

FIG. 12 shows a configuration of a function corresponding to the history information multiplexing device 103 and the counter value multiplexing device 105 when multiplexing the counter value to the encoding parameter multiplexed in the LSB of the baseband digital video signal. FIG.

FIG. 13 shows a history information multiplexing device 103 and a counter value multiplexing device 105 when multiplexing a counter value with an encoding parameter multiplexed in a blanking portion of luminance or color difference of a baseband digital video signal.
FIG. 3 is a diagram illustrating a configuration of a function corresponding to FIG.

FIG. 14 is a block diagram illustrating a configuration of an encoder 301 incorporated in the encoding device 116 in FIG.

FIG. 15 is a diagram showing a state where the transcoding system 1 of FIG. 3 is actually used.

FIG. 16 is a block diagram illustrating a configuration of a transcoding system 1 that is tightly coupled.

FIG. 17 is a diagram illustrating a configuration example of a counter 362.

FIG. 18 is a diagram illustrating the syntax of a stream of a video sequence.

FIG. 19 is a diagram illustrating the configuration of the syntax in FIG. 18.

FIG. 20 is history_stream for recording fixed-length history information.
It is a figure explaining the syntax of ().

FIG. 21 is history_stream for recording fixed-length history information.
It is a figure explaining the syntax of ().

FIG. 22 history_stream for recording fixed-length history information
It is a figure explaining the syntax of ().

FIG. 23: history_stream for recording fixed-length history information
It is a figure explaining the syntax of ().

FIG. 24: history_stream for recording fixed-length history information
It is a figure explaining the syntax of ().

FIG. 25: history_stream for recording fixed-length history information
It is a figure explaining the syntax of ().

FIG. 26 shows history_stream for recording fixed-length history information.
It is a figure explaining the syntax of ().

FIG. 27 shows history_stream for recording variable-length history information.
It is a figure explaining the syntax of ().

FIG. 28 is a diagram for describing the syntax of sequence_header ().

FIG. 29 is a diagram for describing the syntax of sequence_extension ().

Fig. 30 is a diagram for describing the syntax of extension_and_user_data ().

FIG. 31 is a diagram for describing the syntax of user_data ().

Fig. 32 is a diagram illustrating the syntax of group_of_pictures_header ().

FIG. 33 is a diagram for describing the syntax of picture_header ().

Fig. 34 is a diagram for describing the syntax of picture_coding_extension ().

FIG. 35 is a diagram illustrating the syntax of extension_data ().

FIG. 36 is a diagram for describing the syntax of quant_matrix_extension ().

Fig. 37 is a diagram for describing the syntax of copyright_extension ().

[Fig. 38] Fig. 38 is a diagram for describing the syntax of picture_display_extension ().

Fig. 39 is a diagram for describing the syntax of picture_data ().

[Fig. 40] Fig. 40 is a diagram for describing the syntax of slice ().

FIG. 41 is a diagram illustrating the syntax of macroblock ().

Fig. 42 is a diagram for describing the syntax of macroblock_modes ().

Fig. 43 is a diagram illustrating the syntax of motion_vectors (s).

FIG. 44 is a diagram illustrating the syntax of motion_vector (r, s).

[Fig. 45] Fig. 45 is a diagram illustrating a macroblock_type variable-length code for an I picture.

[Fig. 46] Fig. 46 is a diagram for describing a macroblock_type variable length code for a P picture.

[Fig. 47] Fig. 47 is a diagram illustrating a macroblock_type variable length code for a B picture.

FIG. 48 is a block diagram showing another configuration of the transcoding system 1 to which the present invention has been applied.

FIG. 49 is a diagram illustrating a combination of items of history information.

FIG. 50 is a flowchart illustrating the operation of the transcoding system 1 in FIG. 48.

Fig. 51 is a diagram for describing the syntax of re_coding_stream_info ().

FIG. 52 is a view for explaining red_bw_flag and red_bw_indicator.

FIG. 53: Re_Coding Information Bus macroblock for
It is a figure explaining mation.

FIG. 54 is a diagram illustrating Picture rate elements.

FIG. 55 is a diagram illustrating Picture rate elements.

FIG. 56 is a diagram illustrating Picture rate elements.

[Fig. 57] Fig. 57 is a diagram illustrating an area where a Re_Coding Information Bus is recorded.

[Explanation of symbols]

1 transcoding system, 11 video decoding system, 12 video encoding system, 101
Counter, 102 decoding device, 103 history information multiplexing device, 104 history decoding device,
105 counter value multiplexer, 111 counter value separator, 112 format converter, 113
Counter, 114 comparing device, 115 history information separating device, 116 encoding device, 117 history encoding device, 201 user data decoder, 202 converter, 203 history V
LD, 211 history VLC, 212 converter, 213 user data formatter, 251
Decoder, 261 reception buffer, 262 variable length decoding circuit, 263 inverse quantization circuit, 264 IDCT circuit, 265 arithmetic unit, 266 motion compensation circuit, 26
7 frame memory, 271 timing signal generator, 272 counter value format converter, 2
73 encoding parameter format converter, 274
Serial-parallel converter, 275 switch,
281 timing signal generation circuit, 282 switch, 301 encoder, 310 motion vector detection circuit, 311 frame memory, 312 Frame / Fi
eld prediction mode switching circuit, 313 arithmetic unit, 31
5 Frame / FieldDCT mode switching circuit, 316 DC
T circuit, 317 quantization circuit, 318 variable length coding circuit, 319 transmission buffer, 320 inverse quantization circuit, 321 IDCT circuit, 322 arithmetic unit, 323
Frame memory, 324 motion compensation circuit, 330
Controller, 351 SDTI, 361 format converter, 362 counter, 363 comparator, 364 counter, 365 format converter, 381 counter, 382 AND circuit

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) H04N 7/32 F term (Reference) 5C053 FA03 FA14 FA22 GB06 GB08 GB17 GB21 GB29 GB37 LA15 5C059 KK01 KK35 KK39 KK41 MA01 MA15 MA23 PP05 PP06 PP07 RC00 RC24 RC31 RE09 SS02 TA16 TB03 TC27 TC37 TC41 TC42 UA02 UA05 5C063 AB03 AC01 CA11 CA23 DA07 DA13 DB01

Claims (14)

[Claims] 1. A stream generation device for generating and outputting a second stream from an input first stream, wherein a first detection for detecting an encoding history in a past encoding process of the first stream is performed. Means, second detection means for detecting discontinuity of the image of the first stream, and the first detection means using a detection result of the first detection means and a detection result of the second detection means. Generating means for generating the second stream based on the stream. 2. The stream generation apparatus according to claim 1, wherein the generation unit generates the second stream by encoding in an MPEG system. 3. The stream generation apparatus according to claim 1, further comprising: a first description unit that describes an encoding history of the second stream in an encoding process performed so far. 4. The first description means stores the information of the encoding history as a user of the second stream of the MPEG system.
The stream generation apparatus according to claim 3, wherein the stream generation apparatus describes the data as _data. 5. The stream generation apparatus according to claim 1, further comprising: a second description unit that describes continuity information indicating continuation of an image with respect to the second stream. 6. The stream according to claim 5, wherein the second description unit describes the continuity information indicating the continuation of the image as user_data of the second stream of the MPEG system. Generator. 7. The stream generation apparatus according to claim 5, wherein said second description means generates and describes continuous information by addition or subtraction for each access unit. 8. The apparatus according to claim 5, wherein the second description unit describes the continuous information in a blanking portion of a predetermined bit of a luminance signal or a color difference signal of the decoded image signal. Stream generator. 9. The stream generation apparatus according to claim 5, wherein the second description unit describes the continuous information in a blanking portion of the decoded image signal. 10. The stream generation device according to claim 5, wherein the continuous information is packetized. 11. The continuity information is added to the information of the coding history multiplexed in a blanking portion of a predetermined bit of a luminance signal or a chrominance signal of a decoded image signal. The stream generating apparatus according to claim 5, wherein the stream is multiplexed. 12. The apparatus according to claim 1, wherein the second description unit multiplexes the continuous information with information of the coding history multiplexed in a blanking portion of a decoded image signal. 6. The stream generation device according to 5. 13. A second stream from the input first stream.
In the stream generation method of the stream generation device that generates and outputs the stream of the first stream, a first detection step of detecting an encoding history in a past encoding process of the first stream; A second detection step of detecting a discontinuity; and a detection result of the processing of the first detection step and a detection result of the processing of the second detection step, based on the first stream. A generating step of generating the second stream. 14. A second stream from an input first stream.
A program that controls a stream generation device that generates and outputs a stream of a first stream, a first detection step of detecting an encoding history in a past encoding process of the first stream, and an image of an image of the first stream. A second detection step of detecting a discontinuity; and a detection result of the processing of the first detection step and a detection result of the processing of the second detection step, based on the first stream. Generating a second stream, the computer-readable recording medium storing a computer-readable program.