JP3513277B2 - Video encoding device and video decoding device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video encoding apparatus for reducing the amount of information contained in a video (moving image) signal and encoding the same, and decoding the encoded information to generate a video signal. A video decoding device for restoration, in particular, by obtaining a motion vector for each pixel from a representative motion vector,
The present invention relates to a video encoding device and a video decoding device that use a motion vector interpolation prediction method that performs inter-picture prediction in pixel units.

[0002]

2. Description of the Related Art In recent years, with the development of high-speed digital signal processing and LSI technology therefor, and the progress of image processing technology, effective use of image information is expected. In the field of communication, video communication services such as videophone, videoconference, and video database service have been put into practical use due to the development and spread of digital networks represented by ISDN. Furthermore, with the development and popularization of mobile communication networks and the progress of digitalization, realization of mobile video communication services is expected. In general, the amount of information included in video information is very large, and it is not realistic to handle the video signal as it is. However, since the video signal contains a lot of redundancy in the amount of information, it is possible to reduce the amount of information by removing this redundancy. Particularly, in an environment with a small transmission line capacity such as a mobile communication network, a very highly efficient (compression) encoding technique for video is important. To that end, ITU-T and ISO / IEC are energetically working on international standardization of video coding systems at ultra-low bit rates.

A video signal is a spatial information related to the temporal information due to a change in motion contained in the video and the contents of one screen (image frame or image field, both of which are collectively referred to as an image frame) signal. Information is present, and each has redundancy. Therefore, in the motion-compensated inter-frame prediction orthogonal transform coding method which has been widely used in recent years, after the temporal redundancy is removed by the motion-compensated inter-frame prediction, the orthogonal transform coding is further performed on the inter-frame prediction error signal. Has a hybrid configuration that removes spatial redundancy. FIG. 6 shows the principle of the above-described motion compensation interframe prediction orthogonal transform coding method. The motion-compensated inter-frame prediction unit 91 creates a prediction value of the input video signal from the already coded video signal stored in the frame memory 93, and calculates the difference between the prediction value and the input video signal. Output as a prediction error signal. In the prediction error coding unit 92, the prediction error signal is coded by a method such as orthogonal transformation and the redundancy is further suppressed. The encoded prediction error signal is locally decoded, stored in the frame memory 93, and used for prediction of the next image frame.

In ultra low bit rate video coding,
It is necessary to express the video signal with a very small amount of information.
Therefore, the amount of information assigned to the orthogonal transform coding, that is, the coding of the prediction error signal is greatly limited. Therefore, it is considered to be very important to improve the efficiency in the inter-frame prediction, in other words, a prediction method that can more accurately predict the temporal change of the video signal. For this reason,
Recently, interframe prediction methods using affine transformation and bilinear transformation have been actively studied. In the above motion-compensated inter-frame prediction method, the motion included in the video is represented as a parallel movement by the motion vector of each unit area, whereas in the method using the affine transformation or the bilinear transformation, In addition, since rotation, enlargement / reduction, deformation, etc. can be expressed, the motion of the image can be represented more accurately, and the prediction efficiency is improved.

FIG. 7 is a schematic block diagram of an inter-frame prediction unit of a conventional video coding apparatus using affine transformation / bilinear transformation. The inter-frame prediction unit of the conventional video encoding device is
A frame memory unit 31 for storing the already encoded video signal, and a representative motion vector is calculated for each unit area between the input video signal and the video signal read from the frame memory unit 31. Motion vector detector 3
2. A motion vector interpolation unit 33 that calculates a motion vector for each pixel from the representative motion vector, and a predicted image signal is created from the video signal read from the frame memory unit 31 using the motion vector for each pixel Pixel value prediction unit 3
4 and.

The outline of the operation of each unit will be described below. The frame memory unit 31 stores already encoded video signals as reference image frames for inter-frame prediction. The motion vector detecting unit 32 receives the image frame signal that is the current encoding target and reads out the reference image frame stored in the frame memory unit 31. The motion vector detection unit 32 divides the encoding target image frame into unit areas, and searches for a unit area most similar to the area in the reference image frame for each unit area.
As a result, the position of the area within the frame to be encoded,
The displacement with respect to the position of the searched reference image frame area is output as a motion vector. The motion vector is a representative motion vector that represents the inter-frame displacement of a representative point (generally the center of the region) in the unit area. An example of the relationship between the representative point and the motion vector search unit area is shown in FIG. Also,
At the time of the above-mentioned area search, the sum of absolute differences, the sum of squared differences, or the like of each pixel value in the area is used as an evaluation measure of similarity between areas. Further, in order to obtain the displacement relative to the representative point more accurately, the difference between the pixel values is multiplied by a coefficient having a large value in the vicinity of the center of the region and a coefficient having a small value in the vicinity of the area, and then the sum is calculated. Weighting of the central part may be performed.

Next, the representative motion vector is input to the motion vector interpolation unit 33. The motion vector interpolation unit 33
Then, the motion vector for each pixel is obtained using the representative motion vector. At this time, in the case of affine transformation, the motion vector for each pixel in a triangular area surrounded by three neighboring representative points (hereinafter referred to as a transform unit area) solves the affine transformation formula from the representative motion vector of each representative point. Calculated by Further, in the case of the bilinear transformation, the motion vector for each pixel in the transformation unit area of the quadrangle surrounded by four neighboring representative points is calculated by solving the bilinear transformation formula from the representative motion vector of each representative point. Is output.
When the conversion unit area is a square or a rectangle, this is equivalent to proportionally distributing the motion vector value of the representative point in each of the horizontal direction and the vertical direction according to the distance between the pixel of interest and the representative point. . This state is shown in FIG.
The pixel motion vector is input to each pixel (target pixel) to the pixel value prediction unit 34, and the pixel value of the corresponding position from the frame memory unit 31 is used as the displacement of the target pixel from the reference image frame. Is read as the predicted value of the pixel of interest to form a predicted frame. At this time, when a position where no pixel exists in the reference image frame is shown, for example, when the pixel motion vector (displacement) is a decimal value, the neighboring pixel value in the reference image frame is read, The predicted value of the pixel of interest is obtained as an interpolation value by a method such as bilinear interpolation.

FIG. 8 shows a schematic block diagram of an inter-frame prediction unit of a conventional video decoding apparatus using affine transformation / bilinear transformation. The inter-frame prediction unit of the conventional video decoding device is
A frame memory unit 41 for storing the already-decoded video signal, a motion vector interpolation unit 42 for calculating a motion vector for each pixel from a representative motion vector input for each unit area, and for each pixel And a pixel value prediction unit 43 that creates a predicted image signal from the video signal (reference image frame) read from the frame memory unit 41 by using the motion vector of. Frame memory unit 4
The operation of the motion vector interpolation unit 42 and the pixel value prediction unit 43 is performed by the frame memory unit 3 in the conventional video encoding device.
1, the motion vector interpolating unit 33 and the pixel value predicting unit 34 are similar in operation, and a predictive frame is created on the decoding side as well as on the encoding side.

[0009]

However, in the above-mentioned conventional video coding apparatus and video decoding apparatus, when the entire area surrounded by the representative points can be described by the same affine transformation / bilinear transformation parameter. Although it shows good characteristics, when the position of the representative point does not match the change included in the image content, for example, the position or movement of the subject,
The representative motion vector of the representative point represents different motions of different image contents, and the motion vector for each pixel obtained from the representative motion vector also does not represent an appropriate displacement with respect to the pixel. , The inter-frame prediction efficiency, that is, the coding efficiency is significantly reduced,
There was a problem. In view of the above problems, the video encoding device and the video decoding device of the present invention focus on a simple method by weighting each representative motion vector when obtaining a motion vector for each pixel from the representative motion vector. Provided are a video coding device and a video decoding device capable of improving interframe prediction efficiency and coding efficiency by reducing the influence of a motion vector irrelevant to pixels and obtaining a more accurate pixel motion vector. That is the purpose.

[0010]

SUMMARY OF THE INVENTION A first invention of the present application is a frame memory means for storing a video signal which has already been encoded and decoded, a video signal of an input image frame and a video of a reference image frame. Motion vector detecting means for obtaining a representative motion vector of each vertex of the conversion unit area between the signal and the signal, and a representative of the nearest neighborhood of the small area for each small area further divided by the predetermined method. Movement
Coutl and direction consistent with it, or direction and size
The representative motion vector indicating the
Direction, or direction and size
Reduce the weight of the representative motion vector you are pointing to.
In the weighting coefficient control means for determining the weighting coefficient when calculating the motion vector of each pixel in the small area, and using the determined weighting coefficient, the representative motion vector of each vertex of the conversion unit area Using a motion vector interpolation unit that changes the size and, from the representative motion vector whose size has been changed, interpolates the motion vector of each pixel in the small area, and the motion vector of each pixel, And a pixel value prediction unit that creates a predicted image frame from the video signal read from the frame memory unit, and transmits the representative motion vector obtained by the motion vector detection unit. In a second invention of the present application, the weighting coefficient control means prepares a plurality of weighting coefficient patterns in advance, and only the information indicating which weighting coefficient pattern has been selected is included in the motion vector interpolation means. It is characterized by instructing to. The third invention of the present application is characterized in that the weighting factor control means determines the weighting factor according to the direction of each representative motion vector. A fourth aspect of the present invention is characterized in that the weighting factor control means determines the weighting factor based on the vector value of each representative motion vector.

According to a fifth aspect of the present invention, a frame memory means for storing the already decoded video signal and a representative motion vector of each vertex of the transform unit area transmitted from the video coding device are input. Then, for each small area obtained by further dividing the conversion unit area according to a predetermined method ,
The nearest representative motion vector and its matching direction,
Rui is a representative motion vector pointing to direction and magnitude
Is a large weight and, conversely, is not consistent, or
Is a representative motion vector that points in direction and magnitude.
So as to reduce the body, the weight factor control means for determining a weighting factor in calculating the motion vector of each pixel in the small area, using the determined weighting coefficients, each vertex of the conversion unit area Motion vector interpolating means for interpolating and calculating the motion vector of each pixel in the small area from the representative motion vector whose size has been changed, and the motion of each pixel. Pixel value prediction means for creating a predicted image frame from the video signal read out from the frame memory means using a vector. In a sixth aspect of the present invention, the weighting factor control means prepares a plurality of weighting factor patterns in advance, and only the information indicating which weighting factor pattern is selected is sent to the motion vector interpolation means. Characterized by instructing. A seventh invention of the present application is a conversion unit area for a screen which has been dropped without being encoded by the video encoding apparatus, from the representative motion vector of each vertex of the conversion unit area for the screen encoded by the video encoding apparatus. Is provided with motion vector conversion means for calculating a representative motion vector of each vertex, and the representative motion vector calculated by the motion vector conversion means is used to correspond to a frame dropped without being encoded by the video encoding device. It is characterized in that a predicted image frame to be created is generated and this is output as an interpolated image signal.

In the above image coding apparatus, the frame memory unit stores the already coded video signal for use as a reference image frame in the subsequent inter-frame prediction. The motion vector detection unit reads the reference image frame from the frame memory unit, and for each unit region of the input encoding target image frame signal, searches for a portion most similar to the region in the reference image frame,
The displacement of the position of the area within the reference image frame is output as a motion vector. The weighting factor control unit determines a weighting factor for the motion vector for each unit area, and instructs the motion vector interpolating unit for the weighting factor for each representative motion vector. The weighting factors are determined to reduce the effect of each representative motion vector representing different motions of different image content so that an accurate displacement for the pixel is obtained. The motion vector interpolation unit calculates and outputs a motion vector of each pixel using the motion vector output from the motion vector detection unit and weighting to each motion vector instructed by the weighting coefficient control unit. . At this time, by weighting each motion vector value and combining methods such as affine transformation / bilinear transformation,
Since a more accurate pixel motion vector is required as compared with the conventional method, it is possible to improve interframe prediction efficiency and coding efficiency. The pixel value prediction unit reads the prediction frame by reading the pixel value of the corresponding position from the frame memory unit as a prediction value, using the motion vector of each pixel output from the motion vector interpolation unit as the displacement from the reference image frame. Constitute. In addition, the weighting factor control unit does not directly instruct the weighting factor itself for each motion vector, but prepares several weighting factor patterns in advance and selects the optimum one from them, It is also possible to instruct only the selection information of which pattern is selected to the motion vector interpolation means. Further, when determining the weighting factor, the weighting factor control unit considers only the direction of the motion vector, and when the weighting on the motion vector pointing in a peculiar direction is small or the direction of the motion vector has a large variation. By controlling such that the weighting is increased only for the motion vector closest to the pixel of interest, it is possible to accurately determine the weighting coefficient, that is, improve the inter-frame prediction efficiency only by simple processing. Further, the weighting factor control unit uses the vector value of the motion vector to determine the variation between the motion vectors, and to divide the motion vector having a peculiar value that is considered to represent the motion of different image contents,
By determining the weighting coefficient by performing the above, it is possible to obtain an accurate weighting coefficient for obtaining an accurate pixel motion vector, and it is expected that the interframe prediction efficiency is improved.

[0013]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An example of an embodiment of the present invention will be described below with reference to the drawings. FIG. 1 shows a configuration example of an interframe prediction unit of the video encoding device of the present invention. The video encoding device of FIG. 1 includes a frame memory unit 11 for storing an already encoded video signal, and an input video signal and a video signal read from the frame memory unit 11 between A motion vector detection unit 12 that obtains a representative motion vector for each unit area, a weight coefficient control unit 13 that determines and instructs a weight coefficient for the representative motion vector, and a motion vector for each pixel from the representative motion vector and the weight coefficient. A motion vector interpolation unit 14 for calculating, and a pixel value prediction unit 1 for creating a predicted image signal from a video signal read from the frame memory unit 11 using the motion vector for each pixel.
5 and. Of these, the frame memory unit 11, the motion vector detection unit 12, and the pixel value prediction unit 15
The operation of is similar to that of the conventional video encoding device. The video encoding device of the present invention is provided with a weighting coefficient control unit 13 for instructing a weighting coefficient for the representative motion vector, and the motion vector interpolation unit 14 outputs the representative motion vector output from the motion vector detection unit 12 and the weighting coefficient control. This is different from the conventional video encoding device in that the motion vector for each pixel is calculated and output using the weighting coefficient for each vector output from the unit 13. The operations of the weighting factor control unit 13 and the motion vector interpolation unit 14, which are the main parts of the present invention, will be described below. In this example, the motion vector interpolation unit 14 shows a case where the motion vector of a pixel is obtained by bilinear transformation, but the case of using affine transformation can be considered in the same way. Similar to the above-described conventional video encoding device, the processing in the motion vector interpolation unit 14 is performed for each conversion unit area. That is, the conversion unit area of the bilinear conversion is an area surrounded by four neighboring representative points (three representative points in the case of affine transformation) corresponding to the representative motion vector output from the motion vector detection unit 12. The weighting factor control unit 13 determines and instructs a weighting factor for the motion vector of each apex (representative point) of the conversion unit region, which is a processing unit in the motion vector interpolation unit 14. . At this time, regarding the pixels in the vicinity of the representative point, it can be considered that the motion vector of the nearest representative point has the strongest influence. Therefore, the conversion unit area is further divided into small areas, and different weighting coefficients are used for each small area. FIG. 9C shows an example in which the conversion unit area is divided into four rectangular small areas.
Shown in.

The weighting factor control unit 13 determines whether the motion vectors of the vertices of the conversion unit area represent the movement of the same image content (subject), in other words, the conversion unit area is located on one subject. Judge whether or not. if,
When it is determined that the same image content is represented, the motion vector interpolating unit 14 is instructed to equalize the weights of the representative motion vectors of the respective vertices (for example, all weight coefficients = 1). To do. On the contrary, if it is determined that the vertices of the conversion unit area exist on different image contents, the vertices on the same image content as the small area of interest represented by one vertex The weighting coefficient is determined so that the weighting of the motion vector of the above is increased and the weighting of the motion vector of the vertex considered to exist on different image contents is decreased. In extreme cases, do not use motion vectors on different image contents, weighting factor = 0
To the motion vector interpolation unit 14. This weighting factor determination processing is performed for each of the small regions, and a motion vector having a large weighting factor in a certain small region has a small weighting factor in another small region that is considered to correspond to different image content. The control is performed so that This state is shown in FIG. The weight coefficient output from the weight coefficient control unit 13 may be a continuous value represented by a real number, for example, 0, 0.25, 0.5,
It is also possible to select and specify one that is closest to the optimum value from the discretely defined values such as 0.75 and 1.

The motion vector interpolation unit 14 weights the representative motion vector with the weighting factor designated by the weighting factor control unit 13, and then calculates the motion vector of each pixel in the small area. Is output. As an example of the weighting method, the motion vector of each vertex of the transform unit area is MV _i (i = 1,2,3,4), and the weight to the motion vector is w _ij (i, j = 1,2,3). , 4) as

[0016]

[Equation 1]

Weighted motion vector W for
A method of calculating an ordinary bilinear transformation formula using MV _i , or a motion vector for a pixel at the position (x, y) obtained by bilinear transformation from the original motion vector MV _i
For MVP _xy , the weight to the motion vector is w _i (i =
1,2,3,4)

[0018]

[Equation 2]

A method of correcting the pixel motion vector MVP _xy into WMVP _xy in consideration of weighting is also conceivable, where w is a coefficient indicating the proportion of the weight as a whole. These processes in the motion vector interpolation unit 14 are executed for each small area using the weighting coefficient for each small area instructed by the weighting coefficient control unit 13.

Further, the weighting factor control unit 13 preliminarily sets a pattern of M weighting factors, for example, W = {
Wm} (m = 1,2, ... M) Wm = {Wm _ij } (i, j = 1,2,3,4) is prepared in advance, and the conversion unit area or small For each area, one closest to the weighting coefficient determined in the weighting coefficient determining process may be selected and designated as the optimum pattern. At this time, the weighting factor control unit 13 instructs the motion vector interpolating unit 14 only on the information of which pattern is selected. It is also possible to form a pattern by pairing a weighting coefficient for a certain small area and a weighting coefficient for another small area. In this case, by designating one pattern for each conversion unit area, The weighting factors for all the small areas in the conversion unit area can be designated at the same time. At this time, the motion vector interpolation unit 1
4 also holds the same weighting pattern as that of the weighting factor control unit 13, and by using the weighting factor for each motion vector according to the designated weighting factor pattern, the same processing as described above is performed for each pixel. Obtain and output the motion vector.

Next, a first operation example of the weighting factor control unit 13 in the video encoding apparatus of the present invention will be described with reference to the flowchart of FIG. First, the representative motion vector is input from the motion vector detection unit 12 to the weight coefficient control unit 13 (step S1). The weighting factor control unit 13 configures a conversion unit area from four neighboring representative points (step S).
2). Next, the signs of the values of the horizontal component and the vertical component of the representative motion vector of each vertex are examined, and only positive, negative, or zero are extracted, and the direction of each motion vector is roughly classified (step S3). The results are compared to determine whether the motion vectors of the four vertices are roughly matched in the directions (steps S4 and S5). As a result, when the variations are small and it is determined that the respective consistency is sufficient, it is determined that the same image content is represented, and uniform weighting factors are output (steps S6 and S9). on the other hand,
When it is determined that the variation is large, it is determined that the conversion unit area includes a plurality of different image contents, and the motion vector indicating a peculiar direction is separated and The division into groups is performed (step S7). Next, for each small region in the conversion unit region, the motion vector in the nearest neighborhood of the focused small region and the motion vector pointing in the direction matching with this motion vector are heavily weighted, and conversely they are divided into the other sets. A weighting coefficient is determined and output so that the weight on the motion vector or the motion vector determined to be a peculiar direction is reduced (steps S8 and S9).

Next, a second operation example of the weighting factor control unit 13 in the video encoding apparatus of the present invention will be described with reference to the flowchart of FIG. First, similarly to the first operation example, the representative motion vector is input from the motion vector detection unit 12 to the weighting coefficient control unit 13 (step S1), and a conversion unit area is constructed from four neighboring representative points (step S1). S
2). Next, the values of the horizontal and vertical components of the representative motion vector of each vertex are compared (steps S3 and S4), and it is determined whether the directions and sizes of the motion vectors of the four vertices are consistent (step S3). S5). As a result, when the variations are small and it is determined that the respective consistency is sufficient, it is determined that the same image content is represented, and uniform weighting factors are output (steps S6 and S9). on the other hand,
When it is determined that the variation is large, it is determined that the conversion unit area includes a plurality of different image contents, and the motion vectors indicating the peculiar directions and sizes are separated and the consistency is obtained. Division into combinations of motion vectors and the like are performed (step S7). Next, for each small area in the conversion unit area, the motion vector in the nearest neighborhood of the target small area and the motion vector divided into the same combination are given higher weights, and conversely they are divided into the other groups. The weight coefficient is determined and output so that the weight for the motion vector or the motion vector determined to be a peculiar value is reduced (step S8,
S9). In the second operation example, unlike the first operation example, not only the sign of the motion vector component but also the vector value itself is used. Therefore, the detailed direction determination and the magnitude in the same direction are performed. Different motion vectors can be detected, and more strict processing can be performed.

FIG. 2 shows a first configuration example of the interframe prediction section of the video decoding apparatus of the present invention. The video decoding apparatus of FIG. 2 determines a weighting coefficient for a representative motion vector input for each unit area, a frame memory unit 21 for storing a video signal already decoded, and a weighting coefficient control for instructing the weighting coefficient. Unit 22, a motion vector interpolating unit 23 that calculates a motion vector for each pixel from the representative motion vector and the weighting factor, and a predicted image by reading a video signal from the frame memory unit 21 using the motion vector for each pixel And a pixel value prediction unit 24 that creates a signal. Among these, the operations of the frame memory unit 21 and the pixel value prediction unit 24 are the same as those of the conventional video decoding device. The video decoding device of the present invention is provided with a weight coefficient control unit 22 for instructing a weight coefficient for the representative motion vector, and the motion vector interpolation unit 23 inputs the representative motion vector and the weight coefficient control unit 22 to output. In that the motion vector for each pixel is calculated and output using the weighting factor for each vector,
It is different from the conventional video decoding device.

The operations of the weight coefficient control unit 22 and the motion vector interpolation unit 23, which are the main parts of the present invention, will be described below. These operations are similar to those of the video encoding device of the present invention, and the same processing as that of the video encoding device is performed in the video decoding device.

In the weighting coefficient control unit 22, the weighting coefficient for the motion vector of each vertex (representative point) of the conversion unit area, which is the processing unit in the motion vector interpolation unit 23, is converted into the conversion unit area. The area is further divided and determined for each small area and designated. The weighting factor control unit 22 determines whether the motion vectors of the vertices of the conversion unit area represent the movement of the same image content (subject), in other words, whether the conversion unit area is located on one subject. To judge. If it is determined that the same image content is represented, the motion vector interpolating unit 23 is instructed to make the weights of the representative motion vectors of the respective vertices equal. On the contrary, if it is determined that the vertices of the conversion unit area exist on different image contents, the vertices on the same image content as the small area of interest represented by one vertex The weighting coefficient is determined so that the weighting of the motion vector of the above is increased and the weighting of the motion vector of the vertex considered to exist on different image contents is decreased. The processing and output of these weighting factor control units 22 are performed by the weighting factor control unit 1 in the video encoding device of the present invention.
This is the same as the processing and output of No. 3.

The motion vector interpolation unit 23 uses the weighting factor designated by the weighting factor control unit 22 in the same manner as the method of weighting the representative motion vector in the motion vector interpolation unit 14 of the video coding apparatus described above. In this way, the motion vector of each pixel in the small area is calculated and output. In addition, the weighting factor control unit 22 prepares M weighting factor patterns in advance, and selects the weighting factor determined in the above-described weighting factor determination process for each conversion unit region or each small region from these patterns. One closest
It is the same as in the case of the video encoding device that the optimum pattern can be selected and designated. At this time, the weight coefficient control unit 2
2 instructs the motion vector interpolating unit 23 only for information on which pattern is selected. At this time, the motion vector interpolating unit 23 also holds the same weighting pattern as the weighting factor control unit 22, and uses the weighting factor for each motion vector according to the designated weighting factor pattern as described above. By the same process, the motion vector for each pixel is obtained and output.

FIG. 5 shows a second configuration example of the interframe prediction section of the video decoding apparatus of the present invention. The video decoding apparatus shown in FIG. 5 calculates a representative motion vector for a frame dropped screen from a frame memory unit 81 for storing a decoded video signal and a representative motion vector for an encoded screen. A motion vector conversion unit 82, a weight coefficient control unit 83 for determining and instructing a weight coefficient for a representative motion vector input for each unit area, and a motion for calculating a motion vector for each pixel from the representative motion vector and the weight coefficient. A vector interpolation unit 84 and a pixel value prediction unit 85 that reads a video signal from the frame memory unit 81 using the motion vector for each pixel to create a predicted image signal. Of these, the frame memory unit 81,
The operations of the weight coefficient control unit 83, the motion vector interpolation unit 84, and the pixel value prediction unit 85 are the same as those of the video decoding device of the first configuration example. The video decoding device of the present configuration example receives the motion vector for the coded video frame,
This is different from the video decoding apparatus of the first configuration example in that a motion vector conversion unit 82 for outputting a motion vector of a video frame that has been dropped without being encoded is provided.

The operation of the motion vector converter 82 will be described below. The motion vector conversion unit 82 internally divides the value of the representative motion vector for the input coded video frame according to the temporal distance between adjacent coded video frames and the time position of the dropped frame. To do. For example,
It is assumed that the first video frame is coded and then the fifth video frame is coded. At this time, the three video frames from the second to the fourth are dropped. Here, consider a case where the representative motion vector for the fifth video frame is input. Since the first and fifth frames are coded, the temporal distance between the coded frames is 4. The motion vector for the fifth frame is considered to represent the motion for four frame time from the first frame, and the motion vector value is proportionally distributed to the frames dropped. That is, the motion vector value of the dropped frame of the second frame is obtained by multiplying the motion vector value of the fifth frame by 1/4. Similarly, the motion vector value obtained by multiplying 1/2 for the third frame and 3/4 for the fourth frame is output as the motion vector value of each frame-dropped video frame. . The motion vector output in this way is sent to the weight coefficient control unit 83 and the motion vector interpolation unit 84, and by the same processing as the video decoding device of the first configuration example,
Finally, a predicted image signal is obtained from the pixel value prediction unit 85, and the predicted image signal is output as an interpolated image signal for the frame-dropped video frame.

[0029]

As described above, according to the video encoder and the video decoder of the present invention, the following effects can be expected. In the video encoding device of the present invention, the weighting coefficient control means determines the weighting coefficient for each representative motion vector obtained by the motion vector detecting means, and instructs the motion vector interpolating means, thereby the motion vector interpolating means. Even if the conversion unit area does not match the image content, the position and movement of the subject, the effect of the motion vector unrelated to each pixel in the conversion unit area is reduced to obtain a more accurate pixel motion vector. Therefore, it is possible to improve interframe prediction efficiency and coding efficiency. Therefore,
Required for ultra low bit rate video communication,
It is possible to realize a very efficient video coding system. Further, in the video encoding device of the present invention, a plurality of patterns of weighting coefficients for each representative motion vector are prepared in advance in the weighting coefficient control means, and one of the patterns is selected to select the motion vector interpolation means. To the inter-frame prediction efficiency and coding efficiency by obtaining a more accurate pixel motion vector by reducing the influence of the motion vector unrelated to each pixel in the conversion unit area by the motion vector interpolation means. Can be realized by a simple method. Therefore, it is possible to provide a small-sized and inexpensive video encoding device by reducing the processing amount and the hardware / software scale in the highly efficient video encoding device. Furthermore, in the video encoding device of the present invention, in the weighting factor control means, depending on the direction of each representative motion vector,
By determining the weighting coefficient, the processing in the weighting coefficient control means can be greatly simplified. Therefore, it is possible to reduce the size and cost of the video encoding device having high inter-frame prediction efficiency and high encoding efficiency. Further, in the video encoding device of the present invention, the weighting coefficient control means determines the weighting coefficient according to the vector value of each representative motion vector, so that each pixel in the conversion unit area in the motion vector interpolation means is determined. It is possible to strictly determine a motion vector irrelevant to, and improve the accuracy of the pixel motion vector. Therefore, it is possible to further improve the interframe prediction efficiency and the coding efficiency in the video coding apparatus, and it is possible to realize the video coding at a lower bit rate. Further, in the video decoding device of the present invention, the weighting coefficient control means determines the weighting coefficient for each representative vector obtained by the motion vector detecting means, and instructs the motion vector interpolating means to perform the motion vector interpolation. Even if the conversion unit area in the means does not match the image content, the position and movement of the subject, the effect of the motion vector unrelated to each pixel in the conversion unit area is reduced to obtain a more accurate pixel motion vector. Therefore, the inter-frame prediction efficiency and the quality of the decoded video can be improved. Therefore, it is possible to realize a video decoding device that can obtain a high-quality decoded video signal even in ultra-low bit rate video communication. Further, in the video decoding apparatus of the present invention, the weighting coefficient control means prepares a plurality of patterns of weighting coefficients for each representative motion vector in advance, and one of the patterns is selected to select the motion vector interpolation means. By instructing the motion vector interpolation means to reduce the influence of the motion vector irrelevant to each pixel in the conversion unit area, and to obtain a more accurate pixel motion vector, the inter-frame prediction efficiency, decoded It is possible to improve the image quality with a simple method. Therefore,
It is possible to provide a small-sized and inexpensive video decoding device by reducing the processing amount and the hardware / software scale of the high-quality video decoding device. Further, in the video decoding device of the present invention, the motion vector conversion means calculates the representative motion vector for the frame that has been dropped from the encoded representative motion vector for the screen, and drops the frame without being encoded. Image frame corresponding to the displayed screen
By generating the above and outputting this as an interpolated image signal, the interpolated image signal can be obtained by an accurate pixel motion vector in which the influence of the motion vector irrelevant to each pixel in the conversion unit area is reduced. Therefore, it is possible to significantly improve the quality and time resolution of the image that is restored and output. Therefore, it is possible to realize a video decoding device that can obtain a decoded video signal of high quality and smooth motion even in ultra-low bit rate video communication.

[Brief description of drawings]

FIG. 1 is a schematic block diagram showing an embodiment of a video encoding device according to the present invention.

FIG. 2 is a schematic block diagram showing an embodiment of a video decoding device according to the present invention.

FIG. 3 is a flowchart showing a first operation example of the weighting factor control unit 13 in the video encoding device of the present invention.

FIG. 4 is a flowchart showing a second operation example of the weighting factor control unit 13 in the video encoding device of the present invention.

FIG. 5 is a schematic block diagram showing a configuration example of a video decoding device according to the present invention.

[Fig. 6] Fig. 6 is a block diagram for explaining the principle of a motion-compensated inter-frame prediction orthogonal transform coding method.

FIG. 7 is a schematic block diagram showing a configuration example of an interframe prediction unit of a conventional video encoding device.

FIG. 8 is a schematic block diagram showing a configuration example of an inter-frame prediction unit of a conventional video decoding device.

FIG. 9 is a diagram for explaining a relationship between a conversion unit area, a representative point, and a representative motion vector.

[Explanation of symbols]

11 frame memory section 12 Motion vector detector 13 Weighting factor control unit 14 Motion vector interpolation section 15 Pixel value predictor 21 Frame memory section 22 Weighting factor control unit 23 Motion Vector Interpolator 24 Pixel value predictor 81 frame memory section 82 Motion vector converter 83 Weighting coefficient control unit 84 Motion Vector Interpolator 85 Pixel value prediction unit 91 Motion Compensation Interframe Prediction Unit 92 Prediction error signal encoder 93 Frame memory section

Claims (7)

(57) [Claims] 1. A moving picture coding apparatus for coding the difference between a predicted image frame obtained by performing motion compensation interframe prediction and an input image frame as prediction error information, which has already been coded and decoded. Frame memory means for storing the video signal; motion vector detection means for obtaining a representative motion vector of each vertex of the conversion unit area between the video signal of the input image frame and the video signal of the reference image frame; A representative motion vector of the nearest neighborhood of the small area for each small area obtained by further dividing the conversion unit area according to a predetermined method.
And the direction that matches with this, or the direction and size.
The representative motion vector that has
Pointing in a direction that is not smooth, or in the direction and size
Weighting coefficient control means for determining a weighting coefficient when calculating the motion vector of each pixel in the small area so as to reduce the weight of the representative motion vector, using the determined weighting coefficient, A motion vector interpolation means for changing the size of the representative motion vector of each vertex of the conversion unit area, and interpolating the motion vector of each pixel in the small area from the changed representative motion vector; A pixel value prediction unit that creates a predicted image frame from the video signal read from the frame memory unit using the motion vector of each pixel, and transmits the representative motion vector obtained by the motion vector detection unit. A video encoding device characterized by: 2. The weighting factor control means prepares a plurality of weighting factor patterns in advance, and instructs only the information indicating which weighting factor pattern is selected to the motion vector interpolation means. The video coding apparatus according to claim 1, wherein the video coding apparatus is a video coding apparatus. 3. The video encoding device according to claim 1, wherein the weighting factor control means determines the weighting factor according to the direction of each representative motion vector. 4. The video encoding device according to claim 1, wherein the weighting factor control means determines the weighting factor based on the vector value of each representative motion vector. 5. A moving picture decoding apparatus for restoring an image frame from a predicted image frame obtained by performing motion compensation inter-frame prediction and prediction error information transmitted from a video coding apparatus, wherein the decoding is already performed. A frame memory means for storing the converted video signal, and a representative motion vector of each vertex of the conversion unit area transmitted from the video encoding device, and further inputting the conversion unit area according to a predetermined method. Small area for each divided small area
The nearest representative motion vector of and the direction matched with this,
Alternatively, a representative motion vector indicating the direction and size
Has a large weight and, on the contrary, is not consistent
The representative motion vector indicating the direction and magnitude is
Weighting factor control means for determining a weighting factor when calculating a motion vector of each pixel in the small region so as to reduce weight, and each vertex of the conversion unit region using the determined weighting factor Motion vector interpolating means for interpolating and calculating the motion vector of each pixel in the small area from the representative motion vector whose size has been changed, and the motion of each pixel A video decoding device, comprising: a pixel value predicting unit that creates a predicted image frame from a video signal read from the frame memory unit using a vector. 6. The weighting factor control means prepares a plurality of weighting factor patterns in advance, and instructs only the information indicating which weighting factor pattern is selected to the motion vector interpolation means. The video decoding device according to claim 5, characterized in that: 7. The vertices of a conversion unit area for a screen which is not coded by the video coding apparatus from the representative motion vectors of the vertices of the conversion unit area for the screen coded by the video coding apparatus. Motion vector conversion means for calculating a representative motion vector of the motion vector conversion means is provided, and using the representative motion vector calculated by the motion vector conversion means, a prediction image corresponding to a frame dropped without being encoded by the video encoding device. 6. A frame is created and is output as an interpolated image signal.
Alternatively, the video decoding device according to the sixth aspect.