KR0180132B1 - Shape information reducing method using motion estimating - Google Patents

Abstract

The present invention relates to a shape information reduction method using a motion estimation method that reduces the shape information transmission amount by transfer-predicting the contours in a continuous image and transmitting the prediction error for each prediction error region. The shape is reduced by using the proposed method. In this method, the overall contour of the motion region is approximated, then the motion compensation is predicted by the vertices used for the approximation, and the error is transmitted to improve the image quality. In the present invention, by using the redundancy (REDUNDANCY) between the shape information of the motion region by the same object in the continuous image by moving compensation prediction and sending a prediction error to reduce the shape information transmission amount Help with video phones, video conferencing, video encoding and computer vision Shape information is reduced to the motion object in the system in which the shape information of the object being required, it is possible to obtain an improvement in image quality by ensuring that the transmission data is reduced drastically.

Description

Shape information reduction method and apparatus using motion estimation method

1 is an exemplary diagram of video segmentation of a head and shoulder image.

2 is a state diagram of a previous frame and a current frame of a head and shoulder image.

3 is a diagram illustrating a comparison state between an image obtained by moving compensation of a previous frame and a current frame.

4 is a configuration block diagram of a motion compensation contour prediction coding scheme according to the present invention.

5a to c is a polygon / splin approximation process diagram for the present invention.

Figure 6 is a schematic diagram of the polygon / splin approximation method for the present invention.

7 is a diagram illustrating a connection process between two adjacent contour lines in FIG.

* Explanation of symbols for main parts of the drawings

11: subtraction means 12: critical performance means

13: contour prediction means 14: contour reconstruction means

15: addition means 16: movement compensation prediction means

20 Transmission Contour 30 Polygon

40: splin

The present invention relates to a shape information reduction method and apparatus using a motion estimation method of reducing the amount of shape information transmission by moving and predicting a contour in a continuous image and transmitting a prediction error for each prediction error region.

Previously, shape information was reduced using the method proposed in the paper of HOETTER. In this method, the overall contour of the motion region is approximated, and then the motion compensation prediction of the vertices used for approximation is performed, and the error is estimated. Send it.

As described above, the shape information is approximated and the motion compensation is predicted, and the error is transmitted. Therefore, the image quality cannot be improved.

The present invention has been made in view of this point, and an object of the present invention is to use a redundancy (REDUNDANCY) of shape information of a motion region by a same object in a continuous image to perform motion compensation prediction of a contour and to transmit a prediction error. To reduce the amount of shape information transmission.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

First, in the video encoding for each object, the input image is divided into a changed region having a signal different from the previous image by the movement of the object, and a background having no signal change.

The background area does not need any further analysis or transmission of information, and the receiving end reproduces using the signal of the previous image as it is.

In the extracted motion region, the motion information of the object is estimated by using an object model and a motion model, and the motion information and shape information of the motion region are transmitted to the receiver. In this case, the receiving end reproduces the image by using motion compensated prediction with such information.

On the other hand, there are regions where movement compensation prediction errors occur in the movement region. These regions are complex regions where the assumed object model does not fit or the movement model does not fit. .

Since these areas are sensitive to the user's vision, color information is encoded and transmitted so that the receiver can faithfully reproduce the original signal.

In addition, a newly revealed background appears due to the movement of the object, and since the information is not included in the previous image, transmission of information about this is necessary.

FIG. 1 shows an example of dividing a head and shoulder image into a background and dividing the image into a moving compensable area, an uncompensable moving object, and an exposed background.

In the present invention, as a method of encoding shape information of a motion region, shape information of a moving object can be significantly reduced.

Here, the shape information of the motion region may be represented by a binary image representing a region / non-region or a contour of a boundary thereof. redundancy) is large.

The technique of moving compensation predicting the contour using such a property and transmitting the prediction error to reduce the shape information transmission amount is called predictive contour coding.

The contour prediction coding scheme for the present invention performs a thresholding operation and contour approximation to select a transmission prediction error in order to transmit the prediction error.

The contour prediction coding scheme for the present invention will be described with reference to the accompanying drawings.

In object-specific video coding, shape information along with motion information should be transmitted at the highest priority for video prediction. The transmission of shape information gives different motion information to the pixels adjacent to each other around the boundary of the object, thereby avoiding the deterioration of the subjective quality of the block-by-block coding. Is a clue that shows better subjective quality for block-by-block coding.

Here, subjective picture quality, for example, when high quality and low picture quality appear on one screen, the boundary of the block is shown as a mosaic. In this block unit encoding, the boundary of the block is called a blacking phenomenon. When you see it, the subjective picture quality is good.

Methods for expressing boundary of regions are used in computer graphics, character recognition, object synthesis, etc. For example, chain difference coding, S curve, polygon approximation, spline Approximation and Fourier Descriptor are being used.

However, since these methods do not consider transmission, they are difficult to use due to the high data rate when the contour of the motion area must be transmitted every frame.

Since there is much similarity in shape and position between the shape information of the motion area generated by the same object on consecutive frames, it is possible to predict the current shape information from the previous shape information, estimate the motion information of the moving object, and The shape information may be compensated for by moving compensation. If the motion region extraction and the motion information estimation are ideally accurate, the transmission of the shape information is unnecessary.

However, as the data rate decreases, the shape information occupies a larger portion, and thus, a significant reduction of shape information is required to obtain coding gains compared to block-by-block coding methods that do not require shape information transmission. Prediction error regions occur.

FIG. 2 shows shape information of a previous frame (N-1th frame) and a current frame (Nth frame). FIG. 3 shows movement information of shape of a previous frame (N-1th frame) according to motion information. When predicted, nine isolated prediction error areas appear.

In this case, a portion corresponding to shape information of the current frame is encoded and transmitted for each of the nine prediction error regions. Transmission of the prediction error itself may include information that does not affect human vision. Important information affecting subjective picture quality should be transmitted, and transmission of other information can be suppressed to enable low bit rate encoding.

Therefore, information that does not affect the subjective picture quality is removed through the threshold performance so that it cannot be transmitted. At this time, the threshold performance uses the characteristics of the region where the error occurs.

To transmit the contour for transmission of the prediction error region determined to be transmitted, the shape information is reduced by using an approximation method rather than an accurate transmission, and a polygon / sprinkle approximation method is used as the contour approximation method.

As described above, the configuration of the motion compensation contour predictive encoding apparatus of the present invention for reducing shape information is shown in FIG.

That is, motion compensation prediction means 16 for motion compensation prediction of shape information of the previous frame and shape information of the current frame, and subtraction means 11 for obtaining a difference between the output of the motion compensation prediction means 16 and the motion region. By calculating the isolated prediction error areas through the subtraction means 11, critical information affecting the human vision is transmitted by performing critical execution on the isolated prediction error areas, and the transmission of the other information is not performed. Threshold performing means 12 for suppressing, contour prediction means 13 for reducing shape information by using an approximation method rather than an accurate transmission for transmitting the contour for transmission of the prediction error region determined to be transmitted, and performing the threshold Contour reconstruction means 14 and images for reconstructing the encoded contours through means 12 and contour prediction means 13 into isolated prediction error regions. Adder means 15 for forming the shape information of the current frame by adding the isolated prediction error region by the contour reconstruction means 14 and the output of the motion compensation prediction means 16 to input to the motion compensation prediction means 16. Is done.

According to the configuration of the present invention as described above, the threshold performance of selecting false information to be transmitted affects the subjective picture quality and the amount of data to be transmitted.

The critical performance uses characteristics such as the size and shape of the error region, and this requires consideration of shape information prediction errors. Accordingly, the cause of the error region and the threshold performance based thereon will be described.

Shape information and motion information of a motion region of each image are extracted from two consecutive real images. In other words, the current shape information extraction process is independent of the shape information of the previous image. In order for the prediction between the two shape information to be accurate, the extraction of the motion region and the estimation of the motion information must be accurate, and in the ideal case, the transmission of the shape information is unnecessary.

However, due to the limitation of the motion information estimation method and the problem of the signal characteristics of the real image, the imitation information prediction error occurs. In addition, the motion region includes a background exposed with the moving object. Since this region is not related to the motion information, the motion compensation prediction is impossible and an error occurs.

Considering the human visual characteristics that are sensitive to the geometrical shape of the object and the overall movement, the prediction error of the shape information may include information that is not sensitive to human vision. Do not remove or send.

Non-transmission of this error allows effective rate reduction under similar subjective picture quality. The critical performance in the proposed contour prediction coding scheme includes two functions of elimination of small error regions and elimination of meaningless shape information change.

First, the removal of the small error region will be described.

In general, the outline of the shape information can be seen as simple, but it is common to see very complicated changes in the pixel-based observation. Due to the characteristics of the shape information, the difference of the pixel unit by two binary images of the predicted shape information and the actual shape information (PEL-WISED DIFFERENCE) generates many error areas with several hundreds to hundreds of pixels, and even if the prediction is accurate, the added and subtracted areas may appear repeatedly in narrow areas. Elimination of small error areas is necessary.

The selection of the threshold value required for the removal of small error areas is possible up to the maximum size of the area that is not sensitive to human visual characteristics in the whole image, so eliminating small error areas with appropriate thresholds does not affect subjective quality. This reduces the transfer rate and simplifies subsequent performance by dividing the contour of the conveying object into several isolated error areas.

Next, the removal of meaningless shape information change will be described.

Since the human visual characteristics are more sensitive to the shape and motion of the whole object than the local positional error of the object, the rapid change in the shape of the object shape caused by the extraction of the signal characteristics or the movement area, rather than the area changed by the movement of the object, It can be said that the change in shape information is meaningless to the visual characteristics.

In order to transmit the data, a considerable amount of data is required, but it may be disadvantageous in terms of subjective picture quality, so this shape information change is not required to be transmitted.

The error region in which the transmission is determined above performs contour coding for transmission. The contour of the error region includes the contour of the motion compensation prediction shape information. Only the contour from which the prediction contour is removed is encoded and transmitted.

In the head and shoulder images, there are many error domains with narrow bands, and when the chain difference coding without error is used for the coding of the transmission contour of the region, the amount of transmission data becomes significant, so the approximation is expressed. The contour coding method using the SCA is required, and a local position error of the contour occurs.

However, since the human visual characteristics are sensitive to the geometrical shape of the whole object, such small positional errors do not cause undesired deterioration of image quality.

FIG. 5 illustrates a polygon / splin approximation process used for the above-described contour approximation. The transmission contour 20 of the error region in which the transmission is determined is approximated to the polygon 30 first as shown in FIG. 5A. The number of vertices of the polygon 30 depends on the degree of approximation between the actual transmission contour 20 and the approximate polygon 30.

The degree of approximation is represented using the maximum difference between the actual transmission contour 20 and the approximate polygon 30, where the largest difference is a rough approximation, but the number of vertices is reduced, and the smallest difference is a precise approximation. The number of vertices increases.

Then, as shown in FIG. 5B, the splins 40 passing through the obtained approximate vertices are obtained, and the distance to the transmission contour 20 is examined in each pixel on the splins 40, and if the distance is greater than or equal to the threshold value dMAX. As shown in FIG. 5C, an approximation section including the pixel is approximated by the polygon 20 instead of the splin 40, and an index indicating whether the positions of the vertices and the sections between the vertices are the polygon approximation or the splint approximation is shown. Send to the receiver to reproduce the polygon / sprinkle.

Compared to polygon approximation, this contour approximation, combined with polygons and splins, provides a visually natural contour shape with fewer vertices.

The process of approximating several isolated transmission contours using the polygon / splin approximation technique as described above finds the transmission contour in the error region where the transmission is determined after performing a threshold as shown in FIG. 6 (step S1), and between adjacent transmission contours. In order to prevent the occurrence of unnecessary vertices, a process of connecting two adjacent transmission contours to one transmission contour within a predetermined distance is performed (step S2), and then polygon / sprinkle approximation is performed on the connected transmission contours (step S3). .

7 shows a process of connecting two adjacent transmission contours to one transmission contour. When two transmission contours are adjacent to each other as shown in a, the final approximate end point of the upper contour and the initial vertex of the lower contour are represented as b. Since the data must be transmitted in duplicate, if the connection is made like c and then approximated again, one approximate vertex and one initial vertex can be reduced, thereby reducing the amount of transmission.

As described above, the polygon / splin approximation for the present invention divides the entire contour into several transmission contours and approximates it. In addition, the removal of the small error area approximates the reduction of the transmission amount, while approximating several small transmission contours, respectively, resulting in simplicity of seed performance. In addition, the shape information transmission amount can be easily controlled by using a threshold value dMAX that can be given differently for each transmission contour and a threshold value tsm used for a small error area.

As described above, the present invention can reduce the shape information of the moving object in a system requiring the shape information of the moving object such as a video telephone, video conferencing, video encoding, and computer vision, thereby greatly reducing the transmission data, thereby improving image quality. It becomes possible.

Claims (10)

In a method of reducing and transmitting shape information in object-based video encoding, a motion compensation prediction of an outline is performed by using redundancy existing between shape information of a motion region by the same object in consecutive images, and the isolated prediction error is determined. A shape information reduction method using a motion estimation method characterized in that the transmission to reduce the amount of shape information transmission. The shape information of the current frame according to claim 1, wherein when the shape information of the previous frame is moved according to the motion information and the current frame is subjected to the motion compensation prediction, the contour of the moving object is divided into isolated error areas, and the shape information of the current frame is obtained for each prediction error area. Shape information reduction method using a motion estimation method characterized in that only the portion corresponding to the transmission and coding. A motion compensation prediction step of performing motion compensation prediction on the shape information of the current frame using shape information of the previous frame and shape information of the current frame; and subtracting the motion compensation predicted shape information and the current motion region from the motion compensation prediction step. A subtraction step of obtaining independent prediction errors, a threshold performance step of determining a transmission and blocking of information based on time by performing a threshold on the isolated prediction error area through the subtraction step, and a prediction error determined through the threshold performance step A contour prediction step of predicting the contours in the area and reducing and encoding shape information, and a contour reconstructing step of reconstructing the encoded code through the contour prediction step into an isolated prediction error region, and is isolated by the contour reconstruction step. Prediction error region and the chord of the motion compensation prediction step In addition to the shape information of a frame the motion compensation predicting step shape information reduction method using a motion estimation method, characterized by made of an additive comprising: providing a. 4. The method of claim 3, wherein when approximating several transmission contours isolated in the contour prediction step by a polygon / sprinkle approximation technique, a first step of finding a transmission contour in an error region in which transmission is determined after a threshold is performed, and within a predetermined distance. And a third step of connecting two adjacent transmission contours to one transmission contour, and a third step of performing polygon / sprinkle approximation on the connected transmission contours. 5. The method of claim 4, wherein in the second step, when two transmission contours are adjacent to each other, an approximation vertex of the upper contour and an initial vertex of the lower contour scene are connected, and the approximation is performed again to obtain one approximate vertex and one initial vertex. Shape information reduction method using a motion estimation method characterized in that the reduction. 5. The method of claim 4, further comprising: approximating the transmission contour of the error region in which the transmission is determined in the third step to the polygon 30, obtaining a splin 40 passing through the approximate vertices obtained in the step; Examining the distance to the transmission contour 20 at each pixel on the 40 and approximating the approximate section including the pixel to the polygon 20 instead of the spline 40 if the distance is greater than or equal to the threshold value dMAX. And shape information reduction using a motion estimation method, comprising: reproducing a polygon / sprinkle by transmitting an index indicating whether the position of the vertex and the interval between the vertices is a polygon approximation or a splin approximation to a receiver. Way. Motion compensation prediction means 16 for performing motion compensation prediction on shape information of the current frame using shape information of a previous frame and shape information of a current frame, and motion compensation predicted shape information and current from the motion compensation prediction means 16; A subtraction means 11 for subtracting the motion region of the subtractor 11 to obtain an independent prediction error, and performing a critical operation on the isolated prediction error area by the subtraction means 11 to determine information transmission and blocking based on time. Means (12), contour prediction means (13) for predicting contours in the prediction error region determined by the threshold performing means (12), and reducing and encoding shape information, and encoding through the contour prediction means (13). Contour reconstruction means 14 for reconstructing the extracted contour into an isolated prediction error region, and an isolated prediction error region by the contour reconstruction means 14. The motion compensation prediction shape information reduction device in addition to the shape information of the current frame of the means 16 with a motion estimation system, characterized by the addition constituted by any means 15 provided to the motion compensation prediction means (16). The method according to claim 7, wherein the threshold performing means 12 generates a plurality of error regions having several to hundreds of pixels by performing a pixel-by-pixel difference based on two previous images of predicted shape information and actual shape information. A shape information reduction apparatus using a motion estimation method characterized by eliminating insignificant error areas in which added and subtracted portions appear repeatedly in a narrow area. 8. The shape information reduction apparatus according to claim 7, wherein the contour prediction means (13) encodes and transmits only a transmission contour from which a prediction contour included in the contour of the error region is removed. 8. The shape information reduction apparatus according to claim 7, wherein the contour prediction means (13) performs contour approximation using transmission chain difference coding obtained from the threshold performing means (22).