JPH08265778A - Moving amount detection method and moving amount detector - Google Patents

Abstract

PURPOSE: To provide a moving amount detection method and a moving amount detector capable of improving the detection accuracy of a moving amount with simple constitution by applying a pitch method in the case of using hierarchized picture data and detecting the moving amount by a block matching method. CONSTITUTION: Basing on input picture signals, plural hierarchical pictures for indicating the normal components and transient components of the input picture signals are respectively formed, first and second evaluated values for indicating the movement of pictures are obtained by the block matching method for respective hierarchies by using the respective plural hierarchical pictures and then, third evaluated values are obtained for the respective hierarchies by weighting and averaging the first and second evaluated values. Thereafter, by detecting the most certain moving amount from plural moving amount candidates obtained on the basis of the minimum value of the third evaluated values by the pitch method, erroneous detection caused by block completion is prevented and the detection accuracy of the moving amount in the respective hierarchies is improved.

Description

Detailed Description of the Invention

[0001]

[Table of Contents] The present invention will be described in the following order. Field of Industrial Application Conventional Technology (FIG. 7) Problem to be Solved by the Invention (FIGS. 7 and 8) Means for Solving the Problem (FIGS. 1 to 6) Action (FIGS. 1 to 6) Example (FIGS. 1 to 6) Effect of the invention

[0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a motion amount detecting method and a motion amount detecting device, and more particularly to detecting a motion amount after layering two temporally different image data when detecting a motion of an image. Can be applied to.

[0003]

2. Description of the Related Art Conventionally, there is a block matching method as a method for detecting a motion vector of an image. This is shown in Figure 7.
As shown in FIG. 3, a reference block B1 obtained by dividing a current frame (or field) F1 as a reference image into blocks of a size of m pixels × n pixels, and a past frame (or field) F2 as a basic image. Of the absolute value of the difference for each pixel with the candidate block B2 existing in the search area (± s pixel) SA of the

[Equation 1] As shown in, the candidate block B2 in the search area SA
Is shifted for each pixel to calculate an evaluation value P (h, v) of total (2s + 1) ² points. By obtaining the minimum value of these evaluation values, the relative coordinate value (h, v) indicated by the minimum value is used as the motion vector of the reference block B1. The total number of calculations is represented by the following formula: calculation amount = block number × search point × number of evaluation formulas. in this case,
The number of search points is (2s + 1) ² , and the number of evaluation expressions is subtraction (m × n), absolute value calculation (m × n), and addition (m × n).
It is the sum of -1).

However, since the block matching method is a method of performing the matching calculation of the reference block B1 and the candidate block B2 while shifting the pixel in the search area SA pixel by pixel, the amount of calculation for detecting the motion vector becomes enormous.
As a result, there has been a problem that the size of the entire device becomes large and the calculation time becomes long when it is made into an LSI.

In order to reduce the hardware for motion detection in order to solve such a problem, it is necessary to reduce each item in the above equation, but the number of blocks cannot be changed.
Methods for reducing the number of search points include a 3-step method and an orthogonal projection method. Further, there is a hierarchical method as a method of simultaneously reducing the number of search points and the number of evaluation values.

Here, the motion amount detection method by the layering method is a method in which the original image is layered at a plurality of resolutions and the motion amount is detected by the block matching method using the layered images. That is, in this motion amount detection method, first, original image data (hereinafter referred to as layer 1) is hierarchically averaged by averaging or low-pass filter processing to reduce the number of pixels (hereinafter referred to as layer 2). Create). Next, a rough motion amount is detected from the created layer 2 image data, and fine motion amount detection is performed on the layer 1 image data based on the motion amount so that the motion amount can be detected with a small amount of calculation. Has been done. It should be noted that the number of layers here is not limited to two layers, and by repeating the average value layering sequentially, layers 3 having a smaller data amount,
It is also possible to create image data for layer 4, ...

Specifically, FIG. 8 shows an example of generating hierarchical image data. FIG. 8A shows a case where image data in which the original image is hierarchized into three layers is generated, and the lowest layer, layer 1, is the original image. The data of the upper layer than the block (eg 16 × 16) on the original image, the image data of the layer n is M
_{If n} (x, y), then

[Equation 2] Thus, the block size is halved in the horizontal and vertical directions, respectively. Further, when the image data of the layer 2 is generated from the image data of the layer 1 in which the average value is layered as described above, the image data of the layer 2 can be similarly obtained by the equation (2).

By using such a motion amount detecting method, the motion amount can be calculated with a smaller calculation amount as the number of layers increases. That is, since the block size and the search area become smaller as the image data in the higher hierarchy increases, the calculation amount by the evaluation function inevitably becomes small. The block size itself is the same as the normal block matching method for finally obtaining the motion amount, but motion compensation is performed according to the motion amount obtained from the image data of the upper layer, and the search area is calculated. Can be reduced, so that the amount of calculation can be reduced.

[0009]

However, in this motion amount detecting method, there is a problem that the motion amount detecting accuracy deteriorates as the number of layers increases. In practice, in this motion amount detection method, the motion amount for each block is first detected in the upper layer where the image is coarse, and the motion amount is detected in the lower layer based on this detection result. It has a great influence on the motion amount detection in. That is, since the block size of the image data in the upper layer is reduced by the average value layering process, the feature amount of the image differs from that of the original image in the lower layer. In particular, since the edge component is lost due to the average value layering, the correspondence between the motion amount in the average value layered image data and the motion amount due to the original image may shift.

When the motion amount is actually obtained in a layer lower than the current layer, the result of the motion amount in the upper layer is reflected. Therefore, if the shift amount of the correspondence is large and it cannot be covered within the search area, a malfunction occurs. Becomes Therefore, the motion amount detection method based on the average-value-layered image data has a problem that a malfunction may increase due to the lack of the information amount due to the average-value layering. In addition, since the block size becomes smaller as the hierarchy becomes higher, the motion amount detection on the average value hierarchical image data has a problem that the resolution of the motion amount detection with respect to the original image is lowered and malfunction occurs. .

As one method for solving this problem, a plurality of layers of image data representing stationary components of the original image are formed by an average value layering method for the original image, and high frequency components of each layer ( Data representing "activity" will also be layered (hereinafter referred to as "activity layering") to form multiple layers of activity data representing the transient components of the original image, and these layers will be used for movement. Methods to detect quantity have been considered.

For example, the average value of the absolute values of the differences between the average value of the image data of the upper layer and the average value of the image data of the lower layer is set as the activity for each small block, and the second layer image composed of the activity is set. A method of reducing malfunctions by calculating a second evaluation value using the first evaluation value and adding the second evaluation value to the first evaluation value hierarchically averaged to evaluate the second evaluation value is considered. Incidentally, as the activity, not only the sum of absolute values of the differences from the above-mentioned average value but also a method using standard deviation, variance, Laplacian, dynamic range, etc. are considered.

In the conventional motion amount detecting method using a hierarchical image formed by hierarchically dividing images, only the average value layering shown in FIG. 8A is performed. In this method, however, the average value layering is performed by the equation (2). At the same time as generating the image data, the average value hierarchical image data is activity-hierarchized to generate activity data, as shown in FIG. 8B. If the activity data of layer 2 is Δ ₂ (x, y), this activity data Δ ₂ (x, y) is
From the sum of absolute values of the differences of the corresponding images of the original image from the averaged hierarchical image data,

(Equation 3) Similarly, the activity data Δ of layer 3 is obtained.
₃ (x, y) is the following formula

[Equation 4] Ask in. By obtaining all the activity data based on the original image, high frequency components faithful to the original image can be extracted.

When the activity data is obtained in this way, its hierarchical structure is as shown in FIG.
It will have a layer plane other than the lowest layer. This activity data reflects the feature amount that is missing in the image data when the average value is hierarchized.

Here, using the average value hierarchical image data and the activity hierarchical activity data, the amount of motion is detected by the block matching method in each hierarchical layer. That is, the evaluation function of the block matching is expressed by the following equation, where t is the current field.

(Equation 5) It is represented by. However, (u _n , v _n ) indicates the amount of motion in the layer n. _Let V ′ _n = (u _n , v _n ) which gives the minimum of this evaluation function E (Y) _{n be} the motion amount to be obtained. The motion amount V _n in the current layer is calculated by the following equation.

(Equation 6) Therefore, the final motion amount can be obtained by sequentially obtaining each layer.

The evaluation function in this case is the same as the equation (5) for the activity data shown in FIG.
The following formula

(Equation 7) Then, the new evaluation function E (G) _n is

(Equation 8) And However, w ₁ and w ₂ are weighting factors. Then, the amount of motion that gives the smallest evaluation function E (G) _n is obtained. Since there is no activity data in the lowest layer, the evaluation is performed using only the evaluation function E (Y) _{n of the} expression (5).

As described above, the first evaluation value of the image data in which the average value is hierarchized and the second evaluation value of the activity data in which the activity is hierarchized are weighted averaged to create a third evaluation value, and the movement is performed. By detecting the amount, even if the optimum evaluation value of one layer is erroneous detection, correct detection can be performed with the other evaluation value, and the accuracy of motion amount detection in each layer is improved. When the motion amount detection accuracy in each layer is improved as described above, the detection accuracy of the final motion amount obtained by the equation (6) can also be improved, and as a result, malfunctions can be reduced.

However, since such a layering operation (averaging, activity detection) is performed by block completion, there is a possibility that information components will be degenerated in the upper layer and a fine component will be lost. There was a problem that a quantity detection error occurred.

The present invention has been made in consideration of the above points. When the motion amount is detected by the block matching method using hierarchical image data, the gradient method is applied to simplify the operation. An object of the present invention is to propose a motion amount detection method and a motion amount detection device that can improve the motion amount detection accuracy with a configuration.

[0020]

In order to solve such a problem, in the present invention, block compensation is performed in order from the upper layer while performing motion compensation in the lower layer having the higher resolution based on the motion amount obtained in the upper layer having the lower resolution. In a motion amount detecting method and a motion amount detecting device for detecting a motion amount of an image by a ching method, a plurality of hierarchical images representing stationary components of the input image signal are formed based on the input image signal, and the hierarchical image is used. Then, a first evaluation value representing the movement of the image is obtained for each layer by the block matching method, and a plurality of layer images representing the transient component of the input image signal are formed based on the input image signal. After obtaining the second evaluation value representing the movement of the image by the block matching method for each layer using, the weighted average of the first and second evaluation values is applied to each layer. Obtaining a third evaluation value. Then, a plurality of motion amount candidates are obtained based on the minimum value of the third evaluation value, and the most probable motion amount is detected from the motion amount candidates by the gradient method.

[0021]

A plurality of hierarchical images representing stationary components of the input image signal are formed based on the input image signal, and the first hierarchical image is used to represent the motion of the image by the block matching method for each hierarchical layer. Along with obtaining the evaluation value, a plurality of hierarchical images representing the transient component of the input image signal are formed based on the input image signal, and the hierarchical image is used to represent the movement of the image by the block matching method for each hierarchical layer. After obtaining the evaluation value of 2, the weighted average of the first and second evaluation values is obtained to obtain the third evaluation value for each layer. After that, the most probable motion amount is detected by the gradient method from the plurality of motion amount candidates obtained based on the local minimum value of the third evaluation value, thereby preventing erroneous detection due to block completion. As a result, it is possible to improve the accuracy of detecting the motion amount in each layer.

[0022]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described in detail below with reference to the drawings.

In FIG. 1, the motion amount detecting device 1 inputs the image data of the input original image to the block circuit 2 and the frame memory 3. The block circuit 2 sequentially obtains image data obtained by sequentially blocking the original image with a predetermined size (for example, a size including a search area for a block size of 16 × 16), and outputs the evaluation value calculation circuit 4 and the evaluation value calculation circuit 4 of the layer 1. 5, to the average value hierarchy circuit 6 of hierarchy 2, the activity hierarchy circuit 7 (FIG. 2), and the activity hierarchy circuit 8 of hierarchy 3 (FIG. 3).

As shown in FIG. 2, the average value layering circuit 6 of the layer 2 executes the average value layering process by the operation of the equation (2) for a predetermined block of the image data of the input original image. And obtain the image data of layer 2. The image data of layer 2 obtained as a result is layer 2 respectively.
Block circuit 9, frame memory 10, activity hierarchy circuit 7, and average value hierarchy circuit 11 of hierarchy 3 (FIG. 3).

The hierarchy 2 activity hierarchy circuit 7 executes the activity hierarchy processing by the operation of the equation (3) using the input image data of the original image and the hierarchy 2 image data to execute the hierarchy 2 activity. Ask for data. The resulting activity data of the layer 2 is input to the block circuit 12 and the frame memory 13 of the layer 2, respectively.

As shown in FIG. 3, the average value layering circuit 11 of the layer 3 executes the average value layering process by the operation of the equation (2) using the inputted image data of the layer 2 to form the layer. The image data of 3 is obtained. Layer 3 resulting from this
Image data of the block circuit 14 of the third layer,
It is input to the frame memory 15 and the activity hierarchy circuit 8.

The hierarchy 3 activity hierarchy circuit 8 uses the image data of the input original image and the hierarchy 3 image data to execute the activity hierarchy process by the operation of equation (4) to carry out the hierarchy 3 activity. Ask for data. The resulting activity data of layer 3 is input to the block circuit 16 and frame memory 17 of layer 3, respectively.

The image data blocked in this way is averaged into hierarchical levels as image data of layers 1, 2, and 3, and is also activity hierarchical as activity data of layers 2 and 3. .

In the layer 3 which is the uppermost layer, the actual amount of motion is detected. First, the image data and the activity data of the past (that is, one frame before) set in the frame memories 15 and 17 are respectively set according to the search area. The data is read out to the search block circuits 18 and 19. Next, in the evaluation value calculation circuits 20 and 21, the block circuit 14
And 16 and the search block circuits 18 and 19 using the image data and the activity data, the evaluation functions E (Y) ₃ and E of the equations (5) and (7), respectively.
(D) Obtain an evaluation value based on ₃ . This evaluation value is added to the weighting coefficient w in the addition circuit 22 as shown in the equation (8).
₁ and w ₂ are weighted in a predetermined manner and added, and an evaluation value based on a new evaluation function E (G) ₃ obtained as a result is input to the motion amount detection circuit 23.

Here, in the case of this embodiment, the evaluation value calculation circuits 20 and 21 are composed of a circuit 50 as shown in FIG. That is, the evaluation value calculation circuit 50 has a reference block memory 51 corresponding to the block circuits 14 and 16, and a candidate block memory 52 corresponding to the search block circuits 18 and 19, respectively.
The contents of 1 and the candidate block memory 52 are read in the order of the addresses designated by the memory control 53, and are subtracted by the subtraction circuit 56 through the registers 54 and 55, respectively. The difference data obtained as a result is converted into an absolute value by the absolute value conversion circuit 57, and cumulatively added by the addition circuit 58 and the register 59. This cumulative addition result is the evaluation function E (Y).
Adder circuit 22 as an evaluation value based on ₃ and E (D) _3.
(Fig. 3).

Further, in the case of this embodiment, the motion amount detecting circuit 2
3 is composed of a circuit 60 as shown in FIG. That is, in the motion amount detection circuit 60, the evaluation value based on the evaluation function E (G) ₃ output from the addition circuit 22 (FIG. 3) is stored in the evaluation value memory 61 according to the order of the addresses designated by the evaluation value memory control 62. Is entered. In this way, the calculation of the expressions (5) and (7) is performed by the evaluation value calculation circuit 50.
And the evaluation value obtained as a result is evaluated value memory 6
Input to 1. In the actual hierarchy 2 and hierarchy 3, the evaluation values for the average-value-hierarchized image data and the evaluation values for the activity-hierarchical activity data are weighted and added to obtain the evaluation value memory 61.
Is stored.

In the motion amount detecting circuit 60, the evaluation value stored in the evaluation value memory 61 is read out in sequence according to the address designated by the evaluation value memory control 62 and is input to the comparator 63 and the register 64. It The comparator 63 sequentially compares the other input with the evaluation value read from the evaluation value memory 61, and when the input evaluation value is smaller, outputs a signal for updating the contents of the registers 64 and 65.

The register 65 has an evaluation value memory 61.
The addresses for reading are sequentially set. In this way, the evaluation values sequentially stored in the evaluation value memory 61 are evaluated,
The address that gives the smallest evaluation value is register 6
5 is output, and this is output as the output of the motion amount detection circuit 60, that is, the motion amount MV composed of a motion vector.

Here, in the case of the motion amount detecting apparatus 1, the motion amount obtained by the motion amount detecting circuit 23 of the layer 3 is input to the gradient method processing circuit 24. In practice, the gradient method processing circuit 24
Are configured as shown in FIG. In the gradient method processing circuit 24, the minimum value vector detection circuit 80, based on the output of the motion amount detection circuit 23 (that is, the motion vector MV), the evaluation value memory 61 in the highest layer (layer 3).
A plurality of local minimum points of the evaluation value stored in is detected, and these positions are supplied to the candidate vector selection circuit 81 and the motion compensation circuit 82 as the motion vector candidates of the layer 3, respectively.

Incidentally, in this case, the reason why the minimum value is detected without detecting the minimum value of the evaluation values stored in the evaluation value memory 61 in the highest hierarchy is that it is stored in the evaluation value memory in each hierarchy. The evaluation value varies considerably depending on the characteristics of the image in the block (for example, flats and edges). Generally, the value changes slightly in the case of a flat image, while in the case of an image containing edges, Is because it has a characteristic that changes greatly. That is, normally, the evaluation value stored in the evaluation value memory is expressed in the shape of a positive quadratic curve such that the evaluation value at the position corresponding to the motion vector becomes the minimum point, but it is considerably flat depending on the image. It may be touched or have multiple minimum points. In particular, this tendency becomes stronger in the upper layer of the layer image.
Therefore, it is a little dangerous to judge the minimum point.

The motion compensation circuit 82 uses the externally input past frame image data D1 as a minimum value vector detection circuit 80.
The motion compensation is performed on the basis of the motion vector that is a candidate output from the motion compensation image data, and the motion compensation image data thus obtained is transmitted to the difference circuit 83.

Further, the gradient method processing circuit 24 stores the current frame image data D2 externally input in the current frame memory 84.
To be stored. The stored current frame image data D
2 is a differential circuit 83, 85, 86, 87, 88 via a line delay (H) having a delay time of one line and a pixel delay (D) having a delay time of one pixel.
Given to each.

As a result, if the pixel value of the current frame at the coordinates (x, y) is g1 (x, y) and the pixel value of the motion-compensated past frame is g0 (x, y), the difference circuit 83 , The interframe difference Δt is

[Equation 9] In the difference circuit 85, the horizontal spatial gradient Δx _r in the right direction is calculated by the following equation.

[Equation 10] In the difference circuit 86, the horizontal spatial gradient Δx _l in the left direction is calculated by the following equation.

[Equation 11] The vertical space gradient Δy _a in the upward direction is calculated by the following equation in the difference circuit 87.

(Equation 12) In the difference circuit 88, the downward vertical space gradient Δy _u is calculated by the following equation.

(Equation 13) Required by.

Next, the output of the difference circuit 86 and the difference circuit 87
Is given to the vector operation circuit 90 via the absolute value conversion circuits 89A and 89B, respectively, and the outputs of the difference circuit 85 and the difference circuit 88 are respectively supplied to the vector operation circuit 91 via the absolute value conversion circuits 89C and 89D. Given to. Further, the output of the difference circuit 83 is given to the vector operation circuits 90 and 91.

The vector calculation circuit 90 calculates the rightward motion vector V _xr as

[Equation 14] The motion vector V _xl in the left direction is _calculated by

(Equation 15) Of the motion vectors V _xr and V _xl , the larger one of them has the larger absolute value.
Output as _x .

The vector calculation circuit 91 calculates the upward motion vector V _ya as

[Equation 16] And the downward motion vector V _yu is _calculated by

[Equation 17] Of the motion vectors V _ya and V _yu , the one having the larger absolute value is _calculated as the motion vector V in the vertical direction.
Output as _y .

The comparison circuits 92 and 93 compare the horizontal and vertical motion vectors V _x and V _{y respectively} with a predetermined threshold value T.
It is compared with h and the comparison result obtained thereby is output to the candidate vector selection circuit 81. That is, when the absolute value of the motion vector V _x or V _y is larger than the predetermined threshold value Th = 1.0,
That is,

(Equation 18) Or the following formula

[Formula 19] In such a case, it is determined that the motion vector candidate is not applicable, the motion vector is eliminated, and then the absolute values of the motion vectors V _x and V _y are newly compared with the threshold Th.

Thus, the candidate vector selection circuit 81 receives the motion vector input from the minimum value vector detection circuit 80, the horizontal motion vector V _x output from the vector calculation circuit 90, and the vertical motion output from the vector calculation circuit 91. From among the vectors V _y , new motion vectors MV ₁ and M that are the most probable motion vector candidates.
Select V ₂ and output.

Applying the gradient method after performing motion compensation for each of a plurality of motion vector candidates in this way is equivalent to detecting a motion amount of one pixel or less due to the resolution in the highest hierarchy. Therefore, if the motion vector candidate is the most probable, it can be obtained as a motion vector of one pixel or less by the motion vector detection by the gradient method. As a result, the gradient method processing circuit 24 can detect a motion vector with an accuracy of a pixel or less by applying the gradient method after performing motion compensation for each of a plurality of motion vector candidates.

Here, in the case of the motion amount detecting apparatus 1, the motion amounts MV ₁ and MV obtained by the gradient method processing circuit 24 of the layer 3 are obtained.
₂ is supplied to the search block circuits 25 and 26 of the image data of the layer 2 and the search block circuits 27 and 28 of the activity data, respectively, and to the adder circuits 29 and 30, respectively.

First, the motion amount MV ₁ obtained by the gradient method processing circuit 24 of the layer 3 is given to the search block circuit 25 of the image data of the layer 2 and the search block circuit 27 of the activity data, and the search is performed by this motion amount. Areas are motion compensated. That is, when detecting the amount of motion of the layer 2, the past block (that is, one frame before) image data and activity data set in the frame memories 10 and 13 are respectively searched according to the motion-compensated search area. Read at 25 and 27.

Next, the evaluation value calculation circuits 31 and 32 include the block circuits 9 and 12 and the search block circuits 25 and 27.
Using the past and present image data and activity data of and, evaluation values are obtained based on the evaluation functions E (Y) ₂ and E (D) ₂ of the equations (5) and (7), respectively.
In the adder circuit 33, the evaluation values are weighted with a predetermined weighting according to the weighting factors w ₁ and w ₂ and added in the addition circuit 33, and a new evaluation function E (G) obtained as a result is obtained.
_The evaluation value based on ₂ is input to the motion amount detection circuit 34. As a result, the motion amount obtained by the motion amount detection circuit 34 is added to the motion amount MV ₁ of the layer 3 by the addition circuit 29 and is sent as the motion amount MV _1A of the layer 2 as shown in the equation (6). The evaluation value calculation circuits 31 and 32 and the motion amount detection circuit 34 are also configured similarly to the evaluation value calculation circuit 50 and the motion amount detection circuit 60 of FIGS.

On the other hand, the motion amount MV ₂ obtained by the gradient method processing circuit 24 of the layer 3 is given to the search block circuit 26 of the image data of the layer 2 and the search block circuit 28 of the activity data, and this motion amount is used. The search area is motion compensated. That is, when detecting the amount of motion of the layer 2, the past block (that is, one frame before) image data and activity data set in the frame memories 10 and 13 are respectively searched according to the motion-compensated search area. Read at 26 and 28.

Next, the evaluation value calculation circuits 35 and 36 include the block circuits 9 and 12 and the search block circuits 26 and 28.
Using the past and present image data and activity data of and, evaluation values are obtained based on the evaluation functions E (Y) ₂ and E (D) ₂ of the equations (5) and (7), respectively.
In the adder circuit 37, the evaluation values are weighted with a predetermined weighting according to the weighting factors w ₁ and w ₂ and added in the addition circuit 37, and a new evaluation function E (G) obtained as a result is obtained.
_The evaluation value based on ₂ is input to the motion amount detection circuit 38. As a result, the motion amount obtained by the motion amount detection circuit 38 is added to the motion amount MV ₂ of the layer 3 by the addition circuit 30 and is sent as the motion amount MV _2A of the layer 2 as shown in the equation (6). The evaluation value calculation circuits 35 and 36 and the motion amount detection circuit 38 are also configured in the same manner as the evaluation value calculation circuit 50 and the motion amount detection circuit 60 shown in FIGS.

In this way, the motion amount M obtained in the layer 2
V _1A and MV _2A are supplied to the search block circuits 39 and 40 for the image data of the layer 1, and the search area is motion-compensated by the amount of motion. Ie tier 1
In the motion amount detection, the past (that is, one frame before) image data set in the frame memory 3 is read to the search block circuits 39 and 40 in accordance with the motion-compensated search area.

Next, the evaluation value calculation circuit 4 is the block circuit 2
And the search block circuit 39 are used to obtain an evaluation value based on the evaluation function E (Y) ₁ of the equation (5), and the evaluation value is input to the motion amount detection circuit 41 and the determination circuit 45. It The motion amount detection circuit 41 detects the motion amount of the layer 1 based on this evaluation value and adds this to the addition circuit 4
Send to 3. The adding circuit 43 adds the motion amount obtained by the motion amount detecting circuit 41 to the motion amount MV _1A of the layer 2 as shown in the equation (6), and the motion amount MV _{1B of the} original image.
Is detected and sent to the selection circuit 46.

On the other hand, the evaluation value calculation circuit 5 uses the image data of the block circuit 2 and the search block circuit 40,
An evaluation value is obtained based on the evaluation function E (Y) ₁ of the equation (5), and the evaluation value is input to the motion amount detection circuit 42 and the determination circuit 45. The motion amount detection circuit 42 detects the motion amount of the layer 1 based on this evaluation value and sends it to the addition circuit 44. The addition circuit 44 adds the motion amount obtained by the motion amount detection circuit 42 and the motion amount MV _2A of the layer 2 as shown in the equation (6) to obtain the motion amount MV _{2B of the} original image.
Is detected and sent to the selection circuit 46.

The judgment circuit 45 is composed of the evaluation value calculation circuits 4 and 5.
Based on the evaluation value obtained in step 1, which of the evaluation values is the smallest is determined. Thus, the selection circuit 46
On the basis of the result of this determination, determines which of the motion amounts MV _1B and MV _2B of the original image has the smallest evaluation value at the position corresponding to the motion amount as the motion vector of the lowest layer and outputs it.

In the above configuration, the motion amount detecting device 1
When the image is layered, the average value layering process of the pixels in the block is executed to form the average value plane in the layer 3 which is the highest layer, and the average value plane and the layer which is the lower layer. The activity plane in the hierarchy 3 is formed by executing a predetermined calculation process based on the average value plane of 2.

Subsequently, in the hierarchical level 3, after the average value plane and the activity plane are subjected to block matching to calculate the evaluation values, the weighted average value of these evaluation values is obtained, and the minimum point in the weighted average value is determined. A plurality of positions are selected and the positions of the plurality of local minimum points are set as the motion vector candidates of the layer 3. Then, after performing motion compensation for each of the plurality of motion vector candidates and applying the gradient method, it is possible to further select a motion vector candidate for which a value of 1 pixel or less is calculated, As a result, it is possible to prevent erroneous detection caused by the completion of the block.

Thereafter, for each of the selected motion vector candidates of the layer 3, the average value plane and the activity plane are respectively subjected to block matching in the layer 2 which is a lower layer to calculate the evaluation value, and then the evaluation value is calculated. By calculating the minimum value, the motion vector obtained on the basis of the minimum value is output as the motion vector of the layer 1 which is the lowest layer. In this way, the gradient method is applied to multiple motion vector candidates in the top layer, and block matching is performed sequentially from the top layer to the bottom layer to comprehensively evaluate the final motion with a simple configuration. The detection accuracy of the quantity can be improved.

According to the above construction, the evaluation value obtained by performing the block matching on each of the average value plane and the activity plane in the highest hierarchy to calculate the evaluation value, and then weighting these evaluation values. By selecting a plurality of local minimum points from the above and applying the gradient method to the positions of the plurality of local minimum points as motion vector candidates, it is possible to further select a motion vector candidate for which a value of 1 pixel or less is calculated. it can. Therefore, by performing block matching sequentially from the top layer to the bottom layer and comprehensively evaluating it, false detection caused by block completion can be prevented in advance and the accuracy of motion amount detection in each layer can be improved. Therefore, the detection accuracy of the final motion amount can be improved with a simple configuration.

In the above-mentioned embodiment, the case where the image data of three layers is created from the original image by the average value layering is described, but the number of layers is not limited to this, and two or four layers or more can be used. The same effect can be realized. Further, when the average value is hierarchized, the average value is obtained in the block range of 2 × 2 as in the equation (2), but the block range is not limited to this, and the average value is not limited to the smooth value by a low pass filter or the like. You may make it change.

Further, in the above-mentioned embodiment, after the weighted addition of the evaluation value in the average value hierarchy and the evaluation value in the activity hierarchy, a plurality of local minimum points are detected and their positions are respectively determined in the top hierarchy (tier 3). However, the present invention is not limited to this, and for each of the evaluation value in the average value hierarchy and the evaluation value in the activity hierarchy, a plurality of minimum values are set as the highest rank. It may be a candidate for the amount of movement of the hierarchy.

In this case, with respect to the position of the minimum point of the evaluation value in the average value layering and the position of the minimum point of the evaluation value in the activity layering, after overlapping is removed by the OR logic, The positions of the plurality of local minimum points in the evaluation value of the highest layer are set as the candidates for the motion amount of the highest layer.

Further, in the above-described embodiment, the minimum position is selected as the candidate of the motion amount in the hierarchy 2, but the evaluation value in the average value hierarchy and the evaluation value in the activity hierarchy are selected as in the top hierarchy. Select multiple minimum values, increase the number of branches for detection of motion amount in the lower hierarchy, detect the smallest evaluation value of all evaluation values in the lowest hierarchy, and output as the final motion amount. It may be done.

Further, in the above-described embodiment, the case where the gradient method processing circuit 24 of the layer 3 selects and outputs _two motion vectors MV ₁ and MV ₂ as candidates for the motion amount has been described. Not limited to this, three or more motion vectors may be selected and output. In this case, it is necessary to provide the number of search blocks, evaluation value calculation circuits, and motion amount detection circuits in the average value hierarchy and the activity hierarchy in the lower hierarchy according to the number of motion vectors selected and output. .

[0063]

As described above, according to the present invention, a plurality of hierarchical images representing stationary components of the input image signal are formed based on the input image signal, and the hierarchical image is used to block each hierarchical layer. A first evaluation value representing the movement of the image is obtained by the matching method, a plurality of hierarchical images representing the transient components of the input image signal are formed based on the input image signal, and the hierarchical images are used for each hierarchical layer. After the second evaluation value indicating the movement of the image is obtained by the block matching method, the first and second evaluation values are weighted and averaged to obtain the third evaluation value for each layer. After that, the most probable motion amount is detected by the gradient method from the plurality of motion amount candidates obtained based on the local minimum value of the third evaluation value, thereby preventing erroneous detection due to block completion. Then, it is possible to improve the detection accuracy of the motion amount in each layer,
Thus, it is possible to realize a motion amount detection method and a motion amount detection device that can improve the final motion amount detection accuracy with a simple configuration.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a motion amount detecting device according to an embodiment.

FIG. 2 is a block diagram showing a configuration of a motion amount detecting device according to an embodiment.

FIG. 3 is a block diagram showing a configuration of a motion amount detecting device according to an embodiment.

FIG. 4 is a block diagram showing a configuration of an evaluation value calculation circuit in the motion amount detection device of FIGS.

5 is a block diagram showing a configuration of a motion amount detection circuit in the motion amount detection device of FIGS. 1 to 3. FIG.

6 is a block diagram showing a configuration of a gradient method processing circuit in the motion amount detecting device of FIG.

FIG. 7 is a schematic diagram for explaining a method of detecting a movement amount by block matching.

FIG. 8 is a schematic diagram for explaining a hierarchical image.

[Explanation of symbols]

1 ... Movement amount detecting device 2, 9, 12, 14, 16 ...
Block circuit 3, 10, 13, 15, 17 ... Frame memory, 6, 11 ... Average value hierarchical circuit 4, 5, 2
0, 21, 31, 32, 35, 36, 50 ... Evaluation value calculation circuit, 7, 8 ... Activity hierarchy circuit, 18,
19, 25, 26, 27, 28, 39, 40 ... Search block circuit, 22, 29, 30, 33, 37, 43,
44 ... Adder circuit, 23, 34, 38, 41, 42, 6
0 ... movement amount detection circuit, 61 ... evaluation value memory, 24 ...
... Gradient method processing circuit, 45 ... Judgment circuit, 46 ... Selection circuit, 80 ... Minimum value vector detection circuit, 81 ... Candidate vector selection circuit, 82 ... Motion compensation circuit.

Claims (8)

[Claims] 1. A motion amount detecting method for detecting a motion amount of an image by a block matching method in order from the upper layer while performing motion compensation in a lower layer having a higher resolution on the basis of a motion amount obtained in an upper layer having a lower resolution. A first image layering step that forms a plurality of layered images representing stationary components of the input image signal based on the input image signal, and a transient component of the input image signal based on the input image signal A first image layering step for forming a plurality of layered images and a first image layering step for each layer using a block matching method using the layered image obtained in the first image layering step. In addition to obtaining the evaluation value, the second evaluation value representing the motion of the image was obtained for each layer by the block matching method using the layer image obtained in the second image layering step. , An evaluation value calculation step for obtaining a third evaluation value for each layer by weighted averaging the first and second evaluation values, and a plurality of motion amount candidates based on the minimum value of the third evaluation value. And a motion amount detecting step for detecting the most probable motion amount from the motion amount candidates by the gradient method. 2. In the highest hierarchy having the lowest resolution,
The motion amount is detected by executing the motion amount detecting step, and the motion amount is detected in another layer by obtaining the minimum value of the third evaluation value. The motion amount detection method according to claim 1. 3. A motion amount detecting method for detecting a motion amount of an image by a block matching method in order from the upper layer while performing motion compensation in a lower layer having a higher resolution based on a motion amount obtained in an upper layer having a lower resolution. A first image layering step that forms a plurality of layered images representing stationary components of the input image signal based on the input image signal, and a transient component of the input image signal based on the input image signal A first image layering step for forming a plurality of layered images and a first image layering step for each layer using a block matching method using the layered image obtained in the first image layering step. The evaluation value is obtained, and the second evaluation value representing the movement of the image is obtained for each layer by the block matching method using the layer image obtained in the second image layering step. A motion amount for obtaining a most probable motion amount by the gradient method from a plurality of motion amount candidates based on the value calculation step and the minimum value of each of the first and second evaluation values. A motion amount detecting method, comprising: a detecting step. 4. In the highest hierarchy having the lowest resolution,
The motion amount is detected by executing the motion amount detecting step, and in the other layers, the first and second motion amounts are detected.
4. The motion amount detecting method according to claim 3, wherein the motion amount is detected by obtaining a minimum value of the third evaluation values obtained by performing a weighted average of the evaluation values of. 5. A motion amount detecting device for detecting a motion amount of an image by a block matching method in order from the upper layer while performing motion compensation in a lower layer having a higher resolution on the basis of a motion amount obtained in an upper layer having a lower resolution. A first image layering means for forming a plurality of layered images representing stationary components of the input image signal based on the input image signal; and a transient component of the input image signal based on the input image signal. A first image layering means for forming a plurality of layered images, and a first layer representing a motion of the image by the block matching method for each layer using the layered images obtained by the first image layering means. After obtaining the evaluation value, the hierarchical image obtained by the second image layering means is used to obtain the second evaluation value representing the movement of the image for each layer by the block matching method, and then the first image. as well as Evaluation value calculation means for obtaining a third evaluation value for each layer by weighted averaging the two evaluation values, and a plurality of movement amount candidates are obtained based on the minimum value of the third evaluation values, and the movement amount A motion amount detecting device comprising a motion amount detecting means for detecting a most probable motion amount from the candidates by a gradient method. 6. In the highest hierarchy having the lowest resolution,
The motion amount is detected by executing the motion amount detecting means, and the motion amount is detected in other layers by obtaining the minimum value of the third evaluation value. The motion amount detection device according to claim 5. 7. A motion amount detecting device for detecting a motion amount of an image by a block matching method in order from the upper layer while performing motion compensation in a lower layer having a higher resolution on the basis of a motion amount obtained in an upper layer having a lower resolution. A first image layering means for forming a plurality of layered images representing stationary components of the input image signal based on the input image signal; and a transient component of the input image signal based on the input image signal. A first image layering means for forming a plurality of layered images, and a first layer representing a motion of the image by the block matching method for each layer using the layered images obtained by the first image layering means. Evaluation value calculation means for obtaining an evaluation value and for obtaining a second evaluation value representing the movement of the image by the block matching method for each layer using the layer image obtained by the second image layering means; A motion amount detecting means for obtaining a plurality of motion amount candidates based on the respective minimum values of the first and second evaluation values and detecting the most probable motion amount from the motion amount candidates by the gradient method. A motion amount detecting device characterized by being obtained. 8. In the highest hierarchy having the lowest resolution,
The movement amount is detected by executing the movement amount detecting means, and in the other layers, the minimum value of the third evaluation values obtained by weighted averaging the first and second evaluation values is calculated. The motion amount detecting apparatus according to claim 7, wherein the motion amount is detected by obtaining the motion amount.