CN201374785Y - Image processing device for determining motion vectors of interpolated frames - Google Patents

Abstract

The utility model discloses an image processing device for determining a moving vector of an interpolated image, which helps to improve the image quality of performing image update frequency conversion operation. The image processing device includes: a storage unit for at least storing a previous original picture and a subsequent original picture of the interpolation picture and a starting position of a previous interpolation picture; and a calculation unit coupled to To the storage unit, including: a candidate vector generation module, used to determine the plurality of first candidate vectors and the plurality of second candidate vectors of the plurality of blocks of the interpolation picture; and a motion vector determination module, based on the interpolation picture The first candidate vectors of the blocks, the second candidate vectors and the start position of the previous interpolation frame determine a block located in the image coverage area/image exposure area of the interpolation frame A moving vector of .

Description

Image processing device for determining motion vectors of interpolated frames

technical field

The utility model relates to an image processing mechanism, in particular to an image processing device for determining a moving vector of an interpolation block.

Background technique

At present, when the traditional image interpolation mechanism determines the motion vector of an interpolation block in an interpolation frame, it directly determines the motion vector based on the operation result of the block matching algorithm, and generates the motion vector accordingly. The image of the interpolated block. Please refer to FIG. 1 , which is a schematic diagram of the operation of the block comparison algorithm. As shown in Figure 1, the image frames F2 and F3 are the front and rear frames of the input image, and the frame Finter is an interpolation frame generated by the traditional image interpolation mechanism, and A'~L' represent the images For the background images in screens F2 and F3, the dotted arrows in the figure indicate the moving direction of the background images, while the solid arrows indicate the moving direction of a foreground object. In the image screen F2, the background images of F'~I' are drawn by the The foreground object is covered, and in the next image frame F3, the background image of C'~F' is covered by the foreground object; due to the block comparison algorithm, the front and rear two image frames F2, F3 The background image A', B', J', K', L' can be found in all of them, so the correct target moving vector of the corresponding block can be determined, and the background image A', B', J', K ', L' are presented on the screen Finter (as shown in Figure 1); in addition, the block comparison algorithm can also find the image of the foreground object in the front and rear two image frames F2, F3, therefore, can determine the corresponding The correct target movement vector corresponding to the block, and the image of the foreground object is presented on the screen Finter.

However, when determining the target motion vectors of the interpolation blocks in the regions Rinter and Rinter', ideally the target motion vectors of the interpolation blocks in the regions Rinter and Rinter' should be background motion vectors so that the regions Rinter and Rinter The background image can be presented in ', for example, the area Rinter should ideally present the background image C', D', and the area Rinter' should ideally present the background image H', I', however, in practice, due to block comparison The algorithm cannot find background images C', D' (covered by foreground objects) in image frame F3, and cannot find background images H', I' (covered by foreground objects) in image frame F2 ), therefore, the block comparison algorithm cannot determine the correct motion vector in the way of general image comparison, resulting in the distortion of the actual image presented by the interpolation area Rinter and Rinter'. When applied to the frame rate When converting, the traditional image interpolation mechanism will greatly reduce the quality of the output image.

Utility model content

The technical problem to be solved by the utility model is to provide an image processing device for determining the motion vector of the interpolated image, which helps to improve the image quality of the image update frequency conversion operation.

In order to solve the above technical problems, the utility model provides the following technical solutions:

The utility model provides an image processing device. The image processing device is used to determine the motion vector of the image coverage area/image revealing area of an interpolation frame when interpolating the moving image, wherein the interpolation frame is composed of a plurality of blocks, and the image processing device includes a storage The unit and a calculation unit, and the calculation unit further includes a candidate vector generation module and a motion vector determination module. The storage unit is used to at least store a previous original picture and a subsequent original picture of the interpolated picture and a starting position of a previous interpolated picture; the candidate vector generation module is used to determine the interpolated picture A plurality of first candidate vectors and a plurality of second candidate vectors of a plurality of blocks; the motion vector determining module is based on the first candidate vectors, the second candidate vectors and the previous one of the blocks of the interpolation frame The starting position of the interpolation frame determines the motion vector of a block located in the image coverage area/image revealing area of the interpolation frame.

The image processing method and related device used in the present invention to determine the motion vector of the interpolation screen can correctly calculate the target movement of the block for the block belonging to the image coverage area or image exposure area in the interpolation screen Vector, so that a better image can be obtained when performing image interpolation with the target motion vector, which is helpful to improve the image quality of performing the frame refresh rate conversion operation.

Description of drawings

FIG. 1 is a schematic diagram of the operation of a traditional block comparison algorithm.

FIG. 2 is a schematic diagram of the components of the first embodiment of the image processing device of the present invention.

FIG. 3( a ) is an operation flowchart of the image processing device shown in FIG. 2 .

FIG. 3( b ) is an operation flowchart of the second embodiment of the image processing device shown in FIG. 2 .

FIG. 4 is a schematic diagram of an output image generated when the computing unit shown in FIG. 2 performs frame refresh rate conversion.

FIG. 5( a ) is a schematic diagram of an example of the calculation unit shown in FIG. 2 calculating a value representing the variation degree of the motion vector.

FIG. 5( b ) is a schematic diagram of an example of a first numerical curve CV including a plurality of motion vector variation values.

FIG. 6( a ) is a schematic diagram of an example of block comparison difference curves DCV respectively calculated by the calculation unit shown in FIG. 2 by using the candidate vector MV1 to perform a block comparison operation.

FIG. 6( b ) is a schematic diagram of an example of the block comparison difference curve DCV' calculated by the calculation unit shown in FIG. 2 using the candidate vector MV2 to perform the block comparison operation.

FIG. 7 is a schematic diagram of an example of initial positions P1-P4 and P1'-P4' determined by the calculation unit shown in FIG. 2 .

[Description of main component symbols]

200 image processing devices 205 computing units

210 storage unit 215 candidate vector generation module

220 Motion Vector Decision Module 225 Image Generation Module

2151 Image difference value generation module 2152 Candidate vector decision module

2201 Starting position determination module 2202 Vector selection module

Detailed ways

First of all, for the convenience of reading, the background image area that is not covered by the foreground object in the previous image frame but is covered by the foreground object in the next image frame is called the image coverage area (covered area) , and the background image area that is covered by the foreground object in the previous image frame but appears in the next image frame (not covered by the foreground object) is called the image revealing area (uncovered area); for example , the background images C', D', and E' shown in Figure 1 are image coverage areas, while the background images G', H', and I' are image exposure areas; please note that the above definitions are only used to facilitate the description of this The operation of the embodiments of the utility model is not a limitation of the utility model.

Please refer to FIG. 2 and FIG. 3(a), which respectively illustrate the flow of the image processing device 200 and its related operating steps in the first embodiment of the present utility model; for the convenience of description, the following are the steps in conjunction with FIG. 3(a) To illustrate the operation of the image processing device 200 in FIG. 2, it should be noted that, if substantially the same result can be achieved, it is not necessary to follow the sequence of steps in the flow shown in FIG. 3(a), and FIG. 3 The steps shown in (a) do not have to be performed consecutively, ie other steps can also be interposed therein. As shown in FIG. 2, the image processing device 200 includes a calculation unit 205 and a storage unit 210. The image processing device 200 is used to determine the motion vector of at least the image coverage area/image revealing area of an interpolation frame when interpolating the moving image. , and the interpolation picture is composed of a plurality of blocks; wherein the storage unit 210 is used to at least store the previous original picture and the next original picture of an interpolation picture and the image overlay in the previous interpolation picture A starting position (starting position/point) of the area/image revealing area, the calculation unit 205 is then coupled to the storage unit 210, and includes a candidate vector generation module 215, a motion vector determination module 220 and an image generation module 225, the candidate The vector generation module 215 is used to determine the plurality of first candidate vectors (also referred to as left motion vectors) and the plurality of second candidate vectors (also referred to as right motion vectors) of the plurality of blocks of the interpolated picture, and the motion vector determination module 220 The image coverage area/image revealing area located in the interpolation frame is determined according to the left movement vectors, right movement vectors of the blocks of the interpolation frame and the initial position of the previous interpolation frame The corresponding motion vector of each block, and the image generation module 225 generates the image of each block in the image coverage area/image exposure area of the interpolation frame according to the motion vector determined by the motion vector determination module 220 .

In detail, the candidate vector generation module 215 further includes an image difference value generation module 2151 and a candidate vector determination module 2152. The image difference value generation module 2151 is determined according to the complex original motion vectors of a plurality of adjacent blocks of a block An image difference value of the block (step 305), the candidate vector determination module 2152 respectively determines the left motion vectors and the right motion vectors of the multiple consecutive blocks according to the multiple image difference values of the multiple consecutive blocks in one-dimensional space vector (step 310). In addition, the motion vector determining module 220 includes a starting position determining module 2201 and a vector selecting module 2202. The starting position determining module 2201 utilizes the left moving vectors and the right moving vectors of the plurality of blocks of the interpolation frame. One of the moving vectors is used to calculate the comparison difference of the plurality of blocks corresponding to the plurality of blocks, and the starting point of the interpolation frame is determined according to the numerical change of the comparison difference of the plurality of blocks. start position (step 315), and the vector selection module 2202 generates a reference vector according to the start position of the interpolation frame and the start position of the previous interpolation frame, and compares the reference vector with the interpolation For the corresponding left and right motion vectors of each block in the image coverage area/image exposure area of the screen, select one of the corresponding left and right motion vectors that is less similar to the reference vector as the each area Target movement vector for the block (step 320).

Please note that the original motion vector in the first embodiment is obtained by comparing the blocks of the previous original frame and the next original frame, and the step of determining the starting position of the interpolated frame (that is, step 315 ) can first use one of the left movement vectors and the right movement vectors to calculate the comparison difference of the plurality of blocks corresponding to the plurality of blocks, and then compare the calculated blocks according to The starting position is determined by the numerical change of the difference, and the detailed operation will be explained later. It is worth noting that in the first embodiment, the image processing device 200 does not determine the initial position of the image coverage area/image revealing area of the previous interpolation frame, in other words, it assumes that the previous interpolation frame In the case of the initial position of the image coverage area/image exposure area, only the initial position of the image coverage area/image exposure area in the current interpolation frame is calculated, without additional calculation of the image coverage of the previous interpolation frame The starting position of the region/image revealing region.

In addition, in the second embodiment of the present utility model, with reference to the flowchart of FIG. 3(b), the candidate vector generation module 215 in FIG. 2 can also be designed to determine at least one first interpolation frame Candidate vectors (referred to as left and right motion vectors) MV1, MV2 of the interpolation block (respectively referred to as left and right motion vectors) (step 405), and candidate vectors MV1', MV2' of at least one second interpolation block of a second interpolation frame are determined (Step 410). In practice, in this embodiment, the candidate vector generation module 215 generates two corresponding candidate vectors for each interpolation block in the first and second interpolation frames. The size of each block is For example, it includes a pixel range of 8×8; in addition, the motion vector determination module 220 is designed to use at least one candidate vector (such as MV1) of the first interpolation block and one candidate vector of the second interpolation block ( For example, MV1') to calculate a plurality of block comparison differences corresponding to a plurality of interpolation blocks in the first and second interpolation frames (step 415 and step 420), and compare the difference values of these blocks Store in the storage unit 210, and then determine at least one starting position in the first interpolation frame and at least one starting position in the second interpolation frame according to the numerical changes of the above-mentioned block comparison differences (step 425), Wherein the two starting positions are edge positions of an image coverage area or an image revealing area, and the motion vector determination module 220 will determine according to the starting position in the first interpolation frame and the starting position in the second interpolation frame Determine the target motion vector of the second interpolation block (step 430), wherein there is at least one non-image interpolation frame between the first and second interpolation frames, that is, the first interpolation frame can be the second interpolation frame For example, please refer to FIG. 4, which shows the calculation unit 205 shown in FIG. 2 for a plurality of input images (frames or fields, in This only shows Fn-1, Fn, Fn+1) for frame rate conversion to generate multiple output images, for example, frame Fn-1, Fn, Fn+1, etc. are 60Hz input Frames, the calculation unit 205 converts the input frames to the frame update frequency to generate a frame with 120 Hz; the frequency of 60 Hz and 120 Hz is not limited by the present invention. Although the following operations are described by taking the second embodiment as an example, the operations are also applicable to the first embodiment, and will not be repeated for brevity.

In actual processing, the image processing device 200 generates an interpolation frame (the first interpolation frame F') among the frames Fn-1 and Fn, and generates another interpolation frame (the first interpolation frame F') among the frames Fn and Fn+1. Two interpolation frames F"), the second interpolation frame F" is the frame to be interpolated at present and the first interpolation frame F' is the frame produced by interpolation of the previous image; in another embodiment , the first interpolation picture also can be the second interpolation picture F " the picture produced by image interpolation after ", this does not violate the technical spirit of the present utility model. With the picture picture Fn-1, Fn, Fn of Fig. 4 +1 As an example, the moving direction of the foreground object is moving horizontally from right to left, while the background image is moving horizontally from left to right; it should be noted that, for the convenience of explanation, only the example of horizontal movement is used as an illustration However, image movement in any specific direction (vertical direction or diagonal direction of the screen, etc.) can be processed by the embodiments of the present invention.

The following description first explains the actual operation of the candidate vector generating module 215 generating two candidate vectors for each interpolation block of each interpolation frame (including F', F", etc.); here, the frame F' is refers to the first interpolation frame, and frame F" refers to the second interpolation frame. First, the image difference value generation module 2151 calculates the motion vectors of all interpolated blocks according to the block matching algorithm, and then uses the multiple motions of each calculated motion vector and its multiple adjacent blocks vector to calculate the degree of confusion of the moving vector to calculate a numerical curve, wherein, in this embodiment, the numerical curve is a curve formed by the section of the degree of confusion of the moving vector along the horizontal direction of a certain interpolation block , the numerical curve is the difference value of a plurality of consecutive blocks in one-dimensional space, and the degree of confusion represents the variation value of the motion vector (motion vector variance), that is, the numerical curve contains a number representing different A plural number of values for the degree of variation of the motion vector. Please refer to Fig. 5(a), Fig. 5(a) is a schematic diagram of an example of calculating the value representing the degree of variation of the motion vector in the present invention; for example, if the first interpolation block and its multiple adjacent blocks (as shown in Fig. 5(a ), located in the block range of 5×5), the complex motion vectors calculated by the block comparison algorithm are MV00 and MV-2-2~MV22 respectively, and a motion vector can be calculated according to these motion vectors Vector variation value MV_VAR, its algorithm is to take the absolute value obtained by subtracting the smallest horizontal component from the largest horizontal component in the moving vectors, plus the absolute value obtained by subtracting the smallest vertical component from the largest vertical component in the moving vectors, MV_VAR can be represented by the following equation:

MV_VAR＝|MAX(MV _x )-MIN(MV _x )|+|MAX(MV _y )-MIN(MV _y )|equation (1)

Where MVx and MVy respectively represent the horizontal component (x-axis component) and the vertical component (y-axis component). It should be noted that the block range of 5×5 is not a limitation of the present invention, and it can also be implemented by using a block range of N×N or N×M, wherein the parameters N and M are both positive integers and N is not equal to M; in addition, the method of calculating the motion vector variation value MV_VAR can also be realized by using equation (2) or equation (3):

MV_VAR＝|MAX(MV _x )-MIN(MV _x )|+|MAX(MV _y )-MIN(MV _y )|+SAD

Equation (2)

MV_VAR＝α×{|MAX(MV _x )-MIN(MV _x )|+|MAX(MV _y )-MIN(MV _y )|}+β×SAD

Equation (3)

Among them, the numerical value SAD is a block comparison difference calculated by the first interpolation block according to the block comparison method, and the parameters α and β are weighting parameters; A change in implementation all belongs to the category of the present utility model. From the above, according to one of equation (1), equation (2) or equation (3), the calculation unit 205 performs calculations for different interpolation blocks one by one, so that the first value curve CV can be obtained , as shown in Figure 5(b).

FIG. 5( b ) shows the first numerical curve CV of the present invention including a plurality of motion vector variation values. MB00 is the first interpolation block, and in FIG. 5(b), the candidate vector determination module 2152 determines the two candidate vectors MV1 and MV2 of the first interpolation block MB00. From the perspective of the direction (for example, the horizontal direction), the candidate vector determination module 2152 spatially extends a plurality of two sides of the first interpolation block MB00 (for example, six on each side, MB10~MB60 and MB-10~MB-60) For the corresponding motion vector variation value of the block, a maximum value (such as VARmax shown in Figure 5(b)) is taken out within the range of the corresponding motion vector variation value, and within the range of the corresponding motion vector variation value Find the blocks corresponding to the minimum values on the left and right sides of the maximum value VARmax, for example, you can find blocks MB-40 and MB50, and the left and right blocks MB-40 and MB50 are respectively The motion vectors calculated by the block comparison algorithm are the two candidate vectors MV1 and MV2 of the first interpolation block MB00. In other words, the candidate vector MV1 of the first interpolation block MB00 corresponds to the first value The minimum value VARmin on the left side of the maximum value VARmax of the curve CV is also called the candidate vector MV1 as the left moving vector of the first interpolation block MB00, and its candidate vector MV2 corresponds to the pole located on the first value curve CV The minimum value VARmin' on the right side of the large value VARmax is also called the candidate vector MV2 as the right motion vector of the first interpolation block MB00; and one of the candidate vectors MV1 and MV2 corresponds to the background motion vector (background motionvector), and the other corresponds to the foreground motion vector (foreground motion vector), this is because the motion vector variation value around the interpolation block belonging to the image coverage area or image exposure area will be quite large, while the left and right The image block corresponding to the minimum motion vector variation value on both sides corresponds to a foreground motion vector or a background motion vector, depending on whether it is located in the image coverage area or the image exposure area. Therefore, if the first interpolation block MB00 is located in one of the image coverage area and the image exposure area, one of its candidate vectors MV-1 and MV2 corresponds to the background motion vector, and the other corresponds to the foreground Motion vector; it should be noted that the two candidate vectors (or called left and right motion vectors) in the embodiment of the present utility model are substantially a foreground vector and a background vector respectively, and the two candidate vectors of a block during actual operation It may not be exactly the foreground and background vectors, but the image processing device 200 of the present invention is also applicable to this situation. Through the above operations, the candidate vector determination module 2152 can calculate two corresponding candidate vectors MV1 and MV2 for each interpolation block in the first interpolation frame F'.

Please refer to FIG. 6(a), which shows the block comparison operation performed by the starting position determination module 2201 of the present invention using the left candidate vector MV1 of each interpolation block in the first interpolation frame F' A plurality of block comparison differences (represented by the curve DCV shown in FIG. For the numerical change of the difference value DCV, the starting position determination module 2201 can determine a plurality of starting positions of the first interpolation frame F', and these starting positions are located at the edge of the image coverage area (or image revealing area) , so it can be used to determine the actual image area of the image coverage area (or image exposure area). In practice, for the curve DCV (including a plurality of block comparison differences calculated using the candidate vector MV1), the starting position determination module 2201 detects the magnitude of the value change, when the curve DCV (including the candidate vector MV1) When a value change from left to right among the multiple block comparison differences calculated by vector MV1 changes from small to large and includes a block comparison difference reaching a predetermined value TH, the starting position determination module 2201 will use the block position corresponding to the block comparison difference as a starting position in the first interpolation frame, which is also called the starting point of the image coverage area (or image revealing area); for example, as shown in FIG. 6 For (a), the starting position determination module 2201 determines the starting position P1 corresponding to the block comparison difference D1 and the starting position P2 corresponding to the block comparison difference D2 so that the image coverage can be determined. area (or image exposure area), this is because in the horizontal direction of the image frame Fn-1, Fn, the foreground object system moves from right to left, and the background image system moves from left to right, using the image The candidate vector MV1 of the block in the range R1 (actually corresponding to the background motion vector at this time) is used to calculate the comparison difference of a block corresponding to it, which is taken before and after the first interpolation frame F' The images in the image ranges R1' and R1" of the image frames Fn-1 and Fn are calculated. Since the images in the image ranges R1' and R1" are the same/similar background images, the image range R1 corresponds to. The calculated block comparison differences are quite small; when the calculated blocks in the image range are actually located in the image coverage area (for example, the blocks in the image range R2), then use the candidate vector MV1 (actually at this time Still corresponding to the background motion vector) the block comparison difference calculated will start to become larger, this is because no identical or Similar images, wherein the image range R2' is the background image, and the image range R2" is the foreground image; in terms of the calculated block comparison difference corresponding to a starting position between the image ranges R1 and R2, Its numerical value will suddenly increase from left to right, from small to large, and be greater than the predetermined value TH, so the initial position P1 corresponding to the block comparison difference D1 is an edge position of the image coverage area (or image exposure area) , please note that at this moment, it is only known that the starting position P1 is the edge position of the image coverage area or the image exposure area, and the calculation unit 205 needs to perform other calculations to know that the starting position P1 is actually the edge position of the image coverage area .

In addition, using the candidate vector MV1 of the block in the image range R3 (actually corresponding to the foreground motion vector at this time) to calculate the comparison difference of a block corresponding to it, the image frames Fn-1, Fn The images in the image range R3' and R3" are calculated, and because the images in the image range R3' and R3" are the same/similar foreground images, the calculated block ratio corresponding to the image range R3 is poor The values are quite small; when the calculated block in the image range is actually located in the image exposure area (for example, the block in the image range R4), then use the candidate vector MV1 (actually still corresponds to the foreground motion vector at this time) The calculated block comparison difference will start to increase, this is because the same or similar images cannot be found within the image ranges R4' and R4" of the image frames F1 and F2, and the image range R4' is the foreground image , the image range R4" is the background image; taking the calculated block ratio difference corresponding to a starting position between the image range R3 and R4, its numerical value will change suddenly from left to right, from small to large increase and be greater than the predetermined value TH, so the initial position P2 corresponding to the block comparison difference D2 is an edge position of the image coverage area (or image exposure area). Please note that only the initial position P2 is known at this time. The edge position of the image coverage area or the image revealing area needs to wait for the calculation unit 205 to perform other calculations to know that the starting position P2 is actually the edge position of the image revealing area.

Similarly, as shown in FIG. 6(b), the starting position determination module 2201 can use the right candidate vector MV2 of each interpolation block in the first interpolation frame F' to perform a block comparison operation to calculate The plurality of block comparison differences corresponding to the plurality of interpolation blocks of the first interpolation frame F' (represented by the curve DCV' shown in FIG. 6(b)); As the value of DCV' changes, the starting position determining module 2201 can determine two starting positions P3 and P4 in the first interpolation frame F', and these starting positions are located in the image coverage area (or image revealing area) Therefore, the edge can be used to determine the actual image area of the image coverage area (or image exposure area). In practice, for the curve DCV', the initial position determination module 2201 is to detect the magnitude of the value change, when the curve DCV' (including the plurality of block comparison differences calculated using the candidate vector MV2) moves from left to right When the value to the right changes from large to small and includes a block comparison difference reaching a predetermined value TH', the starting position determination module 2201 will use the block position corresponding to the block comparison difference to determine As a starting position in the first interpolation frame, it is also called the starting point of the image coverage area (or image revealing area); for example, as shown in FIG. The starting position P3 corresponding to the difference D3 and the starting position P4 corresponding to the block comparison difference D4 are determined to determine the edge position of the image coverage area (or image exposure area), because when the calculated When the blocks in the image range are actually located in the image coverage area (such as the blocks in the image range R5), the candidate vector MV2 of the block in the image range R5 (actually corresponding to the foreground motion vector at this time) is used to calculate The corresponding block comparison difference is calculated by taking the images in the image ranges R5' and R5" of the image frames Fn-1 and Fn to calculate, and because the images in the image ranges R5' and R5" are different background image, so the calculated block comparison difference corresponding to the image range R5 is larger; and when the calculated block in the image range is located in the foreground object image area (for example, the block in the image range R6 ), then using the candidate vector MV2 (actually still corresponding to the foreground motion vector at this time), the difference value of a block comparison calculated will become smaller, because in the image range R6' of the two image frames F1 and F2 The image in R6" is the same foreground object image; therefore, for the calculated block comparison difference D3 corresponding to a starting position P3 between the image range R5 and R6, the value change will be from left to right. To the right, from large to small suddenly decreases and is less than the predetermined value TH', so the initial position P3 corresponding to the block comparison difference D3 is an edge position of the image coverage area (or image exposure area), please note that, At this moment, it is only known that the initial position P3 is the edge position of the image coverage area or the image exposure area, and the calculation unit 205 needs to perform other calculations to know that the initial position P3 is actually the edge position of the image coverage area.

In addition, when the calculated block in the image range is actually located in the image exposure area (for example, the block in the image range R7), use the candidate vector MV2 of the block in the image range R7 (in this case, it actually corresponds to the background moving vector) to calculate the comparison difference of a block corresponding to it, the images in the image ranges R7' and R7" of the image frames Fn-1 and Fn are used for calculation, and because the image ranges R7' and R7" The images are different images, in which the image range R7' is the foreground object image, and the image range R7" is the background image, so the calculated block comparison differences corresponding to the image range R7 are all the same and when the calculated block in the image range is actually located in the background image area (such as the block in the image range R8), the candidate vector MV2 (actually still corresponding to the background motion vector at this time) is used to calculate The comparison difference of a block will begin to decrease, this is because the same/similar background images are within the image ranges R8' and R8"; therefore, for a starting position P4 between the image ranges R7 and R8 corresponds to For the calculated block comparison difference D4, its numerical change will suddenly decrease from left to right, from large to small, and be smaller than the predetermined value TH', so the initial position P4 corresponding to the block comparison difference D4 That is, an edge position of the image coverage area (or image exposure area), please note that at this moment, only the starting position P4 is known to be the edge position of the image coverage area or image exposure area, and it needs to wait for the calculation unit 205 to perform other calculations to be able to It is known that the starting position P4 is actually the edge position of the image display area.

As mentioned above, the starting position determination module 2201 uses its left and right candidate vectors MV1 and MV2 to perform a block comparison operation for each interpolation block in the first interpolation frame to obtain the block comparison difference value (such as the curve DCV shown in Figure 6(a) and the curve DCV' shown in Figure 6(b)); the starting position determination module 2201 can also be used for each interpolation block in the second interpolation frame The left and right candidate vectors MV1', MV2' perform a block comparison operation to obtain a plurality of other block comparison differences, and also use these other block comparison differences to obtain image coverage areas (or The starting positions (or starting points) P1 ′˜P4 ′ of the edge of the image exposure area); since the calculation process is similar to the above-mentioned calculation process, it will not be repeated here.

After the initial position determination module 2201 calculates the initial positions P1~P4 and P1'~P4', the vector selection module 2202 can already know the first interpolation frame and the The area where an image coverage area (or an image exposure area) actually exists in the second interpolation frame, for example, the vector selection module 2202 goes to the right according to the initial position P1 (or P1') and according to the initial position P3 (or P3 ') to the left is the image coverage area or image exposure area in the first interpolation frame (or the second interpolation frame), and to the right according to the starting position P2 (or P2') And the image area found to the left according to the starting position P4 (or P4') is the image coverage area or image exposure area in the first interpolation frame (or the second interpolation frame). It is worth noting that, The image area found here has not yet been confirmed as an image coverage area or an image exposure area. It is only known to be one of the image coverage area and image exposure area, depending on the candidates of the blocks in the image area found The vector difference of vector MV1', MV2' is used to further judge whether the image area is an image coverage area or an image exposure area, and its detailed operation will be described later; in addition, when the vector difference of candidate vectors MV1', MV2' is more When is larger, the actual image area of the image coverage area (or image exposure area) will also be larger, and vice versa.

As mentioned above, after the initial position determination module 2201 calculates the initial positions P1~P4 and P1'~P4', the vector selection module 2202 can determine the above-mentioned first position according to the initial positions P1~P4 and P1'~P4'. A target motion vector of the second interpolation block, the target motion vector is used to determine/generate the actual image of the second interpolation block. Specifically, please refer to FIG. 7, which is a schematic diagram of an example of the initial positions P1~P4 and P1'~P4' determined by the initial position determination module 2201; the vector selection module 2202 will refer to P1, P1', and P2 respectively and P2', P3 and P3', and P4 and P4' to calculate reference vectors V1-V4, so as to select one of them from the candidate vectors MV1' and MV2' of the second interpolation block according to at least one of the reference vectors V1-V4 One of the reference vectors V1-V4 is used to determine the target motion vector, wherein the reference vectors V1-V4 all correspond to or are close to the foreground motion vector. In practice, the vector selection module 2202 compares at least one of the reference vectors V1-V4 with the candidate vectors MV1', MV2', when one of the reference vectors V1-V4 is close to one of the candidate vectors MV1', MV2', the vector selection module 2202 will select another of the candidate vectors MV1', MV2' as the target motion vector, so , if the second interpolation block is actually located in the image coverage area or image exposure area, the determined target motion vector will correspond to or be close to the background motion vector, for example, if the second interpolation block is as shown in Figure 7 MB, then the candidate vector MV1' corresponds to or is close to the background motion vector and the candidate vector MV2' corresponds to or is close to the foreground motion vector. In this way, the vector selection module 2202 will select the candidate vector MV1' corresponding to or close to the background motion vector As the target motion vector, the image of the interpolation block MB is generated according to the determined target motion vector; and if the second interpolation block is MB' shown in FIG. 7, the candidate vector MV1' corresponds to or It is close to the foreground motion vector and the candidate vector MV2' is corresponding to or close to the background motion vector. The vector selection module 2202 will select the candidate vector MV2' corresponding to or close to the background motion vector as the target motion vector, and the target motion determined accordingly vector to generate the image of the interpolated block MB'.

In addition, if the second interpolation block is not actually located in the image coverage area or the image exposure area, the vector selection module 2202 can still determine the correct motion vector, for example, if the second interpolation block is as shown in FIG. 7 MB", the candidate vectors MV1' and MV2' are all corresponding to or close to the background motion vector. Therefore, no matter whether the target motion vector of the interpolation block MB" is determined based on the reference vectors V1~V4, the vector selection module 2202 arbitrarily Selecting one of the two second candidate vectors MV1', MV2' as its target motion vector is actually a motion vector that can generate a better background image frame for the interpolation block MB"; in addition, if the second The interpolation block is MB"' shown in FIG. 7, and its candidate vectors MV1' and MV2' are both corresponding to or similar to the foreground motion vector. The vector selection module 2202 randomly selects one of the two candidate vectors MV1' and MV2' One is the target motion vector, which is actually a motion vector for generating a better foreground object image frame for the interpolation block MB"'.

In another embodiment, the target motion vector can also be determined by selecting one of the candidate vectors MV1' and MV2' of the second interpolation block only by a single reference vector, which also falls within the scope of the present invention It should be noted that in this embodiment, for example, if the target moving vector is determined only by the reference vector V1, the vector selection module 2202 does not need to calculate the reference vectors V2~V4, however, the starting point still needs to be determined Start positions P1~P4 and P1'~P4', so as to judge the actual image area of the image coverage area and the image exposure area, in other words, the embodiment of the present utility model only determines the two start positions (P1 and P1', P2 and A group of P2', P3 and P3', and P4 and P4') can determine the target motion vector of the second interpolation block.

The actual operation of generating the image of the second interpolation block is described in detail below. First of all, since the embodiment of the present invention mainly takes the example of moving the image horizontally as an illustration, the direction of the moving vector from left to right is positive, otherwise, its relative direction (from right to left) is negative; however , this is not a limitation of the utility model, and any example in which a certain direction is positive in a two-dimensional plane space, and its relative direction is negative, all conform to the spirit of the utility model. The calculation unit 205 determines whether the second interpolation block is located in the image coverage area or in the image exposure area by subtracting a vector difference obtained by subtracting another candidate vector from one candidate vector of the second interpolation block, Please refer to FIG. 7 again. For example, for the interpolation block MB in FIG. 7, the image generation module 225 in the calculation unit 205 subtracts the candidate vector MV2' from the candidate vector MV1' of the interpolation block MB, The obtained vector difference is positive, so it is judged that the interpolated block MB belongs to the image coverage area, and then the corresponding block is obtained from the previous non-interpolated frame Fn of the second interpolated frame according to the determined target motion vector The image of the interpolation block MB is used as the image of the interpolation block MB; in addition, for the interpolation block MB' in FIG. The obtained vector difference is negative, so it is judged that the block MB belongs to the image exposure area, and then according to the determined target moving vector to obtain a corresponding block in a non-interpolation frame Fn+1 after the second interpolation frame The image is used as the image of the interpolation block MB'; Therefore, the image generation module 225 can use the judgment result to select one of the images of the non-interpolation frame before the second interpolation frame and the image of the next non-interpolation frame as the second interpolation area block image.

To sum up, for the block belonging to the image coverage area or the image exposure area in the interpolation frame, the image processing device 200 in the embodiment of the present invention can correctly calculate the target motion vector of the block, so that the When the target moving vector is used for image interpolation, a better image can be obtained, which is helpful to improve the image quality of performing the frame refresh rate conversion operation.

The above descriptions are only preferred embodiments of the present utility model, and all equivalent changes and modifications made according to the patent scope of the present utility model shall fall within the scope of the present utility model.

Claims (3)

1. An image processing device, used for determining a motion vector of an image coverage area/image exposure area of an interpolation image when interpolating a moving image, the interpolation image is composed of a plurality of blocks, characterized in that, The image processing device includes: a storage unit for at least storing a previous original picture and a subsequent original picture of the interpolated picture and a starting position of a previous interpolated picture; and A computing unit, coupled to the storage unit, includes: A candidate vector generation module, used to determine the plurality of first candidate vectors and the plurality of second candidate vectors of the plurality of blocks of the interpolated picture; and A motion vector determination module, based on the first candidate vectors, the second candidate vectors and the initial position of the previous interpolation frame of the blocks of the interpolation frame, determines the location of the interpolation frame A motion vector for a block of the image footprint/image reveal area. 2. The image processing device according to claim 1, wherein the candidate vector generating module comprises: An image difference value generating module, which determines an image difference value of the block according to the plurality of original motion vectors of the plurality of neighboring blocks of the block; and A candidate vector determination module determines the plurality of first candidate vectors and the plurality of second candidate vectors of the plurality of continuous blocks respectively according to the complex image difference values of the plurality of continuous blocks in one-dimensional space. 3. The image processing device according to claim 1, wherein the motion vector determination module comprises: A starting position determination module, using one of the first candidate vectors and the second candidate vectors of the blocks in the interpolation frame to calculate a plurality of block comparisons corresponding to the blocks difference, and determine a starting position of the interpolation frame according to the numerical change of the block comparison difference; and A vector selection module generates a reference vector according to the initial position of the interpolation frame and the initial position of the previous interpolation frame, and compares the reference vector with the image coverage area/image exposure of the interpolation frame The first and second candidate vectors of the block in the region, and one of the first and second candidate vectors that is less similar to the reference vector is selected as the motion vector of the block.

Images