JP3416378B2 - Video camera with image stabilization function - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video camera with a camera shake correction function, and more particularly to a video camera with a camera shake correction function which is applied to, for example, a VTR with a built-in camera for consumer use and corrects the camera shake on the screen based on the correlation value of an image. Regarding video cameras.

[0002]

2. Description of the Related Art One example of a conventional video camera with a camera shake correction function is disclosed in Japanese Patent Laid-Open No. 7-38800 (H04).
N 5/232). This prior art is
The average correlation value and the minimum correlation value are calculated for each of the four detection areas set on the screen, and only the detection area satisfying the condition of the average correlation value / minimum correlation value> threshold is set as the effective area, and the portion obtained in the effective area. The camera shake is corrected from the motion vector.

[0003]

However, in such a conventional technique, when an object passes through the screen, the average correlation value / minimum correlation value varies near the threshold depending on the contrast between the object and the background. As a result, the image stabilization sometimes did not work well. Specifically, when an object enters the screen, the average correlation value / minimum correlation value should be less than or equal to the threshold value, but it continues to exceed the threshold value and incorrect camera shake correction is performed, or the object goes out of the screen. In some cases, even though the average correlation value / minimum correlation value should exceed the threshold value, it did not exceed the threshold value, and the camera shake correction was not restarted forever. Among them, it has been an eyesore for the operator that the camera shake correction is not restarted especially after the object comes out.

[0004]

SUMMARY OF THE INVENTION The present invention forms on a screen.
Image phases obtained individually from the detected areas
Shake correction function to correct the shake of the screen based on the function value
In the attached video camera, average correlation value calculating means for calculating an average correlation value by averaging a plurality of image correlation values, average correlation value
Difference detecting means for detecting the difference between the reference average correlation value and
The condition that minutes are smaller than the specified value for the specified period
To determine whether or not satisfy
If you are satisfied, you must perform image stabilization and satisfy the conditions.
Specially equipped with correction control means for interrupting the image stabilization.
This is a video camera with an image stabilization function.

[0005]

According to the present invention, there is provided a correlation value calculating means for calculating a correlation value of an image for each of a plurality of detection areas set on a screen, and a camera shake correcting means for correcting a shake of the screen based on the correlation value. When the average correlation value calculating means for calculating the average correlation value averaging the correlation values and the variation amount detecting means for detecting the variation amount of the average correlation value are provided, and the variation amount is smaller than the predetermined value for a predetermined period or more, A video camera with a camera shake correction function in which camera shake correction means is activated.

[0006]

The correlation value calculation means calculates the correlation value of the image for each of the plurality of areas, and the average correlation value calculation means calculates the average correlation value. The fluctuation amount detecting means detects the fluctuation amount of the average correlation value, and activates the camera shake correcting means when a period in which the fluctuation amount is smaller than a predetermined value continues for a predetermined period or more.

In the variation detecting means, for example, the difference detecting means detects the difference between the average correlation value and the reference average correlation value, and for example, the first comparing means compares the difference with a predetermined value. For example, the counter counts the time when the difference is smaller than the predetermined value, and, for example, the second comparing means compares the count value with the predetermined period. Then, for example, when the count value is equal to or greater than the predetermined period, the second comparison means activates the camera shake correction means.

The reference average correlation value is stored in the memory means, and for example, when the second comparing means activates the camera shake correction means, the reference average correlation value is updated by the newly detected average correlation value.

[0009]

According to the present invention, when the variation of the average correlation value is smaller than the predetermined value for a predetermined period of time or more, the camera shake correction means is activated. Therefore, the camera shake correction is performed promptly after the object has passed. Can be resumed. The above objects, other objects, features and advantages of the present invention are
It will become more apparent from the following detailed description of the embodiments with reference to the drawings.

[0010]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A video camera 10 having an image stabilization apparatus according to this embodiment shown in FIG. 1 is a C which converts an optical signal from a subject (not shown) input from a lens 14 into an electric signal.
It includes a solid-state image sensor 12 such as a CD. Solid-state image sensor 1
The electric signal from 2 is input to the camera circuit 16. As is well known, the camera circuit 16 includes a sample and hold circuit and samples and holds an electric signal from the solid-state image sensor 12. The sampled and held electrical signal is AGC (A
The gain is adjusted by the utomatic gain controller, and the sync signal is added by the sync signal adding circuit. In this way, the camera circuit 16 converts the image signal from the solid-state image sensor 12 into an analog video signal. This analog video signal is further converted into a digital video signal by the A / D converter 18, and the digital video signal is given to the motion detection circuit 20 and the average value fluctuation amount detection circuit 21. As the motion detection circuit 20, for example, an integrated circuit “L7A0” manufactured by Sanyo Electric Co., Ltd.
948 "is utilized. The integrated circuit also includes a memory control circuit 22 by which digital video signals are written field by field in field memory 24.

The motion detecting circuit 20 uses the well-known representative point matching method to detect each of the four motion detecting areas A, A shown in FIG.
For each of B, C, and D, the minimum correlation value, the average correlation value, and the position of the pixel indicating the minimum correlation value are obtained, and the average correlation value data is given to the average value fluctuation amount detection circuit 21. The value data and the position data are given to the microcomputer 26. Average value fluctuation amount detection circuit 2
1 detects the temporal variation of the average correlation value based on the average correlation value data, and activates the microcomputer 26 when the variation is smaller than a predetermined value and the state continues for a predetermined time or longer.

When activated, the microcomputer 26, based on the minimum correlation value data, the average correlation value data, and the position data, moves the motion vector of the entire screen or image field 30 (hereinafter, "whole motion vector").
Is calculated and the whole motion vector data is given to the memory control circuit 22. The memory control circuit 22 determines the read start address of the field memory 24 based on it, and reads the digital video signal from the address. That is, the memory control circuit 22 determines the field memory 24 as shown in FIG.
The image extraction area 32 formed in the above area is moved to read the digital video signal of the image extraction area 32.

However, since the image extraction area 32 cannot be moved without changing the digital video signal read from the field memory 24, the electronic zoom circuit 28 is used. The electronic zoom circuit 28 (FIG. 1) enlarges the image by using the interpolation method so that the size of the image extraction area 32 matches the image field 30. The digital video signal processed by the electronic zoom circuit 28 is D
The signal is converted into an analog signal by the / A converter 36 and output from the output terminal 38.

When the video camera 10 is shaken due to the vibration of the operator's hand as shown in FIG. 5, the subject moves to the lower left in the image field 30 (upper part in FIG. 5), or the image field 30 The subject moves to the upper right (Fig. 5, bottom). However, the image extraction area 32 is moved for each field according to the overall motion vector,
By enlarging the image in the image extraction area 32, the output image will be one in which the subject is well contained, as shown in the right side of FIG.

Referring to FIG. 6, the motion detection circuit 20 shown in FIG. 1 includes an input end 40 for receiving the digital video signal from the A / D converter 18, and the digital video signal input from the input end 40. Is supplied to the representative point memory 44 and the subtraction circuit 48 through the filter 42. The filter 42 is a digital low-pass filter and has an S / N ratio.
Improve the ratio and secure sufficient detection accuracy with a small number of representative points. The representative point memory 44 stores each motion detection area A shown in FIG.
The position data and luminance data of the representative point set to -D are stored in memory. In this embodiment, each motion detection area A-D is divided into 30 detection areas 46, and each detection area 46 is divided into 30 detection areas 46.
The position data and the brightness data of the representative point are stored.
Each detection area 46 is composed of, for example, 32 pixels × 16 rows as shown in FIG.

The subtraction circuit 48 subtracts the brightness data of the corresponding representative point one field before, which is given from the representative point memory 44, from the brightness data of all the pixels included in each detection area 46 of the current field. By taking the absolute value, the brightness difference between one field is obtained. As a result, the brightness difference between the brightness data of the current field and the brightness data of the previous field is obtained for each detection area 46. The cumulative addition circuit 50 cumulatively adds (30 in this embodiment) the luminance difference for each corresponding detection area 46 of the motion detection areas A to D, and outputs the cumulative addition data as image correlation value data. The arithmetic circuit 52 obtains the minimum correlation value and the average correlation value in each of the detection areas A to D and the position of the pixel indicating the minimum correlation value based on the correlation value data. The minimum correlation value data, the average correlation value data and the position data thus obtained are given to the microcomputer 26 from the output terminal 54. The calculation of such a correlation value is executed by the integrated circuit “L7A0948”.

Referring to FIG. 8, the average correlation value data input to the average correlation value variation detection circuit 21 is applied to the subtraction circuit 56, and the average correlation value data is written into the memory 58 as the reference average correlation value. Subtracted by the value data. The absolute value of the subtraction result is detected by the absolute value detection circuit 60, and the absolute value is compared with a predetermined value by the comparator 62. Here, the predetermined value is, for example, "50", and the comparator 62 outputs a high level signal when the absolute value is smaller than the predetermined value. The counter 64 is incremented for each field only when the high level signal is given from the comparator 62, and is reset when the low level signal is given. Comparator 6
6 compares the count value of the counter 64 with a predetermined value, for example, "30", and outputs a high level signal when the count value is a predetermined value or more. Therefore, the comparator 66 outputs a high level signal as an enable signal for the microcomputer 26 when a period in which the absolute value is smaller than "50" continues for 30 fields or more. Comparator 66
In addition, the switch SW1 is turned on by the high level signal to update the reference average correlation value data stored in the memory 58 with the newly calculated average correlation value data. The switch SW1 is controlled in this way when the subject before panning is different from the subject after panning such as when the video camera 10 is panned. It is also necessary to change. In order to initially store the average correlation value data in the memory 58, the switch SW1 is turned on by the microcomputer 26 for a predetermined period when the power is turned on.

When an object passes through the image field 30, the average correlation value changes as shown in FIG. That is,
The average correlation value becomes a value indicated by a solid line or a broken line depending on the background of the object. First, during the period when the object enters the motion detection area, the average correlation value continues to increase or decrease over time. That is, the temporal fluctuation amount becomes very large. During the period in which the object moves in the motion detection area, camera shake and movement of the object are mixed, and the average correlation value takes various values. However, the fluctuation amount of the average correlation value at this time is smaller than the fluctuation amount of the period. During the period in which the object leaves the motion detection area, the average correlation value changes greatly again, the average correlation value approaches the value before the object entered, and finally becomes the same value.

The average correlation value fluctuation amount detection circuit 21 controls the microcomputer 26 based on the fluctuation amount and the period. Therefore, the microcomputer 26 stops the processing operation during the period when the object enters and leaves the image field 30, that is, during the period, and restarts the processing operation as soon as the object finishes passing.
For this reason, the camera shake correction does not malfunction when entering, as in the prior art. Further, after the object comes out, the microcomputer 26 does not continue to stop the processing, and the camera shake correction can be restarted promptly.

When the average correlation value fluctuation amount detection circuit 21 receives the enable signal, the microcomputer 26 corresponds to the pixel having the minimum correlation value based on the position data from the motion detection circuit 20. The shift with respect to the representative point is obtained, and the shift is used as the partial motion vector.
The microcomputer 26 also determines for each motion detection area A-D whether or not the value obtained by dividing the average correlation value by the minimum correlation value is larger than a certain threshold value, and the motion detection area satisfying this condition is defined as the effective area. To do.

More specifically, when there is a moving object in the motion detection area, the correlation value is different between the part occupied by the moving object and the part not occupied, and the correlation value of the part occupied by the moving object is generally large. It becomes (the correlation is low). Therefore, when there is a moving object in any of the motion detection areas A to D, the minimum correlation value of the motion detection area becomes large, and there is a possibility that a partial motion vector based on camera shake cannot be detected in that motion detection area. Therefore, in this embodiment, if the average correlation value is large, the partial motion vector is reliable even if the minimum correlation value is large to some extent. On the other hand, when the average correlation value is small, if the minimum correlation value is not smaller, The partial motion vector is said to be unreliable. For example, the microcomputer 26
Determines that the motion detection area is valid when (average correlation value) / (minimum correlation value)> 12, and uses only the partial motion vector of the motion detection area satisfying this condition for processing.

The microcomputer 26 processes the flow chart shown in FIG. 7 according to the number of effective areas thus obtained. First, in step S0, it is determined whether or not the enable signal is given, and if not given, the entire motion vector is set to zero in step S2, and the process is terminated. If it is given, in step S1, the above-described method is performed. It is determined whether or not each motion detection area A-D is a valid area. Then, in step S3, the number of effective areas is counted. In step S5, it is determined whether or not the number of valid areas is "4", and if it is not "4", step S7.
At, it is determined whether the effective area is "3". In steps S5 and S7, if the number of valid areas is "4" and "3", respectively, that is, if the number of valid areas is 3 or more, the process proceeds to step S9. In step S9, the horizontal and vertical components of the partial motion vector calculated in the effective area are arithmetically averaged. Then, in step S11, the difference between the horizontal and vertical components of the partial motion vector obtained in each motion detection area A-D and the arithmetic mean value of the horizontal component and the arithmetic mean value of the vertical component obtained in step S9. The absolute value of is calculated and used as the divergence of each detection area A-D. Then, in step S13, two divergence values with small values are selected, and the partial motion vectors of the motion detection areas corresponding to them are arithmetically averaged.
Then, in step S15, the arithmetic mean value is set as the overall motion vector.

On the other hand, if it is determined in step S7 that the number of valid areas is not 3, it is determined in step S17 whether the number of valid areas is 2. 2 effective areas
If so, in step S19, the partial motion vector of the arbitrary invalid area is replaced with the entire motion vector obtained one field before, and the invalid area is set as the valid area, and the process proceeds to step S9. Then, the overall motion vector is determined by the processing of steps S9 to S15. In this embodiment, the partial motion vector of the two valid areas and the global motion vector of the previous field and the global motion vector of the current field replaced in the invalid area are determined.

If it is determined in step S17 that the number of valid areas is not 2, it is determined in step S21 whether the number of valid areas is 1. If the number of effective areas is 1, the process proceeds to step S23. In step S23, the partial motion vector of any two invalid regions is replaced with the whole motion vector obtained one field before or the whole motion vector obtained two fields ago, and then the process proceeds to step S9. Then, the overall motion vector is obtained by the processing of steps S9 to S15. In this embodiment, the partial motion vector of one effective area and the total motion vector replaced in the two invalid areas determine the total motion vector of the current field.

When it is determined in step S21 that the number of valid areas is not 1, that is, when all the motion detection areas are invalid areas, in step S25, the total motion vector obtained one field before is set to 0.9.
It is multiplied by 7 and the result is determined as the overall motion vector in step S15. It should be noted that although the number of motion detection areas is set to four in this embodiment, the present invention is not limited to this.
The detection accuracy improves as the number of motion detection areas increases. Further, in step S13, the divergence selected may be one, and the number thereof is arbitrary. Further, step S2
5, the coefficient for multiplying the overall motion vector obtained one field before is not limited to 0.97, and any coefficient larger than 0 and 1 or less can be used.

The camera shake is corrected using the overall motion vector corresponding to the number of effective areas detected in this way, but this correction itself is well known, and a detailed description thereof will be omitted here. Although the video camera has been described in this embodiment, the video camera is not limited to the one using an analog video tape or a digital video tape as a medium as long as the camera part is digitally processed. It goes without saying that the medium may be a disk.

[Brief description of drawings]

FIG. 1 is a block diagram showing an embodiment of the present invention.

FIG. 2 is an illustrative view showing a principle of electronic zoom and showing a motion detection region in an image field.

FIG. 3 is an illustrative view showing a principle of electronic zoom and showing representative points and sampling points in a detection area.

FIG. 4 is an illustrative view showing the principle of camera shake correction.

FIG. 5 is an illustrative view showing each block in an image field to which the representative point matching method is applied.

6 is a block diagram showing a motion detection circuit shown in FIG. 1. FIG.

FIG. 7 is a flowchart showing the operation of this embodiment.

8 is a block diagram showing an average correlation value fluctuation amount detection circuit shown in FIG. 1. FIG.

FIG. 9 is an illustrative view showing a change in average correlation value when an object passes through.

[Explanation of symbols]

10… Video camera 20 ... Motion detection circuit 21 ... Average correlation value variation detection circuit 22 ... Memory control circuit 24 ... Field memory 26 ... Microcomputer 28 ... Electronic zoom circuit 30 ... Image field 46 ... Detection area

Claims (3)

(57) [Claims] 1. A plurality of detection areas formed on a screen
Based on the multiple image correlation values obtained separately, the screen shake
In a video camera with a shake correction function for correcting this, an average correlation value calculating means for calculating an average correlation value that averages the plurality of image correlation values, and a difference for detecting a difference between the average correlation value and a reference average correlation value.
Minute detection means that a period in which the difference is smaller than a predetermined value continues for a predetermined period
Discriminating means for discriminating whether or not the above condition is satisfied, and
When the above conditions are satisfied, camera shake correction is executed and
If the above condition is not satisfied, the correction control
A video camera with a shake correction function, which is provided with a step . Wherein said discriminating means, the pre-Symbol difference first comparison means for comparing the predetermined value, the comparison result of said first comparison means
A counter that counts time accordingly , and the counter
Second comparing means including, claim 1 shake correction function with video camera according to comparing the count value with the predetermined period of data. 3. The reference average correlation every time the condition is satisfied
The method according to claim 2, further comprising update means for updating the value.
Video camera with the described image stabilization function.