JPH0422280A - Movement detector for picture - Google Patents

Abstract

PURPOSE:To detect only an object moving differently from that of a background by providing a means correcting the entire picture movement and a detection means detecting an area having a different movement from the entire movement through the use of the output of the correction means. CONSTITUTION:A picture of a current frame including a background and a moving object is inputted to an input terminal 34 with respect to a picture of a preceding frame. A moving vector detection means 1 detects a moving vector of a background in each moving vector detection area being each of 1/16 divisions of a pattern. A 0 vector is detected from an area where no pattern is in existence. A representative vector decision means 2 extracts most frequently coincident vectors statistically among the 16 vectors detected from each of movement vector detection areas, decides the result as a background moving vector and outputs it to a read control means 4. A differential threshold level processing means 5 takes a difference between a picture signal of a preceding frame from a frame memory 3a and a picture signal of a current frame from the read control means 4 and outputs a picture element whose absolute exceeds a threshold level as a moving area. A median filter 6 eliminates distortion and only the moving area by the object is outuptted.

Description

[Detailed Description of the Invention] Industrial Application Field The present invention is capable of detecting only objects whose movement differs from the background even when the entire captured image moves due to shaking or vanning of the imaging device. The present invention relates to an image motion detection device configured to allow the motion of an image to be detected.

Conventional technology Conventionally, as a device for detecting moving objects in an image,
There is a device as shown in FIG. This is done by storing the image signal of the previous frame in the frame memory 3, and when the image signal 34 of the current frame is input manually, the difference between the signal of each pixel of the previous frame and the current frame is calculated, and this difference is When the value exceeds a predetermined value, the pixel is determined to be a moving area. For example, as shown in Figure 11(a), when an object moves while the background remains stationary, this device calculates the difference between two frames to calculate the difference in the image as shown in Figure 11(b). Only moving objects can be detected as moving objects.

Problems to be Solved by the Invention However, as shown in FIG. 12(a), in a conventional image motion detection device, when the background is not stationary due to movement of the imaging device such as vanning, this device cannot detect the motion of an image. Figure 12(b) is obtained by taking the difference between two frames.
As shown in , since the background is also detected as a moving area,
Only moving objects cannot be detected in the image.

The present invention has been made to solve the above-mentioned conventional problems, and an object of the present invention is to provide an image motion detection device that can detect only objects that have a motion different from the background even when the background is not stationary. do.

Means for Solving the Problems The image motion detection device of the present invention is capable of detecting only objects that have a motion different from the background even when the background is not stationary. The apparatus includes a correction means, and a detection means for detecting an area having a movement different from the overall movement by using the output of the correction means.

Operation In the image motion detection device of the present invention, the overall motion correcting means corrects the overall motion of the image screen due to a change in the photographing direction of the imaging device. By performing motion detection using this corrected output image signal, it becomes possible to detect only moving objects whose motion is different from the overall motion.

Example Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 shows a block diagram of an image motion detection device according to a first embodiment of the present invention. In Figure 1,
1 is a motion vector detection means, 2 is a representative motion vector determination means, 3.3a is a frame memory, 4 is a readout control means, 5 is a difference and threshold processing means, 6 is a Nudian filter, and 7 is a motion amplitude - Variance calculation means, 34 is an input terminal.

The operation of this image motion detection device will be explained with reference to FIG.

First, in Fig. 2, 8 is the previous frame image, 9 is the current frame - 11 image, lO is the motion vector of the moving object in the image, 11 is the motion vector of the background in the image, and 12 is the motion vector of the background in the image. Moving object, 13 each motion vector detection area in the screen, 1
4 is the 0 vector, 15 is the inverse vector of the background motion vector in the image, 16 is the area of the moving object detected by subtracting and thresholding the two frame images, and 17 is the complete motion correction for the two frames. Therefore, when the image is subjected to differential and threshold processing, the edge parts of the background that are detected as moving areas,
18 is the screen position of the previous frame, and 19 is the screen position of the current frame.

First, as shown in FIG. 2(a), an image 9 of the current frame that has movement in general, including the background and moving objects 12, is input to the input terminal 34 of this device, in contrast to the image 8 of the previous frame 11. Ru. In the motion vector detection means 1, as shown in FIG. 2(b), the motion vector of the image is detected in each motion vector detection area 13 obtained by dividing the screen into 16 parts. In the area where the moving object 12 mainly exists, the motion vector lO of the moving object 12 is detected, and in the area where the background pattern mainly exists, the background motion vector 11 is detected. Also,
In an area where no pattern exists, 0 hectares 14 are detected.6 The representative motion vector determining means 2 determines whether statistically the most vectors of the 16 vectors detected from each motion vector detection area 13 match. The vector that is
It is determined and output as a background motion vector 11. The method for calculating the vector that statistically matches the most vectors is to take a two-dimensional histogram of vector values in the horizontal and vertical directions of the vector, and find the vector value that maximizes the two-dimensional histogram value. It is. This operation will be briefly explained using FIGS. 2(f) and (g). FIG. 2(f) shows a plot of the motion vector points manually entered into the representative motion vector determining means 2 within the detection range. FIG. 2(g) shows this histogram viewed only from the horizontal direction. By detecting this maximum point, a background motion vector 11 is output. The frame memory 3 stores the image signal of the current frame while the background motion vector 11 is detected. The readout control means 4 uses the motion vector 11 as shown in FIG. 2(e).
As shown in FIG. 3, the output position 19 of the current frame is shifted in the direction of the inverse vector 15 of the motion vector 11 based on the output position 18 of the previous frame, and the image signal is read out from the frame memory 3 and output. The frame memory 3a stores the output of the read control means 4 of the previous frame. The difference/threshold processing means 5 takes the difference between the image signal of the previous frame from the frame memory 3a and the image signal of the current frame from the readout control means 4, and processes the pixels whose absolute value exceeds the threshold value. is output as a motion area. The output is indicated by diagonal lines at 16.17 in FIG. 2(d). However, if the movement of the background cannot be completely corrected by parallel movement due to minute errors in motion vector detection, image distortion such as shading, or rotation of the screen, the pattern of the ζ dorsal star may be changed as shown in 17. A motion area may be detected at the edge of the image. This generally occurs in dots or lines only at the edge portion, so it can be removed by the median filter 6, and the second
As shown in Figure (e), only the motion area 16 caused by the moving object is output. The size of the portion detected as a motion region occurring at the edge portion of the background can be predicted from the size of the motion vector of the background, for example, if the cause is image distortion such as shading.

In addition, if the cause is a detection error or screen rotation, the degree of coincidence of motion vectors in each detection area detected by the motion vector detection means 1, that is, the motion that gathers in the background motion vector 11 shown in FIG. 2(g). It can be predicted from the variance 290 size of the vector. The motion amplitude/variance calculating means 7 calculates the amplitude and variance 29 of the background motion vector 11 and outputs them to the median filter 6. The median filter 6 uses a background motion vector 1 inputted from the motion amplitude/variance calculation means 7.
1. Based on the variance 26, the size of the median filter matrix is changed accordingly. As a result, even if there is a large amount of movement and the movement area that occurs at the edge portion of the background becomes large, this can be removed.

Next, a second embodiment of the present invention will be described below with reference to the drawings. FIG. 3 shows a block diagram of an image motion detection device according to a second embodiment of the present invention. In FIG. 3, the same elements as in FIG. 1 are given the same numbers. The operation of this image motion detection device will be explained using FIG. 4. In FIG. 4, the same elements as in FIG. 2 are given the same numbers.

First, connect the input terminal 34 of this device to the input terminal 34 as in the first embodiment.
As shown in FIG. 4(a), an image 9 of the current frame which has movement in general, including the background and moving object 12, is inputted with respect to the image 8 of the previous frame.

The operations of the motion vector detection 1 and representative motion vector determination means 2 are similar to those in the first embodiment. The frame memory 3 stores the image signal of the current frame while the background motion vector 11 is detected.

At the same time, the frame memory 3a stores the image signal of the previous frame. Based on the motion vector 11, the readout control means 4 shifts the output position 18 of the previous frame in the direction of the motion vector 11 with the position 19 of the current frame as a reference, as shown in FIG. 4(C). , reads the image signal from the frame memory 3a and outputs it. The difference and threshold processing means 5 takes the difference between the image signal of the current frame from the frame memory 3 and the image signal of the previous frame output from the readout control means 4, and detects the difference where the absolute value exceeds the threshold. pixels are output as a motion area. The output is the shaded portion 16.17 in FIG. 4(d). As in the first embodiment, if the movement of the background cannot be completely corrected due to minute errors in motion vector detection or image distortion such as shading, the edges of the background pattern may be corrected as shown in 17. It may be detected as a moving area. The amplitude calculation means 7 and the median filter 6 operate in the same manner as in the first embodiment, and remove this to create a motion area 1 caused by a moving object as shown in FIG.
Only 6 is output. Here, in the first embodiment, in order to shift the position of the current frame with reference to the corrected output screen of the previous frame, the movement caused by the moving object is shifted based on the coordinates of the corrected screen as shown in FIG. 2(e). Area 16 is output, but the second embodiment is the current output! In order to shift the position of the current frame using the gnome as a reference, the movement area 16 due to the moving object is output based on the coordinates of both sides of the current field as shown in FIG. 4(e).

A third embodiment of the present invention will be described below with reference to the drawings. FIG. 5 shows a block diagram of an image motion detection device according to a third embodiment of the present invention. Fifth
In the figure, the same elements as in FIG. 2 are given the same numbers. In FIG. 5ζ, 1a is a motion vector detection means. The operation of this image motion detection device will be explained with reference to FIG. In FIG. 6(a), FIG. 2 and
The same elements are given the same number.

In FIG. 6(b), 26 is the number of interphases, 27 is the number of interphases obtained in the background area, and 28 is the number of interphases obtained in the region of the moving object.

First, as shown in FIG. 6(a), an image 9 of the current frame which has movement in general, including the background and moving object 12, is input to this apparatus in contrast to the image 8 of the previous frame. In the motion vector detection means 1a, first, as shown in FIG. 6(b) for the data of the entire screen, the absolute value of the difference in deviation within the motion vector detection range is determined as the interphase number 26.

This correlation number 26 is the correlation function 27 obtained in the background part.
and the correlation number 28 obtained in the region of the moving object. Here, if the area of the moving object is sufficiently small compared to the background, the average amplitude of the correlation function 27 will be sufficiently larger than the correlation number 28, so by finding the minimum point of the correlation number 26, the movement of the background Vector 11 can be found. In this way, the motion vector detection means 1a finds the minimum point of the interphase number 26, detects the background motion vector 11, and outputs it. The frame memory 3 stores the image signal of the current frame in which the background motion vector 11 is detected. At the same time, the frame memory 3a stores the image signal of the previous frame. Thereafter, the operations of the readout control means 4, difference and threshold processing means 5, and median filter 6 are the same as in the second embodiment.

A fourth embodiment of the present invention will be described below with reference to the drawings. FIG. 7 shows a block diagram of an image motion detection device according to a fourth embodiment of the present invention. 7th
In the figure, the same elements as in FIG. 1 are given the same numbers. In FIG. 7, 20 is an acceleration sensor attached to the imaging device, 21 is a zoom magnification sensor, 22 is a motion calculation means, and 26 is a readout control and data interpolation means. The operation of this image motion detection device will be explained with reference to FIG. In FIG. 8, the same elements as in FIG. 2 are given the same numbers. In FIG. 8, 23 is a motion vector of the center part of the screen of the current frame with respect to the previous frame, 24 is the rotation angle of the movement of the peripheral part of the current frame around the center of the screen of the current frame with respect to the previous frame, and 25 is the motion vector of the current frame with respect to the previous frame. This is the rate of change in zoom 11 magnification.

First, the input terminal 334 of this device is connected to the background and moving object 12 for the previous frame image 8, as shown in FIG. 8(a).
An image 9 of the current frame that has movement in general, including , is manually generated. The frame memory 3a stores the image signal of the previous frame.

At this time, the acceleration sensor 20 attached to the imaging device
Accordingly, the change in the angle of the photographing direction of the imaging device and the rotation angle around the photographing direction are manually input to the motion calculating means 22. At the same time, the zoom magnification is manually input from the zoom magnification sensor 21 to the first motion calculation stage 22 . Movement calculation means 22
are the motion vector 23 of the screen center portion of the current frame relative to the previous frame shown in FIG. The rate of change 25 in the zoom magnification is calculated.

As shown in FIG. 8(b), the readout control and data interpolation means 26 reads out the image signal of the previous frame from the frame memory 3a based on the position 19 of the current frame, performs data interpolation, and outputs it. The difference and threshold processing means 5
The difference between the image signal of the current frame from the frame memory 3 and the image signal of the previous frame output from the readout control means 4 is taken, and the pixels whose absolute value exceeds the threshold are output as a motion area. do. The output is the shaded portion 16.17 in FIG. 8(C). This is also the first
As in the example above, if the movement of the background cannot be completely corrected due to minute errors in motion vector detection or image distortion such as shading, the edges of the background pattern are detected as movement areas as shown in 17. may be done. The amplitude calculating means 7 and the median filter 6 operate in the same manner as in the first embodiment, and by removing this, only the motion area 16 caused by the moving object is output as shown in FIG. 8(d).

In the first and second embodiments, the parallel movement of the entire image of the two-frame leap is corrected, but not only the parallel movement but also the overall rotation and enlargement movement are corrected, for example.
Detection is performed as shown in Figures 7 (a) and (b), and Figure 8 (a)
), (b), it is also possible to use a device that rotates and enlarges one of the two frames, corrects the overall movement between the two frames, and detects a region of movement that differs from the overall movement. Conceivable.

Further, in the fourth embodiment, the outputs of the acceleration sensor and the zoom magnification sensor are used to calculate the change in the imaging direction and the zoom magnification, but any method may be used as long as it converts the imaging state into a signal.

For example, an image motion detection device shown in FIG. 9 can be considered as a fifth embodiment. The difference from the fourth embodiment is that it includes means 30 for controlling the imaging direction of the imaging device, means 31 for controlling the zoom magnification, etc., and each outputs an imaging direction control signal and a zoom magnification control signal, and The motion calculating means 22 uses a control signal for the imaging direction instead of the output of the acceleration sensor of the fourth embodiment, and similarly uses a zoom magnification control signal instead of the zoom magnification sensor.

Effects of the Invention As described above, the present invention has means for correcting the overall movement of an image signal, and means for comparing two frames of images output from the correction means and detecting regions of different output values. Even if the entire image is moving due to shaking or panning of the imaging device, it is possible to realize an image motion detection device that can detect only a moving object whose motion is different from the motion of the entire image.

[Brief explanation of drawings]

FIG. 1 is a block diagram of an image motion detection device according to the first embodiment of the present invention, and FIG. 2 is an image diagram for explaining the operation of the image motion detection device according to the first embodiment of the present invention. Third
The figure is a block diagram of an image motion detection device according to the second embodiment of the present invention, and FIG. 4 is an image diagram for explaining the operation of the image motion detection device in the second embodiment of the present invention. FIG. 5 is a block diagram of an image motion detection device according to the third embodiment of the present invention, and FIG. 6 is an image diagram for explaining the operation of the image motion detection device according to the third embodiment of the present invention. FIG. 7 is a block diagram of an image motion detection device according to the fourth embodiment of the present invention, and FIG. 8 is an image diagram for explaining the operation of the image motion detection device according to the fourth embodiment of the present invention. FIG. 9 is a block diagram of an image motion detection device according to a fifth embodiment of the present invention, FIG. 10 is a block diagram of a conventional image motion detection device, and FIGS. 11 and 12 are images of a conventional image motion detection device. FIG. 3 is an image diagram for explaining the operation of the motion detection device of FIG. Lla...Motion vector detection means, 2...Representative motion vector determination means, 3.3a...Frame J1 memory,
4... Readout control means, 5... Difference and threshold processing means, 6... Median filter, 7... Motion amplitude/variance calculation means, 8... Image of previous frame, 9・・・
・Image of the current frame, lO... Motion vector of a moving object in the image, 11... Motion vector of the background in the image, 1
2... Moving object in the image, 13... Each motion vector detection area in the screen, 14... 0 vector, 15... Inverse vector of the background motion vector in the image, 16... Movement Movement area caused by an object, 17...Edge portion of the background detected as a movement area, 18... Screen position of the previous frame, 19... Screen position of the current frame, 20 Acceleration sensor, 21...◆ Zoom magnification sensor, 22... Motion calculation means, 23... Motion vector of background at the center of the screen, 24... Rotation angle of background movement at the center of the screen, 25... Change in zoom magnification, 26...・Readout control, data interpolation means, 29...dispersion of motion vectors,
30... Imaging direction control means, 31... Zoom magnification control means, 34... Image input terminal. Agent Patent Attorney Matsu 1) Tadashi Michi 1 2: Detection means 4: Correction means 5: Comparison detection means Figure (Q) (b) Figure 2 Figure 2 (f) 10 Go 111 Figure 3 Figure 4 (Q) (b) Figure 5 Figure 6 Figure 102\11 (b) Figure 7 Figure 8 (a) (b) Figure 8 (C) (d) Figure 9 Figure 70 Figure 11 (Q) (b)

Claims (3)

[Claims] (1) A correction means for correcting the overall movement of the image signal, and a comparison detection means for comparing at least two or more frames of images corrected by the correction means and detecting areas where the signal values differ. Features: Image motion detection device. (2) a detection means for detecting the overall movement of the image signal using the image signal; a correction means for correcting the overall movement of the image signal based on the movement detected by the detection means; An image motion detection device characterized by comprising a comparison detection means for comparing at least two or more corrected frames and detecting regions where signal values differ. (3) an output means for converting and outputting an imaging state into a signal; a detection means for detecting the overall movement of the image signal using the signal representing the imaging state; It is characterized by comprising a correction means for correcting the overall movement of the image signal, and a comparison detection means for comparing at least two or more frames of images corrected by the correction means and detecting areas where the signal values differ. Image motion detection device.