JPS61153774A - Moving object image extraction method and device - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

[Detailed Description of the Invention] [Field of Application of the Invention] The present invention relates to moving image processing, and is particularly suitable for extracting moving object images from moving images that are continuously input in real time. This invention relates to a method for extracting a moving object image.

[Background of the invention]

Conventional image processing apparatuses for images of moving objects perform processing using values obtained by time differentiation or calculations similar to Fi for each pixel of continuously input moving images. By using an electronic circuit to perform short time differentiation or similar operations, the circuit configuration can be simplified and the operation can be performed at extremely high speed.

However, it is difficult to extract an image of a moving object only from values obtained by time differentiation or similar calculations.

For example, if a circular area moves in front of a plain background, a 93-moon-shaped area with different signs at the front and rear in the direction of movement will be output, but it is difficult to extract the circular area using this information. It is difficult.

Another method has conventionally been to temporarily store the images obtained from the image sensor in a storage device, then take out the images one by one from the storage device, and calculate the corresponding points between each image to identify the moving object [ - Extraction method is used. When a corresponding point is found using this method, the amount of movement thereof is accurately calculated, so a moving object image can be extracted accurately, but the time required for the processing becomes long. Furthermore, if the object is rotated or deformed, it becomes very difficult to detect corresponding points.

Another method is to store in advance an image of only the background that does not include the moving object image in the device, and then extract the moving object image by comparing the continuously input image with the background image. It has been known.

This method cannot be used at all if the background changes. JP-A-59-8087 also describes a mechanism for updating the background image. This is to sequentially capture instantaneous images that do not include images of moving objects into the device and store them as background images. This is an ingenious way to falsely interpret gradual temporal changes in the background as an image of a moving object. However, with this method, if a moving object is always present within the imaging range, the background will not be updated at all.
It is not possible to respond to cases where the background changes significantly instantaneously. In addition, related public knowledge examples include “Information Processing J.
(Vo 1-21. A6. June 1980).

[Purpose of the invention]

An object of the present invention is to provide an image processing method that can extract a moving object image in real time and with high precision from a moving image obtained from an image sensor.

[Summary of the invention]

The present invention captures a constant part or a slowly changing feature component of a moving image obtained from an image sensor, stores it as a background feature amount, and combines the stored data of the background with an image that is input from time to time. By comparing the features of
This is designed to extract information about moving objects.

[Embodiments of the invention]

An embodiment of the present invention will be described in detail below.

FIG. 1 shows a block diagram when the present invention is implemented using an electronic circuit. The illustrated arrows only indicate the flow of data;
Control lines are omitted. First, we will explain the meaning of all the components, and then explain their detailed functions.

101 is an image sensor that captures a predetermined imaging range including a moving object in the background, and 102 stores one frame of image data converted into a digital signal by an A/D converter (not shown). It is an image storage device, and 103 is 10
2 is a mechanism for extracting the features of the image stored in 102. Here, 104 is a mechanism for outputting an edge-enhanced image of the image stored in 102, and 104 is a mechanism for extracting the edge-enhanced image output from 103 for one frame. 105 is an image storage device for storing the edge-enhanced image of the image running in the background, 106 is an image storage device for storing 5 minutes of the edge-enhanced image t-17, and 106 is 104.105
107 is a mechanism that compares corresponding pixels of two images stored in , and outputs information regarding me of the moving object, and at the same time updates the stored data of 105 . It is a mechanism that extracts only me.

In FIG. 1, la is the output from the image sensor and is a normal video signal, 1b is the digital signal of the input image and data is sequentially transferred pixel by pixel to the feature extraction mechanism 103, and 5JJtc is the edge of the input image data 1b. The data related to the enhanced image is sequentially transferred pixel by pixel.D, id is 1 related to the edge enhanced image stored in the image storage device 104.
In the data that is sequentially transferred pixel by pixel, Toshi and Le relate to the background edge-enhanced image stored in the image storage device 105.
The data is transferred pixel by IFL, and If is 1.
1 regarding the background edge-enhanced image updated by 06
It is data that is transferred sequentially pixel by pixel. 1g is information about the moving object image, and on-o data is transferred sequentially pixel by pixel.
ff signal, 1h is data related to the input image that is sequentially transferred pixel by pixel, and 1' is a digital signal related to the moving object image that is sequentially transferred pixel by pixel.

When the image sensor 101 captures scenes such as those shown at 201, 202, and 203 in FIG. 105 stores an edge-enhanced image 204 of the background. Comparison mechanism 106
However, by comparing 201, 202, 203, and 204 and detecting the difference, 201, 202, 203
At , information regarding the location occupied by the moving object is output. 107H1 Based on that information, 201゜2
Moving object image 205, 206° 207t from 02.203
- Output.

Next, detailed structure and operation will be explained. Image sensor 101
captures images and outputs a normal video signal to 102.

The captured image contains a moving object in a static background.

Image storage device 102t! , 101 is passed through an A/D converter (not shown) to convert it into a digital value, and then the digital value is stored for t-1 screens.

A feature extraction mechanism 103 sequentially imports image data from 102, emphasizes edges, and then sequentially stores the results in 104. It is generally known in the field of psychology that contour lines play a very large role when humans recognize objects, and 103 was installed based on this knowledge. This has the effect of preventing contour lines from falling off when extracting an image. In the field of image processing, in order to emphasize contours (areas where brightness changes rapidly spatially),
Conventionally, a spatial filter as shown in FIG. 3(b) has been used. Each numerical value in FIG. 3 represents a coefficient of the convolution integral. Generally, the brightness changes rapidly at the boundary of an object, and when this filter is applied near that boundary, the value of the convolution integral becomes a large positive or negative value. Furthermore, since the brightness changes slowly inside the object, the output of the convolution integral will have a value close to zero. By performing this convolution operation on all pixels of the image data stored in 102, it becomes possible to emphasize the outline. However, when handling moving images such as those handled by the present invention, the following problem occurs with the spatial filter shown in FIG. 3(b).

(1) A shift of about one pixel often occurs in images due to a shift in synchronization signals between frames. For this reason, Figure 3 (
In the contour image processed in b), a change in position occurs due to the synchronization difference between frames, and the contour line of the background image, which is supposed to be stationary, is erroneously recognized as if it were fluctuating.

e) Noise is superimposed on an isolated pixel in the image due to six factors, but the filter shown in FIG. 3(b) acts to emphasize this noise.

(3) The contour-enhanced image obtained by applying the spatial filter t-102 in Fig. 3(b) to the image data includes the contour of the moving object, but information other than the contour is dropped. .

The present invention extracts an image of a moving object based on information obtained from a contour-enhanced image, which is the output of a filter. This is not suitable for the purpose of the present invention, which is to extract the image (shaded image) itself. If a spatial filter with a large spread of convolution coefficients is used as shown in Figure 3(C),
Although it is possible to reduce the effects of image shifts and isolated noise, it is not possible to improve (3) the loss of information inside the outline of a moving body. Therefore, in the present invention, a spatial filter as shown in FIG. 3(a) is applied as a filter suitable for extracting movable objects in moving images. Figure 3 (a
), first of all, as in Fig. 3(C), the weighting coefficients are spread out to prevent inter-frame synchronization and isolated noise in images with a large contrast compared to Fig. 3(b) K. Furthermore, the coefficient at the center was increased so that information other than the contour line, that is, information regarding the gradation value of the image, would not be lost. The filter shown in FIG. 3(a) outputs values that reflect the gradation values of the locations other than the contour line. The spatial filter calculation device is 1
The six multipliers are equipped with multipliers and perform sum-of-products operations on 16 neighboring images at high speed.

An image storage device 104 in FIG. 1 is a storage device that stores 102 edge-enhanced images output from 103 for t-1 screens.

Similarly, the image storage device 105 in FIG. 1 is a normal storage device that stores edge-enhanced images for one screen. 104
stores an edge-enhanced image of an input image consisting of a background and a moving object, and stores an edge-enhanced image of an input image consisting only of a background in ios.

Comparison and background edge-enhanced image updating mechanism 106 in FIG.
has a mechanism for comparing each pixel of 104 with the corresponding pixel of the background image stored in 105 and outputting information regarding the moving object image, and a function of updating the background image stored in 105 based on the image of 104. have 106Fi sequentially input corresponding pixels from 104 and 105, calculate the difference for each pixel, and if the difference is a certain value 0 or more, the pixel is 104
A moving object image is formed in 104 , and if the difference therebetween is less than a certain value θ, it is determined in 104 that the pixel constitutes a background image, and the determination result is outputted in 107 .

In addition, 106 forms an edge-enhanced image of the background by setting the weighted average value for each pixel of 104 and 105 to the new value of 7'[105. Figure 4 shows how the emphasized images are formed. 401, 402, and 403 show the edge-enhanced images of the input image, and 404, 405, 406, and 407 show how the edge-enhanced images of the background are formed. It is assumed that the image storage device that stores the background Oxy-enhanced image is initially cleared.When an input image 401 containing a cat and a house is input, the image is slightly changed as shown in 404. It is stored as a background image.Next, when an input image 402 containing a bird and a house is input, the house image that continues to exist in the background image is emphasized as shown in 405, and the bird image is slightly The image of the cat, which does not exist this time, is weakened.Similarly, when an input image 403 containing people and a house is input, the image of the house is further emphasized.Finally, the background rainbow is emphasized. An image 407 is formed.A detailed view of the image 106 in FIG. 1 is shown in FIG.

This is a mechanism that calculates the difference of 1e and outputs it to 502 (5a). The comparator 502 receives the output 511 from the difference calculator 501.
is compared with the preset value 0,
For example, if θ is larger, 0 is output, and if θ is equal or smaller, 1 is output. A multiplier 503 multiplies the pixel data obtained from 105 and a preset value (2”
-1) is calculated and the result is output to 504 (5d)
do.

The adder 504 adds the pixel data 1d obtained from 104 and the output 5d of the multiplier 503, and divides the result into 50
5 (5e). The n-bit shifter 505 shifts the output from the adder 504 to the right by a preset value n bits, and outputs the result to 105 (1f). A storage device control mechanism 506 is a mechanism that controls address lines and control lines so that reading and writing to and from 104 and 105 are performed correctly. The operation method is first that 506 is sg
, sht to output the address of the pixel to be processed to 104 and 105, and put 104 and 105 in a read state. The corresponding pixel data are ld and 1et'', respectively.
entered through soi.

After being processed in 502, O or 1 is output through 1gt for 107, and at the same time, the pixel data becomes 503.5.
Processed at 04,505. 506 puts 105 in a writing state at an appropriate time while outputting the address. As a result, 105 can be updated through If as shown in the formula below.

Here, s L I IF (x = y): Pixel data regarding the edge-enhanced image of the updated background at the coordinates (x, y) BK*ta (XtY): Edge of the background before updating at the coordinates (x, y) Pixel data related to the enhanced image I (xey): Pixel data related to the edge enhanced image of the input image at the coordinate (xey) n: A preset value. Also, specify which pixel the data output after 1gtA corresponds to. The address information shown is output through 5ft. The above operation is repeated for one screen.

107 is a mechanism for outputting only the pixel data forming the image of the moving object at 102 to the outside based on the information 1g output from 106. Details are shown in FIG. 601 is an information selection mechanism, which is manually operated from 106. This is a mechanism that determines whether to output the input image 102 to the outside or to the outside based on the information 1gt'' about the moving body. 5f is the address information of the pixel being processed and the image storage mechanism. 102.

106 outputs address information regarding the pixel being processed through 5f and places lO2 in a read state. 102 outputs data of the designated pixel through data t-ih.

Outputs l or 0 signal through 106106H1I, 60
1 outputs pixel data t from 102 when 1 is input through 1g, and 0 is output when 0 is input through 1gt'. Ultimately, it is possible to output to the outside image data that includes only address information regarding pixels that are being processed, that is, currently being output, and an image of a moving object.

FIG. 7 shows a configuration that can shorten the overall processing time. Arrows indicate only the flow of data, and control lines are omitted.

After converting the analog value related to luminance outputted from 101 into a digital value by an A/D converter (not shown) and storing it in sequence in 102, the data is simultaneously outputted to 103 and stored in parallel with the edge. Performs emphasis processing. Furthermore, the pixel data sequentially output from 103 is directly input to 106, and the pixel data and the corresponding pixel data of 105 are input sequentially, and the difference is calculated for each pixel, and at the same time, the weighted average is calculated for each pixel. Depending on the situation, the information regarding the moving object image can be updated 1057 at the same time as being output.

As described above, it is possible to construct an image processing apparatus that extracts a moving object image in a shorter processing time than that shown in FIG.

According to this embodiment, it is possible to configure an image processing device that can extract only a moving object image using an electric circuit.

〔Effect of the invention〕

As described above, according to the present invention, since the processing content is simple, images of moving objects can be extracted at high speed without the need for using large-scale equipment such as large-scale computers. Therefore, it can be used as preprocessing for image processing related to moving objects or recognition of moving objects.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing an embodiment of the present invention, and FIG. 2 is a block diagram showing an embodiment of the present invention.
A screen diagram showing how a moving object image is extracted from an input image, FIG. 3 is a spatial filter coefficient used for edge enhancement, and FIG. 4 is a screen diagram showing how a background image is formed.
FIG. 5 is a detailed view of 106 in FIG. 1, FIG. 6 is a detailed view of 107, and FIG. 7 is a block diagram of the embodiment shown in FIG. 1 when the processing speed is increased. 101... Image pickup element, 102... Input image storage device, 103... Edge enhancement mechanism, 104... Edge enhanced image storage device, 105... Background edge enhanced image storage device, 106... Comparison mechanism and background edge-enhanced image updating mechanism, 107...Moving object image extraction mechanism. 11 Fig. 72 Fig. 2 to 3 Fig. (σ,) /4 no CCJ animal 4 Fig. fiis Fig. I1 Fig.

Claims (1)

[Claims] 1. In a method for extracting a moving object image from continuously inputted image data, the feature amount of a background image is stored in advance for the inputted image, and the inputted image data and the Calculates the difference between the stored background feature amount, extracts a moving object image using the calculated difference signal, and performs a predetermined process on the input current image data and the already input current image data. The difference between the input image data and the previously inputted image data is calculated, and for portions where the calculated difference signal is smaller than a preset value, the feature amount of the input image and the feature amount of the background image are calculated. A moving object image extraction method comprising: correcting the background feature amount using a weighted average value of , and using the background feature amount as the background feature amount of the next input image, and extracting a moving object image. 2. A moving object image extraction method as set forth in claim 1, characterized in that an image with emphasized edges is used as the feature quantity of the input image. 3. In claim 1, the input image and the stored image are compared pixel by pixel, an image formed by the difference signal is output, and the input image and the stored image are compared pixel by pixel. A moving object image extraction method characterized in that an image formed by weighted averaging is used as a new stored image. 4. In claim 3, the input image and the stored image are compared pixel by pixel, and an image formed by the difference signal is output, and the signal value is calculated for each pixel of the output image. A moving object image extraction method characterized in that the pixel signal is multiplied by α (α<1) and added to the corresponding pixel signal of the stored image to obtain a new stored background image.