JP2002092618A - Device and method for target detection processing - Google Patents

Abstract

PROBLEM TO BE SOLVED: To sufficiently ensure the continuity of target detection. SOLUTION: Image processing of the input image of frame structure is executed simultaneously in parallel by a static image processing part 11 and a dynamic image processing part 12. A segment collation processing part 20 determines relative relation between image processing systems or between frames on each feature quantity obtained by each image processing system, and determines the identity of a target object on the basis of the determined result. The segment collation processing part 20 is provided with a feature quantity holding function for storing the feature quantity extracted from each frame, in a database, a segment detecting function for detecting whether or not a segment is included in each frame; and a feature quantity interpolation processing function for reading the feature quantity of a front frame corresponding to the segment, from the database, and interpolating it between the present frame and the following frame.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a target determination processing apparatus and a target determination processing method for monitoring and following a target object by using an image processing system.

[0002]

2. Description of the Related Art As is well known, for example, in a monitoring system, a target determination processing device for specifying a target object such as an aircraft is provided in order to detect and follow the target object.

A target determination processing device inputs input image data obtained from an imaging device such as a camera, extracts target candidates from the image data, and sets target targets based on image processing feature amounts corresponding to each target candidate. Detect an area (segment). Here, in order to extract many feature values, the target determination processing device performs image processing on the same image using a plurality of image processing methods. At this time, the position information among the segment feature amounts is used for the segment matching for integrating the feature amounts obtained by the respective processes.

[0004]

However, in the above-described target determination processing device, since the image processing method is different, the extracted area is often different. In particular, a target may not be detected temporarily in some processes due to a change in image status, noise, or the like. In the processing result in which the target cannot be detected, there is no segment and position information, and the integration of the feature amounts is incomplete. At this time, the target segment has insufficient features,
There is a possibility that detection / following will become impossible after deviating from the object candidate.

In order to solve such a problem, conventionally, when a segment cannot be detected temporarily, a target area in the next frame is limited and a target is extracted based on the position / speed information of the segment. Like that. In this case, limiting the processing area increases the possibility of re-detecting the target, but is not sufficient.

SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide a target determination processing apparatus and a target determination processing method capable of ensuring sufficient continuity of target detection even when an input image changes due to a change in image conditions, noise, or the like. It is in.

[0007]

SUMMARY OF THE INVENTION A target determination processing device according to the present invention is a target determination processing device for determining a target object by using image processing, wherein image data having a frame structure including the target object is converted. An image input unit that inputs and processes the input image data by a plurality of types of image processing methods, and extracts each feature amount corresponding to an object candidate obtained by image processing of each image processing method of the image input unit, A determination processing unit configured to determine the identity of the target object based on at least one of a relative relationship of each feature amount between frames and a relative relationship of each feature amount between each image processing method. is there.

The image input means may execute a plurality of types of image processing methods simultaneously and in parallel to generate time-series image processing results including target object extraction results. The processing means is characterized in that the identity of the target object is determined by a correlation process based on overlapping feature amounts obtained between the respective image processing methods or between frames.

According to this configuration, since a plurality of types of image processing methods are executed simultaneously and in parallel, even in a situation where some processing results are invalidated due to environmental changes, noise, or the like, sufficient results can be obtained. A feature amount can be secured, a relative relationship between image processing methods or between frames is determined for each feature amount obtained by each image processing method, and the identity of the target object is determined based on the determination result. As a result, the time required for the target object determination processing can be reduced, and the identity of the target object can be accurately determined without being affected by changes in image conditions or noise, and the target determination processing can be performed. Accuracy can be improved. When determining whether or not there is a correlation with the past image processing result, the degree of overlap between the feature amount corresponding to the past target object candidate and the feature amount corresponding to the current target object candidate is determined. Since the identity of the target object is determined from the processing result, it is possible to determine the identity of the fast moving target object with high accuracy.
Also, when determining whether or not there is a correlation with the image processing results between the image processing methods, the degree of overlap of the feature amounts corresponding to the respective image processing results is correlated, and the same target object is determined from both correlation processing results. Since the gender is determined, the identity of many target objects can be determined with high accuracy.

[0010] The determination processing means may include a feature amount holding means for holding a feature amount extracted from the frame, an object candidate detecting means for detecting whether or not the frame includes an object candidate, When the object candidate in the current frame is not temporarily detected by the object candidate detection means,
Interpolation means for reading out the characteristic amount of the previous frame corresponding to the target object candidate from the characteristic amount holding means and interpolating between the current frame and the next frame. In this way, even if the input image changes and the continuously detected target candidate is not temporarily detected, the retained feature amount of the previous frame is interpolated and the target Continuity of detection can be ensured.

[0011]

Embodiments of the present invention will be described below in detail with reference to the drawings.

FIG. 1 is a block diagram showing a configuration of an embodiment of a target determination processing device according to the present invention. This target determination processing device is roughly composed of an image input system 10 and a segment matching processing unit 20.

The image input system 10 has, for example, a still image processing unit 11 and a moving image processing unit (moving vector processing unit) 12 using a plurality of types of image processing systems. The image input system 10 executes different image processing on an input image having a frame structure simultaneously and in parallel, and executes respective image processing data (S
D) and (MD) are generated. Here, as for the input image, an imaging device (not shown) such as a camera covers a range including the target object.
Is the image data captured by.

The still image processing unit 11 is a processing unit using, for example, a frame smoothing process and a binarizing process. The still image processing unit 11 detects an area having high luminance and contrast from an input image and outputs the area as image processing data (SD). I do. In addition, the moving image processing unit 12 executes a moving vector process from the input image, detects a moving region, and outputs it as image processing data (MD).

Here, the area detected by each image processing is defined as a segment. Each segment belongs to any of the target object, similar object candidates, and their background. Each image processing data (SD),
(MD) includes a feature value of a segment described later.

The segment collation processing section 20 processes the image processing data (SD) and (MD) output from the image input system 10.
Is input, and the feature amount is extracted and output for each image processing method and each frame. In addition, the segment matching processing unit 2
0 has a correlation determination processing function, a feature value holding function, a segment detection function, and a feature value interpolation processing function.

As shown in FIG.
The identity of the target object is determined based on the relative relationship between the feature values of the segments between the respective image processing methods or the relative relationship between the feature values of the segments between the frames.

The feature holding function stores the feature of a segment extracted from each frame in a database prepared in advance. The segment detection function detects whether a segment is included in each frame. When the segment detection function does not temporarily detect a segment in the current frame, the feature amount interpolation processing function reads the feature amount of the previous frame corresponding to this segment from the database and interpolates between the current frame and the next frame. I do.

Next, the operation of the apparatus configured as described above will be described. Here, a case where two types of image processing (for example, still image processing and moving image processing) are performed simultaneously will be described with reference to FIG.

That is, the segment is created by the processing 1,
Perform both processes. In this processing, it is assumed that the processing 2 feature amount is assigned based on the processing 1 segment. If the position of the processing 1 segment is different from the position of the processing 2 segment, the processing 1 segment No. in the database prepared in the segment matching processing unit 20 in advance. After the,
A new No. A frame is added, and information of the processing 2 segment is written.

(Determination of Overlap Between Previous Frame and Current Frame of One Processing Segment) (1-1) Overlap of Single Previous Frame Segment and Single Current Frame Segment For example, as shown in FIG. It is assumed that the previous frame segment overlaps.

In this case, since the same segment No. overlaps, each segment can be determined to be the same target. At this time, the number of continuous detections is also added.

Further, when the same segment No. overlaps even one pixel, it is determined that the same target is set.

(1-2) Overlap between a single current frame segment and a plurality of previous frame segments If, for example, a plurality of previous frame segments overlap one current frame segment as shown in FIG. The segment with the highest overlap ratio is the same target. However, if the overlap ratio is the same,
The segment with the highest evaluation value in the previous frame segment is set as the same target.

(1-3) Overlapping of Multiple Current Frame Segments and Single Previous Frame Segment It is now assumed that the multiple current frame segments and the single previous frame segment overlap as shown in FIG. 6, for example. In this case, since the current frame segment overlaps with one previous frame segment, the feature amount of the previous frame segment is simply inherited.

(1-4) Overlapping of one segment of multiple processing and two segments of multiple processing At this point, it is assumed that, for example, as shown in FIG. 7, one segment of multiple processing and two segments of multiple processing overlap. In this case, the same processing as (1-1) and (1-2) is performed.

(1-5) Flag (Segment Collation Result Record) In the segment collation processing section 20, the segment collation result is stored in a database for use in subsequent processing as shown in FIG.

(1-6) When there is no current frame segment corresponding to the previous frame segment It is assumed that, for example, a continuously detected segment is not temporarily detected. In this case, the continuity of the segment is interrupted, and the evaluation value calculation and the like are reset. Therefore, in this embodiment, the information of the previous frame segment is copied as the current frame segment, and pseudo continuity is ensured. However, if the segment is not detected for several consecutive frames, the segment is erased.

Although the above operation has been described with reference to one processing segment, the same processing as the above (1-1) to (1-6) can be performed for the second processing segment.

(Determination of Overlap Between Process 1 Segment and Process 2 Segment) (2-1) Overlap of Single Process 1 Segment and Single Process 2 Segment Now, for example, as shown in FIG. 9, single process 1 segment and single process 2 Assume that the segments overlap.

Here, when the processing 1 segment and the processing 2 segment overlap by 30% or more, they are determined as the same target, and when they are 15% or less, they are determined as different targets.

(2-2) Overlap of one singular processing one segment and two or more processing two segments It is assumed that a plurality of two processing segments overlap one processing one segment, for example, as shown in FIG.
In this case, the segment having the highest overlap ratio with respect to the processing 1 segment among the processing 2 segments is determined to be the same target.

(2-3) Overlapping of one segment of multiple processing and two segments of single processing At this point, it is assumed that a plurality of processing one segments overlap one processing two segment as shown in FIG. 11, for example.
In this case, as viewed from the processing 1 segment, one processing 2
Since it overlaps with the segment, the vector of the process 2 segment is simply integrated.

(2-4) Overlapping of one segment of multiple processing and two segments of multiple processing Now, for example, as shown in FIG. 12, it is assumed that one segment of multiple processing and two segments of multiple processing overlap. In this case, the same processing as the above (2-1) and (2-2) is performed.

(2-5) Flag (Segment Collation Result Record) In the segment collation processing section 20, the segment collation result is stored in a database as shown in FIG.

As described above, in this embodiment, the still image processing unit 11 and the moving image processing unit 12 perform the image processing on the input image simultaneously and in parallel, so that environmental changes, noise, etc. Therefore, even in a situation in which some processing results become invalid, a sufficient feature amount can be secured, and the segment matching processing unit 20 determines whether each feature amount obtained by each image processing method is different between image processing methods or Since the relative relationship between the frames is determined and the identity of the target object is determined based on the determination result, it is not necessary to evaluate whether or not the target object is provided for each segment. It can contribute to shortening the time required for the process of determining the target object, and accurately determine the identity of the target object without being affected by changes in image conditions or noise. Thereby improving the physical accuracy.

When it is determined whether or not there is a correlation with the past image processing result, the degree of overlap between the feature amount corresponding to the past segment and the feature amount corresponding to the current segment is determined. Since the identity of the target object is determined from the result, it is possible to determine the identity of the fast moving target object and the identity of many segments with high accuracy.

Further, when judging the presence or absence of a correlation with the image processing results between the image processing methods, the degree of overlapping of the feature amounts corresponding to the respective image processing results is correlated, and the target object is determined based on both correlation processing results. Since the identity of an object is determined, it is possible to determine the identity of many segments with high accuracy. Determination can be prevented.

In the above-described embodiment, the segment matching processing unit 20 has a feature value holding function of storing feature values extracted from each frame in a database, and detects whether a segment is included in each frame. When a segment in the current frame is not temporarily detected by the segment detection function, the feature amount of the previous frame corresponding to the segment is read out from the database and interpolated between the current frame and the next frame. A feature amount interpolation function is provided. For this reason, even if a segment that has been continuously detected due to a change in the input image is not temporarily detected, the retained feature amount of the previous frame is interpolated to ensure pseudo target continuity. Accordingly, it is possible to prevent the segment being detected from being temporarily interrupted, thereby preventing the determination as the background.

In the above embodiment, the identity of the target object is determined by the correlation processing based on the overlap of the feature values obtained between the image processing methods or between the frames. For example, a reference point is determined in a segment. The identity of the target object can be determined based on the distance correlation between the reference point positions of the segments between the image processing methods or between the frames. In addition, the present invention is not limited to the above-described embodiment, and can be similarly implemented by various modifications without departing from the gist of the present invention.

[0041]

As described in detail above, according to the present invention,
Even if the input image changes due to a change in image conditions, noise, etc., the identity of the target object can be accurately determined, and sufficient continuity of target detection can be ensured even if the target is temporarily interrupted. A target determination processing device and a target determination processing method can be provided.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of an embodiment of a target determination processing device according to the present invention.

FIG. 2 is a functional block diagram showing a main configuration of the target determination processing device shown in FIG. 1;

FIG. 3 is a view for explaining a feature amount input operation in the segment matching processing unit shown in FIG. 1;

FIG. 4 is a diagram illustrating a segment matching processing unit illustrated in FIG.
The figure for demonstrating the overlap correlation determination process of the singular previous frame of process 1 segment, and the singular current frame.

FIG. 5 is a diagram illustrating a segment collation processing unit shown in FIG.
FIG. 11 is a diagram for explaining an overlap correlation determination process between a single current frame segment and a plurality of previous frame segments.

FIG. 6 is a diagram illustrating a segment collation processing unit shown in FIG.
FIG. 9 is a diagram for explaining an overlap correlation determination process between a plurality of current frame segments and a single previous frame segment.

FIG. 7 is a diagram illustrating a segment matching processing unit illustrated in FIG.
The figure for demonstrating the overlap correlation determination process of the multiple processing 1 segment and the multiple processing 2 segment.

FIG. 8 is a diagram showing recorded contents of a segment collation result in a segment collation processing unit shown in FIG. 1;

FIG. 9 is a diagram illustrating a segment matching processing unit illustrated in FIG.
The figure for demonstrating the overlap correlation determination processing of one singular process 1 segment and singular process 2 segment.

FIG. 10 is a diagram for explaining an overlap correlation determination process between one singular process and two multi-process segments in the segment matching processing unit shown in FIG. 1;

FIG. 11 is a view for explaining an overlap correlation determination process between one segment for multiple processing and two segments for single processing in the segment matching processing unit shown in FIG. 1;

FIG. 12 is a diagram for explaining an overlap correlation determination process between a multi-process 1 segment and a multi-process 2 segment in the segment matching processing unit illustrated in FIG. 1;

FIG. 13 is a diagram showing recorded contents of a segment collation result in the segment collation processing unit shown in FIG. 1;

[Explanation of symbols]

 10: Image input system, 11: Still image processing unit, 12: Moving image processing unit, 20: Segment matching processing unit.

Claims (10)

[Claims] 1. A target determination processing device for determining a target object by using image processing, comprising inputting image data of a frame structure including the target object,
Image input means for processing the input image data by a plurality of types of image processing methods; and extracting feature amounts corresponding to object candidates obtained by image processing of the respective image processing methods of the image input means. Determining processing means for determining the identity of the target object based on at least one of a relative relationship between the respective feature amounts and a relative relationship between the respective feature amounts between the respective image processing methods. Target determination processing device. 2. The image input device according to claim 1, wherein the plurality of types of image processing methods are executed simultaneously and in parallel to generate time-series image processing results including target object candidate extraction results. 2. The target determination processing device according to 1. 3. The target determination processing device according to claim 1, wherein the determination processing unit determines the identity of the target object by a correlation process based on the overlap of the feature amounts obtained between the respective image processing methods. . 4. The target determination processing device according to claim 1, wherein said determination processing means determines the identity of the target object by a correlation process based on overlapping feature amounts obtained between frames. 5. The determination processing unit includes: a feature amount holding unit that holds a feature amount extracted from a frame; and an object that detects whether an object candidate according to each image processing method is included in the frame. Candidate detecting means, and when the object candidate in the current frame is not temporarily detected by the object candidate detecting means, the characteristic amount of the previous frame corresponding to the object candidate is read out from the characteristic amount holding means and the current amount is read. 2. The target determination processing device according to claim 1, further comprising an interpolation unit that performs interpolation between a frame and a next frame. 6. A target determination processing method for determining a target object using image processing, comprising the steps of: inputting image data of a frame structure including a target object, and inputting the input image by a plurality of types of image processing methods. Processing the data; extracting each feature amount corresponding to the candidate object obtained by the image processing of each image processing method; the relative relationship of each feature amount between frames and each feature between the image processing methods A step of determining the identity of the target object based on at least one of the relative amounts of the objects. 7. The step of processing the input image data, wherein a plurality of types of image processing methods are executed simultaneously and in parallel to generate respective time-series image processing results including target object extraction results. 7. The target determination processing method according to claim 6, wherein 8. The method according to claim 1, wherein the step of determining the identity of the target object determines the identity of the target object by a correlation process based on overlapping feature amounts obtained between the respective image processing methods. 6. The target determination processing method according to 6. 9. The method according to claim 6, wherein the step of determining the identity of the target objects determines the identity of the target objects by a correlation process based on overlapping feature amounts obtained between frames. Target judgment processing method. 10. The step of judging the identity of the target object includes holding a feature amount extracted from the frame and detecting whether or not the frame includes an object candidate according to each image processing method. 7. The method according to claim 6, wherein when a candidate object in the current frame is not temporarily detected, the feature amount of the previous frame corresponding to the candidate object is interpolated between the current frame and the next frame. Target determination processing method.