JP3365182B2 - Video surveillance equipment - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image monitoring apparatus for reproducing a moving object to be monitored as a virtual interpolation image having a desired field of view / angle or displaying the image with an optimum camera.

[0002]

2. Description of the Related Art As shown in FIG. 12, a conventional video monitoring apparatus disclosed in, for example, Japanese Patent Laid-Open No. 334572/1990, a fixed camera 51 always takes an image of the entire interior of a room under a fixed field of view. When the tracking camera 51a detects an object to be monitored (intruder or the like), the tracking camera 51a moves from the calculation unit 52 to the activation unit 53 according to the movement of the object.
The start-up and tracking command and the magnification command via the tracking unit 54 and the tracking command are performed by the built-in zooming motor so that detailed features (human phase, etc.) can be grasped by the command and the magnification command, and an enlarged image corresponding to the field of view centered on the monitored object. The image of. The image section 52 is
The object to be monitored is detected based on the video signal from the fixed camera 51, and the start and tracking commands are issued and the magnification command of the desired enlarged image is issued. The recording unit 55 includes fixed and tracking cameras 51 to 5
The captured image is recorded from 1a, reproduced if necessary, and examined.
Further, for example, as a conventional technique of detecting / tracking a moving monitored object (intruder or intruding object into the monitoring area) generally shown, a feature point is recognized with respect to image information captured by two video cameras, and the image is detected after the coordinates are detected. A method of generating an interpolated intermediate image, a method of acquiring the three-dimensional position and size of the target object by triangulation from the center coordinates of the extracted object and the azimuth angle of the camera by pattern matching or the like, and JP-A-6-32518.
There is a method of predicting the position of a tracking target object after a predetermined time from captured image information as in Japanese Patent No. 0.

The above-mentioned conventional video monitoring apparatus adopts a system of video monitoring by moving the camera itself in the direction of a moving object to be monitored or switching several cameras.

[0004]

In the conventional image monitoring apparatus as described above, since the camera itself is moved in the direction of the moving object to be monitored or several cameras are switched,
The monitoring area is limited by the rotation speed of the camera platform and the number of cameras. In addition, it is necessary to increase the number of cameras when it is desired to display an image with a desired field of view / angle (when there are a plurality of observers or when it is desired to confirm from a plurality of fields of view). In addition, when a natural disaster such as an earthquake or lightning strike or a system abnormality occurs, the combination of automatic start conditions for video surveillance is fixed and limited.

The problem to be solved by the present invention is to provide the following system in a video surveillance device. (1) A method of performing video monitoring using a virtual interpolated image in which the extracted feature points of each captured image captured from a plurality of cameras are weighted and combined according to the distance from the virtual viewpoint (virtual video monitoring method) (2) Wide area A system that does not require recognition and tracking of the monitored object within the range that can be captured by the camera, and that autonomously performs video monitoring operation (autonomous video monitoring system) (3) In the inter-observation communication that arranges multiple devices on the network Video monitoring system that enables wide-range monitoring and measures load distribution through autonomous cooperative operation (autonomous cooperative distributed video monitoring system) (4) Corresponding to each system, the required monitoring information is constantly monitored and multiple elements are combined. Video monitoring method under the auto-start condition set (auto-start video monitoring method)

[0006]

In order to solve the above-mentioned problems, the video monitoring apparatus of the present invention is provided with the following means,
It is characterized by adopting autonomous / autonomous cooperative decentralized / auto-start video monitoring system. That is, the video monitoring apparatus according to the present invention includes an image recognition unit that extracts a feature point of each captured image captured from three or more cameras installed on the same plane,
The plane specifying unit that specifies a plane formed by the camera positions on the same plane, the desired virtual viewpoint set on the plane specified by the plane specifying unit from the input unit, and the 3
A plane distance calculation unit that extracts each plane distance from each camera position on a table or more, and each plane relative distance from the plane distance calculation unit with respect to each position coordinate of each feature point extracted by the image recognition unit. And a plane virtual interpolation image synthesizing unit for synthesizing the plane virtual interpolation image having the desired virtual viewpoint as a viewpoint by repeating the weighting by the above. In addition, a second image recognition unit that extracts a characteristic point of each captured image captured from a second camera that is not on the same plane as the camera installed on the same plane, and a space set by the input unit. A straight line connecting a desired virtual viewpoint and the second camera position, and a plane viewpoint specifying unit that specifies an intersection of the plane specified by the plane specifying unit as a virtual viewpoint on the plane, and a virtual viewpoint on the plane. position of a desired virtual viewpoint and the the spatial distance calculator for extracting each spatial distance between the second camera position, the feature points extracted by the second image recognition unit on the space
For each coordinate and the monitored object position coordinates of the plane virtual interpolation image output from the plane virtual interpolation image synthesizing unit, by repeating weighting by each spatial relative distance from the spatial distance calculating unit, And a spatial virtual interpolation image synthesizing unit for synthesizing a spatial virtual interpolation image having a desired virtual viewpoint in space as a viewpoint. In addition, the preceding image storage unit that temporarily stores the captured image captured from the camera as the preceding captured image, and the difference between the preceding captured image stored in the preceding image storage unit and the current captured image captured from the camera is calculated. Thus, the change range recognizing unit that specifies the change range in the time axis direction and outputs only the change range to the plane virtual interpolation image synthesizing unit as the respective feature points from the image recognizing unit is provided. . In addition, a three-dimensional graphic image generation unit that draws and creates a visual field image with the virtual viewpoint on the plane set by the input unit as a viewpoint, and the three-dimensional graphic image generation unit. A three-dimensional image synthesis for generating a three-dimensional image with a virtual viewpoint on the plane as a viewpoint by image-synthesizing a three-dimensional graphic image and a plane virtual interpolation image of only the change range from the plane virtual interpolation image synthesis unit. And a section. Further, the input unit sets two virtual viewpoints for the right eye and the left eye that are separated by an interval corresponding to the parallax of both eyes, and the plane virtual interpolation image synthesizing unit corresponds to each of the two virtual viewpoints. Thus, two plane interpolation images for the right eye and the left eye are generated.

It should be noted that the image recognition units are located on the same plane.
Characteristic points of each captured image captured from one or more cameras are extracted. Alternatively, the characteristic points of each captured image captured from three or more cameras on the same plane and one or more cameras not on the same plane are extracted.

The plane specifying section specifies a plane formed by the positions of three or more cameras on the same plane.

The plane distance calculation unit uses the desired virtual viewpoint on the plane specified by the plane specifying unit from the input means to be set or the left or right eye desired virtual image on the plane converted by using the two-eye difference. Extract each plane distance with each of three or more camera positions on the same plane with respect to the viewpoint.

The plane virtual interpolation image synthesizing unit repeats weighting for each feature point from the image recognizing unit by each plane relative distance from the plane distance calculating unit to synthesize a plane virtual interpolation image.

The plane view point specifying section is a plane as an intersection of a straight line uniquely determined by a straight line passing through a camera not on the same plane as the desired virtual view point in an arbitrary space from the input means to be set and the plane specified by the plane specifying section. The virtual viewpoint for the left or right eye on the plane converted for the left or right eye is specified using the above virtual viewpoint or the two-eyes visual difference.

The spatial distance calculation unit extracts each spatial distance between the virtual viewpoint on the plane from the planar viewpoint specifying unit and the desired virtual viewpoint on the space and the camera position not on the same plane.

The spatial virtual interpolation image synthesizing unit calculates a spatial distance between the feature points of the captured image captured by the camera which is not on the same plane from the image recognizing unit and the planar virtual interpolation image from the planar virtual interpolation image synthesizing unit. The spatial virtual interpolated image is synthesized by repeating weighting according to each spatial relative distance from the part.

The preceding image storage unit temporarily stores each captured image captured from each camera as each preceding captured image.

The change range recognition section determines the change range in the time axis direction by obtaining the difference between each current captured image captured from each camera and each preceding captured image from the preceding image storage section. The respective feature points from the image recognition unit are output to the plane virtual interpolation image synthesis unit only for the change range.

The three-dimensional graphic image generation unit draws and creates the created three-dimensional data for the visual field image from the virtual viewpoint on the plane from the input means to be set.

The three-dimensional monitoring image synthesizing unit performs image synthesizing of the three-dimensional graphic image from the three-dimensional graphic image generating unit and the plane virtual interpolation image of only the change range from the plane virtual interpolation image synthesizing unit to obtain the three-dimensional monitoring image. To generate.

The object position specifying unit extracts the characteristic points of the wide area imaged image captured by the wide area camera for observing the entire observed area, and generates the specific position information of the monitored object.

The myopic camera control unit fetches a myopic image which is selected and controlled by the myopic camera selection control means from one or more myopic cameras in the observation area according to the specific position information from the object position specifying unit, and the image is taken. Display on the display. Alternatively, according to the specific position information from the object position specifying unit, from a myopic camera having one or more movable mechanisms in the observation area,
First, the myopia camera selection control means controls the selection,
Next, the near-sighted image whose operation is controlled by the movable camera control means is captured and displayed on the image display unit.

The virtual camera control section is provided in place of the myopic camera control section, and does not use the myopic camera according to the specific position information from the object position specifying section, but at the optimum position where the monitored object is myopic by the virtual interpolation image synthesizing means. A virtual myopia image to be combined is captured as a virtual interpolation image from a virtual viewpoint, and is displayed on the image display unit.

A plurality of moving object recognition units are provided and extract characteristic points of each captured image taken from a plurality of cameras that observe a part of the entire observed area, and generate current position and traveling direction information of the monitored object. .

A plurality of picked-up image data transmitters / receivers are provided, the picked-up image data are taken in from a plurality of cameras via the picked-up image data transmission / reception control unit, and are displayed on the image display unit.

A plurality of monitored object traveling direction information transmitters are provided, and it is judged whether or not the monitored object is within the image capturing range based on the current position and traveling direction information from the plurality of moving object recognition units, and the outside of the range is determined. In that case, the fact is notified to the adjacent monitored object traveling direction receiver in the traveling direction of the monitored object via the communication line.

When the monitored object traveling direction information receiver receives the notification from the monitored object traveling direction information transmitter and the monitored object enters the image capturing range, the captured image data transmission / reception control section communicates with the communication line. It will be notified via.

The input information processing unit extracts only the required monitoring information from the input information from the monitoring camera / sensor.

The transmission start condition holding unit holds a condition judgment expression having a parameter for presetting an arbitrary transmission start condition according to required monitoring information or downloading an arbitrary transmission start condition. Alternatively, a transmission condition automatic changing means for automatically optimizing the transmission start condition parameter to be downloaded is provided.

The condition evaluation unit compares and evaluates the condition determination formula referred from the transmission start condition holding unit and the required monitoring information from the input information processing unit.

The transmission unit starts transmission of the picked-up image captured from the surveillance camera to the image display unit according to the judgment result from the condition evaluation unit.

[0029]

BEST MODE FOR CARRYING OUT THE INVENTION As shown in FIG. 1, a video surveillance apparatus showing an embodiment of the present invention is a surveillance camera 1 for A, B, and C.
Is on the same plane and images the monitored object. The image recognition unit 2 for A, B, and C recognizes the monitored object (feature point) from each image data captured by the monitoring cameras 1 for A, B, and C, and detects the position coordinates. The plane specifying unit 3 specifies a plane formed by the three points A, B, and C of the monitoring cameras 1. The input unit 4 sets a desired virtual viewpoint on the plane specified by the plane specifying unit 3 with a keyboard, a mouse or other input means, and inputs it to the plane distance calculation unit 3. The plane distance calculation unit 5 extracts each plane distance between the positions of the monitoring cameras 1 of A, B, and C with respect to the desired virtual viewpoint on the plane from the input unit 4. The plane virtual interpolation image synthesizing unit 6 performs image interpolation processing for weighting each monitored object position coordinate from the A, B, and C image recognizing units 2 with each plane relative distance from the plane distance calculating unit 5. By repeating this, the monitored object captured image (planar virtual interpolation image) from a virtual viewpoint on the plane is synthesized.
The image display unit 7 displays the plane virtual interpolation image from the plane virtual interpolation image synthesis unit 6 as necessary. Plural observers can monitor from a desired viewpoint without operating or moving the camera itself in the captured image of the monitored object from a desired virtual viewpoint on a plane.

The video monitoring apparatus according to the above-mentioned embodiment carries out video monitoring using a virtual interpolation image in which the extracted feature points of the captured images captured from a plurality of cameras are weighted by the distance from the virtual viewpoint and synthesized. (Virtual video monitoring method) is adopted.

As shown in FIG. 2A, the plane virtual interpolated image synthesizing unit 6 converts the position coordinates of each monitored object from the image recognizing units 2 of A, B, and C into (x ₁ , y ₁ ) and (x ₂ ). , Y ₂ ) and (x ₃ ,
y ₃ ), and the relative distances between the monitoring cameras 1 of A, B, and C and the virtual viewpoint from the plane distance calculating unit 5 are a, b, and c, first, the monitored object position coordinates from the corresponding virtual viewpoints. (X, y) is calculated as follows, and then the calculation is repeated to synthesize the plane virtual interpolation image as shown in FIG. x = (a × x ₁ + b × x ₂ + c × x ₃ ) / 3 y = (a × y ₁ + b × y ₂ + c × y ₃ ) / 3

In the embodiment of the invention shown in FIG. 1, as shown in FIG. 3, the surveillance camera 1a not on the same plane as the surveillance cameras 1 for A, B, and C, and the monitored object are recognized from the captured image data. In addition to the image recognition unit 2a that detects the position coordinates, the plane view point identification unit 3a, the input unit 4a, the spatial distance calculation unit 5a, the spatial virtual interpolation image synthesis unit 6a, and the image display unit 7a.
And may be separately provided. A plurality of observers can monitor a desired viewpoint without operating or moving the camera itself in a captured object captured image from a desired virtual viewpoint in space. The plane viewpoint specifying unit 3a is a keyboard of the input unit 4a instead of the input unit 4,
The first virtual viewpoint on the plane is defined as an intersection of the first desired virtual viewpoint in the space set by the mouse or other input means and the straight line uniquely determined through the surveillance camera 1a and the plane specified by the plane specifying unit 3. Identify. The spatial distance calculating unit 5a extracts each spatial distance between the first desired virtual viewpoint in the space set by the input unit 4a and the position of the surveillance camera 1a for the first virtual viewpoint on the plane from the planar viewpoint specifying unit 3a. To do. The spatial virtual interpolation image synthesizing unit 6a uses the spatial relative distances from the spatial distance calculating unit 5a for the monitored object position coordinates from the image recognizing unit 2a and the plane virtual interpolating image from the plane virtual interpolating image synthesizing unit 6. The spatial virtual interpolation image is synthesized by repeating the image interpolation processing for weighting. Image display section 7a
Instead of the image display unit 7, the spatial virtual interpolation image synthesizing unit 6a
The spatial virtual interpolation image is displayed as required.

In the embodiment of the invention shown in FIG. 1, as shown in FIG. 4, the preceding image storage units 8 of A, B and C, and A and B are stored.
It is also possible to separately provide the change range recognition unit 9 for C and C and synthesize the planar virtual interpolation image as shown in FIG. High-speed image interpolation processing can be performed only with the image change range. The preceding image storage unit 8 of A, B, and C performs image processing on the imaged image data of the monitoring cameras 1 of A, B, and C by the image recognition unit 2 of A, B, and C, and also each preceding captured image. Temporarily store as data. The change range recognizing unit 9 for A, B, and C recognizes each current captured image data and A and B input from the image recognizing unit 2 for A, B, and C.
And C, the difference between each of the preceding captured image data from the preceding image storage unit 8 is obtained, the change range in the time axis direction is specified, and the monitored object position coordinates from the A, B, and C image recognition units 2 are calculated. Only the change range is output to the plane virtual interpolation image synthesizing unit 6.

In the embodiment of the invention shown in FIG. 4, as shown in FIG. 6, the visual field image from the desired virtual viewpoint on the plane from the input unit 4 is drawn and created with the three-dimensional data already created. Three-dimensional (3D) graphic image generation unit 10
Then, the 3D graphic image data and the plane virtual interpolation image synthesizing unit 6 perform image synthesis on the plane virtual interpolation image data only in the change range to generate a 3D monitoring image by synthesizing the monitoring image with the current 3D desired visual field image. The 3D monitoring image synthesizing unit 11 may be separately provided. 3D monitoring images can be easily combined with existing 3D data.

Further, instead of the desired virtual viewpoint on the plane in the embodiment of the invention shown in FIG. 1, the left on the plane converted for the left (or right) eye by utilizing both visual differences as shown in FIG. A virtual viewpoint for the eye (or the right) may be used. If the desired virtual viewpoint coordinates on the plane are (x, y) and the visual difference between them is αcm, the virtual viewpoint for the left / right eyes on the plane (when the origin is on the left) is approximately (x−α × θ / 2, y) / (x + α × θ /
2, y) (θ is an error or a shift due to a distance), and a stereoscopic image can be easily created by obtaining a left / right eye plane virtual interpolation image.

Although the embodiment of the invention shown in FIG. 4 and FIG. 6 or FIG. 7 has been described as the image interpolation processing in the embodiment of the invention shown in FIG. 1, the embodiment shown in FIG. 2 or FIG. Alternatively, it is needless to say that the present invention can be applied to all the video surveillance methods in which the image interpolation processing is performed by the input of two or more cameras, in addition to the embodiment of the invention shown in FIG.

As shown in FIG. 8 (a), the myopic cameras 21 of A, B, and C of the image monitoring apparatus according to the embodiment related to the present invention image the monitored object at a specific position. A and B
The wide area camera 21a constantly observes the entire observed area. The A and B object position specifying units 22 recognize the monitored object from the wide area image data captured by the A and B wide area cameras 21a, detect the position coordinates, and generate specific position information. The myopia camera control unit 23 automatically or manually selects and controls the myopia camera 21 from the myopia camera 21 of the A, B or C according to the specific position information from the A and B object position specifying units 22. Is captured and displayed on the near-sighted camera image display unit 24. It repeatedly and autonomously tracks the monitored object by the optimal camera every fixed frame,
The monitoring operation load can be reduced.

The image monitoring apparatus of the above embodiment does not require recognition or tracking of a monitored object within a range captured by a wide area camera, and autonomously performs image monitoring operation (autonomous image monitoring method). Take.

In the embodiment of the invention shown in FIG. 8A, instead of the myopic camera 21 and the myopic camera control section 23, a movable mechanism for zooming, panning, tilting, etc. is provided as shown in FIG. 8B. According to the myopic camera (movable camera) 21b and the specific position information from the object position specifying unit 22 of A and B, first the myopic camera selection control means performs automatic or manual selection control, and then the movable camera control means. A myopic camera control unit 23 that takes in myopic imaged image data from the A, B, or C myopic camera 21b that controls operations such as zooming, panning, and tilting, and displays the data on the myopic camera image display unit 24.
It may be configured as a. The monitoring operation load can be reduced by repeatedly and autonomously tracking the monitored object in the optimum field of view by the optimum camera every fixed frame.

Further, in place of the myopic camera control unit 23 in the embodiment of the invention shown in FIG. 8 (a), as shown in FIG.
According to each specific position information (monitored object position coordinates) from the object position specifying unit 22 of B and B, the myopic camera 21 is not used, and the virtual interpolation image synthesizing means performs a myopic view of the monitored object from the virtual viewpoint at the optimum position. Image of monitored object (virtual interpolation image)
Alternatively, the virtual camera control unit 23b may be configured to fetch virtual myopic imaged image data to be combined by image interpolation processing and display the image on the myopic camera image display unit 24. It is possible to autonomously track a monitored object from a virtual viewpoint repeatedly every fixed frame and reduce the monitoring operation load.

As shown in FIG. 10, an image monitoring apparatus according to another embodiment related to the present invention has observation cameras 31 for A, B, C and D for observing a part of the entire monitored area.
The moving object recognition unit 32 of A, B, C, and D recognizes the moving monitored object from the image data captured by the observation cameras 31 of A, B, C, and D, and specifies the position and the traveling direction. A
The captured image data transmitter / receiver 33 of B, C, and D captures the captured image data from the observation cameras 31 of A, B, C, and D via the captured image data transmission / reception control unit 37, and displays the captured image data on the image display unit 38. . The monitored object traveling direction information transmitters 34 of A, B, C, and D use the current position and traveling direction information from the moving object recognition units 32 of A, B, C, and D to detect the monitored object within the image capturing range. If it is out of the range, the adjacent A, B, C in the traveling direction of the monitored object,
Alternatively, it notifies the monitored object traveling direction information receiver 34 of D to that effect via the communication line 36. When the monitored object traveling direction information receiver 35 of A, B, C, or D receives the notification and enters the monitored object within the image capturing range, the captured image data transmission / reception control unit 37 transmits the monitored image through the communication line. Notify that. It is possible to monitor a wide range by the above continuous operation, and it is possible to autonomously cooperate and distribute the load. It goes without saying that the observation camera may be a movable camera having a movable mechanism such as zoom, pan and tilt.

The video monitoring apparatus according to the above-described embodiment enables a wide range of monitoring through inter-observation system communication arranged on a network, and performs video monitoring by measuring load distribution by autonomous cooperative operation (autonomous cooperation). Distributed video surveillance method).

The picked-up image data transmitter / receiver 33 and the monitored object traveling direction information transmitter / receiver 35/36 are as shown in FIG.
For example, if the monitored object is moving to the right at the position shown in the figure, the observation camera 3 of B within the imageable range concerned.
The picked-up image data transmission / reception control unit 37 takes in the picked-up image data transceiver 33 of B for the picked-up image data from
The image is displayed on the image display unit 38 via. Next, when it is predicted that the monitored object moves to the right and is out of the image capturing range of the observation camera 31 of B, the monitored object traveling direction information transmitter 34 of B transmits the monitored object traveling direction of C. When the current position and the traveling direction information of the monitored object are transmitted / received to / from the information receiver 35 via the communication line 36, for the image data captured from the observation camera 31 of C in the image capture range to the right of B, The captured image data transmitter / receiver 33 of C loads the image display unit 38 via the captured image data transmission / reception control unit 37.
To display.

An image monitoring apparatus showing another embodiment related to the present invention is as shown in FIG.
1 images the monitored object. The monitoring sensor 41a detects, for example, a change in temperature. Input information processing unit 42
Extracts only the required monitoring information such as the image change range, the magnitude of vibration, and the temperature rise from the input information from the monitoring camera 41 or the monitoring sensor 41a. The transmission start condition holding unit 43 uses a condition determination formula (for example, image change of 20% or more ‖ (or) seismic intensity that presets an arbitrary transmission start condition based on required monitoring information (for example, image change range, vibration magnitude, etc.). 2 or more). The condition evaluation unit 44 uses the condition determination formula referred from the transmission start condition holding unit 43 and the required monitoring information from the input information processing unit 42 (for example, image change 5).
%, Seismic intensity 3, etc.) to determine whether or not “start transmission” is necessary. The transmission unit 45 starts transmission of the captured image data captured from the surveillance camera 41 to the image display unit 46 according to the determination result of “transmission start” from the condition evaluation unit 44. The automatic start condition for video surveillance can be easily determined by combining multiple elements corresponding to each system.

The video monitoring apparatus according to the above-described embodiment adopts a system (automatic startup video monitoring system) that constantly monitors required monitoring information corresponding to each system and performs video monitoring under an automatic startup condition that sets a combination of a plurality of elements. .

In the embodiment of the invention shown in FIG. 11 (a), the transmission start condition holding unit 43 has been described as a method of presetting an arbitrary transmission start condition in the condition judgment expression. 11 (b), an arbitrary transmission start condition (for example, A: “Image change 2
0% or more ", B:" Seismic intensity of 2 or more ", C:" Keywords such as "takes""(meaning" or ")" and is specified as the transmission start condition holding unit 43b. Good. The automatic start condition can be easily set and changed by the condition judgment formula created with any transmission start condition parameter.

In the embodiment of the invention shown in FIG. 11 (b), the transmission start condition holding section 43b automatically sets the transmission start condition parameters downloaded by the transmission condition automatic changing means as shown in FIG. 11 (c). You may comprise as the transmission start condition holding part 43c which optimizes dynamically. For example, when the image change clearly exceeds 20% between the daytime and nighttime under outdoor installation, the transmission start condition parameter of "image change 20% or more" is automatically changed to "image change 20% with preceding image for 10 seconds". To do. The automatic start condition can be easily set and changed by the condition judgment formula created by the transmission start condition parameter that optimizes in response to the environmental change that can be predicted to some extent.

[0048]

As described above, the video monitoring apparatus according to the present invention adopts the virtual / autonomous / autonomous cooperative distributed / auto-start video monitoring system. Therefore, the camera itself should be moved toward the moving monitored object as in the conventional case. Compared with the method of monitoring video by switching several cameras, the drawbacks are solved and the following effects are obtained for each invention. (1) A plurality of observers can monitor from a desired plane viewpoint without operating or moving the camera itself in a captured image of a monitored object from a desired virtual viewpoint on a plane. (2) A plurality of observers can monitor from the desired space viewpoint without operating or moving the camera itself in the captured image of the monitored object from the desired virtual viewpoint in space. (3) High-speed image interpolation processing can be performed only in the image change range. (4) A three-dimensional monitoring image can be easily combined with existing three-dimensional data. (5) A stereoscopic image can be easily created by obtaining a plane virtual interpolation image for the left / right eyes .

[Brief description of drawings]

FIG. 1 is a configuration diagram of a video surveillance device showing an embodiment of the present invention.

FIG. 2 is a diagram illustrating a function of a plane virtual interpolation image synthesizing unit shown in FIG.

FIG. 3 is a configuration diagram showing another form of the embodiment of the present invention.

FIG. 4 is a configuration diagram showing another form of the embodiment of the present invention.

FIG. 5 is a diagram showing a process of synthesizing the plane virtual interpolation image shown in FIG.

FIG. 6 is a configuration diagram showing another embodiment of the present invention.

FIG. 7 is a configuration diagram showing another embodiment of the present invention.

FIG. 8 is a configuration diagram of a video surveillance device showing an embodiment related to the present invention.

FIG. 9 is a configuration diagram showing another embodiment related to the present invention.

FIG. 10 is a configuration diagram of a video surveillance device showing another embodiment related to the invention.

FIG. 11 is a configuration diagram of a video surveillance device showing another embodiment related to the invention and a configuration diagram showing two other forms of the embodiment.

FIG. 12 is a configuration diagram of a video surveillance device showing a conventional technique.

[Explanation of symbols]

1 surveillance cameras, 1a surveillance cameras not on the same plane,
2, 2a Image recognition unit, 3 Plane specifying unit, 3a Plane viewpoint specifying unit, 4, 4a Input unit, 5 Plane distance calculating unit, 5
a Spatial distance calculator, 6 Plane virtual interpolation image synthesizer, 6
a space virtual interpolation image synthesizing unit, 7, 7a image display unit,
8 preceding image storage unit, 9 change range recognition unit, 10 three-dimensional (D) graphic image generation unit, 11 surveillance image synthesis unit, 21 myopia camera, 21a wide area camera, 21b movable camera, 22 object position specification unit, 23, 23a Myopia camera control unit, 23b virtual camera control unit, 24 myopia camera image display unit, 31 observation camera, 32 moving object recognition unit, 33 imaged image data transceiver, 34 monitored object traveling direction information transmitter, 35 monitored object progress Direction information receiver, 36 communication line, 37 captured image data transmission / reception control unit, 38 image display unit, 41 surveillance camera, 41a
Monitoring sensor, 42 input information processing unit, 43, 43a,
43b Transmission start condition holding unit, 44 Condition evaluation unit, 45
Transmitter, 46 image display. In the drawings, the same reference numerals indicate the same or corresponding parts.

Continuation of the front page (56) Reference JP-A-7-129792 (JP, A) JP-A-7-274063 (JP, A) JP-A-7-287761 (JP, A) JP-A-6-325180 (JP , A) JP-A-5-334572 (JP, A) (58) Fields investigated (Int.Cl. ⁷ , DB name) G06T 1/00 315 G08B 13/196 H04N 7/18

Claims (5)

(57) [Claims] 1. An image recognition unit for extracting a feature point of each captured image captured from three or more cameras installed on the same plane, and a plane formed by each camera position on the same plane is specified. A plane specifying unit; a plane distance calculating unit that extracts a plane distance between the desired virtual viewpoint set on the plane specified by the plane specifying unit from the input unit and each of the three or more camera positions; A plane having the desired virtual viewpoint as a viewpoint from the captured image by repeating weighting by the relative distance of each plane from the plane distance calculating unit for each position coordinate of each feature point extracted by the recognition unit. And a plane virtual interpolation image synthesizing unit for synthesizing virtual interpolation images. 2. A second image recognition section for extracting a feature point of each captured image captured from a second camera which is not on the same plane as the camera installed on the same plane, and is set by the input means. A plane viewpoint specifying unit that specifies an intersection point between a desired virtual viewpoint in space and the second camera position and a plane specified by the plane specifying unit as a virtual viewpoint on the plane, and a plane viewpoint specifying unit on the plane. A spatial distance calculation unit that extracts spatial distances between the desired virtual viewpoint in the space and the second camera position with respect to the virtual viewpoint, and position coordinates of the feature points extracted by the second image recognition unit. By repeating weighting with respect to the monitored object position coordinates of the plane virtual interpolation image output from the plane virtual interpolation image synthesizing unit by each spatial relative distance from the spatial distance calculating unit, The video monitoring device according to claim 1, further comprising: a spatial virtual interpolation image synthesizing unit that synthesizes a spatial virtual interpolation image having the desired virtual viewpoint as a viewpoint. 3. A preceding image storage unit for temporarily storing a captured image captured from the camera as a preceding captured image,
By determining the difference between the preceding captured image stored in the preceding image storage unit and the current captured image captured from the camera, the change range in the time axis direction is specified, and only the change range from the image recognition unit is specified. 3. The video monitoring apparatus according to claim 1, further comprising: a change range recognition unit that outputs the plane virtual interpolation image synthesis unit as each feature point. 4. A three-dimensional graphic image generation unit for drawing and creating a visual field image having a virtual visual point on the plane set by an input means as a viewpoint, and the three-dimensional graphic image generation. 3D image from the virtual viewpoint on the plane is generated by image-compositing the 3D graphic image from the image plane and the plane virtual interpolation image from the plane virtual interpolation image synthesizing unit 3 The image monitoring apparatus according to claim 3, further comprising a three-dimensional image combining unit. 5. The input means sets two virtual viewpoints for the right eye and the left eye separated by an interval corresponding to the parallax of both eyes, and the plane virtual interpolation image synthesizing unit sets each of the two virtual viewpoints. 5. The video monitoring apparatus according to claim 1, wherein two plane interpolation images for the right eye and the left eye are generated corresponding to the above.