JPH04148282A - Image pickup state detecting method for picture image pickup device - Google Patents

Abstract

PURPOSE:To accurately specify a physical feature in a scene by detecting the image pickup state of a picture image pickup device from the relative size and direction of each velocity vector and the convaerging point position of each velocity vector in a velocity vector distribution. CONSTITUTION:At a fix operation time, since the relative size of each velocity vector in the velocity vector distribution of an optical flow is zero, there is no direction and there is no converging point position or no diverging point position of the velocity vector in the velocity vector distribution as well, at that case, the operation of the picture image pickup device is specified as the fix from the calculated optical flow. Similarly, from the combination of all and the part of the converging point position or the diverging point position of each velocity vector in the velocity vector distribution, either of the image pickup states of a panning, a tilt, a dolly, a track, a booming, a zooming, a zoom or the like can be detected. Thus, in the analysis of the more interactive dealing technology of a moving image in accordance with the rapid connection of a computer and the moving image, the physical feature in the scene can be specified.

Description

Detailed Description of the Invention [Industrial Application Field] The present invention provides an imaging state of an image capturing device that detects the capturing state of the image capturing device when the image is manually operated from an image input from the image capturing device. This relates to a detection method.

[Prior Art] Due to recent rapid developments in technology for combining computers and video images, in addition to conventional video technology (storage, access, and display), the possibility of interactive video handling technology has been studied. There is.

Along with this, there is an increasing need for technology that automatically defines the unique characteristics of moving images.

Recently, a method for detecting video change points from changes in histograms between frames has been reported as a method for extracting moving image features. “Foreign Materials, Branch: Study on Video Database Handling, Institute of Electronics, Information and Communication Engineers, IE89-33
, pp. 49-56° 1989 J0 A method of detecting a scene change from a change in difference between frames or between fields has also been reported. "
Kamanaka, Endo, Ogura, Ishibashi: Video signal browsing device;
JP-A-56-68084, pp561-566.
See 1989. The above report is concerned with scene change points among the unique characteristics of moving images, and does not report on the physical characteristics of scenes.

The unique characteristics of moving images include the physical characteristics of the scene, and how to define the physical characteristics of the scene is thought to hold the key to improving techniques for handling moving images. . However, there is a problem in that reports regarding physical features in the scene have not yet been made.

[Problems to be Solved by the Invention] As described above, with the rapid combination of computers and moving images, there is a need to define physical features in scenes in the study of more interactive techniques for handling moving images.

Generally, images are input from an image capture device using a combination of the image capture device operations shown in FIGS. 3 and 4(a)-(g). The quality of the technique used to operate the image capturing device directly affects the quality of the screen and the satisfaction of the viewer. Images for broadcast programs are created with sufficient consideration given to the operation of the image capturing device. On the other hand, today, the speed with which video cameras have become widespread is astounding. The environment is such that even ordinary people can easily create moving images. When ordinary people create images casually, it is thought that they casually create images by combining the image capturing device operations shown in FIGS. 3 and 4 (a) to (g). This casual operation of the image capturing device can be understood to indicate some kind of intention on the part of the photographer.

Based on the above, it is recognized that it is appropriate to be able to detect the imaging state of an image capturing device as a physical feature in a scene for examining techniques for handling moving images.

[Means for Solving the Problems] An imaging state detection method of an image imaging device according to the present invention includes:
The relative magnitude and direction of each velocity vector in the apparent velocity vector distribution on the screen caused by the relative motion between the image capturing device and the subject, and the convergence of all and part of each velocity vector in the velocity vector distribution. The image capturing state of the image capturing device at the time of image capturing of the screen is detected from the point position or the divergent point position.

[Operation] In the present invention, the relative magnitude of each velocity vector in the velocity vector distribution in the apparent velocity vector distribution on the screen caused by the relative motion between the image capturing device and the subject, and the Imaging by fixing, panning, tilting, dolly, tracking, booming, zooming, etc., or a combination of multiple of these, from a combination of convergence point positions or divergence point positions of all and some of each velocity vector. It is possible to detect whether the state is

[Example] In the imaging state detection method of an image imaging device according to the present invention, dynamic pressure is obtained from optical flow. Optical flow is the apparent velocity vector distribution on the ζ screen caused by the relative movement between the image capturing device and the subject. As a typical technique for determining optical flow, temporal changes in image gradation and spatial Methods based on concentration gradients (gradient method) [i'B, P, Horn & B, G, 5chunk
:Determining Optical Flow,
Artifi, Intel,, Vol. 17. pi
), 185-203 (1981) Jl and a method for understanding motion in the spatiotemporal frequency domain ffD, J, Heer: 0
ptical Flow Using a Spati
otempolar Filters, Int, J, C
output, Vision, Vol. 1. pp, 279
~302 (19881, !1, etc. have been reported. Optical flow is generally used as a motion analysis method for segmenting objects in images and restoring three-dimensional structures. Recently, optical flow has been used for image communication coding. Applications have also been reported.

The operations of the image capturing device can be broadly classified as shown in Figures 3 and 4 (a).
) to (g), there are fixed shots, camera direction, camera position, camera height changes, and lens screen changes.

The specifications for the image input operation from optical flow (Figure 3) are the relative magnitude and direction of each velocity vector in the velocity vector distribution of optical flow, and the convergence point of all and some of the velocity vectors in the velocity vector distribution. This is done by calculating and analyzing the position or divergence point position.

Image input operation from optical flow (Figure 3) Regarding optical flow analysis method and detection of image capturing state from analysis in accordance with regulations of image input operation from optical flow (Figure 3)
A detailed explanation will be given below based on the drawings.

○Fix Fix operation refers to a camera operation that causes the camera 1 to stand still relative to the subject 2, as shown in FIGS. 3 and 4(a).

A typical optical flow for an image during a fix operation is shown in FIG. 1(a). The relative magnitude of each angle vector in the velocity vector distribution of optical flow is zero,
There is no direction. There is also no convergence or divergence point location of velocity vectors in the velocity vector distribution.

From the above, in optical flow, the relative magnitude of each velocity vector is zero and the direction is not fixed (if there is no convergence point position or divergence point position of each velocity vector in the velocity vector distribution, it can only be seen during fix operation). This can be considered to be an optical flow unique to a fixed operation.Therefore, from the calculated optical flow, it can be determined that the operation of the image capturing device is a fixed operation.

○Panning Panning operation refers to a camera operation in which the camera head is directed in the left/right (horizontal) direction, as shown in FIGS. 3 and 4(b).

A typical optical flow for an image during a panning operation is shown in FIG. 1(b). The relative magnitude of each velocity vector in the velocity vector distribution of the optical flow is equal, and the direction is oriented in a fixed direction (right or left). The position of the convergence point or the position of the divergence point in the velocity vector distribution is infinite.

From the above, in optical flow, the relative magnitude of each velocity vector is equal, the direction is constant (right or left), and the convergence point position or divergence point position of the velocity vector in the velocity vector distribution is at infinity on the left and right. If this occurs, it is seen only during panning operations, and can be considered to be an optical flow unique to panning operations. Therefore, from the calculated optical flow, it can be determined that the operation of the image capturing device is panning.

○Tilt The tilt operation refers to a camera operation in which the camera head is directed up and down (vertically) as shown in FIGS. 3 and 4 (C).

A typical optical flow for an image during a tilt operation is shown in FIG. 1(C). The relative magnitude of each velocity vector in the optical flow velocity vector distribution is equal,
It faces in a certain direction (up or down). The convergence point position or divergence point position of the velocity vector in the velocity vector distribution is at infinity above and below.

From the above, in optical flow, the relative magnitude of each velocity vector is equal, the direction is constant (up or down), and the convergence point position or divergence point position of the velocity vector in the velocity vector distribution is infinite. is seen only during tilt operation, and can be considered to be an optical flow unique to tilt operation. therefore,
From the calculated optical flow, it can be determined that the operation of the image capturing device is a tilt.

○Dolly Dolly operation refers to a camera operation that moves the camera 1 in two directions (back and forth directions) of the subject, as shown in FIGS. 3 and 4(d). A typical optical flow for an image during a dolly operation is shown in FIG. 1(d). The relative magnitude of each velocity vector in the velocity vector distribution of the optical flow is different; the nearby subject 2 has a fast vector velocity (the distant subject 2 has a slow vector velocity). facing outward from the screen.

The position of the convergence point or the position of the divergence point of the velocity vectors in the velocity vector distribution is at the center of the screen.

From the above, in optical flow, the relative size of each velocity vector is different, the direction is toward the screen center or from the center to the outside of the screen, and the convergence point position or divergence point position of the velocity vector in the velocity vector distribution is with respect to the screen center. If this is the case, it is seen only during dolly operation, and can be considered to be an optical flow unique to dolly operation.

Therefore, from the calculated optical flow, it can be determined that the operation of the image capturing device is dolly.

○Track Track operation refers to a camera operation in which the camera 1 is moved left and right (laterally) with respect to the subject 2, as shown in FIGS. 3 and 4(e).

A typical optical flow for an image during truck operation is shown in FIG. 1(e). The relative magnitude of each velocity vector in the optical flow velocity vector distribution is different; the nearby subject 2 has a fast vector velocity, and the far subject 2 has a slow vector velocity. The position of the convergence point or divergence point of the velocity vector in the velocity vector distribution is infinite.

From the above, in optical flow, the relative magnitude of each velocity vector is different, there is no regularity (random), the direction is fixed (right or left), and the position of the convergence point or divergence of velocity vectors in the velocity vector distribution When the point position becomes infinite, this is seen only during track operation, and can be considered to be an optical flow unique to track operation. Therefore, from the calculated optical flow, it can be determined that the operation of the image capturing device is a track.

○Booming Booming operation refers to a camera operation in which the camera height is raised or lowered using the camera pedestal to change the camera angle, as shown in FIGS. 3 and 4(f).

A typical optical flow for an image during a booming operation is shown in FIG. 1(f). The relative magnitude of each velocity vector in the velocity vector distribution of the optical flow is different, with the nearby subject 2 having a fast vector velocity and the far subject 2 having a slow vector velocity. The direction is a certain direction (
facing up or down). The position of the convergence point or the position of the divergence point of the velocity vector in the velocity vector distribution is infinite.

From the above, in optical flow, the relative size of each velocity vector is different, there is no regularity (random), the direction is fixed (up or down), and the position of the convergence point of the velocity vector in the velocity vector distribution If the divergence point position is infinite, this is seen only during the booming operation, and can be considered to be an optical flow unique to the booming operation. Therefore, from the calculated optical flow, it can be determined that the operation of the image capturing device is booming.

○Zoom Zoom operation means to continuously change the size of the subject by changing the angle of view of the zoom lens without changing the relative positions of camera 1 and subject 2, as shown in Figures 3 and 4 (g). Refers to the camera operation that changes the

A typical optical flow for an image during a zoom operation is shown in FIG. 1(g). Although the relative magnitude of each velocity vector in the velocity vector distribution of the optical flow is different, vectors located on a concentric circle around the center of the screen have the same relative magnitude and are oriented toward the center of the screen or toward the outside of the screen from the center.

The position of the convergence point or the position of the divergence point of the velocity vectors in the velocity vector distribution is at the center of the screen.

From the above, in optical flow, the relative magnitude of each velocity vector is different, but the vectors located on a concentric circle at the center of the screen have the same relative magnitude and are oriented in a fixed direction (
If the position of the convergence point or divergence point of the velocity vector in the velocity vector distribution is at the center of the screen, this can only be seen during zoom operation, and the optical It can be considered as a flow. Therefore, based on the calculated optical flow, it can be determined that the operation of the image capturing device is zooming.

Figure 2 shows the relationship between the relative magnitude and direction of each velocity vector of the optical flow explained above, the convergence point position or divergence point position of all and some of the velocity vectors in the velocity vector distribution, and the operation of the image input device. show. In any operation, there is no overlap in the relationship between vector magnitudes, directions, and positions of convergence or divergence points.

Generally, the above seven basic operations are combined to configure the operation of the image capturing device. The relative magnitude, vector direction, convergence point position, or divergence point position of the optical flow vector calculated by the combination type changes. For example panning and zoomin
In an operation in which g is combined, the magnitude of the vector is different, the direction is constant (right or left), and the position of the convergence point or divergence point is changed. That is, when an operation with an infinite convergence point or divergence point position is combined with a screen operation, the convergence point or divergence point position of the operation becomes finite outside the screen. As in the example above, when the operation is composed of several basic operations, the constituent basic operations can be separated from the relative magnitude, direction, and position of the convergence or divergence point of the optical flow vector. It can be confirmed.

[Effects of the Invention] As explained above, the present invention can be applied to Since the image transport state of the image capturing device at the time of capturing the screen is detected from the convergence point positions of all and part of each velocity vector in the velocity vector distribution, it is possible to study a more interactive technique for handling moving images. This has the advantage of being able to accurately define the physical features in the scene.

[Brief explanation of drawings]

Figures 1 (a) to (g) are diagrams showing the types of camera operations according to the present invention, and Figure 2 shows the types of camera operations according to the present invention, convergence points or divergence points, vector magnitudes, and vector directions. FIG. 3 is a diagram showing the relationship, FIG. 3 is a diagram showing the types of camera operations, and FIGS. 4(a) to (g) are diagrams explaining the types of camera operations. In the figure, 1 is a camera and 2 is a subject. Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure figure (e) truck and going to the track (0) - Mi) group' (9) Swimming part (c) Drying l letter zI photo cup (υha"shini 2ri" (d) de no

Claims (1)

[Claims] The relative magnitude and direction of each velocity vector in the apparent velocity vector distribution on the screen caused by the relative movement between the image capturing device and the subject, all of the velocity vectors in each velocity vector distribution, and 1. A method for detecting an imaging state of an image capturing apparatus, characterized in that the imaging state of the image capturing apparatus at the time of capturing the screen is detected from a part of a convergence point position or a divergence point position.