JP7128695B2 - Shooting operation data collection device and program - Google Patents

Description

The present invention relates to a photographing operation data collecting device and a program.

BACKGROUND ART With the progress of artificial intelligence (AI) technology, the application of machine learning, which attempts to realize a function similar to the learning ability that humans naturally perform in computers, is spreading. As one of such applications, it is being considered to collect data of camera shooting operations (camerawork) by a cameraman and have a computer perform machine learning.

The technology for collecting shooting operation data includes a technology that collects shooting operation data by installing a measurement unit such as a rotary encoder on a pan head that changes the angle of the mounted camera and measuring the angle of the pan head. disclosed (see Patent Document 1).

In addition, while viewing a partial image (clipped image) cut out from the wide-field image, a virtual shooting operation is performed, and the position and size of the image cutout are adjusted according to the detected shooting direction and shooting angle of view of the camera. A technique for generating a changed image has been disclosed (see Patent Document 2).

Japanese Patent No. 5771117 JP 2005-223487 A

However, the technique disclosed in Japanese Patent Application Laid-Open No. 2002-200000 collects data of one shooting operation in which the same subject is shot by the same device in order to measure the actual shooting operation in real time. Therefore, it may be difficult to collect various photographing operation data such as the same subject being photographed by the same cameraman with different photographing operations, or the same subject being photographed by a different cameraman.

In addition, in the technique of Patent Document 2, since the shooting operation is performed while visually recognizing the clipped video on a fixed screen, the operational feeling of the clipping size and position is not related to the movement of the subject in the real space. , the shooting operation may feel uncomfortable. Such discomfort may reduce the quality of the collected shooting operation data.

SUMMARY OF THE INVENTION The present invention has been made in view of the above points, and an object of the present invention is to collect various and high-quality shooting operation data.

A photographing operation data collection device according to an aspect of the disclosed technology is a photographing operation data collection device that collects photographing operation data associated with a photographing operation of a camera, and includes attitude angle and zoom operation operations in accordance with the photographing operation. a movable part in which at least one of the amounts changes; a measurement part that measures an operation amount of a first shooting operation with respect to the movable part; and an image processing unit that generates a first video by cutting out a part of the cut-out partial video and geometrically transforming the cut-out partial video, and the progress from the start of input of the first video or the input video A video synchronized with the first video based on time, and in another shooting operation data collecting device different from the shooting operation data collecting device, a second shooting operation different from the first shooting operation is performed. a display unit for displaying a second image generated by cutting out a part of the input image according to the operation amount and performing image processing for geometrically transforming the cut out partial image; a switching unit for switching the image displayed on the unit to either the first image or the second image; an operation amount of the first shooting operation measured by the measurement unit; and the input image. and a recording unit that records at least one of the first images in association with the elapsed time.

According to the embodiment of the present invention, it is possible to collect various and high-quality shooting operation data.

1 is a diagram illustrating an example configuration of a photographing operation data collection system according to a first embodiment; FIG. It is a block diagram showing an example of the configuration of a measurement unit according to the first embodiment. 3 is a diagram showing an example of a configuration of a processing unit according to the first embodiment using functional blocks; FIG. FIG. 4 is a diagram for explaining a spatial relationship between an input image and a clipped image according to the embodiment; FIG. 4 is a diagram for explaining the temporal relationship between an input video and a clipped video according to the embodiment; 4 is a flowchart showing an example of processing according to the first embodiment; FIG. 10 is a diagram illustrating an example of a configuration of a photographing operation data collection device according to a second embodiment; FIG. FIG. 10 is a diagram showing an example of the configuration of a video storage unit according to the second embodiment in terms of functional blocks; It is a figure which shows an example of a structure of the process part which concerns on 2nd Embodiment with a functional block. 9 is a flowchart showing an example of processing according to the second embodiment; FIG. 12 is a diagram illustrating an example of a configuration of a photographing operation data collection device according to a third embodiment; FIG. It is a figure which shows an example of a structure of the operation amount memory|storage part which concerns on 3rd Embodiment with a functional block. FIG. 11 is a diagram illustrating an example of shooting operation data stored in an operation amount storage unit according to the third embodiment; FIG. It is a figure which shows an example of a structure of the process part which concerns on 3rd Embodiment with a functional block. FIG. 11 is a flow chart showing an example of processing according to the third embodiment; FIG. FIG. 14 is a diagram illustrating an example of a configuration of a photographing operation data collection device according to a fourth embodiment; FIG. FIG. 12 is a block diagram showing an example of the configuration of a measurement unit according to the fourth embodiment; FIG.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In each drawing, the same components are denoted by the same reference numerals, and redundant description may be omitted.

The shooting operation data collection device according to the embodiment is a device that collects operation amount data (shooting operation data) such as a shooting posture angle and a zoom according to a shooting operation (camerawork) of a camera. Moreover, it is a device that collects shooting operation data by a virtual shooting operation simulating camera shooting without using a real camera.

[First embodiment]
In the first embodiment, an example of a photographing operation data collection system including two photographing operation data collection apparatuses according to the embodiment will be described.

<System configuration>
First, the overall configuration of the photographing operation data collection system according to this embodiment will be described with reference to FIG. FIG. 1 is a diagram illustrating an example of the configuration of a shooting operation data collection system 1 according to this embodiment. The photographing operation data collection system 1 has a photographing operation data collecting device 10a and a photographing operation data collecting device 10b.

The photographing operation data collecting device 10a has a measuring section 11a, a processing section 12a, a display section 13a, a recording section 14a, and a switching section 15a. It should be noted that each of these units is an example of "each functional unit". A display unit 13a is mounted on a platform 17a fixed to a tripod 16a. The camera platform 17a is provided with a measuring section 11a, a processing section 12a, a recording section 14a, and the like.

The pan head 17a is a mechanical member whose attitude angle changes according to the shooting operation. Conventionally, a pan head is used to change the attitude angle of a mounted camera. It is used to change the posture angle of the display unit 13a placed on 17a. The camera platform 17a also includes a focus demand 18a and a zoom demand 19a.

The focus demand 18a is an input device for giving a rotational angular velocity command to a servomotor that rotates the focus ring of the zoom lens (photographing lens). The zoom demand 19a is an input device for giving a rotational angular velocity command to a servomotor for rotating the zoom ring of the zoom lens. In this embodiment, since a real camera and zoom lens are not used, the focus demand 18a and the zoom demand 19a are used to extract angular velocity command signals from simulated focus and zoom operations.

Further, the focus demand 18a and the zoom demand 19a have a function as an operation bar for changing the attitude of the platform 17a and the attitude angle of the display section 13a.

The operator 2a can perform a photographing operation via the focus demand 18a and the zoom demand 19a. The shooting operation includes an operation of changing the attitude of the camera platform 17a using the focus demand 18a and the zoom demand 19a as an operation bar, a focus operation using the focus demand 18a, and a zoom operation using the zoom demand 19a. In addition, below, the posture angle of the pan head 17a is referred to as the photographing posture angle for the sake of convenience. The camera platform 17a is an example of a "movable part", and the angular velocity command signal extracted by the zoom demand 19a is an example of a "zoom operation amount".

The measurement unit 11a measures shooting operation data Da by the operator 2a. The photographing operation data Da is data indicating the photographing posture angle, zoom, focus, and the like. The shooting attitude angles are the pan axis rotation angle in the pan direction indicated by arrow 131a in FIG. 1, the tilt axis rotation angle in the tilt direction indicated by arrow 132a, and the roll direction indicated by arrow 133a. shaft rotation angle and the like. The measurement unit 11a outputs the measured shooting operation data Da regarding the shooting posture angle and the like to the processing unit 12a and the recording unit 14a.

The photographing operation by the operator 2a is an example of the "first photographing operation", and the photographing operation data Da by the operator 2a is an example of the "operation amount of the first photographing operation".

The processing unit 12a receives the shooting operation data Da from the measuring unit 11a, and performs image processing on the input image B according to the shooting operation data Da. Then, the clipped image Ca, which is an image after image processing, is output to the display unit 13a. The clipped image Ca is an image obtained by changing the photographing attitude angle, the zoom, etc. in accordance with the photographing operation performed by the operator 2a with respect to the input image.

The display unit 13a displays either the input clipped image Ca or a clipped image Cb, which will be described later. As the display unit 13a, a display such as a liquid crystal display (LCD), an organic light emitting diode (OLED), a plasma display panel (PDP), or a cathode ray tube (CRT). device can be used.

The clipped image Ca and the clipped image Cb displayed on the display unit 13a simulate images captured by a camera. The operator 2a can visually recognize the image displayed on the display unit 13a and perform a virtual photographing operation as if he were visually recognizing the image photographed by a real camera.

In this embodiment, an example in which the display unit 13a is placed on the camera platform 17a and the posture angle of the camera platform 17a is changed to change the posture angle of the display unit 13a has been described, but the present invention is not limited to this. The display unit 13a may not be placed on the pan head 17a, and the posture angle of the display unit 13a itself may not change even if the posture angle of the pan head 17a changes.

The recording unit 14a records the shooting operation data Da measured by the measuring unit 11a moment by moment. An HDD (Hard Disk Drive) or the like can be used as the recording unit 14a.

The recording unit 14a may record time data together with the shooting operation data Da. The time data is, for example, the elapsed time from the start of input of the input video when the shooting operation data Da was acquired. The recording unit 14a may record such time data based on a calendar time such as Coordinated Universal Time or Japan Standard Time, or it may be synchronized with the video frame or video field of the input video B like a time code. may be recorded based on timestamps counted by

The recording unit 14a may record at least one of the input image B and the clipped image Ca together with the shooting operation data Da. In FIG. 1, a broken line 141a indicates a signal line through which the recording unit 14a inputs and records the input image, and a broken line 142a indicates a signal line through which the recording unit 14a inputs and records the clipped image Ca.

When the recording unit 14a records at least one of the input image B and the clipped image Ca, it is preferable to record these images in a format in which there is time correspondence between these images and the shooting operation data Da. The format with time correspondence is a format in which the shooting operation data Da and the video are synchronized by a time code.

The switching unit 15a switches the video displayed on the display unit 13a to either the clipped video Ca generated by the shooting operation data collecting device 10a or the clipped video Cb generated by the shooting operation data collecting device 10b. It has a switching function.

The switching unit 15a has a switching switch 151a and a switching electric circuit 152a. As the changeover switch 151a, a push button provided on the zoom demand 19a of the photographing operation data collection device 10a can be used. The switching electric circuit 152a is provided in the platform 17a and electrically connected to the switching switch 151a.

When the push button of the changeover switch 151a is pressed, the changeover electric circuit 152a operates to switch the video signal line. Then, the image displayed on the display unit 13a is switched from the clipped image Ca generated by the photographing operation data collection device 10a to the clipped image Cb generated by the photographing operation data collection device 10b.

FIG. 1 shows an example in which the terminal 153a of the switching electric circuit 152a is connected to the processing section 12a, but when the push button of the changeover switch 151a is pressed, the terminal 153a is switched to be connected to the processing section 12b.

When an operator 2b different from the operator 2a performs a photographing operation with the photographing operation data collection device 10b, the operator 2a operates the changeover switch 151a so that the image displayed on the display unit 13a is displayed by the operator 2b. It is possible to switch to the image by operation. If the operator 2b is a veteran cameraman, the operator 2a can perform the photographing operation while referring to the photographing operation of the veteran cameraman. Alternatively, while comparing the clipped image Ca and the clipped image Cb, the operator 2b can intentionally perform a photographing operation different from that of the operator 2b.

As the switch 151a, a return switch used for photographing television broadcasts may be used. A return switch is provided in a zoom demand of a broadcasting camera or the like, and is a switch that switches the image displayed on the camera monitor to either the captured image monitor or the image after distribution to the television receiver. By using such a return switch as the changeover switch 151a, the photographing operation data collection device 10a can realize the function of the above changeover switch 151a without modifying equipment such as a platform for camera photography. can.

The photographing operation data collecting device 10b has a measuring section 11b, a processing section 12b, a display section 13b, a recording section 14b, and a switching section 15b. A display unit 13b is mounted on a platform 17b fixed to a tripod 16b. The camera platform 17b is provided with a measuring section 11b, a processing section 12b, a recording section 14b, and the like.

The processing unit 12b receives the photographing operation data Db by the operator 2b from the measuring unit 11b, and performs image processing on the input image based on the photographing operation data Db, using image extraction and coordinate conversion as will be described later. to run. Then, the clipped image Cb, which is the image after the image processing, is output to the display section 13b. The input image input to the processing unit 12b is the same image as the input image input to the processing unit 12a of the photographing operation data collection device 10a.

Here, since the clipped image Cb is an image in which the shooting posture angle and the shooting angle of view are changed according to the shooting operation performed by the operator 2b, the shooting operation data collection device 10a and the shooting operation data When the same input image is used in the collection device 10b, the clipped image Cb is synchronized with the clipped image Ca based on the elapsed time from the start of input of the input image. Therefore, the operator 2a and the operator 2b can perform shooting operations on the same input image at the same timing.

The functions of the other parts of the photographing operation data collecting device 10b than those described above are the same as those of the photographing operation data collecting device 10a, and thus descriptions thereof will be omitted.

The photographing operation by the operator 2b is an example of the "second photographing operation", and the photographing operation data Db is an example of the "operation amount of the second photographing operation". The shooting operation data collection device 10b is an example of "another shooting operation data collection device".

In this manner, the photographing operation data collection system 1 can collect photographing operation data when different photographers photograph the same subject with different photographing operations, and collect various kinds of photographing operation data. Here, the clipped image Ca is an example of the "first image", and the clipped image Cb is an example of the "second image".

In the above-described embodiment, the recording unit 14a and the recording unit 14b are provided separately. By recording the shooting operation data from the operation data collection device in one recording unit, it is possible to easily manage the collected shooting operation data.

Further, in the above-described embodiment, an example in which the processing unit 12a and the recording unit 14a are provided in the camera platform 17a has been shown, but these components are provided separately from the camera platform 17a, and the processing unit 12a, the measurement unit 11a, and the processing unit 12a and the display unit 13a may be electrically connected by wire or wirelessly.

Furthermore, in the above-described embodiment, an example in which the changeover switch 151a is provided in the zoom demand 19a has been shown. If it is provided in the zoom demand 19a or the focus demand 18a, the changeover switch 151a can be operated while the photographing operation is being performed, and the image switching operation can be facilitated.

Further, the place where the switching electric circuit 152a is installed is not limited to the pan head 17a, and may be the zoom demand 19a, the display unit 13a, or the like.

<Configuration of measurement unit>
FIG. 2 is a block diagram showing an example of the configuration of the measuring section 11a according to this embodiment. In addition, although the measuring unit 11a is described as an example in FIG. 2, the same applies to the measuring unit 11b.

The measurement unit 11a has a rotation angle detection unit 111a and a zoom operation amount detection unit 112a. The rotation angle detection unit 111a has a pan direction rotary encoder 113a, a tilt direction rotary encoder 114a, and a roll direction rotary encoder 115a.

The pan direction rotary encoder 113a is provided in a rotation mechanism of the pan head 17a around the pan axis (see the pan direction 131a in FIG. 1), and detects the rotation angle of the pan head 17a around the pan axis. Data indicating the detected rotation angle of the camera platform 17a around the pan axis is output as the rotation angle P to the processing unit 12a and the recording unit 14a.

The tilt direction rotary encoder 114a is provided in a rotation mechanism portion of the pan head 17a around the tilt axis (see the tilt direction 132a in FIG. 1), and detects the rotation angle of the pan head 17a around the tilt axis. Data indicating the detected rotation angle of the camera platform 17a about the tilt axis is output as the rotation angle T to the processing unit 12a and the recording unit 14a.

The roll direction rotary encoder 115a is provided in a rotation mechanism portion of the camera platform 17a around the roll axis (see the roll direction 133a in FIG. 1), and detects the rotation angle of the camera platform 17a around the roll axis. Data indicating the detected rotation angle of the camera platform 17a about the roll axis is output as the rotation angle R to the processing unit 12a and the recording unit 14a.

The rotation axis in the pan direction, the rotation axis in the tilt direction, and the rotation axis in the roll direction are orthogonal to each other. The rotation angle in the pan direction is, in other words, the azimuth angle or yaw angle, and the rotation angle in the tilt direction is, in other words, the elevation angle or pitch angle.

As a rotary encoder, an incremental type or an absolute type may be used. The incremental type outputs a two-phase pulse signal, and the absolute type outputs a code signal. Each of them is used as data indicating the rotation angle.

The zoom operation amount detection unit 112a measures the zoom operation amount by the zoom demand 19a, and outputs a zoom amount Z indicating the zoom operation amount to the processing unit 12a and the recording unit 14a. The zoom operation amount detection unit 112a outputs the angular velocity command value from the zoom demand 19a as the zoom amount Z. FIG. Based on the zoom amount Z, the processing unit 12a calculates a focal length or a shooting angle of view corresponding to the zoom.

The zoom operation amount detection unit 112a has a calculation function by a CPU or the like. The zoom operation amount detection unit 112a calculates data indicating the focal length or the shooting angle of view based on the angular velocity command value, and determines the zoom amount Z from the calculated data. can be converted to and output.

Further, the zoom operation amount detection unit 112a has a calculation function by a CPU or the like, and calculates the depth distance and the like to the in-focus position based on the angular velocity command value input by the zoom operation amount detection unit 112a from the focus demand 18a. The data may be converted into a zoom amount Z and output.

Further, the measurement unit 11a detects markers and transmitters placed around the installation location of the shooting operation data collection device 10a, or natural objects existing in the surroundings, by means of cameras and receivers installed on the camera platform 17a. It is also possible to measure the attitude angle and the shooting angle of view.

<Structure of Processing Unit>
FIG. 3 is a diagram showing an example of the configuration of the processing unit 12a (computer) according to this embodiment in conceptual functional blocks. All or part of each functional block can be functionally or physically distributed and combined in arbitrary units.

Each functional block of the processing unit 12a is implemented by a CPU (Central Processing Unit) executing a program stored in a ROM (Read Only Memory) using a RAM (Random Access Memory) as a work area. be. Part or all of the processing performed by the CPU may be realized by hardware based on wired logic such as FPGA (Field-Programmable Gate Array) or ASIC (Application Specific Integrated Circuit).

The processing unit 12a includes a shooting attitude angle acquisition unit 121a, a zoom data acquisition unit 122a, and an image processing unit 123a.

The photographing posture angle acquisition unit 121a is electrically connected to the measurement unit 11a, and inputs the rotation angle P, the rotation angle T, and the rotation angle R detected by the rotation angle detection unit 111a. The photographing posture angle acquisition unit 121a calculates a pan direction posture angle based on the rotation angle P, a tilt direction posture angle based on the rotation angle T, and a roll direction posture angle based on the rotation angle R. Then, these are output to the image processing unit 123a.

The shooting attitude angle acquisition unit 121a multiplies the two-phase pulse signal of the rotation angle P by a coefficient (degrees/pulse) and converts it into angle information to calculate the panning direction attitude angle. Similarly, the tilt direction posture angle is calculated by multiplying the two-phase pulse signal of the rotation angle T by a coefficient (degrees/pulse) and converting it into angle information, and the two-phase pulse signal of the rotation angle R is converted to the coefficient (degrees/pulse). is converted into angle information to calculate the roll direction attitude angle.

The zoom data acquisition unit 122a is electrically connected to the measurement unit 11a, and inputs the zoom amount Z detected by the zoom operation amount detection unit 112a. The zoom data acquisition unit 122a calculates the focal length based on the zoom amount Z, and outputs it to the image processing unit 123a.

The zoom data acquisition unit 122a inputs an angular velocity command value as the zoom amount Z, and executes time integration calculation. Also, the zoom data acquisition unit 122a converts the result of the integral calculation into a focal length by referring to a model of zoom ring rotation angle versus focal length or rotation angle versus angle of view. The model of the zoom ring rotation angle versus focal length is a function, lookup table, or the like that indicates the relationship between the zoom ring rotation angle versus focal length.

Note that the zoom data acquisition unit 122a may calculate the shooting angle of view instead of the focal length and output it to the image processing unit 123a. In this case, the zoom data acquiring unit 122a refers to a model of zoom ring rotation angle vs. shooting angle of view, and converts the result of the integral calculation into the shooting angle of view. The model of the zoom ring rotation angle versus the shooting angle of view is a function, lookup table, or the like that indicates the relationship between the zoom ring rotation angle versus the shooting angle of view.

The image processing unit 123a receives the shooting posture angle from the shooting posture angle acquisition unit 121a and inputs the focal length from the zoom data acquisition unit 122a. The image processing unit 123a performs geometric transformation and cropped image processing on the input image B based on the shooting attitude angle and the focal length, and outputs the cropped image Ca, which is the image after the processing, to the switching unit 15a.

As described above, the clipped image Ca is an image displayed by simulating the image captured by the camera when the shooting operation is being performed, and is an image that is generated by being clipped from the input image B. Therefore, it is preferable that the input image B has a wider field of view and higher definition than the clipped image Ca so that the shooting operation can be performed in an operation range as wide as possible. The input video B preferably includes the entire range from which the clipped video Ca can be clipped by a virtual shooting operation. In addition, it is preferable that the maximum distance between adjacent pixels of the input image B is not larger than the minimum distance between corresponding points on the input image B corresponding to the pixels forming the clipped image Ca.

The image processing unit 123a outputs the clipped image Ca to the display unit 13a via the switching unit 15a, and outputs the clipped image Ca to the display unit 13b via the switching unit 15b. The clipped image Ca output from the image processing unit 123a is input to the display unit 13a according to a switching signal from the changeover switch 151a of the switching unit 15a. Further, it is input to the display section 13b according to a switching signal from the changeover switch 151b of the switching section 15b. The display section 13a and the display section 13b display the input clipped image Ca.

Note that the processing unit 12a may include a switching processing unit that realizes the functions of the switching electric circuit 152a by software. In this case, the switching processing unit receives a switching signal from the switch 151a, and executes processing by software to switch the output destination of the clipped image Ca to the display unit 13a or the display unit 13b according to the switching signal.

The function of the processing unit 12a provided in the photographing operation data collecting device 10a is the same as that of the processing unit 12b provided in the photographing operation data collecting device 10b, so the description of the processing unit 12b is omitted.

Here, FIG. 4 is a diagram for explaining the spatial relationship between the input image and the clipped image.

In FIG. 4, the coordinates on the image plane Q in the image frame of the input image B are (X, Y), and the pixel value at the coordinates (X, Y) is Ib(X, Y). The X-axis and the Y-axis are orthogonal in the plane of the image plane Q. FIG. The optical axis Sb of the optical system that captures the input image B extends from the optical principal point O with the subject direction as the positive direction, and is orthogonal to the image plane Q at the coordinates (0, 0) on the image plane Q. there is The X-axis, Y-axis, and optical axis Sb form a right-handed system in this order, and the coordinate system is called a real camera coordinate system Σ(G).

On the other hand, let (x, y) be the coordinates on the image plane W in the image frame of the clipped image Ca obtained by executing image processing on the input image B according to the shooting operation data Da, and the coordinates (x, y) is Ica(x, y). The x-axis and the y-axis are orthogonal in the plane of the image plane W. FIG. The optical axis Sca of the optical system that virtually captures the clipped image Ca extends from the optical principal point O with the direction in which the subject is present as the positive direction, and is located at the coordinates (0, 0) on the image plane W. is orthogonal to The x-axis, y-axis, and optical axis Sca form a right-handed system in this order, and the coordinate system is called a virtual camera coordinate system Σ(V).

The origin of the real camera coordinate system Σ(G) and the origin of the virtual camera coordinate system Σ(V) are the same as the optical principal point O. Let F be the focal length of the optical system that captures the input image. The shooting operation data Da includes the pan direction attitude angle α, the tilt direction attitude angle δ, the roll direction attitude angle φ, and the focal length f of the optical system that captured the clipped image Ca. δ, φ, f).

The position vector r(G) in the real camera coordinate system Σ(G) is observed from the virtual camera coordinate system Σ(V) in the state of the shooting operation data Da(α, δ, φ, f). Let r(V) be The relationship between r(G) and r(V) is represented by the following equations (1) and (2).

また画像平面Ｗ上の座標（ｘ、ｙ）に対応する画像平面Ｑ上の座標（Ｘ（ｘ、ｙ、α、δ、φ）、Ｙ（ｘ、ｙ、α、δ、φ））は、次の（３）式で表すことができる。

The coordinates (X (x, y, α, δ, φ), Y (x, y, α, δ, φ)) on the image plane Q corresponding to the coordinates (x, y) on the image plane W are It can be expressed by the following formula (3).

（３）式のＸ（ｘ、ｙ、α、δ、φ）は（４）式で表すことができ、（３）式のＹ（ｘ、ｙ、α、δ、φ）は（５）式で表すことができる。

X (x, y, α, δ, φ) in formula (3) can be expressed by formula (4), and Y (x, y, α, δ, φ) in formula (3) can be expressed by formula (5) can be expressed as

以上から、画像処理部１２３ａは、入力映像の入力開始時からの経過時間ｔにおいて出力すべき切出映像Ｃａの画像フレームにおける画素値Ｉｃａ（ｘ、ｙ）は、次の（６）式で決定することができる。

From the above, the image processing unit 123a determines the pixel value Ica(x, y) in the image frame of the clipped image Ca to be output at the elapsed time t from the start of inputting the input image by the following equation (6). can do.

画像処理部１２３ａは切出映像Ｃａを構成する全画素の画素値Ｉｃａ（ｘ、ｙ）を、（６）式により算出することで、切出映像Ｃａを生成し、出力することができる。

The image processing unit 123a can generate and output the clipped video Ca by calculating the pixel values Ica(x, y) of all the pixels forming the clipped video Ca according to Equation (6).

Note that the functions of the image processing unit 123a realized by the formulas (3) to (5) are equivalent to homography conversion and image clipping.

Also, due to the pixel structure and sampling of the image, on the image plane Q, the effective If the pixel does not exist, the pixel value of the existing pixel closest to the coordinates (X (x, y, α, δ, φ), Y (x, y, α, δ, φ)) can be interpolated. good. Such interpolation is so-called nearest neighbor interpolation. Bilinear interpolation, bicubic A value obtained by performing interpolation processing such as interpolation, Lanczos interpolation, or spline interpolation may be used.

The coordinates (X, Y) on the image plane Q, the coordinates (x, y) on the image plane W, the focal length F, and the focal length f are all expressed by a common scale (unit of length). shall be As such a scale, the unit may be the pixel interval (the length of one side of the pixel arrangement grid, the length of the diagonal, etc.), or the unit may be some absolute scale (meters, millimeters, etc.). .

FIG. 5 is a diagram for explaining the temporal relationship between an input video and a clipped video. In FIG. 5, the horizontal axis represents time. Time t0 indicates the input start time (time) of input video B, and times t1, t2, and t3 indicate elapsed times from the input start of input video B, respectively.

Input image B at time t0 is input image B(t0), input image B at time t1 is input image B(t1), input image B at time t2 is input image B(t2), and input image B at time t3 is Each image is displayed as an input image B(t3).

A clipped image Ca indicates a clipped image generated based on the input image B according to the shooting operation data Da by the operator 2a. A clipped video based on the input video B(t0) is indicated as a clipped video Ca(t0), and a clipped video based on the input video B(t1) is indicated as a clipped video Ca(t1). A cutout image based on the input image B(t2) is indicated as a cutout image Ca(t2), and a cutout image based on the input image B(t3) is indicated as a cutout image Ca(t3).

A clipped image Cb represents a clipped image generated based on the input image B according to the photographing operation data Db by the operator 2b. A clipped video based on the input video B(t0) is indicated as a clipped video Cb(t0), and a clipped video based on the input video B(t1) is indicated as a clipped video Cb(t1). A clipped video based on the input video B(t2) is indicated as a clipped video Cb(t2), and a clipped video based on the input video B(t3) is indicated as a clipped video Cb(t3).

The clipped image Ca is displayed smaller than the input image B because it is an image generated by clipping a part of the input image B according to the photographing operation performed by the operator 2a. Similarly, the cropped image Cb is displayed smaller than the input image B because it is generated by cropping a portion of the input image B according to the shooting operation performed by the operator 2b.

As shown in FIG. 5, the cut-out video Ca and the cut-out video Cb start at the same time as the input start of the input video B, and every elapsed time from the input start of the input video B. The timing at which the video is generated matches. In other words, the clipped image Ca and the clipped image Cb are synchronized based on the elapsed time from the input of the input image B starting.

Also, the clipped image Ca and the clipped image Cb are generated based on the common input image B. FIG. Both the clipped video Ca(t0) and the clipped video Cb(t0) are generated based on the input video B(t0), and both the clipped video Ca(t1) and the clipped video Cb(t1) are generated based on the input video B (t1). Both the clipped video Ca(t2) and the clipped video Cb(t2) are generated based on the input video B(t2), and both the clipped video Ca(t3) and the clipped video Cb(t3) are generated based on the input video. It is generated based on B(t3).

When the frame period of input video B is 30 (frame per second; fps), t0 = 0 (second; s), t1 = 1/30 (s), t2 = 2/30 (s), t3 = 3/30 (s). The clipped images Ca and Cb are also generated and displayed at a frame cycle of 30 (fps). Display of at least one of the input image B and the clipped images Ca and Cb may be electrically delayed in order to match the timing of displaying the images. Since the image processing executed by the image processing unit 123a is required to be fast, the image processing unit 123a is preferably configured by hardware or the like based on wired logic.

The clipped image Ca and the clipped image Cb generated in this manner are switched by the switching unit 15a and displayed on the display unit 13a in the photographing operation data collection device 10a. Also, in the photographing operation data collection device 10b, it is switched by the switching unit 15b and displayed on the display unit 13b.

<Operation of shooting operation data collection device>
FIG. 6 is a flowchart showing an example of processing according to this embodiment. In FIG. 6, the processing of the photographing operation data collection device 10a will be described as an example, but the processing of the photographing operation data collection device 10b is the same.

First, in step S61, the photographing posture angle acquisition unit 121a receives the rotation angle P, the rotation angle T, and the rotation angle R output by the measurement unit 11a, and based on these, obtains the pan direction posture angle, the tilt direction posture angle, and the roll direction. A direction/posture angle is acquired and output to the image processing unit 123a.

Subsequently, in step S62, the zoom data acquisition unit 122a receives the zoom amount Z output by the measurement unit 11a, acquires zoom data such as the focal length based on the zoom amount Z, and outputs the zoom data to the image processing unit 123a. .

Subsequently, in step S63, the image processing unit 123a performs geometric transformation and cropped image processing on the input image B based on the input photographing attitude angle and zoom data to generate a cropped image Ca. The generated clipped image Ca is output to the switching unit 15a.

Subsequently, in step S64, the switching unit 15a switches the video displayed on the display unit 13a according to the switching signal from the changeover switch 151a.

Subsequently, in step S65, if the switching signal is not a signal that enables (ON) video switching (step S64, No), the switching electric circuit 152a provided in the switching section 15a does not operate, and the display section 13a is switched off. Display the image Ca.

On the other hand, if the switching signal is a signal that enables (ON) video switching (step S64, Yes), the switching electric circuit 152a provided in the switching section 15a operates. The connection is switched so that the clipped video Cb output by the image processing unit 123b is displayed on the display unit 13a. In step S66, the display unit 13a displays the clipped image Cb.

In this manner, the photographing operation data collection device 10a switches the image displayed on the display unit 13a by the switching unit 15a to the clipped image Ca generated by the image processing unit 123a or the clipped image Cb output by the image processing unit 123b. You can switch to one or the other.

Note that the processing unit 12a may include a determination unit that determines start or end of collection of shooting operation data. The image processing unit 123a starts generating the clipped image Ca based on the input image B based on the start determination by the determination unit. Also, the display unit 13a starts displaying the clipped image Ca, and the recording unit 14a starts recording the photographing operation data.

The determination unit determines to start collecting the shooting operation data based on the operation signal of the start switch provided on the camera platform 17a or the like. Alternatively, it may be determined to start collecting the shooting operation data by detecting that the signal output from the measurement unit 11a in response to the shooting operation or the input image B is input to the processing unit 12a.

On the other hand, the image processing unit 123a terminates the generation of the clipped image Ca based on the input image B due to the end determination by the determination unit. Also, the display unit 13a finishes displaying the clipped image Ca, and the recording unit 14a finishes recording the photographing operation data.

The judging unit judges the end of collecting the photographing operation data based on the operation signal of the end switch provided on the camera platform 17a or the like. Alternatively, it may be determined to end the collection of the shooting operation data by detecting that the signal output from the measurement unit 11a in response to the shooting operation or the input image B is not input to the processing unit 12a for a predetermined time or longer. By providing such a determination unit, it is possible to prevent recording more shooting operation data than necessary in the recording unit 14a.

As described above, in the present embodiment, geometric transformation and clipped image processing are performed on the input video in conjunction with the operator's shooting operation, and the generated clipped video is displayed on the display unit. As a result, it is possible to prevent a sense of incompatibility in the photographing operation due to the operational feeling of the size and position of the clipping being unrelated to the movement of the subject in the real space. Then, it is possible to perform a virtual photographing operation with an operational feeling as if the photographing operation were performed with a real camera, and to collect high-quality photographing operation data accurately simulating an actual photographing environment.

In this embodiment, a plurality of photographing operation data collection devices are provided, and a plurality of persons can perform photographing operations using the same input image B, and collect the photographing operation data.

Further, shooting operation data can be recorded for shooting operations repeatedly performed on the same input image B. FIG. Even when the subject is a natural object (including humans and animals), the situation of the same subject such as a human performance can be repeatedly reproduced, and photographing operation data can be collected by different photographing operations for each reproduction.

Further, the switching unit 15a causes the display unit 13a to display the clipped image Cb by the photographing operation of another operator 2b, so that the operator 2a can refer to or compare the photographing operation of the other operator 2b. Shooting operation data can be collected by performing a shooting operation while holding the camera.

As a result, various shooting operation data can be collected and the shooting operation data can be enriched.

In this embodiment, the pan head 17a whose posture angle changes according to the shooting operation is provided with the measurement unit 11a to measure the posture angle of the pan head 17a. By measuring the attitude angle of the camera platform 17a, it is possible to simulate the photographing attitude angle corresponding to the photographing operation and collect the photographing operation data.

In this embodiment, the display unit 13a is placed on the camera platform 17a, and the posture angle of the display unit 13a is changed in accordance with the posture angle of the camera platform 17a, so that the display unit 13a behaves like a viewfinder of a camera. . As a result, it is possible to reduce the sense of incompatibility felt by the operator and improve the quality of the photographing operation data.

In the present embodiment, a centrally projected clipped image Ca is generated from an input image B, which is a photographed image by central projection, or an input image B of computer graphics, by homography transformation. As a result, it is possible to reproduce the projection distortion when the photographing operation is performed by an ideal camera having a flat imaging device and a photographing lens conforming to the pinhole model, and to generate a natural clipped image. In addition, the visual discomfort felt by the operator can be reduced, and the quality of the shooting operation data can be improved.

As described above, according to the present embodiment, it is possible to collect diverse and high-quality shooting operation data.

[Second embodiment]
Next, an example of a photographing operation data collection device according to the second embodiment will be described. Note that the description of the same components as those of the already described embodiment will be omitted.

The shooting operation data collection device 10c according to the present embodiment includes a video storage unit 20 that stores the clipped video Cb, and stores the video displayed by the display unit 13c as the clipped video Ca generated according to the shooting operation, or It is a device that can switch to any one of the clipped images Cb stored in the image storage unit 20 .

<Configuration of shooting operation data collection device>
The overall configuration of the photographing operation data collection device according to this embodiment will be described with reference to FIG. FIG. 7 is a diagram illustrating an example of the configuration of the shooting operation data collection device 10c according to this embodiment.

The shooting operation data collection device 10c has a video storage unit 20 and a switching unit 15c. The image storage unit 20 is provided in the camera platform 17c. However, the image storage unit 20 may be provided separately from the platform 17c, and the image storage unit 20 and the display unit 13c may be electrically connected by wire or wirelessly.

The video storage unit 20 stores the clipped video Cb. The clipped video Cb is a video generated by performing geometric transformation and clipped image processing on the input video B according to the shooting operation performed by the shooting operation data collection device 10c. The clipped video Cb is a video generated by performing a shooting operation with the shooting operation data collection device 10c at a time past the current time when shooting operation data is to be collected. The video storage unit 20 can output the stored clipped video Cb to the display unit 13c via the switching unit 15c and display it on the display unit 13c.

When the input of the input image B to the processing unit 12c is started, the image storage unit 20 starts outputting the stored clipped image Cb to the switching electric circuit 152c provided in the switching unit 15c in synchronization with this. do. This is continued until the input of the input image B to the processing unit 12c is completed.

The switching unit 15c has a function of switching the image displayed on the display unit 13c to either the clipped image Ca generated by the shooting operation data collection device 10c or the clipped image Cb stored in the image storage unit 20. have.

The switching unit 15c has a switching switch 151c and a switching electric circuit 152c. As the changeover switch 151c, a push button provided on the zoom demand 19c of the photographing operation data collection device 10c can be used. The switching electric circuit 152c is provided in the platform 17c and electrically connected to the switching switch 151c.

When the push button of the switch 151c is pressed, the switching electric circuit 152c is activated to switch the video signal line. Then, the image displayed on the display unit 13c is switched from the clipped image Ca generated by the shooting operation data collection device 10c to the clipped image Cb stored in the image storage unit 20. FIG.

The above "current time" is an example of the "first time", and the above "past time" is an example of the "second time". The clipped image Ca is an example of the "first image", and the clipped image Cb stored in the image storage unit 20 is an example of the "second image". The shooting operation at the current time by the shooting operation data collection device 10c is an example of the "first shooting operation", and the shooting operation at the past time by the shooting operation data gathering device 10c is an example of the "second shooting operation". be.

However, the clipped image Cb stored in the image storage unit 20 is not limited to the image corresponding to the shooting operation performed by the shooting operation data collection device 10c. A clipped image corresponding to a photographing operation performed by another photographing operation data collecting device different from the photographing operation data collecting device 10c may be stored. Also, the operator at the “past time” may be the same as or different from the operator at the “current time”.

Also, the input image B used when the shooting operation is performed at the "current time" and the input image B used when the shooting operation is performed at the "past time" are common.

<Structure of Video Storage Unit>
FIG. 8 is a diagram showing an example of the configuration of the video storage unit 20 according to this embodiment in terms of functional blocks. The image storage unit 20 has an input/output unit 201 , an output image storage unit 202 , a retained image storage unit 203 and an I/F (Inter/Face) unit 204 .

Each processing function performed in each functional block of the video storage unit 20 is realized by the CPU executing a program stored in the ROM using the RAM as a work area. Also, the output image storage unit 202 and the retained image storage unit 203 are realized by an HDD or the like.

The input/output unit 201 inputs a command signal for instructing the image storage unit 20 to start or end outputting the cut image Cb. Further, the input/output unit 201 outputs the clipped image Cb stored in the output image storage unit 202 to the switching unit 15c according to the command signal.

The output image storage unit 202 stores an image to be output to the switching unit 15c among the clipped images Cb stored in the image storage unit 20. FIG. On the other hand, the held video storage unit 203 stores video that is held without being output to the switching unit 15c among the clipped video Cb stored in the video storage unit 20 .

When the clipped video Cb is stored in a moving image file format and different clipped videos Cb are stored as a plurality of moving image files, the output video storage unit 202 selects the switching unit 15c from among the plurality of moving image files. The retained image storage unit 203 stores moving image files to be displayed on the display unit 13c via the video storage unit 203, and stores moving image files that are not output to the switching unit 15c but are retained in the image storage unit 20, among the plurality of moving image files.

As a result, the video storage unit 20 can output only a specific clipped video Cb to the switching unit 15c and display it on the display unit 13c via the switching unit 15c while storing a plurality of different clipped videos Cb. can.

The I/F unit 204 is an interface for connecting with an external device such as a PC (Personal Computer) 205 , and the clipped image Cb stored in the image storage unit 20 can be rewritten, added, or edited via the PC 205 . The clipped video Cb in the video file format can be deleted, and can be rewritten, added, or deleted in the video storage unit 20 . In addition, it is possible to move the clipped image Cb stored in the retained image storage unit 203 to the output image storage unit 202 and change the clipped image Cb output from the image storage unit 20 via the PC 205 . .

The recording unit 14c may have the function of the video storage unit 20 so that data can be easily transferred between the recording unit 14c and the video storage unit 20. FIG.

<Structure of Processing Unit>
FIG. 9 is a diagram showing an example of the configuration of the processing unit 12c according to this embodiment in terms of functional blocks. The processing unit 12c has a determination unit 124c.

The determination unit 124c determines start or end of collection of shooting operation data by the shooting operation data collection device 10c. The image processing unit 123c starts generating the clipped image Ca based on the input image B based on the start determination by the determination unit 124c. Also, the image storage unit 20 starts outputting the clipped image Cb to the switching electric circuit 152c provided in the switching unit 15c. Further, the display unit 13c starts displaying the clipped image Ca or the clipped image Cb, and the recording unit 14c starts recording the photographing operation data Da.

The determination unit 124c detects that the input image B has started to be input to the processing unit 12c, and determines to start collecting the shooting operation data. Alternatively, the start of collection of shooting operation data may be determined based on an operation signal of a start switch provided on the camera platform 17c or the like, or a signal output by the measurement unit 11c in response to shooting operation.

When the determination unit 124 c determines to start collecting the shooting operation data, it outputs a start determination signal indicating the start to the image processing unit 123 c and the video storage unit 20 . At the same time, a start determination signal may be output to the display section 13c and the recording section 14c. According to the start determination signal, the image processing unit 123c starts generating the clipped image Ca, and the video storage unit 20 starts outputting the clipped image Cb to the switching electric circuit 152c. The start determination signal output to the video storage unit 20 corresponds to a command signal instructing the video storage unit 20 to start outputting the clipped video Cb.

On the other hand, the image processing unit 123c terminates the generation of the clipped image Ca, and the image storage unit 20 terminates the output of the clipped image Cb due to the end determination by the determination unit 124c. Also, the display unit 13c finishes displaying the clipped image Ca, and the recording unit 14c finishes recording the photographing operation data.

The determination unit 124c detects that the input image B has not been input to the processing unit 12c for a predetermined time or longer, and determines the end of the shooting operation data collection. Alternatively, the end of the photographing operation data collection may be determined based on an operation signal of an end switch provided on the pan head 17c or the like, or a signal output by the measurement unit 11c in response to the photographing operation. When the determination unit 124c determines that the collection of the shooting operation data has ended, the determination unit 124c outputs a termination determination signal indicating the termination to the image processing unit 123c and the video storage unit 20. FIG. At the same time, an end determination signal may be output to the display section 13c and the recording section 14c. The end determination signal output to the video storage unit 20 corresponds to a command signal that instructs the video storage unit 20 to stop outputting the clipped video Cb.

For simplification, FIG. 7 omits illustration of a signal line from the determination unit 124c to the video storage unit 20 and a signal line from the determination unit 124c to the recording unit 14c.

<Operation of the device according to the second embodiment>
FIG. 10 is a flowchart showing an example of processing according to this embodiment.

First, in step S101, the determination unit 124c detects that the input image B has started to be input to the processing unit 12c, and determines to start collecting shooting operation data.

When collection of shooting operation data has started (step S101, Yes), the determination unit 124c outputs a start determination signal indicating that to the image processing unit 123c and the video storage unit 20. FIG. According to the start determination signal, the image processing unit 123c starts generating the clipped video Ca, and the video storage unit 20 starts outputting the clipped video Cb. On the other hand, if shooting operation data collection is not started (step S101, No), the process returns to step S101 and continues.

Subsequently, in step S102, the shooting posture angle acquisition unit 121c obtains the pan direction posture angle, the tilt direction posture angle, and the roll direction posture angle based on the rotation angle P, the rotation angle T, and the rotation angle R output by the measurement unit 11c. It acquires and outputs a signal indicating these to the image processing unit 123c.

Subsequently, in step S103, the zoom data acquisition unit 122c acquires zoom data such as the focal length based on the zoom amount Z output by the measurement unit 11c, and outputs a signal indicating the zoom data to the image processing unit 123c. .

Subsequently, in step S104, the image processing unit 123c performs geometric transformation and cropped image processing on the input image B based on the input shooting attitude angle and zoom data, and generates a cropped image Ca. The generated clipped image Ca is output to the switching electric circuit 152a provided in the switching section 15c.

Subsequently, in step S105, the switching unit 15c switches the video displayed on the display unit 13c according to the switching signal from the changeover switch 151c.

Subsequently, in step S106, if the switching signal is not a signal to enable (ON) the switching (step S105, No), the switching electric circuit 152c provided in the switching section 15c does not operate, and the display section 13c displays the clipped image Ca. display.

On the other hand, if the switching signal is a signal that enables (ON) switching (step S105, Yes), the switching electric circuit 152c provided in the switching section 15c operates. Then, the connection of the video signal line is switched so that the clipped video Cb output from the video storage unit 20 is displayed on the display unit 13c. In step S107, the display unit 13c displays the clipped image Cb.

Subsequently, in step S108, the determination unit 124c detects that the input image B has not been input to the processing unit 12c for a predetermined time or longer, and determines the end of collection of the shooting operation data. When the photographing operation data collection is to be terminated (step S108, Yes), the determination section 124c outputs a termination determination signal to the image processing section 123c and the video storage section 20 to that effect. According to the termination determination signal, the image processing unit 123c terminates the generation of the cutout image Ca, and the image storage unit 20 terminates the output of the cutout image Cb. On the other hand, if the shooting operation data collection has not ended (step S108, No), the process returns to step S102 and continues.

In this way, the photographing operation data collection device 10c switches the image displayed on the display unit 13c to the clipped image Ca generated by the image processing unit 123c or the clipped image Cb output by the image storage unit 20 by the switching unit 15c. can be switched to either

As described above, in the present embodiment, the clipped video displayed on the display unit is either the clipped video Ca corresponding to the current shooting operation or the clipped video Cb corresponding to the previous shooting operation. switch to one side. As a result, it is possible to perform the shooting operation and collect the shooting operation data while visually recognizing the images of the previous shooting operation performed on the same input image B. FIG. Then, various shooting operation data can be collected and the shooting operation data can be enriched.

Further, in the present embodiment, since the video storage unit 20 stores the clipped video Cb, processing for generating the clipped video Cb is unnecessary. This reduces the processing load and speeds up the processing.

Effects other than those described above are the same as those described in the embodiments already described.

[Third embodiment]
Next, an example of a photographing operation data collection device according to the third embodiment will be described. Note that the description of the same components as those of the already described embodiment will be omitted.

The photographing operation data collection device 10d according to the present embodiment includes the operation amount storage unit 21 that stores the photographing operation data Db. It is a device capable of switching to one of the clipped images Cb according to the photographing operation data Db stored in the operation amount storage unit 21 .

<Configuration of Apparatus According to Third Embodiment>
The overall configuration of the photographing operation data collection device according to this embodiment will be described with reference to FIG. FIG. 11 is a diagram illustrating an example of the configuration of the shooting operation data collection device 10d according to this embodiment.

The shooting operation data collection device 10d has an operation amount storage unit 21 and a switching unit 15d. The manipulated variable storage unit 21 is provided in the pan head 17d. However, the operation amount storage unit 21 may be provided separately from the platform 17d, and the operation amount storage unit 21 and the display unit 13d may be electrically connected by wire or wirelessly.

The operation amount storage unit 21 stores shooting operation data Db from the shooting operation data collection device 10d. The photographing operation data Db, like the photographing operation data Da, is data indicating the photographing attitude angle, zoom, focus, and the like. The photographing operation data Db is the photographing operation data recorded by the photographing operation data collection device 10d at a time in the past with respect to the current time at which the photographing operation data is to be collected. The operation amount storage unit 21 can output the stored photographing operation data Db to the processing unit 12d via the switching unit 15d.

When the input of the input image B to the processing unit 12d is started, the operation amount storage unit 21 outputs the stored photographing operation data Db to the switching electric circuit 152d provided in the switching unit 15d in synchronization with this. Start. This is continued until the input of the input image B to the processing unit 12d is completed.

The switching unit 15d switches the image displayed on the display unit 13d to the clipped image Ca generated according to the current photographing operation data or the past photographing operation data stored in the operation amount storage unit 21. It has a function of switching to one of the clipped images Cb.

The switching unit 15d has a switching switch 151d and a switching electric circuit 152d. As the changeover switch 151d, a push button provided on the zoom demand 19d of the photographing operation data collection device 10d can be used. The switching electric circuit 152d is provided in the platform 17d and electrically connected to the switching switch 151d.

When the push button of the changeover switch 151d is pressed, the changeover electric circuit 152d is activated to switch the signal line. Then, the image displayed on the display unit 13d is a clipped image generated according to the shooting operation data Db stored in the operation amount storage unit 21 from the clipped image Ca generated according to the current shooting operation data Da. The video is switched to the video Cb.

The above "current time" is an example of the "first time", and the above "past time" is an example of the "second time". The shooting operation data Da is an example of the "operation amount of the first shooting operation", and the shooting operation data Db is an example of the "second shooting operation amount".

By the way, the photographing operation data Db is not limited to the photographing operation data obtained by the photographing operation data collection device 10d. The photographing operation data obtained by another photographing operation data collecting device different from the photographing operation data collecting device 10d may be used. Also, the operator at the “past time” may be the same as or different from the operator at the “current time”.

However, the input image B used when the shooting operation is performed at the "current time" and the input image B used when the shooting operation is performed at the "past time" are common.

<Configuration of operation amount storage unit>
FIG. 12 is a diagram showing an example of the configuration of the operation amount storage unit 21 according to this embodiment in functional blocks. The manipulated variable storage unit 21 has an input/output unit 211 , an output manipulated variable storage unit 212 , a held manipulated variable storage unit 213 , and an I/F (Inter/Face) unit 214 .

Each processing function performed in each functional block of the manipulated variable storage unit 21 is realized by the CPU executing a program stored in the ROM using the RAM as a work area. Also, the output manipulated variable storage unit 212 and the held manipulated variable storage unit 213 are implemented by an HDD or the like.

The input/output unit 211 inputs a command signal instructing the operation amount storage unit 21 to start or stop outputting the photographing operation data Db. The input/output unit 211 also outputs the shooting operation data Db stored in the output operation amount storage unit 212 to the switching unit 15d in response to the command signal.

The output operation amount storage section 212 stores the shooting operation data to be output to the switching section 15 d among the shooting operation data Db stored in the operation amount storage section 21 . On the other hand, the held operation amount storage unit 213 stores, among the shooting operation data Db stored in the operation amount storage unit 21, the shooting operation data that is held without being output to the switching unit 15d.

It is assumed that the shooting operation data Db is stored in a data file format and different shooting operation data Db are stored in a plurality of data files. The output manipulated variable storage unit 212 stores data files to be output to the processing unit 12d via the switching unit 15d among the plurality of data files. On the other hand, the retained manipulated variable storage unit 213 stores a data file that is retained in the manipulated variable storage unit 21 without being output to the switching unit 15d among the plurality of data files.

With this configuration, the operation amount storage unit 21 outputs only the specific shooting operation data Db to the switching unit 15d while storing a plurality of different shooting operation data Db. 12d.

The I/F unit 214 is an interface for connecting with an external device such as the PC 205 . The shooting operation data Db stored in the operation amount storage unit 21 can be rewritten, added, or deleted via the PC 205 . The shooting operation data Db in the data file format is rewritten, added, or deleted in the operation amount storage unit 21 . By moving the shooting operation data Db stored in the holding operation amount storage unit 213 to the output operation amount storage unit 212 via the PC 205, the shooting operation data Db output from the operation amount storage unit 21 can be changed. can.

The recording unit 14d may have the function of the operation amount storage unit 21 so that data can be easily transferred between the recording unit 14d and the operation amount storage unit 21. FIG.

FIG. 13 is a diagram illustrating an example of the shooting operation data Db stored in the operation amount storage unit 21. As shown in FIG. As shown in the figure, the operation amount storage unit 21 stores shooting operation data in association with the elapsed time. The operation amount storage unit 21 outputs shooting operation data Db to the switching electric circuit 152d in synchronization with the input of the input image B to the processing unit 12d based on the elapsed time from the start of inputting the input image.

<Structure of Processing Unit>
FIG. 14 is a diagram showing an example of the configuration of the processing unit 12d according to this embodiment in terms of functional blocks. The processing unit 12d has a determination unit 124d.

The determination unit 124d determines start or end of collection of shooting operation data by the shooting operation data collection device 10d. Based on the start determination by the determination unit 124d, the image processing unit 123d starts generating the clipped image Ca based on the input image B. Start outputting Db. Also, the display unit 13d starts displaying the clipped image Ca or the clipped image Cb, and the recording unit 14d starts recording the photographing operation data Da.

The determination unit 124d detects that the input image B has started to be input to the processing unit 12d, and determines to start collecting the shooting operation data. Alternatively, the start of collection of shooting operation data may be determined based on an operation signal of a start switch provided on the platform 17d or the like, or a signal output by the measurement unit 11d in response to shooting operation.

When the determination unit 124 d determines to start collecting the shooting operation data, the determination unit 124 d outputs a start determination signal indicating the start to the image processing unit 123 d and the operation amount storage unit 21 . At the same time, a start determination signal may be output to the display section 13d and the recording section 14d. According to the start determination signal, the image processing unit 123d starts generating the clipped image Ca, and the operation amount storage unit 21 starts outputting the photographing operation data Db to the switching electric circuit 152d. The start determination signal output to the operation amount storage unit 21 corresponds to a command signal instructing the operation amount storage unit 21 to start outputting the shooting operation data Db.

On the other hand, the image processing unit 123d terminates the generation of the clipped image Ca, and the operation amount storage unit 21 terminates the output of the photographing operation data Db by the end determination by the determination unit 124d. Also, the display unit 13d finishes displaying the clipped image Ca, and the recording unit 14d finishes recording the photographing operation data.

The determination unit 124d detects that the input image B has not been input to the processing unit 12d for a predetermined time or longer, and determines the end of collection of the shooting operation data. Alternatively, the end of collection of the shooting operation data may be determined based on an operation signal of an end switch provided on the camera platform 17d or the like, or a signal output by the measurement unit 11d in response to the shooting operation. When the determination unit 124d determines that collection of the shooting operation data has ended, the determination unit 124d outputs a termination determination signal indicating the termination to the image processing unit 123d and the operation amount storage unit 21 . At the same time, an end determination signal may be output to the display section 13d and the recording section 14d. The termination determination signal output to the operation amount storage section 21 corresponds to a command signal for instructing the operation amount storage section 21 to terminate the output of the photographing operation data Db.

For simplification, FIG. 14 omits illustration of a signal line from the determination unit 124d to the operation amount storage unit 21 and a signal line from the determination unit 124d to the recording unit 14d.

In FIG. 14, for the sake of convenience, the signal output from the measurement unit 11d to the switching electric circuit 152d is shown as one signal line. Angle T, rotation angle R and zoom amount Z are included. Similarly, the signal output from the operation amount storage unit 21 to the switching electric circuit 152d includes the rotation angle P, the rotation angle T, the rotation angle R, and the zoom amount Z. The signals output from the switching electric circuit 152d include the rotation angle P, the rotation angle T, the rotation angle R, and the zoom amount Z. The rotation angle P, the rotation angle T, and the rotation angle R are input to the photographing posture angle obtaining section 121d, and the zoom amount Z is input to the zoom data obtaining section 122d.

The image processing unit 123d outputs the clipped video Ca to the display unit 13d when the shooting operation data Da is input, and outputs the clipped video Cb to the display unit 13d when the shooting operation data Db is input.

<Operation of the device according to the third embodiment>
FIG. 15 is a flowchart showing an example of processing according to this embodiment.

First, in step S151, the determination unit 124d detects that input of the input image B to the processing unit 12c has started, and determines to start collection of shooting operation data.

When collection of shooting operation data is started (step S151, Yes), the determination unit 124d outputs a start determination signal indicating that to the image processing unit 123d and the operation amount storage unit 21. FIG. According to the start determination signal, the image processing unit 123d starts generating the clipped image Ca, and the operation amount storage unit 21 starts outputting the shooting operation data Db. On the other hand, if collection of shooting operation data is not started (step S151, No), the process returns to step S151 and continues.

Subsequently, in step S152, the switching unit 15d switches the shooting operation data to be output to the processing unit 12d according to the switching signal from the switching switch 151d.

Subsequently, in step S153, if the switching signal is not a signal for enabling (ON) the switching (step S152, No), the switching electric circuit 152d provided in the switching section 15d does not operate, and the processing section 12d outputs the photographing operation data Da. Enter

On the other hand, if the switching signal is a signal that enables (ON) switching (step S152, Yes), the switching electric circuit 152d provided in the switching section 15d is activated. The connection of the signal line is switched so that the shooting operation data Db output from the operation amount storage unit 21 is input to the processing unit 12d. Subsequently, in step S154, the processing unit 12d inputs shooting operation data Db.

Subsequently, in step S155, the shooting posture angle acquisition unit 121d obtains the pan direction posture angle, the tilt direction posture angle, and the roll direction based on the rotation angle P, the rotation angle T, and the rotation angle R of the input shooting operation data Da. Get the attitude angle. Then, a signal indicating these is output to the image processing unit 123d.

Subsequently, in step S156, the zoom data obtaining unit 122d obtains zoom data such as the focal length based on the zoom amount Z in the input shooting operation data Da, and outputs a signal indicating the zoom data to the image processing unit 123d. output to

Subsequently, in step S157, the image processing unit 123d performs geometric transformation and clipped image processing on the input image B based on the input photographing attitude angle and zoom data, and extracts the clipped image Ca or the clipped image Cb. to generate The image processing unit 123d generates a clipped image Ca when the photographing operation data Da is input, and outputs the clipped image Ca to the display unit 13d. Further, the image processing unit 123d generates a clipped image Cb when the photographing operation data Db is input, and outputs the clipped image Cb to the display unit 13d.

Subsequently, in step S158, the display unit 13d displays the cutout image Ca when the cutout image Ca is input, and displays the cutout image Cb when the cutout image Cb is input.

Subsequently, in step S159, the determination unit 124d detects that the input image B has not been input to the processing unit 12d for a predetermined time or longer, and determines the end of collection of shooting operation data. When the collection of shooting operation data is to be terminated (step S159, Yes), the determination section 124d outputs a termination determination signal to that effect to the image processing section 123d and the operation amount storage section 21. FIG. According to the end determination signal, the image processing unit 123d ends the generation of the clipped video Ca, and the operation amount storage unit 21 ends the output of the photographing operation data Db. On the other hand, if the collection of shooting operation data has not ended (step S159, No), the process returns to step S152 and continues.

In this manner, the photographing operation data collection device 10d switches the photographing operation data output to the processing unit 12d to the photographing operation data Da output by the measuring unit 11d or the photographing operation data output by the operation amount storage unit 21 by the switching unit 15d. It is possible to switch to either one of the data Db. The image displayed on the display unit 13d can be switched to either the clipped image Ca or the clipped image Cb according to the shooting operation data input to the processing unit 12d.

As described above, in the present embodiment, the shooting operation data to be output to the processing unit 12d is either the shooting operation data Da output by the measuring unit 11d or the shooting operation data Db output by the operation amount storage unit 21. switch to Then, the image displayed on the display unit 13d is switched to either the clipped image Ca corresponding to the current photographing operation data or the clipped image Cb corresponding to the past photographing operation data. As a result, it is possible to perform the shooting operation and collect the shooting operation data while visually recognizing the images of the previous shooting operations performed on the same input image B. FIG. Then, various shooting operation data can be collected and the shooting operation data can be enriched.

In addition, in the present embodiment, the operation amount storage unit 21 stores shooting operation data having a small storage capacity with respect to video, so the storage capacity can be reduced compared to the case of storing video.

Effects other than those described above are the same as those described in the embodiments already described.

[Fourth embodiment]
Next, an example of a photographing operation data collection device according to the fourth embodiment will be described. Note that the description of the same components as those of the already described embodiment will be omitted.

<Configuration of Device According to Fourth Embodiment>
The overall configuration of the photographing operation data collection device according to this embodiment will be described with reference to FIG. FIG. 16 is a diagram illustrating an example of the configuration of the shooting operation data collection device 10e according to this embodiment.

A photographing operation data collection device 10e according to the present embodiment has a measurement unit 11e. The measurement unit 11e measures the photographing operation data Da by the operator 2. FIG. The photographing operation data Da is data indicating the photographing posture angle, zoom, focus, and the like. The shooting attitude angles are the pan axis rotation angle in the direction indicated by arrow 131e in FIG. 16, the tilt axis rotation angle in the direction indicated by arrow 132e, and the roll axis rotation angle in the direction indicated by arrow 133e. etc. The measuring unit 11e outputs the photographing operation data Da to the processing unit 12e and the recording unit 14e.

<Structure of Measuring Unit According to Fourth Embodiment>
FIG. 17 is a block diagram illustrating an example of the configuration of the measuring section 11e. As shown in FIG. 17, the measurement unit 11e has a rotation angle detection unit 111e, a zoom operation amount detection unit 112e, and a photographing attitude angle calculation unit 116e. The rotation angle detection unit 111e has a pan direction gyroscope 113e, a tilt direction gyroscope 114e, and a roll direction gyroscope 115e.

A pan direction gyroscope 113e, a tilt direction gyroscope 114e, and a roll direction gyroscope 115e are each incorporated inside the platform 17e. The panning direction gyroscope 113e measures the angular velocity of the camera platform 17e in the panning direction (see the panning direction 131e in FIG. 16), and outputs it as velocity data to the shooting posture angle calculator 116e. The tilt direction gyroscope 114e measures the angular velocity of the camera platform 17e in the tilt direction (see the tilt direction 132e in FIG. 16) and outputs it as velocity data to the photographing posture angle calculator 116e. Also, the roll direction gyroscope 115e measures the angular velocity of the pan head 17e in the roll direction (see the roll direction 133e in FIG. 16) and outputs it as velocity data to the photographing attitude angle calculator 116e.

The shooting attitude angle calculator 116e is realized by a CPU or the like. The photographing posture angle calculation unit 116e integrates the velocity data in the panning direction to calculate the photographing posture angle in the panning direction, and outputs it as the rotation angle P. FIG. Also, the velocity data in the tilt direction is integrated to calculate the photographing attitude angle in the tilt direction, which is output as the rotation angle P. Further, the velocity data in the roll direction is integrated to calculate the photographing attitude angle in the roll direction, which is output as the rotation angle R.

Note that when the shooting attitude angle calculation unit 116e is provided, the shooting attitude angle acquisition unit 121e of the processing unit 12e does not need to perform calculations for calculating the shooting attitude angle. The angle R may be directly output to the image processing unit 123e. Further, the arithmetic function of the photographing posture angle calculation unit 116e may be configured to be realized by the photographing posture angle acquisition unit 121e of the processing unit 12e.

A MEMS (Micro Electro Mechanical System) gyroscope can be used as the gyroscope. The MEMS gyroscope utilizes the Coriolis force that is generated in a direction perpendicular to both the velocity direction and the rotation axis when angular velocity is applied to an object having velocity. A MEMS gyroscope can detect an angular velocity by detecting a voltage generated by a change in vibration mode due to a Coriolis force associated with rotational motion of an object. As the gyroscope, in addition to the MEMS gyroscope, an optical fiber gyroscope or a ring laser gyroscope may be used.

In addition to the gyroscope, the rotation angle detection unit 111e includes an acceleration sensor, a geomagnetic sensor, or a plurality of Global Navigation Satellite System (GNSS) receivers, etc., and uses these to detect the rotation angle. good too.

When an acceleration sensor is used, the directional vector of acceleration due to gravity can be obtained from the outputs of a plurality of acceleration sensors. Alternatively, the elevation angle can be obtained by measuring the component value of acceleration due to gravity in a predetermined axial direction using one or more acceleration sensors.

When a geomagnetic sensor is used, the attitude angle (azimuth angle) of the pan head 17e with respect to magnetic north or magnetic south can be acquired by the geomagnetic sensor.

When GNNS receivers are used, a plurality of GNNS receivers are installed separately from each other, and the attitude (azimuth angle and elevation) can be obtained.

Note that the function of the zoom operation amount detection unit 112e is the same as that of the zoom operation amount detection unit 112a, so description thereof will be omitted.

In this embodiment, by using a gyroscope or the like as the rotation angle detector 111e, the rotation angle of the camera platform 17e can be measured with high accuracy. Further, when a MEMS gyroscope is used, since the MEMS gyroscope is small, it can be compactly housed inside the camera platform 17e or the like, and the apparatus can be miniaturized.

Note that the gyroscope, acceleration sensor, etc. of the rotation angle detection unit 111e may be attached to the display unit 13e or the like, the posture angle of which changes according to the shooting operation, without being limited to the camera platform 17e.

Effects other than the above are the same as those described in the already described embodiments.

Although the preferred embodiments of the present invention have been described in detail above, the present invention is not limited to such specific embodiments, and various modifications, Change is possible. It is also possible to combine part or all of the above-described embodiments.

Embodiments also include programs. The program is a program for causing a computer to function as each functional unit of the photographing operation data collection device according to claim 1.

With such a program, it is possible to obtain effects similar to those of the photographing operation data collection device described above.

1 Shooting Operation Data Collection Systems 2, 2a, 2b Operators 10a, 10b, 10c, 10d, 10e Shooting Operation Data Collection Devices 11a, 11b, 11c, 11d, 11e Measurement Units 111a, 111e Rotation Angle Detection Units 112a, 112e Zoom Operation amount detector 113a pan direction rotary encoder 113e pan direction gyroscope 114a tilt direction rotary encoder 114e tilt direction gyroscope 115a roll direction rotary encoder 115e roll direction gyroscopes 12a, 12b, 12c, 12d, 12e processing units 121a, 121b, 121c, 121d, 121e shooting posture angle acquisition units 122a, 122b, 122c, 122d, 122e zoom data acquisition units 123a, 123b, 123c, 123d, 123e image processing units 124c, 124d determination units 13a, 13b, 13c, 13d, 13e display unit 131a , 131b, 131c, 131d, 131e Pan directions 132a, 132b, 132c, 132d, 132e Tilt directions 133a, 133b, 133c, 133d, 133e Roll directions 14a, 14b, 14c, 14d, 14e Recording units 15a, 15b, 15c, 15d , 15e switching units 151a, 151b, 151c, 151d, 151e switching switches 152a, 152b, 152c, 152d, 152e switching electric circuits 153a, 153b, 153c, 153d, 153e terminals 16a, 16b, 16c, 16d, 16e tripods 17a, 17b , 17c, 17d, 17e pan head (an example of a movable part)
18a, 18b, 18c, 18d, 18e Focus demand 19a, 19b, 19c, 19d, 19e Zoom demand 20 Image storage unit 201 Input/output unit 202 Output image storage unit 203 Retained image storage unit 204 I/F unit 205 PC
21 operation amount storage unit 211 input/output unit 212 output operation amount storage unit 213 holding operation amount storage unit 214 I/F unit B input images Ca, Cb cropped images Da, Db shooting operation data P rotation angle around pan axis T tilt Rotation angle R around the axis Rotation angle Z around the roll axis Zoom amount

Claims (5)

A photographing operation data collection device for collecting photographing operation data accompanying photographing operation of a camera,
a movable part in which at least one of an attitude angle and an operation amount of a zoom operation changes according to the photographing operation;
a measurement unit that measures an operation amount of a first photographing operation with respect to the movable unit;
an image processing unit that receives an image, cuts out a portion of the image according to the operation amount of the first shooting operation, and geometrically transforms the cut out partial image to generate a first image; ,
The first image, or an image synchronized with the first image based on the elapsed time from the start of input of the input image, and other shooting operation data different from the shooting operation data collecting device. In the collection device, image processing for cutting out a part of the input video and geometrically transforming the clipped partial video in accordance with the operation amount of a second shooting operation different from the first shooting operation. a display unit that displays a second image generated by applying;
a switching unit that switches an image displayed on the display unit to either the first image or the second image;
a recording unit that records the operation amount of the first shooting operation measured by the measuring unit and at least one of the input video and the first video in association with the elapsed time; data collection device. The first photographing operation is a photographing operation performed by the photographing operation data collection device at a first time, and the second photographing operation is performed at a second time different from the first time. 2. The photographing operation data collecting device according to claim 1, wherein the photographing operation is a photographing operation performed by the photographing operation data collecting device. The shooting operation data collection device has a video storage unit that stores the second video,
3. The photographing operation data collection device according to claim 2, wherein the switching unit switches the image displayed by the display unit to either the first image or the second image stored in the image storage unit. . The photographing operation data collection device has an operation amount storage unit that stores an operation amount of the second photographing operation,
The switching unit switches to either the operation amount of the first shooting operation or the operation amount of the second shooting operation,
The image processing unit performs the image processing on the input image according to one of the operation amount of the first shooting operation and the operation amount of the second shooting operation, and performs the display. 3. The photographing operation data collection device according to claim 2, wherein the unit generates an image to be displayed. A program for causing a computer to function as each functional unit of the photographing operation data collection device according to claim 1.

Images