JPH07143471A - Image pickup device and electronic communication conference system - Google Patents

Abstract

PURPOSE:To revise an image pickup object whose shape is recognized from the outside of a camera or not recognized from the outside of the camera. CONSTITUTION:The device is provided with a shift mechanism 34 shifting laterally an image pickup element 30 with respect to an optical axis of an image forming optical system 32 and with a turning mechanism 36 turning the image pickup element 30, the image forming optical system 32 and the shift mechanism 34 within a horizontal plane integrally. The shift mechanism 34 and the turning mechanism 36 are used depending on the image pickup situation.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup apparatus and an electronic communication conference system, and more specifically, to an image pickup apparatus suitable for inputting an image in an electronic communication conference performed by connecting a plurality of different places with a communication line. And an electronic communication conference system using the imaging device.

[0002]

2. Description of the Related Art In an electronic communication conference, an image communication system is used as a means for realizing smooth communication, in which mutual talkers and their surroundings are mutually transmitted as images. This allows the conference to proceed while mutually confirming the atmosphere of the conference site and the facial expressions of the other party.

For image communication, at least a camera for inputting an image, a monitor for displaying an image, and a communication device for transmitting an image signal are required. The cameras and monitors that meeting attendees face directly are particularly important as man-machine interface devices. As an elemental technology for that purpose, a technology of inputting and displaying an image so that the lines of sight of the speakers match each other so that the speakers can naturally talk with each other has been studied. Among them, some technologies have been put into practical use so as to prevent the movement of the camera from being noticed when a plurality of speakers rotate the camera. One of them is a method of hiding the camera by covering the entire camera or the front part of the camera with a half mirror.

Further, as a technique in the image pickup apparatus itself,
There is a technique for apparently reducing the movement of the image pickup apparatus itself by shifting and tilting some elements of the image pickup system in the housing of the image pickup apparatus. For example, there is a camera shake correction function that corrects image vibration caused by camera shake. This is to optically or electronically stabilize the subject in the image signal, and is basically a function added to the image pickup apparatus alone.

A configuration has also been proposed in which two images captured in a state of being shifted by 1/2 pixel from each other are combined to obtain a high-definition image. For example, a part of the image pickup system (optical system or image pickup device) is slightly vibrated in synchronization with one frame period for scanning one screen, and two images shifted by 1/2 pixel can be obtained.

[0006]

When the camera is completely hidden, smooth communication in a conference may be excluded instead of being alienated. For example, in a system capable of remotely controlling a camera of a communication partner, it becomes impossible to determine who the communication partner is paying attention to. Also, if there is a subject that you want to display on the other party's monitor in this conference hall, you have to ask the other party to shoot that subject and have the other person operate this camera, which is very troublesome. .

As described above, in an electronic communication conference using image communication, it is better not to be aware of the presence of the camera.
Sometimes it is better to be conscious.

An object of the present invention is to provide an image pickup apparatus and an electronic communication conference system that solve such problems.

[0009]

An image pickup apparatus according to the present invention is an image pickup apparatus having an image forming optical system and an image pickup element for converting an optical image formed by the image forming optical system into an electric signal. A first subject area changing unit that changes a subject area projected on the image pickup device by moving the image pickup device laterally with respect to the optical axis of the system, the image forming optical system, and the image pickup unit. It is characterized in that a second subject area changing means for changing the subject area by moving integrally and a control means for controlling the first and second subject area changing means are provided.

An electronic communication conference system according to the present invention is an electronic communication conference system for mutually communicating images taken by image pickup devices. The imaging device, an imaging optical system, an imaging element that converts an optical image by the imaging optical system into an electrical signal,
A first subject area changing unit that changes a subject area projected on the image pickup device by moving the image pickup device laterally with respect to the optical axis of the image forming optical system; And a second subject area changing means for changing the subject area by moving the image pickup means integrally. Then, each terminal is provided with control means for controlling the first and second subject area changing means of the imaging device of one or more communication partners.

[0011]

When the subject area is changed by the first subject area changing means, the orientation of the image pickup device does not change in appearance,
Therefore, it is apparently impossible to know the subject range to be photographed.

On the other hand, when the subject area is changed by the second subject area changing means, the main part of the image pickup apparatus moves integrally, so that the change in the object to be photographed can be known from the appearance.

[0013]

Embodiments of the present invention will now be described in detail with reference to the drawings.

FIG. 1 is a schematic block diagram of the entire electronic communication conference system according to an embodiment of the present invention. In this embodiment, the two conference halls #A and #B are interconnected by a communication line. At both conference rooms #A and #B, at least one image input device (camera or the like) 10A and 10B, images from the image input devices 10B and 10A of the communication partner, and images from the own image input devices 10A and 10B. Image display devices 12A and 12B for displaying a message, voice input devices (microphones and the like) 14A and 14B, and a voice input device 14 of a communication partner.
Audio output devices (speakers, etc.) 16A and 16B for outputting audio from B and 14A, and operation control units 18A and 18B for remotely operating the device of the communication partner are provided.

A communication device 20 is installed between the conference halls #A and #B and transmits both signals to the other party.

Each of the operation control units 18A and 18B is provided with a camera control unit for controlling the photographing direction and the photographing range of the image input devices 10B and 10A of the communication partner, and the control signals are provided in the image input devices 10A and 10B. A camera drive mechanism for changing the shooting direction, the shooting range, etc. is provided.

2A and 2B are schematic configuration diagrams of the camera drive mechanism of the image input devices 10A and 10B, FIG. 2A showing a plan view thereof, and FIG. 2B showing a side view thereof. Reference numeral 30 denotes an image pickup element (for example, CCD type image sensor), 32 denotes an image forming optical system for forming an optical image of a subject on the image pickup surface of the image pickup element 30, and 34 denotes light of the image pickup optical system 32. The shift mechanism 36 that moves in the horizontal lateral direction with respect to the axis integrally includes the image sensor 30, the imaging optical system 32, and the shift mechanism 34.
It is a rotating mechanism that rotates in a horizontal plane. Needless to say, the control signal from the operation control unit 18B (or 18A) is applied to the image input device 10A (or 10B) via the communication device 20.

In accordance with a control signal from the operation control unit 18B (or 18A), the image input device 10A (or 10)
In the shift mechanism 34 of B), the image sensor 32 is connected to the image forming optical system 32.
By moving a specified amount in the horizontal direction with respect to the optical axis, the rotating mechanism 36 rotates the image pickup device 30, the imaging optical system 32, and the shift mechanism 34 integrally by a specified angle in the horizontal plane.

The camera control unit in the operation control units 18A and 18B has two modes for camera rotation control. The first camera operation mode is the rotation mechanism 3
6 is a mode in which only the image input devices 10A and 10B are driven to rotate in the horizontal plane. In the second camera operation mode, only the shift mechanism 34 is driven and only the image pickup device 30 is perpendicular to the photographing optical axis. It is a mode that shifts laterally. In either mode, the image input device 10A, 10
Whether to operate B can be freely selected by the operation control units 18A and 18B.

Next, the operation of this embodiment will be described. At one of the conference halls #A (or #B), the speaker or the operator operates the operation control unit 18A (or 18B) to select the camera operation mode. The choice is conference room #B
(Or #A) image input device 10B (or 10A). The information transmitted is the selected mode and the movement amount (shift amount or rotation angle) of the movement mechanism (shift mechanism 34 or rotation mechanism 36) corresponding to the selected mode. According to the information on the amount of movement, the shift mechanism 3
4 moves the image pickup device 30 laterally, and the rotation mechanism 36
The image pickup device 30, the imaging optical system 32, and the shift mechanism 34 are integrally rotated in a horizontal plane. As a result, the shooting range is changed.

Image display device 1 at conference hall #A (or #B)
An image input by the image input device 10B (or 10A) of the conference hall #B (or #A) is displayed in 2A (or 12B), so that the speaker of the conference hall #A (or #B) or The operator T operates the operation control unit 18A (or 18B) while watching the displayed image to display the image of the desired area of the conference hall #B (or #A) on the image display device 12A (or 12B). Can be displayed.

At this time, when the first camera operation mode is selected, the image input device 10B (or 10A) rotates as a whole, so that the speaker at the conference hall #B (or #A) is
From the appearance of the image input device 10B (or 10A), the shooting direction and the shooting range can be known. This is convenient, for example, when adjusting the shooting direction and shooting range of the image input devices 10A and 10B.

On the other hand, when the second camera operation mode is selected, only the image pickup device 30 is moved by the shift mechanism 34 in the lateral direction of the optical axis, so that the speaker at the conference hall #B (or #A) is From the appearance of the image input device 10B (or 10A), the shooting direction and the shooting range cannot be recognized. Therefore, the conference hall #
The shooting range can be changed without being understood by the speaker B (or #A). That is, it is possible to photograph an arbitrary speaker in the conference hall #B (or #A) without being aware of being photographed.

In this way, in this embodiment, the image input devices 10A and 10B for image communication can be remotely operated by either a method which is clear in appearance or a method which is unknown in appearance.

In the example shown in FIG. 2, the shift mechanism 34 shifts the image pickup device 30 in the horizontal and horizontal directions, but it may shift the image pickup device 30 in the vertical and horizontal directions. FIG. 3 shows a schematic configuration diagram of a camera drive mechanism of such image input devices 10A and 10B. 3A is a plan view, FIG. 3B is a side view, and FIG. 3C is a front view. 3, the same components as those in FIG. 2 are designated by the same reference numerals. Three
Reference numeral 8 denotes a shift mechanism that shifts the image sensor 30 in a vertical and horizontal direction orthogonal to the optical axis of the imaging optical system 32.

FIG. 4 shows the effective image pickup area 30 of the image pickup device 30.
The relationship between a and the good image forming area 32a of the image forming optical system 32 is shown. While the effective image pickup area 30a of the image pickup device 30 is usually a rectangle whose horizontal sides are long (here, the aspect ratio is 1: 2), the good image pickup area 32a of the image forming optical system 32 is normally formed as an image. It is a circle whose center is located on the optical axis of the optical system 30. FIG. 4A shows areas 30a and 32 at the initial position where the image sensor 30 is centered on the optical axis of the imaging optical system 32.
4B shows the relationship between the regions 30a and 32a when the image sensor 30 is vertically shifted.

The effective image pickup area 30a of the image pickup device 30 is a rectangle whose horizontal sides are long (here, the aspect ratio is 1:
Since it is 2), even if the image sensor 30 is shifted in the vertical direction, the effective image pickup area 30a is less likely to deviate from the good image formation area 32a than in the horizontal shift. Figure 4 for reference
FIG. 4C shows the relationship between the regions 30a and 32a when the image pickup device 30 is horizontally shifted by the same distance as in FIG.
In the horizontal shift, the end portion on the short side of the effective image pickup area 30a is directly displaced from the good image formation area 32a. Of course, if the good image formation area 32a is sufficiently wider than the effective image pickup area 30a, such inconvenience is not a problem in practical use.

In the embodiment shown in FIGS. 2 and 3, the image pickup device 30 is shifted one-dimensionally, but it is possible to shift the image pickup device 30 two-dimensionally in the plane orthogonal to the optical axis of the imaging optical system 32. It is clear that it is okay.

In the above embodiment, the image pickup device itself is movable in the direction orthogonal to the photographing optical axis, but the photographing optical axis may be displaced by the prism in the orthogonal direction. A schematic configuration diagram of the embodiment is shown in FIG. Figure 5 (a)
Shows a plan view thereof, and FIG. Reference numeral 40 denotes an image sensor, which is fixed. Reference numeral 42 denotes an image forming optical system, and 44 denotes an image pickup device 40 in which the optical axis of the image forming optical system 42 is laterally shifted.
Is a shift mechanism for making the light incident on the optical axis, and by adjusting the distance Dz between the two prisms 46 and 48 arranged along the optical axis between the image pickup device 40 and the imaging optical system 42, the lateral shift of the optical axis is performed. You can change the amount. Reference numeral 50 denotes a rotating mechanism that integrally rotates the image pickup device 40, the image forming optical system 42, the shift mechanism 44, and the prisms 46 and 48 in a horizontal plane.

FIG. 6 is an explanatory diagram of the optical axis lateral shift function of the prisms 46 and 48. The prisms 46 and 48 having the same apex angle are arranged such that their incident surfaces are parallel to each other. Obviously, if the distance between the prisms 46 and 48 is changed, the optical axis incident on the image pickup device 40 is laterally shifted. Since the incident surfaces of the prisms 46 and 48 are parallel to each other, even if the prism 46 or 48 moves in the optical axis direction, the image forming surface does not deviate from the photoelectric conversion surface of the image pickup device 40.

Although the prisms 46 and 48 are used in FIG. 5, the optical axes can be shifted in the lateral direction even if two parallel plates are arranged instead of the prisms 46 and 48 and their inclinations are changed. Further, even if the prisms 46 and 48 are tilted together with respect to the optical axis, the optical axis can be laterally shifted.

Next, an embodiment in which the photographing optical axis is laterally shifted by the reflecting mirror will be described. FIG. 7 shows a schematic configuration diagram of the embodiment. FIG. 7A shows a plan view thereof, and FIG. 7B shows a side view thereof. An image sensor 60 is fixed. 62
Is an image forming optical system, and 64 and 66 are reflection mirrors that reflect the light beam from the image forming optical system 62 and enter the image pickup element 60.
The reflection mirrors 64 and 66 are arranged in parallel with each other.
The shift mechanism 68 shifts the reflection mirror 66 in a direction toward and away from the center line of the imaging optical system 62. 70
Is an image pickup device 50, an imaging optical system 52, a reflection mirror 64,
This is a rotating mechanism that integrally rotates 66 and the shift mechanism 68 in a horizontal plane.

FIG. 8 is an explanatory view of the optical axis lateral shift function by the reflection mirrors 64 and 66. 66 as the reflection mirror 64
By changing the distance Dx of the reflection mirror 6
When the reflection mirror 66 is moved closer to or farther from the fourth mirror 4, the light rays incident on the image sensor 60 are laterally shifted. The shift amount is proportional to Dx.

The same thing can be achieved by simultaneously changing the angles of the reflecting mirrors 64 and 66.

In each of the above embodiments, the lateral shift mechanism of the optical axis and the rotation of the entire camera are controlled independently, but by combining these, various operational effects can be obtained. An example will be described below.

For example, the rotation mechanism of the entire camera is used for slowly changing the object to be photographed, and the lateral shift mechanism of the optical axis is used for quick change. FIG. 9 shows a schematic block diagram of the control mechanism. 80 is the operation control unit 18
A camera rotation control circuit installed in A and 18B, 82
Is a camera drive mechanism equipped in the image input devices 10A and 10B.

The camera drive mechanism 82 includes a shift mechanism 82a having the same function as the shift mechanisms 34, 38, 44 and 68 described above and a rotation mechanism having the same function as the rotating mechanisms 36, 50 and 70 described above. And a moving mechanism 82b.

In the camera rotation control circuit 80, 84
Is a pointing device for operating the camera such as a mouse and a joystick, 86 is an acceleration detecting circuit that receives the position pointing signal from the pointing device 84, and detects the acceleration of the motion of the camera operation, 88 is the detection result of the acceleration detecting circuit 86 It is a drive distribution circuit that controls the shift mechanism 82a and the rotation mechanism 82b at a sharing ratio according to. The shift mechanism 82a and the rotation mechanism 82b each include an encoder that detects the shift amount and the rotation amount and outputs a signal indicating the shift position and the rotation position. The output of the encoder is supplied to the drive distribution circuit 88.

The drive distribution circuit 88, when the acceleration of the camera operation by the pointing device 84 is a predetermined value or more, the pointing device 84.
Is supplied to the shift mechanism 82a exclusively, and when it is less than the predetermined value, the camera operation signal from the instruction device 84 is exclusively supplied to the rotating mechanism 82b. That is,
For slow camera operation, the rotating mechanism 82b changes the object to be imaged, and for camera operation at a certain speed or more, the shift mechanism 82a changes the object to be imaged.

The drive distribution circuit 88 also includes a shift mechanism 8
When 2a reaches or reaches its moving end, the rotating mechanism 82b is used, and when the rotating mechanism 82b reaches or reaches its moving end, the shift mechanism 82a is used. Then, the drive amounts of the shift mechanism 82a and the rotation mechanism 82b are distributed.

Further, when the direction of the camera is changed according to the line of sight of the person watching the monitor screen, when the face of the person watching the monitor screen moves, the movement of the camera body corresponds to it, and only the eyes move. Sometimes I move by moving inside the camera,
When both the face and eyes move, they may be distributed to both the camera body and inside the camera according to the ratio. In order to determine the distribution amount between the movement of the camera body and the movement inside the camera, a means for detecting the movements of the face and eyes of the person watching the monitor screen is required. The detection method will be briefly described with reference to FIG.

In FIG. 10, an alternate long and short dash line 90 is a center line connecting the image display devices 12A and 12B and the observer's head, and FIG.
Looking at the center of the display screen of A and 12B, the same (b)
[Fig. 4] is each plan view of a state in which a position deviated from the center is viewed.

This observer uses the image input devices 10A and 10B.
Is captured by the eyeball 9 in the head 92 from the input image.
4, 96 and the relative position of the nose 98 are determined. The angle θ of the head 92 with respect to the above-mentioned alternate long and short dash line is calculated under the assumption that the head 92 is basically bilaterally symmetric and is substantially spherical. Further, the position of the pupil part (colored part) in the eyeballs 94, 96 is obtained from the input images from the image input devices 10A, 10B, and the angle θ of the head 92 is also taken into consideration to determine the line-of-sight direction of the eyeballs 94, 96. .

Head portion 92, eyeballs 94, 9 from the input image
6 and its pupils and each part of the nose 98 can be easily extracted and determined by using a combination of the extraction methods of edges, shadows and specific colors (skin color etc.).

The present invention can also be used to grasp the state of the person to be photographed and change the rotation method. For example, when the camera is not in the field of view of the subject, the entire camera is moved, and when it is in the field of view, only the inside of the camera is moved as much as possible. In order to determine this distribution amount, the movement of the face of the person to be photographed and the eye gaze direction may be detected from the input image as described in FIG.

Of course, the person to be photographed may operate a predetermined operation means to selectively operate the movement of the camera body and the movement inside the camera.

[0047]

As can be easily understood from the above description, according to the present invention, it is possible to select the driving of the camera body or the driving inside the camera with respect to the change of the photographing symmetry, and as a result, various cameras can be selected. Work becomes possible. In particular, camera operation that makes the presence and movement of the camera conscious,
It is possible to provide a video conference system capable of selecting a camera operation that does not make the camera movement conscious according to a shooting situation.

[Brief description of drawings]

FIG. 1 is an overall configuration block diagram of an embodiment of the present invention.

FIG. 2 is a schematic configuration diagram of a first example of image input devices 10A and 10B.

FIG. 3 is a schematic configuration diagram of a second example of the image input devices 10A and 10B.

4 is a relationship diagram between an effective image pickup area 30a of the image pickup device 30 and a good image pickup area 32a of the image pickup optical system 32. FIG.

FIG. 5 is a schematic configuration diagram of a third example of the image input devices 10A and 10B.

6 is an explanatory diagram of optical axis lateral shift in FIG.

FIG. 7 is a schematic configuration diagram of a fourth example of the image input devices 10A and 10B.

FIG. 8 is an explanatory diagram of optical axis lateral shift in FIG.

FIG. 9 is a schematic block diagram of a camera drive system that uses both a shift mechanism and a rotation mechanism.

FIG. 10 is an explanatory diagram of a method of detecting movements of a face and eyes from a captured image.

[Explanation of symbols]

10A, 10B: Image input device 12A, 12B: Image display device 14A, 14B: Voice input device 16A, 1
6B: Audio output device 18A, 18B: Operation control unit 20: Communication device 30: Imaging device 32: Imaging optical system 34: Shift mechanism 36: Rotation mechanism 38: Shift mechanism 30a: Effective imaging area of the imaging device 30 32a:
Good imaging area 40 of imaging optical system 32: image sensor 4
2: Imaging optical system 44: Shift mechanism 46, 48: Prism 50: Rotating mechanism 60: Imaging device 62: Imaging optical system 64, 66: Reflecting mirror 68: Shift mechanism 7
0: Rotation mechanism 80: Camera rotation control circuit 82: Camera drive mechanism 82a: Shift mechanism 82b: Rotation mechanism
84: Instructing device for camera operation 86: Acceleration detection circuit 88: Drive distribution circuit 90: Image display devices 12A, 12
Center line connecting B to the observer's head 92: Head 9
4,96: Eyeball 98: Nose

Front page continuation (72) Masayoshi Sekine, Inventor Masayoshi Shimomaruko 3-30-2, Canon Co., Ltd. (72) Inventor Katsumi Iijima 3-30, Shimomaruko, Ota-ku, Tokyo Canon Inc.

Claims (7)

[Claims] 1. An imaging apparatus having an imaging optical system and an imaging device for converting an optical image formed by the imaging optical system into an electrical signal, wherein the imaging device is arranged horizontally with respect to the optical axis of the imaging optical system. By moving in the direction, the first subject area changing means for changing the subject area projected on the image pickup device, and the image pickup optical system and the image pickup means are moved integrally to change the subject area. An image pickup apparatus comprising: a second subject area changing means; and a control means for controlling the first and second subject area changing means. 2. The image pickup apparatus according to claim 1, wherein the first subject area changing unit is an image pickup device moving unit that moves the image pickup device laterally with respect to a photographing optical axis. 3. The first subject area changing means comprises a movable prism arranged between the imaging optical system and the image pickup device, and a driving means for rotating or translating the movable prism. Item 1. The image pickup device according to item 1. 4. The first subject area changing means comprises a reflecting member arranged between the imaging optical system and the image pickup device, and driving means for rotating or translating the reflecting member. Item 1. The image pickup device according to item 1. 5. The image pickup apparatus according to claim 1, wherein the control means is separable from the main body of the image pickup apparatus, and a communication means is provided for communicating predetermined information between the control means and the main body of the image pickup apparatus. 6. The image pickup apparatus according to claim 1, wherein the first subject area changing unit moves the subject area substantially in the minor axis direction of the effective light receiving portion of the image pickup device. 7. An electronic communication conference system for mutually communicating images picked up by an image pickup device, wherein the image pickup device comprises an image forming optical system, and an image pickup element converting an optical image formed by the image forming optical system into an electric signal. A first subject area changing means for changing the subject area projected on the image sensor by moving the image sensor laterally with respect to the optical axis of the image forming optical system; And second subject area changing means for changing the subject area by moving the image pickup means as a unit, and each terminal has one or more subject areas of the image pickup device of one or more communication partners. An electronic communication conference system comprising a control means for controlling the changing means.