JP2588970B2 - Multipoint conference method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Overview] Regarding a multi-point conference system in which a video conference is simultaneously performed between three or more conference terminals via a center device, an appropriate image switching of a conference room terminal can be performed with a configuration as simple as possible. Each conference room terminal has a speaker detection unit that performs speaker detection from a group of microphones provided for the participants, and controls a person camera toward the speaker whose speaker is detected. Transmitting the image of the person camera together with the voice of the speaker and the speaker detection signal to the center device, and the center device transmits a conference room terminal related to the speaker detection signal each time the speaker detection signal is received. The output image of the central conference room terminal is held for a certain period of time by a timer and distributed to monitors of other conference room terminals.

[Industrial applications]

The present invention relates to a multipoint conference system, and more particularly to a multipoint conference system in which video conferences are simultaneously performed between three or more conference room terminals via a center device.

2. Description of the Related Art Video conferences are used between conference room terminals located at remote locations in order to quickly transmit information and to save time and money when people and things move (business trips).

In a case where three or more conference room terminals are connected and a video conference is held at the same time, the method of distributing an image of one conference room terminal to another conference room terminal is best in consideration of a data transmission amount. At this time, the conference room terminal that is the center of this conference is usually
It is desirable to select the conference room terminal where the speaker exists, and it is desirable that this selection control be performed automatically.

[Conventional technology]

FIG. 8 shows a conventional example of a multipoint conference system in which video conferences are simultaneously performed between three or more conference room terminals via a center device. Each conference room terminal 1 faces a conference participant. A plurality of microphone groups 11, a microphone mixer 12 for synthesizing outputs from these microphone groups 11, and a speaker for outputting sound from another conference room terminal.
13, an audio control unit 14 that performs an input / output audio switching operation or an echo canceling operation or the like to prevent the output audio of the speaker 13 from sneaking into the microphone group 11, and an audio control unit from or from the audio control unit 14. An encoding / decoding unit (hereinafter abbreviated as CODEC) 15 for audio to 14, and a camera 16 for displaying the whole view of the conference room terminal 1, a person or a material
And a monitor 17 for displaying images of other conference room terminals
And CODEC 18 for images of camera 16 and monitor 17,
It comprises a multiplexing / demultiplexing unit (MUX / DMUX) 19 for input / output signals of an audio CODEC 15 and an image CODEC 18.

Each conference room terminal 1 having such a configuration is interconnected to the center device 2. The center device 2 includes a multiplexing / separating unit 21 for audio / video from each conference room terminal 1 or to each conference room terminal 1, an audio CODEC 22 connected to the multiplexing / separating unit 21, and each pitch. An audio level comparator 23 for comparing the level of the audio signal from the CODEC 22;
A mixer 24 for synthesizing the audio signal after the comparison and sending it to the speaker 13 of each conference room terminal 1 via the audio CODEC 22 and the multiplexing / demultiplexing unit 21 and an image separated by the multiplexing / demultiplexing unit 21 An image data selector 25 for selecting data,
The control unit 26 receives the comparison result from the audio level comparator 23 and supplies a selection signal to the selector 25, and the image data selected by the selector 25 via the multiplexing / demultiplexing unit 21 to each conference room terminal 1
And a distribution unit 27 for sending to the monitor 17.

In the operation of this conventional example, the level of the synthesized voice from the microphone group 11 of each conference room terminal 1 is set to the center device 2.
, The conference room terminal outputting the maximum audio level among them is determined as the central conference room terminal, the image of the conference room terminal is selected by the selector 25, and the
At 27, it is distributed to other conference room terminals.

Such a multipoint conference system is disclosed in, for example, Japanese Patent Application Laid-Open No. 61-4568.
This is shown in JP-A-9.

[Problems to be solved by the invention]

However, in the conventional multipoint conference system shown in FIG. 8, since the synthesized speech for each conference room terminal is used, the speech is recognized as a speech due to noise or noises other than the speech.
As a result of distributing the unnecessary images of the conference room terminals that are not speaking to each conference room terminal, there is a problem in operation such that the image of the central conference room cannot be distributed.

Accordingly, the present invention is to enable appropriate image switching of conference room terminals with a configuration as simple as possible in a multipoint conference system in which a video conference is simultaneously performed between three or more conference room terminals via a center device. With the goal.

[Means and actions for solving the problem]

FIG. 1 is a block diagram showing the principle of a multipoint conference system according to the present invention for achieving the above-mentioned object. As shown in FIG. , The speaker is detected from the group of microphones 11 provided to the camera, and the person camera 4 is controlled and directed to the speaker. And send it out.

In the center device 2, as shown in FIG. 2, each time the speaker detection signal is received, the conference room terminal related to the speaker detection signal is used as the central conference room terminal, and the output image is held for a fixed time by the timer 6, It is distributed to the monitor 17 of another conference room terminal 1. Therefore, if the speaker detection signal continues to be received even after the elapse of the holding time, the holding time is updated, and if the same speaker detection signal is not input within the holding time, the next talk is performed. The holding time of the timer 6 is started by the person detection signal.

As a result, the conference room terminal where the speaker is present becomes the central conference room terminal, and the person image is sequentially distributed to the other conference room terminals, and the conference proceeds.

In the present invention, the speaker's voice level is detected in two stages by the conference room terminal 1 and the center device 2, and the speaker is detected for each microphone input. There is no end.

In addition, in the present invention, when there is a conference room terminal that outputs a material image by the material camera 5, the center device 2 distributes the material image to another conference room using the conference room terminal as a central conference room terminal. At this time, even if another conference room terminal sends a speaker detection signal, the conference room terminal that is outputting the material image is still the central conference room terminal, and the currently distributed material image is not switched. To

This allows the conference to proceed with the conference room terminal that outputs the material image always as the central conference room terminal.

Further, in the present invention, the center device 2 can send a composite image of the person image and the material image to each conference room terminal 1, and in this case, as shown in FIG. By switching the central conference room terminal to combine and display its own material image and the person image of the conference room terminal that generated the speaker detection signal, the material image is prioritized and the user speaks while viewing this material image The participant's image can be viewed on the central conference room side.

〔Example〕

FIG. 4 shows an embodiment of the multipoint conference system according to the present invention. In this embodiment, a speaker detector 3 is provided in each conference room 1 in comparison with the conventional example shown in FIG. A data transmission unit 8 for including a speaker detection signal from the speaker detection unit 3 in transmission data is provided, and a data transmission unit 28 for extracting the speaker detection signal in the center device 2 is multiplexed. The second embodiment is different from the first embodiment in that the second embodiment is connected to the multiplexing / separating unit 21 and that the audio level comparator is omitted. Note that the panoramic camera 7 is shown in the conference room in addition to the person camera 4 and the document camera 5, but the present invention may not be particularly used.

In the operation of this embodiment, the audio output from the microphone group 11 provided one by one for each attendee is sent to the speaker detector 3 via the microphone mixer 12. The speaker detector 3 determines whether or not a voice has been input to the microphone, and sends a speaker detection signal (whether or not a speaker has been detected and a conference room number) to the data transmitter 8 and the speaker. A speaker identification number signal is sent to the person camera 4 to perform turning control so that an image of a person corresponding to the detection is obtained.

This speaker detection operation is performed by the present applicant, for example, in Japanese Patent Application No. 62-3218.
This is already disclosed in No. 46, but will be briefly described here.

FIG. 5 shows a specific configuration of the speaker detection unit 3. Each output signal corresponding to each microphone from the mixer 12 (only the synthesized output signal is sent to the voice control unit 14) is sampled by a sampling circuit. Sample at 30. This sampling circuit 30 is a circuit for converting into a binary digital signal which is turned on when the microphone output is equal to or higher than a predetermined threshold level and turned off otherwise.
Are distributed and accumulated via the input buffer 31 to the accumulation buffers 32 prepared for the respective microphones.

Each bit number of the accumulation buffer 32 corresponds to a predetermined number of seconds, for example, a sampling number of 4 seconds, and the total time of the audio output of the microphone is confirmed by the number of bits set in the accumulation buffer 32. .

The total time indicated by the number of bits is calculated by the processing circuit 33.
The number of bits corresponding to the total time of about 2 seconds, that is, the speaker of the microphone corresponding to the accumulation buffer in which almost half of the bits are set is detected as the speaker.

In this case, when there are a plurality of speakers detected as speakers, the number of bits set in the accumulation buffer 32, that is, the speaker of the microphone corresponding to the buffer having the longest total time is recognized as the speaker. And outputs a speaker detection signal (speaker position number signal).

In this way, from each conference room 1, the microphone group
The synthesized voice data of 11, the image data from the human camera 4, and the speaker detection signal are multiplexed by the multiplexing / demultiplexing unit 19 and sent to the center device 2 through a digital line. still,
The speaker detection signal from the data transmission unit 8 is continuously transmitted cyclically while the speaker is present.

In the center device 2, a speaker detection signal (speaker presence signal and conference room number) is extracted via the multiplexing / demultiplexing unit 21 and the data transmission unit 28 and is provided to the control unit 26. The selector 25 is switched so as to select the output image of the conference room, and the same image is distributed to conference rooms other than the conference room in which the speaker is detected by the distribution unit 27. The audio is synthesized by the mixer 28 via the multiplexing / demultiplexing unit 21 and the audio CODEC 22, and is transferred to each conference room 1.

Here, the operation when the control unit 26 receives a plurality of speaker detection signals will be described with reference to the flowcharts of FIGS.

First, when any signal from each conference room terminal is received, an interrupt signal is sent to the control unit 26, and each conference room terminal is sequentially scanned as shown in FIG. When a signal is input, the speaker detection control shown in FIG. 7 is performed.

In the speaker detection control shown in FIG. 7, it is first determined whether or not a speaker is currently detected (step S).
1). If the speaker is detected, it is determined whether the source of the speaker presence signal (input conference room terminal) is the same as the station that has been recognized as the speaker (step S2). If they are the same, the speaker detection holding timer conceptually shown in FIG. 1 is updated (step S3), otherwise, this signal itself is ignored.

As shown in FIG. 6 (b), this timer operation is interrupted in step S3, and when a talker presence signal is received from one conference room terminal, an initial value is set, and a speech is made to any conference room terminal. When no user is present, the value is decremented by "1" every cycle of the program.

This holding timer is a timer for measuring the minimum holding time provided in order to prevent the displayed conference room terminal from being changed each time when the speaker presence signal frequently comes up from each conference room terminal. (Currently the initial value is about 4 seconds). Also, when the speaker detection state is set (when the hold timer is $ 0), even if a speaker presence signal is received from another conference room terminal, the signal is ignored and the conference room terminal which is currently recognized as a speaker is received. Updates the holding timer (resets the initial value) (see FIG. 2).

On the other hand, if it is not in the speaker detection state in step S1, it is determined whether to switch the image selector 25 (step S4). Here, if the sender of the speaker detection signal has already sent the speaker detection signal and is the central conference room terminal, there is no need to switch the image selector 25 and the timer is updated (step S3).

When the transmission source is not the central conference room terminal, the monitor screen state of the central conference room terminal is checked (step S5).

As a result, when the central conference room terminal is transmitting the material image from the material camera 5, since all the conference room terminals are paying attention to the material image, the automatic switching of the central conference room terminal is not performed. Not performed.

However, when a person image of an arbitrary conference room terminal is synthesized and displayed on the monitor of the central conference room terminal in addition to the material image,
As shown in FIG. 3, the sub-conference room terminal (other than the central conference room terminal) in the image selector table is updated in order to switch only the partner person image to the speaker's person image (step S6).

When the central conference room terminal is transmitting an image other than the material, the central conference room terminal and the secondary conference room terminal in the image selector table are updated to perform automatic switching by speaker detection (steps S6 and S7). And outputs a selection signal to the image selector 25 (step S9).

It should be noted that the person image of the conference room terminal which has been the center until immediately before this can be the image of the new center conference room terminal selected by the speaker detection.

〔The invention's effect〕

As described above, according to the multipoint conference system according to the present invention,
Each of the conference room terminals performs speaker detection and outputs an image of the person, and each time the center device receives the speaker detection signal, the timer is held for a certain period of time to center the conference room terminal related to the speaker detection signal. Since the output image is distributed to the other conference rooms as the conference room terminals, the appropriate and smooth progress of the conference can be realized without distributing an erroneous person image due to noise or the like. In addition, by using a timer to recognize the speaker state, the continuity of speech is emphasized, and the switching of conference room terminals is prevented from becoming too frequent.

[Brief description of the drawings]

FIG. 1 is a block diagram showing the principle configuration of a multipoint conference system according to the present invention, FIGS. 2 and 3 are diagrams for explaining the system of the present invention, and FIG. 4 is an embodiment of the system of the present invention. FIG. 5 is a diagram showing an embodiment of a speaker detection unit used in the system of the present invention. FIGS. 6 and 7 show control algorithms of a control unit used in the center device of the system of the present invention. FIG. 8 is a flow chart, and FIG. 8 is a block diagram for explaining a conventional example. In FIG. 1, 1... Conference room terminal, 2... Center device, 3... Speaker detector, 4... Human camera, 5... Material camera, 6... Timer, 11. ……monitor. In the drawings, the same reference numerals indicate the same or corresponding parts.

Claims (3)

(57) [Claims] In a multipoint conference system in which a video conference is simultaneously performed between three or more conference room terminals (1) via a center device (2), each conference quality terminal (1) is connected to a participant. A speaker detection unit (3) for detecting a speaker corresponding to each microphone from the microphone group (11) provided is provided, and a person camera (4) is controlled toward a speaker whose speaker has been detected. An image of the person camera (4) is transmitted to the center device (2) together with the voice of the speaker and the speaker detection signal, and the center device (2) receives the speaker detection signal each time the speaker detection signal is received. A multipoint conference system characterized in that the conference room terminal according to (1) is used as a central conference room terminal, and the output image is held for a fixed time by a timer (6) and distributed to monitors (17) of other conference room terminals. 2. When the center device (2) receives a material image from the material camera (5), the conference room terminal that has output the material image is used as a central conference room terminal and the material image is transferred to another conference room. 2. The multipoint conference system according to claim 1, wherein the material image is distributed, and the material image is not switched even after receiving the speaker detection signal from another conference room terminal. 3. The center apparatus (2) outputs its own station material image and the speaker detection signal to the central conference room terminal (1) that has output the material image by the material camera (5). 3. The multipoint conference system according to claim 2, wherein switching is performed so as to synthesize and display a person image of the conference room terminal.