JPH08130723A - Video conference system speaker discrimination device - Google Patents

Abstract

PURPOSE: To display a conversation state on a television screen when two speakers speak and to advance a video conference provided with presence. CONSTITUTION: A conversation state detection part 62 compares voice level data 251 , 252 ,...25N. obtained from voice level detection parts 191 , 192 ,...19N corresponding to respective video conference terminals 111 , 112 ,...11N and extracts the two video conference terminals 11 indicating a maximum voice level and the next voice level. A speaker discrimination part 63 discriminates the two speakers based on them. When the two speakers are discriminated, a picture processing part bisects pictures for instance, displays the speakers and increases the presence of the conversation.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video conference system for realizing a video conference by connecting a plurality of points, and in particular, a video conference effective for distinguishing a speaker when a video conference is performed by connecting a plurality of points. The present invention relates to a system speaker discrimination device.

[0002]

2. Description of the Related Art Many companies and the like have come to employ a video conference system for conducting a video conference by connecting video conference terminals at a plurality of points using a public line or a dedicated line. When such a video conference is held at multiple points, it is convenient to display only the speaker who is making a statement on the television or a larger screen than those at other points. Therefore, by analyzing the signals sent from the voice lines of each speaker divided into these multiple points, the speaker is identified and only that speaker is distinguished from other conference participants as described above. It is designed to be highlighted.

In distinguishing the speaker from each TV participant, the sound level is measured for each TV terminal as disclosed in, for example, Japanese Patent Laid-Open No. 4-150590, and the background level is measured from here. It has been conventionally performed to eliminate the sound level, obtain each voice level, compare the obtained voice levels for each television terminal, and discriminate one speaker having the highest level. Then, the one speaker thus identified is emphasized on the television screen.

However, when the conference participant having the highest audio level among the television terminals is simply selected as the speaker and the speaker to be emphasized is switched at each time, for example, 1 of the conference participants is selected. When a person coughs, it causes inconvenience that the display on the screen changes.

Therefore, in Japanese Unexamined Patent Publication No. 2-4095, the presence or absence of sound is detected for each corresponding voice input, and even if the presence of sound is detected, it is not immediately determined to be the speaker and when this exceeds a certain time. It is decided to distinguish from the speaker. Further, with regard to a person who is once determined to be a speaker, even if silence is detected, the person is not immediately determined to be a non-speaker, and is determined to be a non-speaker only when the number of silences exceeds a certain value.

[0006]

As described above, in the conventional video conference system speaker discrimination device, the speaker discrimination is performed by comparing the magnitudes of the voice levels, so that the two parties are parallel to each other. When it became possible to continue speaking, there was a problem that if one of the speakers had a difference in voice level, only one speaker was identified as the speaker and only that person was highlighted on the TV screen. . In addition, if the voice level changes each time each of the two utterances changes, and the difference between the voice levels of the two changes or the larger one is replaced, the speaker cannot be accurately discriminated. Of course, in such a case where there are two speakers, the speaker on the side determined to be the non-speaker is not emphasized, and depending on the emphasizing mode, the person is not displayed on the TV screen. Such a situation also occurred.

Further, when a certain time interval is set to newly determine that the speaker is a speaker as in Japanese Patent Laid-Open No. 2-4095, the person who is identified as the speaker continues to speak. Sometimes, when someone else speaks in a way that interrupts, or when there are multiple speakers in addition to the one who is identified as the speaker, the new speaker can be accurately identified. There was a problem that I could not do it.

Therefore, an object of the present invention is to provide a video conference system speaker discrimination apparatus capable of accurately discriminating each speaker when a speaker interrupts or a plurality of speakers exist.

Another object of the present invention is to provide a teleconferencing system capable of providing a realistic teleconferencing by displaying the interactive state of these two speakers on a television screen when two speakers are speaking. It is to provide a speaker discrimination device.

[0010]

According to a first aspect of the present invention, (a) audio level detecting means for detecting the audio level of each video conference terminal participating in the video conference for each terminal,
(B) utterance state detecting means for detecting whether or not the speaker is speaking at each video conference terminal in a predetermined cycle unit,
(C) A speaker discrimination unit that discriminates the speaker for each cycle described above based on the voice level detected by the voice level detection unit, and (d) a detection result of the speech state detection unit and a speaker discrimination unit. Speaker change situation determination means for determining the change situation of the speaker using the determination result, and (e) When this speaker change situation determination means detects the change of the speaker, the above-mentioned cycle is set short and the change is performed. The video conference system speaker discrimination device is provided with a cycle changing means for shortening this cycle when the section up to is long.

That is, according to the first aspect of the present invention, the change situation of the speaker is determined when the speaker is speaking for a long time by one speaker, or when another speaker starts to speak in a way that interrupts. ,
In response to this, by changing the period for distinguishing the speaker, it is possible to prevent the speaker from being mistaken for the unnecessary voice, while the speaker interrupts or multiple speakers When present, each speaker can be accurately discriminated.

According to the second aspect of the invention, (a) a voice level detecting means for detecting the voice level of each video conference terminal participating in the video conference for each terminal, and (b) the voice level detecting means. A voice level comparing means for comparing the maximum voice level and the next highest voice level for each terminal, and (c) When the difference between the voice levels compared by this voice level comparing means is within a predetermined range, the two parties can talk with each other. A dialogue discriminating means for discriminating that the conversation is being conducted, and (d) a display emphasizing means for emphasizing the two speakers discriminated that the dialogue is discriminated by the screen display as compared with other video conference participants. And the video conference system speaker discrimination device.

That is, according to the second aspect of the present invention, it is determined whether or not there are two speakers who are speaking by extracting the one with the maximum voice level and the one with the next voice level and comparing the difference between these levels. I decided to do it. Then, when there are two speakers, these persons are highlighted on the screen to enhance the realism of the dialogue.

According to the third aspect of the invention, the period as a unit for switching the screen display is changed by detecting the utterance state of the speaker, and the invention according to the second aspect is modified. I decided to incorporate the advantages.

In the invention according to claim 4, (a) voice level detecting means for detecting the voice level of each video conference terminal participating in the video conference for each terminal, and (b) the voice level detecting means. A plurality of speaker extracting means for extracting one having a maximum voice level for each terminal and one or a plurality of voices having a relatively high voice level similar to the maximum voice level;
(C) The video conference system speaker discrimination device is provided with a display emphasizing unit for emphasizing the plural speakers extracted by the plural speaker extracting units on the screen display as compared with other video conference participants.

That is, according to the invention of claim 4, the speaker of the terminal having the maximum voice level and the speakers of one or a plurality of terminals close to the terminal are simultaneously extracted and the image is emphasized. 3 like a crosstalk show
A state in which more than one person speaks incandescently can be realized as a display form on the screen even in a video conference.

According to the fifth aspect of the present invention, the utterance state detection means, when the utterance of the speaker is divided into all the intervals in the cycle, when the sum of the intervals occupies most of the cycle,
When speaking at the end of the cycle, it is assumed that speaking is done in that cycle, which makes it possible to effectively identify the speaker even if the cycle is relatively long. Become.

[0018]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below with reference to embodiments.

First embodiment

FIG. 1 is a block diagram of a first embodiment of the present invention.
Levi conferencing system A video conferencing system using a speaker identification device.
This is an overview of the stem. Tokyo headquarters and Nago
Stores located at various locations such as shops or branch offices in Osaka.
Levi conference terminal 11₁, 11₂, …… 11_NIs a video line
12 such as voice and voice lines₁, 12₂, …… 12_NThrough
Video conferencing system Communication interface of speaker identification device
Connected to the switch 13. From communication interface 13
Is each video conference terminal 11₁, 11₂, …… 11 _NFor each
Voice information 14₁, 14₂, ...... 14_NIs taken out,
The corresponding voice level measuring unit 15₁, 15₂, ……
Fifteen_NEntered in. Voice information here 14₁, 14₂,
...... 14_NIs each video conference terminal 11₁, 11₂, …… 1
1_NOf the sound information detected by a microphone (not shown)
Yes, including voice and other background sounds
There is.

Voice level measuring section 15 ₁ , 15 ₂ , ... 1
5 _N receives the measurement cycle control signal 17 sent from the audio level measurement cycle control unit 16 and analyzes the measurement cycle for analyzing the audio information 14 ₁ , 14 ₂ , ... 14 _N of the corresponding video conference terminal. Increase or decrease by. Then, these results are used as voice level measurement data 18 ₁ , 18 ₂ , ... 18 _N , and voice level detecting units 19 ₁ , 19 arranged corresponding to the respective video conference terminals 11 ₁ , 11 ₂ , ... 11 _{N. 2} ,
...... 19 _N and background level detectors 21 ₁ and 2
1 ₂ , ... 21 _N and speech state detection unit 22 ₁ , 2
2 ₂ , ... Supply to 22 _N. Background level detection unit 21 _1, 21 _2, ...... 21 _N detects the background level of the sounds other than speech, the background level measurement data 23 _1, 23 _2, the sound level detecting unit ...... 23 _N 19 _1, 19 _2, he said ...... 19 _N state detection unit 22 _1, 22 _2, and supplies the ...... 22 _N.

Voice level detector 19 ₁ , 19 ₂ , ... 1
9 _N is the input voice level measurement data 18 ₁ , 18
_2, ...... 18 _N and the background level measured data 2
The difference between the measured data of each pair of 3 ₁ , 23 ₂ , ... 23 _N is detected, and the voice level from which the influence of the background level is eliminated is detected. The voice level data 25 ₁ , 25 ₂ , ... 25 _N representing these voice levels are
It is sent to the speaker discrimination unit 26 which discriminates the speaker. The method of discriminating the speaker will be described later.

The speaker discrimination result is the speaker discrimination result data 27.
Is output as The speaker discrimination result data 27 is sent to the image processing unit 28, and image processing for emphasizing the discriminated one speaker is performed. Video signal 29 after this image processing
From the communication interface 13 to each video conference terminal 1
11 ₁ , 11 ₂ , ... 11 _N , and the image is displayed on each TV screen.

On the other hand, the speech state detection units 22 ₁ , 22 ₂ , ...
22 _N is the input voice level measurement data 18 ₁ ,
18 ₂ , ... 18 _N and background level measurement data 23 ₁ , 23 ₂ , ... 23 _N are used to measure the presence / absence of a speech state of each video conference terminal 11 ₁ , 11 ₂ , ... 11 _N. Detect each. Specifically, the rate of change of each of the audio level measurement data 18 ₁ , 18 ₂ , ... 18 _N is measured, and the individual video conference terminals 11 ₁ , 11 ₂ ,.
The presence or absence of a utterance is detected for each cycle by detecting the rising and falling of the voice level for each 11 _N and determining the start state and end state of the utterance.

Here, each voice level measurement data 18 ₁ ,
18 ₂ , ・ ・ ・ 18 _N , when no change is observed, the voice level measurement data 18 ₁ , 18 ₂ , ...... 1
8 _N and background level measurement data 23 ₁ , 23
₂ , ... 23 _N are compared, and it is determined that the video conference terminal 11 having a level difference within one cycle has a speaking state, and if there is no level difference, it is determined that a speaking is not being performed. .

When one TV conference terminal 11 detects rising and falling edges of a plurality of speech states within this period, it is determined whether the integrated value of the time during speech in one period exceeds a predetermined threshold value. Make a distinction. As a result, when the threshold value is exceeded, it is determined that the utterance was made in that cycle. If it does not exceed the threshold value, it may be judged as non-speaking, but considering the case where the section of the speech spans the boundary of the cycle, if the speech state is at the timing of switching the cycle, the cycle It is determined that the statement was made in.

In this way, each speech state detection unit 22 ₁ ,
The speech result data 31 ₁ , 31 ₂ , ... 31 _N detected by 22 ₂ ... 22 _N for each cycle are commonly input to the voice level measurement cycle control unit 16. On the other hand, the speaker discrimination unit 26 detects one or a plurality of video conference terminals 11 that are making a speech by using the speech result data 31 ₁ , 31 ₂ , ... 31 _N obtained from the voice level measurement cycle control unit 16.
Next, the voice level data 25 are compared between the talking videoconference terminals 11 to determine the maximum one as the speaker. It has already been described that the speaker discrimination result is output as the speaker discrimination result data 27.

Further, the voice level measurement cycle control unit 16 is
The speaker discrimination data is acquired from the speaker discrimination unit 26 and stored, and it is discriminated whether the speakers are the same or change in a cycle unit. As a result, (a) when it is determined that the speaker changes, control is performed by the measurement cycle control signal 17 so that the measurement cycle of the voice level is shorter than the current value by a predetermined time. The history may be set to be slightly longer, the frequency at which the speaker changes is calculated, and the frequency information may be used to control the value for shortening the cycle. In addition, ROM (read only
Values for such addition and subtraction may be stored in a memory) as a table, and one cycle interval may be set while reading the addition and subtraction values according to the situation.

(B) When it is determined that the number of speakers is constant, it is determined whether or not there is any speaker other than the speaker. This determination is to determine the presence / absence of a person other than the speaker by calculating the number of data items in the starting and speaking states from the remaining speech result data after deleting the speech result data determined to be the speaker. Done by. As a result, when it is determined that “there is a speaker in addition to the speaker”, the measurement period control signal 17 for shortening the measurement period of the voice level is output to the voice level measuring units 15 ₁ , 15 ₂ , ......
Send to 15 _N. At this time, the larger the number of data, that is, the number of speakers, the shorter the measurement cycle.

On the other hand, when it is determined that "there is no speaker other than the speaker", a measurement cycle control signal 17 for lengthening the measurement cycle of the voice level is provided to each voice level measuring section 15 _1. , 15 ₂ , ... 15 _N.
Note that this state means that the same speaker continues as a speaker. Therefore, the speaker discrimination result data 27
The time as a speaker is determined from the history of, and the longer the time is, the longer the measurement cycle is. The measurement cycle is set with reference to a scheduled break time of one speaker.

In this way, each video conference terminal 1
1 ₁ , 11 ₂ , ... 11 _N identified as speaker 1
The human speaker is emphasized and displayed on the screen, and even when the speaker change occurs, the speaker highlighted on the screen is appropriately changed. As a method of emphasizing the speaker, a method of displaying only the person who is determined to be the speaker on the entire screen, or a small screen for all speakers is set, and the screen of the person who is determined to be the speaker is It is possible to use various known methods such as a method of enlarging the screen of the person larger than the screen of the person or a method of changing the frame of the screen of the person who is determined to be the speaker to a color different from the frame of the screen of another person. it can.

Second embodiment

FIG. 2 shows an outline of a video conference system using a video conference system speaker discrimination apparatus according to the second embodiment of the present invention. The same parts as those in FIG. 1 are designated by the same reference numerals, and the description thereof will be appropriately omitted. From the communication interface 13, each video conference terminal 1
11 ₁ , 11 ₂ , ... 11 _N voice information for each 14 ₁ , 14
₂ , ... 14 _N are taken out and input to the corresponding voice level measuring units 61 ₁ , 61 ₂ , ... 61 _N.
Here, the voice information 14 ₁ , 14 ₂ , ... 14 _N is the information of the sound detected by the microphone (not shown) of each video conference terminal 11 ₁ , 11 ₂ , ... 11 _N. It also includes sound.

Voice level measuring unit 61₁, 61₂, …… 6
1_NSound level measurement data measured by 18₁, 18₂,
…… 18_NIs each video conference terminal 11₁, 11₂, ……
11 _NAudio level detection unit 19 arranged corresponding to₁,
19₂, …… 19_NAnd background level detector 2
1₁, 21₂, …… 21_NAnd speech state detection unit 2
2 ₁, 22₂, …… 22_NIs supplied to. Voice level detection
Debu 19₁, 19₂, …… 19_NAre respectively entered
Voice level measurement data 18₁, 18₂, …… 18 _NWhen
Background level measurement data 23₁, 23₂, ...
… 23_NTo remove each of the background levels.
Voice level data 2 representing the removed voice level
5₁, 25₂, ...... 25_NCreate and interact these with
It is supplied to the detection unit 62 and the speaker discrimination unit 63. Speaker discrimination unit 6
3 is the dialogue state data 64 detected by the dialogue state detection unit 62.
And each voice level data 25₁, 25₂, ...... 25_NFor
Discriminate one or more speakers and judge the result
The data 65 is sent to the image processing unit 66. Image processing
The section 66 emphasizes the one or more determined speakers.
Image processing is performed. Video signal 67 after this image processing
From the communication interface 13 to each video conference terminal 1
1₁, 11₂, …… 11_NSent to each tele
The video will be displayed on the screen.

In such a television conference system speaker discrimination device, the dialogue state detection unit 62 uses the voice level data 2
5 _1, 25 _2, enter the ...... 25 _N, sound level measurement unit 61 _1, 61 _2, each of the integral of the sound level at each measuring period ...... 61 _N to measure the television conference terminal 1
It is calculated for each of 1 ₁ , 11 ₂ , ... 11 _N. Then, two types of the maximum voice level as the integrated value and the next voice level are extracted.

However, depending on the device, when there are two or more voice levels that are close to the maximum voice level to some extent, they may be extracted together. This is because, when the measurement cycle is set to be relatively long, a phenomenon may occur in which three or more persons have conversations in rapid succession during one cycle. In such a case, the measurement period control signal 17 is sent from the conversation state detecting section 62 to each of the voice level measuring sections 61 ₁ , 61 ₂ , ... 6 for the same purpose as in the first embodiment.
It is also effective to control the measurement cycle by adopting a configuration for sending to 1 _N. In the present embodiment, in order to simplify the description, the description will be limited to a dialogue between two parties.

The dialogue state detection unit 62 compares the integrated values of the voice levels of the two persons, and when they are a small difference within the range of a predetermined value, the video conference currently being conducted is in the dialogue state. Is determined. In other cases, that is, when there is a large difference, it is determined that the current video conference is in the non-interactive state. Further, the dialogue state detection unit 62 similarly determines the non-dialogue state when there is one speaker or when there is no speaker.

Dialog state data 6 representing the presence or absence of a dialogue state
4 is each voice level data 25 ₁ , 25 ₂ , ... 25 _N
It is also input to the speaker discrimination unit 63. When the conversation state data 64 indicates the non-conversation state, the speaker discrimination unit 63 determines the respective voice level data 25 ₁ , 25 ₂ , ... 25 _N.
And the video conference terminal 11 having the highest audio level is determined as the "speaker A pattern".

Further, when the conversation state data 64 indicates the conversation state, the one of the video conference terminal 11 which similarly represents the maximum voice level is discriminated as the "speaker A pattern" and is in the conversation state. The one of the video conference terminal 11 showing the next highest voice level as another speaker is discriminated as the "speaker B pattern".
These determination results are sent to the image processing unit 66.

The image processing section 66 has a CPU (central processing unit) (not shown), and also a ROM (not shown).
A control program stored in the (read only memory) is used to perform screen enhancement processing according to the presence / absence of a conversation state and the presence / absence of a speaker.

FIG. 3 shows how the image processing section is controlled. The image processing unit 66 uses the speaker discrimination data 65.
Is received and it is checked whether or not a speaker is present (step S101). The speaker exists when the "speaker A pattern" exists. When no speaker is present (N), video processing is performed such that the video of all participants of the video conference is divided into small screens and displayed (step S102), and this video signal 67 is sent to the communication interface 1
Each video conference terminal 11 ₁ , 11 ₂ , ... 11 via 3
Send to _N.

When the speaker exists and the "speaker B pattern" exists (step S103; Y), the screen is changed to 2
A video signal 67 that divides and displays both speaker A and speaker B is created (step S104), and this is sent to the communication interface 13. If the speaker B does not exist (step S103; N), a video signal 67 for displaying only the speaker A on the entire screen is created (step S105) and sent to the communication interface 13. become.

The above screen display mode is checked and switched at each measurement cycle. There may be various emphasis modes other than this. For example, the screen of all participants is displayed in a small frame, and only the person or the speaker who is having a dialogue is displayed in a frame of a color different from the others,
The frames of these persons may be displayed relatively larger than the frames of other persons.

Although the first and second embodiments have been described above, the present invention is not limited to these. For example, it is possible to realize a video conference system speaker discrimination device based on a dialogue by incorporating the voice level measurement cycle control unit for changing the measurement cycle of the first embodiment into the second embodiment. is there.

[0045]

As described above, according to the first aspect of the invention, the speaker when one speaker speaks for a long time, or when another speaker starts to speak in a way that interrupts the talk, etc. By distinguishing the change situation of the speaker and changing the period for distinguishing the speaker accordingly, it is possible to prevent false recognition of the change of the speaker for unnecessary voice, while When there is an interruption or a plurality of speakers, each speaker can be accurately discriminated.

According to the second aspect of the present invention, it is determined whether or not there are two speakers who are speaking by extracting the one with the maximum voice level and the one with the next voice level and comparing the difference between these levels. Since it is decided to make a distinction, when there are two speakers, these persons can be highlighted on the screen to enhance the realism of the dialogue.

Furthermore, in the invention according to claim 3, since the period as a unit for switching the screen display is changed by detecting the utterance state of the speaker, the talk is performed even when the image is displayed in the interactive mode. The person can be appropriately identified.

In the invention according to claim 4, since the speaker of the terminal having the highest voice level and the speaker of one or a plurality of terminals similar thereto are simultaneously extracted and the image is highlighted, three or more persons are selected. It is possible to realize a state in which a person speaks incandescently as a display form on the screen even in a video conference.

Further, in the invention according to claim 5, when the utterance of the speaker is divided into all the intervals in the cycle and the sum of the intervals occupies most of the cycle, Then, it was decided to pretend that speech was being made in that cycle when speech was being made at the end of the cycle, so it is possible to effectively identify the speaker even when the set cycle is relatively long. There is an effect that it becomes possible.

[Brief description of drawings]

FIG. 1 is a block diagram showing a video conference system speaker identification device according to a first embodiment of the present invention and respective video conference terminals connected thereto.

FIG. 2 is a block diagram showing a video conference system speaker identification device according to a second embodiment of the present invention and respective video conference terminals connected thereto.

FIG. 3 is a flowchart showing how the image processing unit in the second embodiment of the present invention is controlled.

[Explanation of symbols]

 11 video conference terminal 15, 61 voice level measuring unit 16 voice level measuring cycle control unit 19 voice level detecting unit 21 background level detecting unit 22 speech state detecting unit 25 voice level data 26, 63 speaker discriminating unit 28, 66 image processing Part 62 Dialog state detecting part

Claims (5)

[Claims] 1. A voice level detecting means for detecting a voice level of each video conference terminal participating in a video conference for each terminal, and detecting whether or not a speaker is speaking at each video conference terminal in a predetermined cycle unit. Utterance state detecting means, a speaker deciding means for deciding a speaker for each cycle based on the voice level detected by the voice level detecting means, a detection result of the utterance state detecting means and a speaker deciding means. Speaker change situation determination means for determining the change situation of the speaker using the determination result of the means, and when this speaker change situation determination means detects the change of the speaker, the cycle is set to be short and A teleconferencing system speaker discrimination apparatus comprising: a period changing means for shortening this period when a section is long. 2. A voice level detecting means for detecting the voice level of each video conference terminal participating in the video conference for each terminal, the maximum voice level of each terminal detected by the voice level detecting means, and next. A voice level comparing means for comparing a large one, a dialogue discriminating means for discriminating that both parties are having a dialogue when the difference between the voice levels compared by the voice level comparing means is within a predetermined range, and the dialogue discriminating means. A video conference system speaker discrimination device comprising: a display emphasizing unit for emphasizing the two speakers who have discriminated to each other on the screen display as compared with other video conference participants. 3. The speaker determining apparatus according to claim 2, wherein the speaking state of the speaker is detected and the period as a unit for switching the screen display is varied. 4. A voice level detecting means for detecting the voice level of each video conference terminal participating in a video conference for each terminal, and the maximum voice level of each terminal detected by the voice level detecting means and Speaker multiple extraction means for extracting one or a plurality of similar relatively high voice levels, and a plurality of speakers extracted by the speaker multiple extraction means are emphasized on the screen display as compared with other video conference participants. A video conference system speaker identification device, comprising: 5. The utterance state detection means makes a statement when a speaker's utterance is divided into all sections within a cycle, when the sum of the sections occupies most of the cycle, and at the end of the cycle. 2. The video conference system speaker discrimination device according to claim 1, wherein the video conferencing system speaker discrimination device imitates that a speech is being made in that cycle.