JP2000236330A - Audio distribution method and apparatus for client-server type digital audio conference system - Google Patents

Abstract

(57) [Problem] In a digital audio conference system, a digital audio distribution system that prevents delays in digital audio distribution and does not hinder the flow of conversation, and that is used for virtual space communication when used for virtual space communication. Provided is a sound distribution method and apparatus for giving a sound image impression according to the positional relationship of avatars in space. SOLUTION: When a digital audio conference server 1 receives audio data from a plurality of clients 21 to 2n at the same time, a priority list in which each client has notified the server and a voice of the client which has notified the server from each client. The number of audio data to be distributed to each client is determined according to the data receiving ability. Further, when utilizing for virtual space communication, a priority list notified by each client to the server is determined based on the positional relationship between the own avatar and other avatars in the virtual space.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a digital audio distribution method and apparatus for use in a client-server type digital audio communication conference where a server distributes digital audio received from a plurality of clients to a plurality of points. It is.

[0002]

2. Description of the Related Art Conventional digital audio distribution methods are roughly divided into the following two methods.

The first method is a method in which digital audio data transmitted from a plurality of clients to a server is mixed by the server, and the mixed digital audio data is transmitted to each client.

[0004] A second method is to assign a transmission right to a specific client by exchanging control messages, and the server distributes voice data received from the client to each client.

[0005]

Among the conventional digital audio distribution methods, in the method of performing mixing, even if audio data is received simultaneously from a plurality of clients, they are mixed and distributed to each client. A natural conversation becomes possible in a digital voice communication conference using. Further, unlike the method of assigning the transmission right, there is an advantage that the flow of the conversation is not hindered because the exchange of the control message is not required.

However, in this method, since processing for mixing digital audio data received from a plurality of clients occurs, a delay occurs in the distribution of digital audio, and a server requires mixing equipment. There was a problem that.

[0007] When this method is used in a virtual space communication system in which a plurality of participants participate and communicate through an avatar, a participant receiving the same mixed voice data must be in the virtual space of the avatar. The inconvenience is that even if the client has stereo speakers, the voice of another participant on his / her right side in the virtual space may not always be heard from the right side, regardless of the position of the participant. In order to solve this problem, it is necessary to mix audio data individually for each participant, and there is a problem that a processing amount of a server required for mixing increases.

On the other hand, among the conventional digital audio distribution methods,
The method of assigning a transmission right to a specific client by exchanging control messages has the advantage that there is no delay in digital voice distribution and no mixing equipment is required.

However, in this method, since a control message needs to be exchanged in order to determine which of a plurality of clients is to be given a transmission right, a multipoint digital voice communication conference using this requires a user. However, it is necessary to perform some operation for acquiring the transmission right prior to making a comment, and there is a problem that the flow of conversation is hindered.

In addition, since only voice from a specific client having transmission right is generally sent to all clients, it is suitable for use in a conference or the like where the chairman exists, but is not suitable for free discussion. there were.

SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and provides digital voice distribution that does not hinder the flow of conversation while preventing delay in digital voice distribution, and uses the present invention for virtual space communication. In this case, it is an object of the present invention to provide an audio distribution method and apparatus for a digital audio conference system that can give a sound image impression according to the positional relationship of an avatar in a virtual space of each participant.

[0012]

In order to solve such a problem, the invention according to the first aspect of the present invention is directed to a case where, when a server receives voice data from a plurality of clients at the same time, voice data to be distributed to each client is transmitted. Is determined based on the priority list notified to the server. According to a second aspect of the present invention, in the first aspect, when the server receives voice data from a plurality of clients at the same time, the voice distributed to each client according to the voice data receiving capability of the client notified from each client to the server. The number of data is determined. The invention of claim 3 is claim 1
In this case, the priority list notified by each client to the server is determined based on the positional relationship between the own avatar and another avatar in the virtual space.

In the digital audio distribution method according to the present invention,
Since no mixing processing is performed in the server, there is no delay in digital audio distribution.

According to the first aspect of the present invention, in the digital audio distribution method according to the present invention, when the server receives audio data from a plurality of clients at the same time, the audio data to be transmitted according to a priority list notified in advance from each client. Does not require the exchange of control messages. Thus, in the multipoint digital voice communication conference using the present invention, the user does not need to perform an operation for acquiring the transmission right before speaking.

According to the first aspect of the present invention, since each client individually creates a priority list, in a multi-person communication conference, some members form a subgroup by changing the priority list and are members of the subgroup. , And various communication settings such as setting a high priority for a special participant such as a chairman or a teacher.

According to a second aspect of the present invention, in the digital audio distribution method according to the present invention, the server determines the number of audio data to be distributed to each client according to the audio data receiving capability of the client. In other words, when a plurality of participants speak at the same time, only one voice reaches a client with a small line capacity or voice data processing ability, and a large number of clients, such as two or three, according to the ability. You ’ll receive voices for each person, and if your client ’s ability is high,
The control message of the conventional method enables a more natural conversation as compared with the case where only one voice arrives.

According to a third aspect of the present invention, in the digital voice distribution method using the present invention for virtual space communication, the priority of voice data distributed to each client is such that a participant having a closer avatar position in the virtual space. As a result, when a plurality of participants speak at the same time, each participant is given priority from the voice of the nearby participant, and the spatial sound image impression is not spoiled.

Also, unlike the case of performing mixing by the server of the conventional method, since the voice of each participant is individually delivered to each client, the voice data received by the client is adjusted according to the positional relationship of the avatar. By performing the sound image localization processing, a sense of reality in a virtual space can be enhanced.

The digital audio conference server of the client-server type digital audio conference system of the present invention comprises a plurality of clients which are terminal devices provided with audio input / output means such as microphones and speakers, and a server which is a centralized device. The client digitizes the voice input from the voice input / output means and transmits it to the server, and the server including a buffer for receiving voice data from the client and a buffer for transmitting voice data to the client receives from each client. In a client-server type digital audio conference system for realizing a conversation of many people by distributing voice data to a plurality of clients, a storage means for storing the priority of another client with respect to the client, and distributing the received voice data. Decide which client to voice Characterized in that it has a digital audio selective distribution process of making the distribution of over data.

The storage means further stores a digital audio receiving capability obtained by integrating the line capacity and digital audio processing capability of the client.

Further, the digital voice selection and distribution process includes:
A priority data processing unit for storing the priority or the priority and the digital voice receiving capability in the storage means; a message handler for distributing the data in the reception buffer to the voice data distribution unit and the priority data processing unit; And an audio data distribution unit for distributing the audio data to a transmission buffer with reference to the priority of each client or the priority of each client and the digital audio receiving capability.

The digital audio conference terminal of the client-server type digital audio conference system of the present invention has a microphone
An audio input / output unit such as a speaker, an encoder that digitizes and compresses an audio signal input from the audio input unit, a transmission buffer that sends encoded audio data to a server, and a reception buffer that receives audio data from the server. And a decoder for decompressing the received voice data, and a priority notification unit for notifying the digital voice conference server of the priority of the other client with respect to the client and the receiving capability of the client.

Further, in the case of the terminal device of the virtual space communication system, the digital audio conference terminal further adds the voice data expanded by the decoder according to the positional relationship between the avatar of the caller and the avatar of the user in the virtual space. And a sound image localization processing unit for performing sound image localization processing.

Further, the digital audio conference terminal allows the client to join and leave the digital audio conference,
The virtual space communication system has a communication conference control unit for controlling the position of each participant's avatar in the virtual space.

[0025]

Next, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of the digital audio conference system of the present invention. As shown in the figure, a digital audio conference system according to the present invention includes a digital audio conference server 1 that distributes audio from a client to another client with reference to a priority specified by the client, and a plurality of digital audio conference terminals (clients). ) 21 to 2n
And a communication conference control server (process) 3.

FIGS. 2 to 5 show components of the digital audio conference system of the present invention. FIG. 2 is a block diagram showing an embodiment of the digital audio conference server, and FIG. 3 shows another embodiment of the digital audio conference server. FIG. 4 is a block diagram showing one embodiment of the digital audio conference terminal, and FIG. 5 is a block diagram showing another embodiment of the digital audio conference terminal.

The digital audio conference server 1 shown in FIG.
Receiving buffers 111 to 11n for temporarily receiving digital audio data from a plurality of clients 21 to 2n; a digital audio selecting and distributing process 14 for deciding to which client the digital audio data of the receiving buffers 111 to 11n is to be distributed and performing distribution; A priority list 131 to 13n for each client used to determine which audio data is distributed to which client;
2 to 2n.

The digital voice selection / distribution process 14 includes a priority data processing unit 142 for storing the priority list passed from the message handler 141 in the priority lists 131 to 13n, and a client for transmitting the voice data passed from the message handler 141 to the client. Priority list 13 for each
An audio data distribution unit 143 that distributes the data received by the reception buffers 111 to 11n to the audio data distribution unit 143 and the priority data processing unit 142 by referring to 1 to 13n.
41.

Next, the digital audio conference server 1 shown in FIG. 3 includes reception buffers 111 to 11n for temporarily receiving digital audio data from a plurality of clients 21 to 2n.
A digital audio selection / distribution process 14 for deciding to which client the digital audio data of the reception buffers 111 to 11n is to be distributed and performing distribution, and a priority for each client used for determining which audio data to distribute to which client The order lists 131 to 13n and the clients 21 to 21 which are also used to determine the distribution to the clients
2n client voice data receiving capability memories 151 to 15n storing the voice data receiving capability notified from 2n;
Transmission buffer 1 for outputting to clients 21 to 2n
21 to 12n.

The digital voice selection / distribution process 14 compares the priority list passed from the message handler 141 and the receiving capability of the client with the priority lists 131 to 13.
n and client voice data receiving capability memory 151
15n to the transmission buffer with reference to the priority list 131 to 13n for each client and the client voice data reception capability memories 151 to 15n. Distributing audio data distribution unit 1
43, the data received by the reception buffers 111 to 11n, and the audio data distribution unit 143 and the priority data processing unit 1
42 and a message handler 141 for distributing the message.

The digital audio conference terminal shown in FIG.
Includes an encoder 212 for digitizing and compressing the input audio signal, a transmission buffer 213 for transmitting the encoded audio data to the server, a reception buffer 214 for receiving the audio data from the server, and expanding the received audio data. , A decoder 215 for outputting as a sound, a priority notification unit 219 for notifying the digital audio conference server 1 of the reception capability and priority of the client, a client participating in the digital audio conference, leaving the conference, and virtual space communication. The system includes a communication conference control unit 210 that controls the position of each participant in the virtual space of the avatar, and a communication control unit 218.

The digital audio conference terminal shown in FIG.
Includes an encoder 212 for digitizing and compressing the input audio signal, a transmission buffer 213 for transmitting the encoded audio data to the server, a reception buffer 214 for receiving the audio data from the server, and expanding the received audio data. , A decoder 215 for outputting a sound, a priority notification unit 219 for notifying the digital audio conference server 1 of the receiving capability and priority of the client, a client participating in the digital audio conference, leaving the conference, and each participant. The communication conference control unit 210 for controlling the position of the avatar in the virtual space, the communication control unit 218, and in the case of the terminal device of the virtual space communication system, the voice data expanded by the decoder 215, Performs sound image localization processing according to the positional relationship between the user and its avatar in the virtual space Image localization processing unit 216 is also provided.

The terminal devices 22 to 2n are also digital audio conference terminal devices similar to the terminal device 21.

The communication conference control server (process) 3 shown in FIG. 1 controls the participation of clients 21 to 2n in a communication conference and the withdrawal from the conference. In the virtual space communication system of the present invention, the position of each participant's avatar in the virtual space is also controlled.
The communication conference control process 3 may be a software process in the same device as the digital audio conference server 1, or may be another device. The internal configuration is not described because it does not relate to the essence of the present invention.

Next, the operation of the embodiment of the digital audio conference system of the present invention will be described with reference to FIGS. FIG. 6 is a flowchart of the digital audio conference server, and FIG. 7 is a flowchart of the digital audio conference terminal (client).

First, the operation of the digital audio conference server will be described. In the audio conference server 1, the digital audio selection / distribution process 14 monitors the reception buffers 111 to 11n (S101), and if the data received from the clients 21 to 2n by the message handler 141 is a priority list or audio data reception capability, priority is given. The data is passed to the rank data processing unit 142, and if the received data is voice data, it is passed to the voice data distribution unit 143 (S102). The priority data processing unit 142 stores the priority list and the voice data receiving capability passed from the message handler 141 in the priority lists 131 to 13n and the client voice data receiving capability memories 151 to 15n, respectively (S103, S104).

When only the audio data of one buffer is passed from the message handler 141 (S105), the audio data distribution unit 143 distributes the data to the transmission buffers 121 to 12n (S106). When the audio data data of the plurality of reception buffers is transferred at the same time, the client audio data reception capability memories 151 to 1 are output.
The voice data to be distributed to each client is determined from the values stored in 5n and the priority lists 131 to 13n (S107), and distributed to the clients 21 to 2n through the transmission buffers 121 to 12n (S108). For example, if the value of the client voice data receiving capability memory 151 is 2
Then, of the receiving buffers 111 to 11 n, the one receiving the audio data is referred to the priority list 131, and the audio data of the two receiving buffers having the highest priority are distributed to the client 21 through the transmission buffer 121.

Next, the operation of the client will be described. The clients 21 to 2n communicate with the communication control process 3 to obtain information on other clients participating in the conference, create a priority list of the other clients, and transmit the priority list to the audio conference server 1 (S201). . Also,
Clients 21 to 2n are communication conference control units 210 to 2n
It determines the voice data receiving capability of its own terminal based on 0, and transmits it to the voice conference server 1 (S202). In the virtual space communication system, the position of each participant's avatar in the virtual space is controlled (S20).
3).

The clients 21 to 2n input voice through voice input means such as a microphone (S204). The voice input by the user is digitized and compressed by the encoders 212 to 2n2 (S205).
Through 2n3, the reception buffer 11 of the audio conference server 1
The data is transmitted to the addresses 1 to 1n1 (S206). The clients 21 to 2n receive the audio data from the audio conference server 1 in the reception buffers 214 to 2n4 (S207),
It is expanded by decoders 215-2n5 (S208),
The sound is output to the user by the sound output means 217 to 2n7 (S209). The client performs the above five operations asynchronously.

When the digital voice distribution method of the present invention is applied to a virtual space communication system (S21)
0), before the audio data expanded by the decoders 215 to 2n5 is output by the audio output means 217 to 2n7, the sound image localization units 216 to 2n6 use the audio data localization unit 216 to 2n6 to obtain the audio data from the communication conference control process 3 A sound image localization process based on the avatar's positional relationship is performed (S211).

[0042]

As described above, in the digital voice distribution method according to the present invention, when the server receives voice data from a plurality of clients simultaneously, the server receives the voice data from the clients in advance based on the priorities notified from each client. Determines the audio data to be transmitted to each client, and distributes the audio data without adding a mixing process. Therefore, there is an effect that it is possible to prevent an increase in the amount of processing required for mixing and a delay in digital audio distribution, which are problems in the conventional method using mixing. In addition, there is an effect that the problem that the flow of a natural conversation is hindered, which has been a problem in the conventional method using control message exchange, is solved.

When the present invention is used for virtual space communication, the voice of another person reaches each client without being mixed, so that sound image localization processing reflecting the positional relationship of avatars in the virtual space is possible. Thus, it is possible to enhance the sense of reality in the virtual space. In addition, since the voice of another person is preferentially received from a close person in the virtual space, there is an effect that even when many participants utter simultaneously, the conversation with the nearby participant is not hindered.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a digital audio conference system of the present invention.

FIG. 2 is a configuration diagram illustrating an embodiment of a digital audio conference server.

FIG. 3 is a configuration diagram showing another embodiment of a digital audio conference server.

FIG. 4 is a configuration diagram showing one embodiment of a digital audio conference terminal.

FIG. 5 is a configuration diagram showing another embodiment of the digital audio conference terminal.

FIG. 6 is a flowchart of the digital audio conference server.

FIG. 7 is a flowchart of a digital audio conference terminal (client).

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 Digital audio conference server 111-11n Receiving buffer 121-12n Transmission buffer 131-13n Priority list 14 Digital audio selective distribution process 141 Message handler 142 Priority data processing unit 143 Audio data distribution unit 151-15n Client audio data receiving capacity memory 21 to 2n client 210 communication conference control unit 211 audio input means 212 encoder 213 transmission buffer 214 reception buffer 215 decoder 216 sound image localization processing unit 217 audio output means 218 communication control unit 219 priority notification unit 3 communication conference control process

Continued on the front page F term (reference) 5C064 AA06 AC06 AC16 AC22 AD06 5D062 CC13 5K015 AA00 AB00 AB01 AB02 JA00 JA01 JA05 JA10 JA11 5K030 GA02 GA18 HA07 HB01 HB17 JT10 KA01 KA03 KA06 KA19 LA07 LD08 LE05

Claims (8)

[Claims] 1. A plurality of clients, which are terminal devices provided with voice input / output means such as microphones and speakers, and a server, which is a centralized device, are connected by a communication line, and the client digitally inputs voice input from the voice input / output means. In a voice distribution method in a client-server type digital voice conference system that realizes conversation of many people by distributing voice data received from each client to a plurality of clients by transmitting to a server, A client-server type digital device, comprising: a step of notifying a server of a priority order of another client with respect to its own client; and a step of determining sound data to be transmitted to each client based on the priority order. A voice distribution method for a voice conference system. 2. The step of notifying the server of the priority order comprises: a step in which each client gives the server a digital audio reception capability based on the line capacity and digital audio processing capability of its own client in addition to the priority order of other clients. 2. The server according to claim 1, wherein the server determines audio data to be transmitted to each client based on the priority of each client, with the digital audio reception capability of each client as an upper limit. Audio distribution method for a client-server type digital audio conference system of the present invention. 3. The step of notifying the server of the priority order includes: when using a virtual space communication system in which a plurality of participants participate in a virtual space through an avatar and communicate with each other, according to a distance between the avatars in the virtual space. 3. The voice distribution method for a client-server type digital voice conference system according to claim 1, further comprising a step of determining a priority order for virtual space communication. 4. A plurality of clients, which are terminal devices having voice input / output means such as microphones / speakers, and a server, which is a centralized device, are connected by a communication line, and the client digitally converts voice input from the voice input / output means. A server having a buffer for receiving audio data from the client and a buffer for transmitting audio data to the client distributes the audio data received from each client to a plurality of clients, thereby transmitting data to a plurality of clients. In a digital audio conference server of a client-server type digital audio conference system for realizing, a storage means for storing the priority of another client with respect to its own client, a client to which received audio data is to be distributed, Digital voice selection for data distribution Digital audio conference server client-server digital voice conference system, comprising processes and, a. 5. The client-server type digital audio conference system according to claim 4, wherein said storage means is further a storage means for storing a digital audio receiving ability obtained by integrating a line capacity and a digital audio processing ability of each client. Digital audio conferencing server. 6. A priority data processing unit for storing the priority or the priority and the digital voice reception capability notified from each client in the storage unit, wherein the digital voice selection and distribution process includes: A message handler for distributing the voice data to the voice data distribution unit and the priority data processing unit; and transmitting a voice buffer passed from the message handler by referring to the priority of each client or the priority and the digital voice receiving capability of each client. 6. An audio data distribution unit for distributing the audio data to
A digital audio conference server of the client-server type digital audio conference system according to the above. 7. An audio input / output unit such as a microphone / speaker, an encoder for digitizing and compressing an audio signal input from the audio input unit, a transmission buffer for sending encoded audio data to a server, and audio from the server. A plurality of clients, which are terminal devices including a reception buffer for receiving data, and a decoder for expanding received audio data, and a server, which is a centralized device, are connected by a communication line;
In a digital audio conference terminal of a client-server type digital audio conference system that realizes multi-party conversation by distributing audio data received from a client to a plurality of clients, the priority of other clients with respect to their own client A client-server type digital having a priority notification unit for notifying a digital audio conference server of its own client's reception capability, and a communication conference control unit for controlling participation in and departure from the digital audio conference. Digital audio conference terminal of the audio conference system. 8. In the case where the digital audio conference terminal is a terminal device of a virtual space communication system, the audio data expanded by the decoder is added to the positional relationship between the avatar of the sender and the avatar of the own avatar in the virtual space. 8. The digital audio conference terminal of the client-server type digital audio conference system according to claim 7, further comprising a sound image localization processing unit that executes a sound image localization process and controls a position of the avatar in the virtual space accordingly.