CN101001678B - SIP based VOIP multiplayer network games - Google Patents

Abstract

This invention is directed to a system and method for the close coupling of Session Initiation Protocol (SIP) based Voice over Internet Protocol (VoIP) with multiplayer network games. Game state dependent voice communications are used as another method of player interaction in the game environment. The system uses SIP enhanced game servers and conference servers with audio mixing capabilities, andthe game server controls the creation, maintenance and changing of the voice conferences. Participation in a voice conference is determined by the game state features associated with each player including distance and direction to determine how the voices are mixed in a voice conference.

Description

SIP-based VOIP multi-player online game

technical field

The present invention relates to the field of network-based computer games. More specifically, the present invention relates to the use of Session Initiation Protocol (SIP) to integrate Voice over IP (VoIP) to enhance network-based multiplayer games

Background technique

Multiplayer online games have become increasingly popular. These games are played across a computer network, such as the Internet or the World Wide Web, and allow a large number of geographically dispersed and remote users to simultaneously participate in one game. Participating in a game requires maintaining game state for each participant. Since the game environment is dynamic, these game states include the user's eligibility to join, participate, and leave the game. Additionally, these games provide for interaction between users within the gaming environment.

Traditionally, these interactions have been limited to pre-defined game actions such as shooting, running, fighting, and text messaging. A handful of online gaming systems have recently attempted to provide a modicum of voice communication between game participants. The voice function facilitates conversation between players, but the voice chat is not based on the game context. For example, voice communication is used as a messaging mechanism between players in place of text messaging. Often, voice messaging is completely outside the scope and role of the game, and is initiated by the user independently of the game. Furthermore, although VoIP is used to facilitate these conversations, the use of VoIP is limited and fails to utilize the full potential and functionality of VoIP.

VoIP has traditionally been used primarily as an alternative telecommunications solution to the standard Public Switched Telephone Network (PSTN); however, VoIP has potential in many other applications as well.

Therefore, what is needed is a system and method for enhancing the gaming experience between players utilizing full VoIP functionality. VoIP will be embedded and integrated into the game and maintained by the game in conjunction with the game state, eliminating the need for the user to independently and individually initiate voice communication. Voice data exchange will not be limited to conversations, but will incorporate game-based context to enhance the gaming experience.

Contents of the invention

The present invention relates to systems and methods for using Voice over IP (VoIP) to supplement and enhance the experience of participants in network-based multiplayer games. VoIP provides the ability to send voice data along with or in combination with other data on the same network.

The present invention utilizes VoIP over Session Initiation Protocol (SIP) to provide game-driven, context-based voice interaction between participants in a network-based gaming environment. For example, in addition to allowing players to talk to each other, form alliances, discuss strategy, and coordinate teammates and actions, the present invention imposes limitations on communication between players with a single shared context (e.g., the same physical location or room in a game environment). VoIP can also be used to provide more complex audio mixing to convey information about the distance and location of other players relative to the recipient of the voice communication, eg, players who are further away sound farther away.

To achieve the above goals, the present invention utilizes game server controlled sessions between different sets or groups of players, and dynamically switches players from one session to another based on game state. Additionally, more sophisticated conferencing mechanics, such as media mixing that exploit other game state characteristics such as the distance of sound sources from the listener, are used to control conferencing parameters such as speech amplitude.

Each user (in addition to a gaming system such as a computer) also uses a VoIP-enabled device capable of receiving and sending audio signals. In addition to receiving the visual state of the game, each player also receives related audio communications from the game or from other players. For example, players in the same game room or on the same team can talk to each other. Similar to how their visual state changes, the audio changes seamlessly as the player changes game rooms. The present invention uses the game server to automatically indicate the need for such changes and switch players dynamically without additional player input.

Description of drawings

Figure 1 is a schematic diagram of an embodiment of a gaming system according to the present invention;

Figure 2 is a schematic diagram of another embodiment of the game system according to the present invention;

Figure 3 is a schematic diagram illustrating communication between components of the gaming system;

Figure 4 is a schematic diagram of a room-based environment used in the gaming system of the present invention;

FIG. 5 is a schematic diagram showing the creation of an audio conference according to the present invention; and

FIG. 6 is a schematic diagram illustrating a modified audio conference according to the present invention.

Detailed ways

Referring first to FIGS. 1 and 2, the present invention relates to a system 10 for providing simultaneous context-based audio interaction among multiple participants or players 12 in a network-based gaming environment. Audio interaction includes the transmission of audible communications or signals, including voice communications, between participants. In one embodiment as shown in FIG. 1 , system 10 includes a single centralized game server 14 in communication with multiple game participants 12 . In another embodiment as shown in FIG. 2 , the system includes a plurality of distributed game servers 14 , wherein each game server 14 is associated with at least one game participant 12 . Game server 14 may be any type of server known and available in the art and capable of hosting a dynamic multi-user network-based gaming environment and maintaining a game state profile for each game participant 12. When system 10 includes multiple game servers 14 , each game server maintains a game state profile for each game participant 12 . This information may be stored in one or more databases 16 in communication with game server 14.

Any protocol suitable for implementing Voice over IP (VoIP) conferencing can be used with the present invention. Preferably, the protocol is Session Initiation Protocol (SIP), which stands out in VoIP applications. Thus, in one embodiment, game server 14 includes a SIP stack. SIP is a Hypertext Transfer Protocol (HTTP)-like protocol for establishing sessions between groups of users or locations. SIP provides functions between users, such as allowing a user to subscribe to events of interest hosted at another user's location, and to receive notifications whenever these events occur. More details on SIP can be found in Request for Recommendation 3261 (RFC 3261) of the Internet Architecture Board (IAB).

Game server 14 communicates with participants via one or more local or wide area networks 18 (eg, the Internet or the World Wide Web). Other suitable networks include secure area networks, private networks, virtual private networks, and combinations thereof. In one embodiment, game server 14 communicates with game participants 12 using SIP.

The system also includes one or more audio conferencing servers 20 in communication with game servers 14. In one embodiment, game servers 14 communicate with audio conferencing servers 20 using SIP. Audio conferencing servers 20 can be any type of A server that hosts a VoIP-based audio conference between one or more game participants 12. Suitable servers are available and well known in the art. The conference server 20 and game server 14 are also networked together to provide communication. In one embodiment, conference server 20 and game server 14 communicate over wide area network 18. This may be the same network through which game server 14 communicates with game participants 12, or it may be a separate network. Other suitable networks include local area networks, Secure Area Networks, Private Networks, Virtual Private Networks and combinations thereof.

System 10 also includes at least one audio mixer 22 in communication with audio server 20 and capable of providing audio mixing for an audio conference. In one embodiment, audio mixer 22 and audio server 20 communicate over wide area network 18 . This may be the same network through which the game server 14 communicates with the game participants 12, or it may be a separate network. Other suitable networks include local area networks, secure area networks, private networks, virtual private networks, and combinations thereof. In another embodiment, audio mixer 22 may be centrally located at audio server 20 , including integrated into audio conferencing server 20 .

To participate in the audio conference, each participant 12 has an audio communication device 24 capable of sending and receiving audio communications, such as voice communications. Suitable audio communication devices 24 include IP phones 26 , software-based phones 28 and conventional telephony devices 30 interfaced through a public switched telephone network (PSTN) 32 using a SIP gateway. An audio mixer 22 is in communication with each audio communication device 24 . In one embodiment, audio mixer 22 and communication device 24 communicate over wide area network 18 . This may be the same network through which the game server 14 communicates with the game participants 12, or it may be a separate network. Other suitable networks include local area networks, secure area networks, private networks, virtual private networks, and combinations thereof. In an alternative embodiment, multiple distributed (eg, geographically dispersed) audio conferencing servers 20 and audio mixers 22 may be used to improve the performance of the audio conferencing. In this embodiment, each audio mixer 22 and audio server 20 communicates with at least one audio conference participant 12 . In U.S. Patent Application No. entitled "System and Apparatus for Geographically Distributed VoIP Service with Enhanced QoS" (System and Apparatus for Geographically Distributed VoIP Service with Enhanced Quality of Service), the entire contents of which are hereby incorporated by reference. A suitable distributed arrangement of audio conferencing servers 20 and mixers 22 is described in 10/697,834.

In response to instructions from game server 14, audio conference server 20 establishes an audio conference between two or more participants. The audio mixer 22 controls the audio signal and transmits the audio signal directly to each audio communication device 24 according to the instructions of the audio conferencing server 20 . The game server 14 initiates and controls the audio conference according to the game state profiles of the game participants. For example, game server 14 uses the game state profile to determine which participant 12 to include in a given audio conference. Participant 12 does not initiate or control the audio conference. Thus, as shown in FIG. 3 , game participants 12 communicate with game server 14 and audio mixer 22 , but do not send instructions or queries directly to conference server 20 .

For a given audio conference, game server 14 determines or selects a group of game participants 12 based on each game participant's 12 current game state profile. Alternatively, game server 14 manages multiple simultaneous and independent audio conferences, each audio communication involving a group of two or more game participants 12 . The number of participants in a group can range from one (for example, to hear an echo) up to the current number of all game participants 12 . Communication can be two-way or one-way (eg, when eavesdropping). Each game participant 12 may be simultaneously a member of two or more groups, participating in two or more independent audio conferences. In general, game participants 12 in a given group have one or more shared game contexts. As used herein, a shared context defines a set of parameters or attributes in a game state profile that allow audio communication between two or more participants.

For example, as shown in FIG. 4, the shared context may be in physical proximity. As shown, the game environment includes a physical structure 34 , such as a house with interior and exterior areas and a number of rooms 36 . The audio conferencing group will include game participants 12 located in the same area or room. This may be referred to as room-based conferencing or view-based conferencing. For example, players in the same game room can chat with each other (chat, strategize).

As game participants 12 move between rooms or areas, game server 14 monitors game state profiles for changes and dynamically switches game participants 12 between appropriate groups. Game server 14 may also Delete groups or create new ones. When a game participant 12 changes rooms, the audio changes seamlessly along with the visual state of the game. Changes require no additional input from the game participant 12.

Other shared contexts include possession of a communication medium (such as a two-way radio), membership in a group (such as belonging to the same squadron or army), telepathy, and a shared language. Thus, the present invention adds game-specific context to communications between participants, enhancing the gaming experience.

In another embodiment of the present invention, the gaming experience is further enhanced by the use of complex audio mixing of the audio signals delivered to each game participant. In general, each game participant 12 receives an audio signal that simply includes cumulative input from all other game participants 22 in the group for a particular audio session. However, according to the invention, the received audio is not just a direct combination of audio signals, but game state dependent. For example, in a soccer game, players hear all the other players on the soccer field. This is the group that contains all game participants on the soccer field. However, the loudness of the sound from each player depends on factors such as the distance from the listener. As a result, the striker hears the other strikers and midfielders and the other team's defenders louder than his own team's defenders. This provides a more realistic simulation experience and integrates well with games.

As shown in FIG. 4, a first participant 38 receives an audio signal containing Σ (audio signals or sounds from all other participants 12 in the same room 36). To add additional context, a feature vector X is defined between the first participant and every other participant in the room. This feature vector is determined by the game server 14 based on game context features including, but not limited to, distance from the first participant, communication medium (air, water, telephone, etc.), frequency of transmission, amplitude of transmission, speaker's Tone, room acoustics, and room layout (eg presence of partial barriers 40). Game server 14 sends feature vectors for each game participant 12 to audio mixer 22 via audio conferencing server 20 .

A separate eigenvector can be specified for each game participant 12, or different eigenvectors can be calculated, some applying to all players in a group and some targeting a certain player or pair of players.

Thus, the audio signal received by the first participant is Σ( _X2 (audio from second participant 42)+ _X3 (audio from third participant 44)+ _X4 (audio from fourth Audio from participant 46)+ _X5 (audio from fifth participant 48)). Similar calculations can be made for other group members.

Furthermore, feature vectors can indicate not only scalar properties of audio signals (such as distance and loudness), but also directional information (including left-right, up-down, and front-back). Thus, based on the audio signal received by the first player 38, that player will be able to know that the second and fourth players 42, 46 in the gaming environment are to the left and upstairs, and that the second player 42 is further than the fourth participant 46 . Orientation information is especially suitable for stereo surround sound, virtual reality or holographic applications. The exact value of the feature vector can be set to any value suitable for providing the desired game environment effect.

In order to provide simultaneous context-based audio interaction between multiple game participants 12 in a network-based gaming environment, a network-based gaming environment containing multiple game participants 12 will be established. Referring to FIG. 5, in order to join the game environment, each game participant 12 contacts 50 the game server 14 and gives the address of its audio communication device 24 for audio or voice communication in conjunction with the game. After the game participant 12 has contacted the game server, the game participant 12 can play the game. As the game progresses, the game server (whether centralized or distributed) maintains a game state profile for each game player 12 participating.

For the centralized game server 14 embodiment, the game server 14 acts as a link between all game participants 12 and maintains a game state profile for each game participant 12. In a distributed game server 14 embodiment, each A distributed game server 14 is used to complete the maintenance and exchange of game state profile information. In an embodiment where a game server 14 is associated with each participant 12, the participant 12 maintains and exchanges game state information by itself. In this embodiment, Whenever a participant 12 performs an action (move, shoot, leave, etc.), that participant 12 notifies all other participants 12 of this action, and the other participants update their game state profiles accordingly.

After the game participant 12 has contacted the game server 14 and joined the game, the game server 14 brings the participant into the game and activates the audio communication device and game device 52 identified by the game participant, e.g., using a SIP "INVITE" command to invite The device enters the game. The participant responds 53 by sending its Signal Description Protocol (SDP) (eg, by using a SIP "OK" command).

The game server 14 maintains a game state profile for each participant in the game, when a participant 12 joins or participates in a game, according to the present invention, the game server 14 communicates among the game participants 12 according to the game state profile. Establish one or more VoIP-based audio conferences. Preferably, the game server establishes a SIP-based VoIP audio conference. Establishing an audio conference includes creating a new conference and modifying or terminating an existing conference.

In the embodiment shown in FIG. 5, the game server 14 obtains the media or audio path information of the game participant 12 from the received SDP. This information is forwarded to the audio conferencing server 54, which passes it on to the audio mixer 56, both using eg the SIP "INVITE" command. The audio mixer responds to the conference server 58, which responds to the game server 60, both using eg a SIP "OK" command. These responses contain SDPs for the conference server and audio mixer. The game server extracts the necessary media information from these SDPs and forwards this information to the game participants 62 (eg, using the SIP "ACK" command). The game server also contacts the audio conferencing server 64, which in turn contacts the audio mixer 66, using eg a SIP "ACK" command. This establishes the audio path 68 between the game participant 12 and the audio mixer 22 and enables the audio path 68 to be used in the VoIP audio conference.

In an embodiment with multiple distributed game servers, the game server associated with each game participant maintains the session directly with the conference server and, depending on the game state, initiates the establishment and dynamic switching of the audio conference. In this embodiment, the interaction performed by the game server is done by a client-side utility, and the communication with the conference server comes from the game server associated with each participant.

When establishing an audio conference, game server 14 determines at least one or more groups of game participants that possess a shared context that allows audio communications to be transmitted or received between game participants in the group. Game server 14 maintains or modifies game participants in each group according to changes in their game state profiles. This includes adding or removing participants to or from a group and switching at least one participant between two different groups. Switching participants between groups is done dynamically and seamlessly by the game server without any disruption to the game or audio conference or the need for additional input from the participants. In the room-based conferencing example, as soon as a game participant moves to another room, the game server notices the change and switches the participant to the audio conference associated with the new room. Typically, the game participants do not do any SIP signaling with the conference server itself. Game servers act as continuous user agents and maintain their sessions on their behalf, setting up media paths to each audio communication device.

Referring to FIG. 6, this information is transmitted to the game server 70 as the game participant 12 moves or acts in the game environment. If the action or movement of the game participant 12 indicates the need to transfer the participant from one conference group to another, the game server 14 updates the media information at both the audio mixer 22 and the game participant 12 . Game server 14 initiates the transfer process by removing game participants from the audio conference, for example, by sending a SIP “BYE” command to conference server 72 which sends a SIP “BYE” command to audio mixer 74 . The change is confirmed back to the game server 14, for example, by sending a SIP "OK" command to the conference server 76, which sends a SIP "OK" command to the game server 78.

Game server 14 then obtains media or audio path information for game participant 80 using, for example, a SIP "RE-INVITE" command. This information is forwarded to the audio conferencing server 54, which passes it on to the audio mixer 56, both using eg the SIP "INVITE" command. The audio mixer responds to the conference server 58, which responds to the game server 60, both using eg a SIP "OK" command. These responses contain the SDPs of the conference server and audio mixer. The game server extracts the necessary media information from these SDPs and forwards this information to the game participants 62 (eg, using the SIP "ACK" command). The game server also contacts the audio conferencing server 64, which in turn contacts the audio mixer 66, using eg a SIP "ACK" command. This establishes the audio path 68 between the game participant 12 and the audio mixer 22 and enables the audio path 68 to be used in the VoIP audio conference.

In one embodiment, establishing an audio conference includes transmitting to each audio conference participant an audio signal comprising a sum of all received audio signals from all other audio conference participants. Preferably, the game server determines audio feature vectors for each pair of audio conference participants based on game state profiles associated with the participants, and modifies audio transmitted between the pair of audio conference participants based on the audio feature vectors. Signal. In one embodiment, the game server modifies the audio feature vector in response to a change in the audio conference participant's game state profile.

In one embodiment, the game server communicates the feature vector to the conference server and audio mixer. This can be done by the conference server subscribing to such information at the game server (using e.g. the SIP "SUBSCRIBE" command), and the game server sending a message containing such feature vectors of the players to the conference server (using e.g. SIP "NOTIFY" command).

The present invention also relates to a computer-readable medium containing computer-executable code that, when read by a computer, causes the computer to execute a Provides methods for simultaneous context-based audio interaction. In one embodiment, this code is maintained, read and executed by the game server.

While it will be apparent that the exemplary embodiments of the invention disclosed herein achieve the objectives of the invention, it is to be understood that numerous modifications and other embodiments will be devised by those skilled in the art. Furthermore, feature(s) and/or element(s) from any embodiment may be used alone or in combination with one or more other embodiments. It is, therefore, to be understood that the appended claims are intended to cover all such modifications and embodiments as fall within the spirit and scope of the invention.

Claims (25)

1. A system for providing simultaneous context-based audio interaction between a plurality of participants in a network-based gaming environment, the system comprising: a game server in communication with a plurality of game participants, the game server capable of hosting the network-based gaming environment and maintaining a game state profile for each game participant; and an audio conferencing server in communication with the game server, the audio conferencing server capable of hosting a voice-over-IP audio conference between two or more game participants; Wherein the audio conference server establishes the audio conference in response to an instruction from the game server. 2. The system of claim 1, wherein the game server determines the participants in the audio conference based on the game state profiles of the game participants. 3. The system of claim 1, wherein the participants in the audio conference include a shared context. 4. The system of claim 1, wherein the game server includes a session initiation protocol stack, and the audio conference is a session initiation protocol based voice over IP communication. 5. The system of claim 1 , wherein each participant includes an audio communication device, and the system includes at least one audio mixer in communication with the audio conferencing server and each participant audio communication device to provide The audio conferencing provides audio mixing. 6. The system as recited in claim 5, wherein the audio communication device comprises an IP phone, a software-based phone, or a traditional phone device. 7. The system of claim 5, wherein the audio mixer is centrally located at the audio conferencing server. 8. The system of claim 5, further comprising a plurality of distributed audio mixers, each audio mixer in communication with one of the audio conference participants. 9. The system as recited in claim 1, wherein the game server is capable of managing multiple simultaneous and independent audio conferences, each audio conference communication involving two or more game participants. 10. The system as claimed in claim 9, wherein each game participant is capable of simultaneously participating in two or more independent audio conferences. 11. The system of claim 1, further comprising a plurality of distributed audio servers in communication with the game server, each audio server capable of hosting an audio conference between two or more game participants. 12. The system of claim 1, further comprising a plurality of distributed game servers. 13. A method for providing simultaneous context-based audio interaction between a plurality of participants in a network-based gaming environment, the method comprising: Create a web-based gaming environment with multiple game participants; maintaining a game state profile for each of the game participants; and One or more voice-over-IP audio conferences are established between the game participants based on the game state profile. 14. A method as claimed in claim 13, wherein said step of maintaining a game state profile comprises maintaining a game state profile for each participant in a single centralized game server. 15. The method as recited in claim 13, wherein said step of maintaining a game state profile comprises maintaining a game state profile for each participant in each of a plurality of distributed game servers associated with each participant. Game state profile. 16. The method as recited in claim 13, wherein said step of establishing an audio conference comprises establishing a Session Initiation Protocol-based Voice over IP audio conference. 17. The method as recited in claim 13, wherein said step of establishing an audio conference includes determining at least one group of game participants that possess a shared context that allows audio communications to be transmitted or received between game participants in the group. 18. The method of claim 17, further comprising modifying the group of participants based on changes in the game state profiles of game participants within the group. 19. A method as claimed in claim 18, wherein said step of modifying a group of participants comprises removing or adding participants. 20. The method as recited in claim 17, further comprising determining groups of players, wherein each group of game players owns a shared context that allows audio communications to be transmitted or received between game players in the group. 21. A method as claimed in claim 20, further comprising dynamically switching at least one participant between two different groups. 22. A method as claimed in claim 13, wherein said step of establishing an audio conference includes transmitting to each audio conference participant an audio signal comprising a sum of all received audio signals from all other audio conference participants. 23. The method of claim 13 , wherein said step of establishing an audio conference includes determining audio feature vectors for each pair of audio conference participants based on said game state profiles associated with the participants; and An audio signal communicated between the pair of audio conference participants is modified according to the audio feature vector. 24. The method as claimed in claim 23, wherein the audio feature vector includes information related to distance, direction, communication medium, transmission frequency or transmission amplitude. 25. The method of claim 23, further comprising modifying the audio feature vector in response to a change in the game state profile of the audio conference participant.

Images