JP2006345103A - Media conference system, and control method, program and storage medium therefor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a media conference system that enables many users to participate in a conference and lets a user who audits the conference have a say. <P>SOLUTION: The media conference system is equipped with: a media conference server device 12 which allows a plurality of utterance terminal device to mutually send and receive media information; and a streaming server device 11 which distributes the media information sent and received between the utterance terminal devices to audit terminal devices. The media conference server device 12 once receiving a request for a connection as an utterance terminal device from one of the audit terminal devices makes a choice of whether the audit terminal device as the connection request source is registered as a connection reserving terminal device, or connected as the utterance terminal device to the media conference server device 12 according to the number of the utterance terminal devices currently connected to the media conference server device 12. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a media conference system that performs a conference by classifying a plurality of terminal devices participating in a conference into a speech terminal device and a listening terminal device, a control method thereof, a program, and a storage medium.

  In general, a media conference system is a terminal device such as a personal computer of each user connected via a communication line or a server device, respectively, and predetermined information transmitted from each terminal device, such as character information or audio and video. It is a system that performs a conference by sharing media information such as.

  In a conference system that shares character information, each user operates each terminal to send character information to each other on a system such as a bulletin board system (BBS) or chat (immediate character dialogue). Thus, character information from each user is shared. Here, the electronic bulletin board is a system in which character information input by a user from a keyboard is stored in a server on a network, and a plurality of users browse the posted information stored in the server via the network. In this way, by storing character information input by the user on the server and disclosing it, the character information can be shared among an unspecified number of users. The electronic bulletin board basically requires other users to perform an operation for browsing in order to browse a message input by a certain user. Therefore, the exchange of character information on the electronic bulletin board is asynchronous.

  Next, messages exchanged in the electronic bulletin board system will be described with reference to FIGS. FIG. 11 is a diagram showing an example of time series display of messages exchanged on the electronic bulletin board, and FIG. 12 is a diagram showing an example of thread display of messages exchanged on the electronic bulletin board.

  In an electronic bulletin board, messages exchanged between users can be displayed in time series. In this case, as shown in FIG. 11, the title of the electronic bulletin board, the latest utterance date and time, and the current date and time are displayed on the user terminal device (21). For example, when the user name “Taro Yamada” speaks (inputs a message), the user name, the character information input by the user, and the input date and time are displayed (22). When another user speaks in response to the message, the user name, the character information input by the user, and the input date and time are displayed (23). Thereafter, each time a user speaks, the user name, the character information input by the user, and the input date and time are displayed (24 to 27).

  Moreover, it is possible to perform thread display for the purpose of easily browsing comments from other users on a statement instead of the time series display. This can be selected according to user settings. In the case of thread display, for example, as shown in FIG. 12, the title of the electronic bulletin board, the latest utterance date and time, and the current date and time are displayed (31). Then, for example, the utterance (32) of the user name “Taro Yamada” and the utterances of other users corresponding to the utterance are displayed in association with each other (33, 37). Similarly, an utterance of a certain user and an utterance of another user corresponding to the utterance are displayed in association with each other (33 to 36).

  The chat is a conversation in real time (immediate character dialogue) by sending character information input from the keyboard by the user in the network. Usually, in order to perform a chat, the user logs in to the chat system, and the character information spoken there is immediately notified to the participant who has logged in to the chat system and displayed on the display. Therefore, the exchange of character information in chat is synchronous, and the participants can exchange messages immediately with each other like a normal conversation.

  In a conference system that shares media information, a videophone device is used to conduct a conference using audio and video. The videophone device of each user is connected to an MCU device (multipoint connection device) that can mix audio and video, thereby realizing a video and audio conference between users.

  In recent years, the spread of the Internet network and intranet networks within the organization, the spread of flat-rate billing systems on the premise of always-on connection in the use of the Internet network, the use of broadband networks, and inexpensive video capture devices for personal computers (video cameras) As a result of the widespread use, a personal computer to which a video capture device and an audio device are connected can be used as an alternative to a videophone device using a communication line network such as the Internet network or an intranet network.

  In addition, there is a telephone conference system using a telephone in order to perform a conference only by voice between users. FIG. 13 is a block diagram showing a configuration of a conventional telephone conference system.

  As shown in FIG. 13, the telephone conference system connects telephone devices 41 </ b> A to 41 </ b> C used by each user to a telephone conference bridge 43 that can perform voice mixing via a circuit switching network. An audio conference is performed between users connected to the bridge 43. Further, if the mobile phone device 42 is connected to the conference call bridge 43, it is possible to participate in the conference from outside.

  In order to participate in the conference call, each user makes a call to the conference call bridge 43 from the respective phone devices 41A to 41C and the mobile phone device 42 at a designated time, and the conference is started from the phone devices 41A to 41C and the mobile phone device 42. ID (identification information) etc. for participating in is input. Thereby, each user can participate in the same audio conference. Conversely, a call is made from the conference call bridge 43 to each of the telephone devices 41A to 41C and the mobile phone device 42 at a designated time, and each user responds to the call, so that each user has the same audio conference. Can participate.

  In such a telephone conference system, in order to use a circuit switching network, a large communication cost is required when a long-time conference is performed by a large number of users. In addition, since the number of users that can be simultaneously connected to one telephone conference bridge 43 is limited, in order to increase the number of participants in the conference, a telephone conference bridge that can accommodate a larger number of telephone lines is necessary. At the same time, a large number of telephone lines need to be laid for the conference call bridge. As a result, it is very expensive.

  As such a system in which many users can participate in a conference call, there is, for example, Patent Document 1.

  In the conference system using this streaming server device, as shown in FIG. 13, for example, the streaming server device 44 is connected to a circuit switching network and a network such as the Internet and a LAN. The streaming server device 44 is connected to the teleconference bridge 43 via a circuit switching network, and can acquire the audio of the teleconference performed by the telephony devices 41A to 41C and the mobile phone device 42. In addition, the streaming server device 44 receives connections from the user terminal devices 45A to 45D via the network, and performs streaming distribution of the acquired audio of the conference call to the user terminal devices 45A to 45D. By using such a streaming server device 44, more users can listen to the contents of the conference call.

  In general, for the Internet network, the cost of using a line is low due to the spread of a flat-rate billing system based on a constant connection, and the streaming server device is intended for one-way distribution of multimedia data. In addition, since it is possible to connect with more users than with the conference call bridge, a system in which more users can listen to the contents of the conference call can be easily constructed.

Also, in a video and audio conference, in order to dramatically increase the number of people who can participate at the same time, the video and audio of an MCU device (multi-point connection device) capable of mixing video and audio are similarly streamed. It is possible to increase the number of people who can listen to the conference by distributing according to.
JP-A-7-162824

  However, in the conference system using the above-described streaming server device, a user connected to the streaming server device can only listen to the conference and cannot speak in the conference. That is, only users connected to the conference call bridge can speak at the conference, and the number of users that can be connected to the conference call bridge is limited. The number is limited. Therefore, there is a case where it is not possible to give an opportunity to speak to a user who wants to actively speak in a meeting.

  An object of the present invention is to provide a media conferencing system, a control method thereof, a program, and a storage medium capable of allowing a large number of users to participate in a conference and giving an opportunity to speak to a user who listens to the conference. There is to do.

  In order to achieve the above object, the present invention provides a media conference server device in which a plurality of speech terminal devices are connected via a communication line and media information can be transmitted and received between the speech terminal devices, and the communication line. And a streaming server device that is connected to the media conference server device and that is connected to one or more listening terminal devices and distributes media information transmitted and received between the speaking terminal devices to the listening terminal device. When the media conference server device receives a connection request to the speech terminal device from any of the listening terminal devices via the communication line, the media conference server device depends on the number of speech terminal devices currently connected to the media conference server device. Whether to register the connection request source listening terminal device as a connection reservation terminal device or to connect to the media conference server device as a speech terminal device. To provide a media conference system which is characterized in that.

  In order to achieve the above object, the present invention provides a media conference server device in which a plurality of speech terminal devices are connected via a communication line and media information can be transmitted and received between the speech terminal devices, and the communication line. And a streaming server device that is connected to the media conference server device and that is connected to one or more listening terminal devices and distributes media information transmitted and received between the speaking terminal devices to the listening terminal device. A control method for a media conference system, wherein when the media conference server device receives a connection request from any one of the listening terminal devices to a speech terminal device via the communication line, the media conference server device is currently connected Depending on the number of speaking terminal devices to be registered, the listening terminal device of the connection request source is registered as a connection reservation terminal device, or the speaking terminal device A control method of a media conference system and selects whether to connect to the media conference server device to.

  In order to achieve the above object, the present invention provides a media conference server device in which a plurality of speech terminal devices are connected via a communication line and media information can be transmitted and received between the speech terminal devices, and the communication line. And a streaming server device that is connected to the media conference server device and that is connected to one or more listening terminal devices and distributes media information transmitted and received between the speaking terminal devices to the listening terminal device. A program executed by the media conference server device to control a media conference system, and when receiving a connection request from any one of the listening terminal devices to the speech terminal device via the communication line, the media conference Depending on the number of speech terminal devices to which the server device is currently connected, the connection request source listening terminal device is registered as a connection reservation terminal device. Either provides a program characterized by having a module for selecting whether to connect to the media conference server apparatus as saying terminal.

  In order to achieve the above object, the present invention stores the above-mentioned program in a computer-readable manner.

  As described above, according to the present invention, a large number of users can participate in a conference, and an opportunity to speak can be given to a user who listens to the conference.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram schematically showing a configuration of a media conference system according to an embodiment of the present invention.

  In the media conference system according to the present embodiment, as shown in FIG. 1, a plurality of IP (Internet Protocol) telephone terminal devices 15A to 15G are simultaneously configured by an IP configured by a LAN (Local Area Network) and a WAN (Wide Area Network). A media conference is realized by connecting to the media conference server device 12 or the streaming server device 11 via a network. Here, the media conference server apparatus 12 controls the media conference. The IP telephone terminal devices 15A to 15G, the media conference server device 12, and the streaming server device 11 perform a media session in a media conference using SIP (Session Initiation Protocol, RFC3261) formulated by the IETF (The Internet Engineering Task Force). Establish.

  Specifically, IP telephone devices 15A to 15G having a communication function, IP telephone adapter 16, portable wireless IP telephone device 19, various terminal devices 110, 111A, 111B, streaming server device 11, media conference server device 12, SIP The server 13 and the gateway device 14 are connected to an IP network composed of a LAN and a WAN. The telephone device 17A and the mobile phone device 18 are connected to a circuit switching network. The gateway device 14 can communicate with each other in order to mediate the IP network and the circuit switching network.

  Here, in the following description, the IP telephone devices 15A to 15G, the IP telephone adapter 16, the portable wireless IP telephone device 19, and the various terminal devices 110, 111A, and 111B are collectively referred to as terminal devices. Unless otherwise specified, the terminal device includes any or all of the above.

  In the present embodiment, terminal devices that are scattered in remote locations by bidirectionally transmitting and receiving media information such as video and voice, text chat, and application shared data between terminal devices connected to the media conference server device 12. It is possible to hold a meeting between. Here, a terminal device connected to the media conference server device 12 and capable of bidirectionally transmitting and receiving media information is referred to as a speech terminal device. However, the maximum number of connectable terminal devices that can be connected to the media conference server device 12 is limited due to performance limitations of the media conference server device 12 and bandwidth limitations in the network. That is, the number of speaking terminal devices is limited.

  The streaming server device 11 receives the above-described media information in the media conference handled by the media conference server device 12 from the media conference server device 12 and distributes it to the terminal device connected to the streaming server device 11 in one direction. To do. That is, when more than the maximum connectable terminal devices of the media conference server device 12 participate in the media conference, some terminal devices that cannot connect to the media conference server device 12 , It is possible to receive one-way media information from the streaming server device 11. As a result, the terminal device connected to the streaming server device 11 can participate in the same media conference, although only listening to the media conference.

  Hereinafter, a terminal device connected to the streaming server device and listening to the media conference will be referred to as a listening terminal device. Moreover, in order to participate in a conference as a speech terminal device that transmits a speech and information to the media conference among the listening terminal devices, a terminal device that reserves connection with the media conference server device 12 is connected to the reservation terminal. I will call it.

  For example, in a media conference, when three IP telephone devices 15A to 15C are connected to the media conference server device 12 and the other three IP telephone devices 15D to 15F are connected to the streaming server device 11, each IP telephone device The devices 15 </ b> A to 15 </ b> C can listen to the content of the media conference and make a statement in the media conference. On the other hand, each of the IP telephone devices 15D to 15F can listen to the contents of the media conference, but cannot speak. In this case, each of the IP telephone devices 15A to 15C is a speech terminal device, and each of the IP telephone devices 15D to 15F is a listening terminal device.

  In this way, by providing the streaming server device 11, the number of terminals that can participate in the same media conference can be dramatically increased.

  The media conference server device 12 is a server that controls the entire media conference system, and details of the control will be described later.

  The SIP server 13 is a call control for the IP telephone devices 15A to 15G, the IP telephone adapter 16, the portable wireless IP telephone device 19, the various terminal devices 110, 111A, 111B, the streaming server device 11, the media conference server device 12, and the gateway device 14. Which is normally called IP-PBX. When the SIP server 13 receives a call from the IP telephone devices 15A to 15G, the IP telephone adapter 16, the portable wireless IP telephone device 19, and the various terminal devices 110, 111A, and 111B, the SIP server 13 sends the call to the called terminal device. Send.

  Next, when a call is made from one IP telephone apparatus to another IP telephone apparatus, a procedure until a call is established between the IP telephone apparatuses by call control of the SIP server will be described with reference to FIGS. To do. FIG. 2 is a sequence diagram showing a procedure until a call is established between each IP telephone apparatus by call control of the SIP server when a call is made from one IP telephone apparatus to another IP telephone apparatus. FIG. It is a figure which shows the content of Description Protocol) Offer and SDPAnswer.

  As shown in FIG. 2, for example, when IP telephone apparatus 15A calls IP telephone apparatus 15B, IP telephone apparatus 15A transmits an SIP INVITE method to SIP server 13 as a call request to IP telephone apparatus 15B (S501). ). Here, the INVITE method includes SDP, which is information necessary for establishing a session, such as the IP address of IP telephone device 15A, the port number used in the call, and the codec that can be used. This SDP, that is, SDPOffer has contents as shown in FIG.

  The SIP server 13 that has received the INVITE method recognizes that the INVITE method is a call to the IP telephone device 15D, and transmits the INVITE method that is a call request to the IP telephone device 15B (S502). A provisional response indicating that the request is being processed is returned to the caller IP telephone apparatus 15A (S503). This provisional response is a response to the INVITE method, and its response code is 100.

  The IP telephone apparatus 15B that has received the INVITE method sounds a ringing tone and returns a provisional response indicating that the call request is being processed (calling) to the SIP server 13 (S504). This provisional response is a response to the INVITE method (S502) from the SIP server 13, and the response code is 180.

  The SIP server 13 that has received the provisional response from the IP telephone apparatus 15B recognizes that the provisional response is a response to the call from the IP telephone apparatus 15A, and sends the provisional response, which is a call response, to the IP telephone apparatus 15A. Transmit (S505).

  When picking up the handset of IP telephone apparatus 15B that is ringing, IP telephone apparatus 15B transmits to SIP server 13 a success response indicating that the call request has been understood and accepted (S506). This success response is a response to the INVITE method from the SIP server 13 and the response code is 200. This successful response includes SDP, which is information necessary for establishing a session, such as the IP address of IP telephone device 15B, the port number used in the call, and the codec that can be used. This SDP, that is, SDP Answer, has contents as shown in FIG.

  The SIP server 13 that has received the success response from the IP telephone apparatus 15B recognizes that the success response is a response to the call from the IP telephone apparatus 15A, and sends the success response, which is a call response, to the IP telephone apparatus 15A. Transmit (S507).

  The IP telephone apparatus 15A that has received the success response from the SIP server 13 confirms that the call request to the IP telephone apparatus 15B has been accepted, and sends an ACK method to the SIP server 13 indicating that the response to the call request has been received. Transmit (S508).

  The SIP server 13 that has received the ACK method recognizes that the ACK method is a response confirmation to the IP telephone device 15B, and transmits the ACK method to the IP telephone device 15B (S509).

  In this way, a session is established between the IP telephone apparatuses 15A and 15B, and media data (voice data in this example) is transmitted between the IP telephone apparatus 15A and the IP telephone apparatus 15B using RTP (Real-time Transport Protocol). Voice communication is performed between the two. Then, when the IP telephone device 15B puts the handset, a BYE method that is a request to end the call is transmitted to the SIP server 13 (S510).

  The SIP server 13 that has received the BYE method recognizes that the BYE method is a call termination request to the IP telephone device 15A, and transmits the BYE method, which is a call termination request, to the IP telephone device 15A (S511).

  The IP telephone apparatus 15A that has received the call termination request from the SIP server 13 understands the call termination request and transmits a success response indicating that the call termination has been accepted to the SIP server 13 (S512). This success response is a response to the BYE method, and its response code is 200.

  The SIP server 13 that has received the success response from the IP telephone apparatus 15A recognizes that the success response is a response to the call termination request from the IP telephone apparatus 15B, and transmits the success response to the IP telephone apparatus 15B (S513). ). As a result, the media session ends.

  Next, the configuration of the media conference server device 12 will be described with reference to FIG. FIG. 4 is a block diagram showing a configuration of the media conference server apparatus 12 of FIG.

  As shown in FIG. 4, the media conference server device 12 includes a CPU 71, a RAM 72, a ROM 73, an HDD (Hard Disk Drive) 74, a network I / F (interface) 75, an input device I / F 76, a display I / F 77, and an imaging device. It has an I / F 78, an external media drive device I / F 79, an audio input / output device I / F 80, and a system bus 88 for connecting them. A keyboard 81 and a mouse 82 are connected to the input device I / F 76. A display 83 is connected to the display I / F 77. An imaging device 84 is connected to the imaging device I / F 78. An external media drive device 85 is connected to the external media drive device I / F 79. A voice input device 86 and a voice output device 87 are connected to the voice input / output device I / F 80.

  In accordance with a program stored in the ROM 73 or the HDD 74, the CPU 71 controls the entire media conference server device 12 and controls the media conference system while using the RAM 72 as a work area. The HDD 74 stores a program for realizing control in the media conference system, and data such as user database information and terminal information necessary for the operation of the program.

  The network I / F 75 performs communication control for connecting to the network. Various communication interfaces are applied to the network I / F 75 according to the environment. For example, when connecting directly to the access network, the network I / F 75 is configured by various modems, terminal adapters, and the like. Further, when connecting to an ADSL modem or cable modem of an external device, the network I / F 75 includes an Ethernet interface or the like.

  The input device I / F 76 processes input from a keyboard 81, a mouse 82, a pen input device (not shown), or the like, and is used to operate the media conference server device 12. These are optional devices provided as necessary.

  The display I / F 77 controls display on the display 83. The display 83 is composed of, for example, a CRT or a liquid crystal display. On the display 83, for example, the operation status of the media conference server apparatus 12 is displayed. The display I / F 77 and the display 83 are optional devices provided as necessary.

  The imaging device I / F 78 is for inputting imaging data from the imaging device 84. The imaging device 84 is used for imaging the surrounding situation of the media conference server device 12. These are optional devices provided as necessary.

  The external media drive device I / F 79 inputs / outputs data to / from the external media drive device 85. The external media drive device 85 is a device that reads / writes data from / to an external storage medium such as a floppy (registered trademark) disk drive, CD-ROM drive, MO drive, DVD-ROM drive or the like.

  The voice input / output device I / F 80 performs input / output from a voice input device 86 such as a microphone and a voice output device 87 such as a speaker. These are optional devices provided as necessary.

  The streaming server device 11 has the same configuration as the media conference server device 12. Therefore, the description of the configuration of the streaming server device 11 is omitted here.

  Next, the functional configuration of the media conference server device 12 will be described with reference to FIG. FIG. 5 is a block diagram showing a functional configuration of the media conference server apparatus 12 of FIG.

  The media conference server device 12 realizes a media conference by receiving a connection from a terminal device connected to a network and transmitting media conference information to the terminal device. Here, as described above, the terminal device includes the IP phone devices 15A to 15G, the IP phone adapter 16, the portable wireless IP phone device 19, and various terminal devices 110, 111A, and 111B.

  As shown in FIG. 5, the media conference server apparatus 12 includes a media conference information storage unit 801, a media conference information management unit 802, a media conference control unit 803, a media conference identification unit 804, a media conference mixing unit 805, and a media conference receiving unit. 806 and a media conference transmission unit 807.

  The media conference information storage unit 801 stores media conferences such as session information of media conferences, media information in media conferences, participation information in media conferences, participating user information, and participating terminal information in accordance with requests from the media conference information management unit 802. Media conference information about is written. Information written in the media conference information storage unit 801 is read as necessary.

  The media conference information management unit 802 operates and manages the media conference information storage unit 801 in accordance with a request from the media conference control unit 803. Specifically, the media conference information management unit 802 writes the media conference information in the media conference information storage unit 801, deletes the media conference information, or deletes the media conference information according to the request from the media conference control unit 803. Read from the information storage unit 801.

  The media conference control unit 803 receives a media conference participation request from the terminal device and controls connection to the media conference. This control method will be described later.

  The media conference identifying unit 804 performs processing based on the media conference information received by the media conference receiving unit 806. For example, when the received media conference information is a request regarding a media conference session by SIP, the media conference identification unit 804 sends the media conference information to the media conference control unit 803 and requests processing. If the received media conference information is media information related to a media conference such as audio and video, the received media conference information is sent to the media conference mixing unit 805.

  The media conference mixing unit 805 decodes media information such as a plurality of compressed video and audio received from a plurality of terminal devices connected to the media conference server device 12, and mixes the media information from each terminal device. . The mixed media information is encoded again, and the encoded media information is sent to the media conference transmission unit 807. Through the mixing process of the media conference mixing unit 805, each terminal device connected to the media conference server device 12 and performing the media conference can share the media information.

  The media conference receiving unit 806 receives media conference information transmitted from the terminal device, and sends the received media conference information to the media conference identifying unit 804.

  The media conference transmission unit 807 transmits the media conference information to the designated terminal device in accordance with transmission commands from the media conference identification unit 804 and the media conference mixing unit 805.

  Next, connection control by the media conference control unit 803 (CPU 71) of the media conference server device 12 will be described with reference to FIG. FIG. 6 is a flowchart showing a connection control procedure by the media conference control unit 803 of the media conference server apparatus 12 of FIG.

  The media conference server device 12 starts connection control in the media conference system by starting a server process. The media conference control unit 803 of the media conference server device 12 monitors requests from each terminal device to the media conference server device 12, and monitors the speech status of speech terminal devices participating in the media conference.

  When the media conference receiving unit 806 receives media conference information from each terminal device, the received media conference information is sent to the media conference identifying unit 804. When the received media conference information is a request regarding a media conference session by SIP, the media conference identification unit 804 sends the media conference information to the media conference control unit 803. In addition, when the received media conference information is media information such as voice and video, the media conference identification unit 804 sends the media conference information to the media conference mixing unit 805, and each message based on the media conference information. Data indicating the speech status of the terminal device (data indicating the speech interval, the number of speeches, etc.) is generated and sent to the media conference control unit 803. The media conference control unit 803 is a terminal device based on the media conference information from the media conference identification unit 804 (request regarding a media conference session by SIP), data indicating the speech status of each speech terminal device, and the presence / absence of a connection reservation terminal device. Control connection.

  Specifically, as shown in FIG. 6, the media conference control unit 803 first determines whether there is a registered connection reservation terminal device (listening terminal device registered as a connection reservation terminal device). (Step S90). Here, if there is a connection reservation terminal device, the media conference control unit 803 determines whether or not the received media conference information is a disconnection request (step S91). If the received media conference information is not a disconnection request, the media conference control unit 803 determines whether the received media conference information is a connection request from a listening terminal device (step S92). The connection request from the listening terminal device is a request to change to the speaking terminal device so that the listening terminal device can speak in the media conference. This connection request is made, for example, by pushing a dial of the listening terminal device and transmitting a DTMF signal. Alternatively, it may be assumed that a connection request is made when the voice of the above-described auditory terminal device is detected by voice recognition. Further, the DTMF signal may be transmitted as a voice or an RTP payload, or may be transmitted by a SIP INFO method.

  When it is determined that the media conference information received in step S92 is not a connection request from the attending terminal device, the media conference control unit 803 transmits the received media conference information from a terminal device that does not participate in the media conference. It is determined whether or not the SIP INVITE method means a connection request to the media conference (step S93).

  Here, when it is determined that the media conference information is not a connection request to the media conference transmitted from a terminal device that does not participate in the media conference, the media conference control unit 803 indicates data indicating the speech status of each speech terminal device. Based on the above, it is determined whether or not there is a speech terminal device that can be changed to a listening terminal device among the speech terminal devices (step S94). That is, here, when a utterance terminal device having no utterance for a predetermined time or a small number of utterances is detected among the utterance terminal devices based on data indicating the utterance status of each utterance terminal device, the detected utterance is The terminal device is a speech terminal device that can be changed to a listening terminal device. In this determination, the number of speaking terminal devices connected to the media conference server device 12 has reached the maximum number of connectable terminal devices of the media conference server device 12, and the listening terminal is connected to the streaming server device 11. In some cases, it is performed to give an opportunity to participate in a media conference as a speech terminal device to any of the listening terminal devices. If there is no utterance terminal device that can be changed to the listening terminal device, the media conference control unit 803 returns to step S90.

  If it is determined that there is a connection reservation terminal device registered in step S90, the media conference control unit 803 determines whether or not the connection reservation terminal device (listening terminal device) can be connected to the media conference server device 12. Is determined (step S95). This determination is made according to whether or not the number of speech terminal devices currently connected to the media conference server device 12 is less than the maximum number of connectable terminal devices of the media conference server device 12. Here, if the number of speaking terminal devices currently connected to the media conference server device 12 is less than the maximum connectable terminal device number of the media conference server device 12, the reserved connection terminal device can be connected to the media conference server device 12. It is determined that there is. On the other hand, if the number of speech terminal devices currently connected to the media conference server device 12 is equal to or greater than the maximum number of connectable terminal devices of the media conference server device 12, the reservation connection terminal device can connect to the media conference server device 12 It is determined that it is not.

  When it is determined that the reservation connection terminal device can be connected to the media conference server device 12, the media conference control unit 803, via the media conference transmission unit 807, of the listening terminal device registered as the connection reservation terminal device. Connection processing is performed so as to change the connection destination from the streaming server device 11 to the media conference server device 12 (step S96). Thereby, the above-described listening terminal device becomes a speaking terminal device capable of speaking in a media conference. In addition, the media conference control unit 803 updates the information on the listening terminal device in the media conference information storage unit 801 to information as a speech terminal device connected to the media conference server device 12. 802 is instructed.

  When it is determined that the media conference information received in step S91 is a disconnection request from the media conference of the terminal device participating in the media conference, the media conference control unit 803 establishes a connection with the terminal device that is the disconnect request source. Media information transmission / reception or distribution of media information to the terminal device that requested disconnection is stopped, and the media conference session is terminated (step S97). Here, when ending the media conference session, the media conference control unit 803 deletes the media conference information of the disconnection request source terminal device stored in the media conference information storage unit 801 so as to delete the participation information. The management unit 802 is instructed.

  When it is determined that the media conference information received in step S92 is a connection request from the listening terminal device, the media conference control unit 803 can connect the listening terminal device of the connection request source to the media conference server device 12. It is determined whether or not there is (step S98). This determination is the same as the determination in step S95. Here, when the connection request source listening terminal device is connectable to the media conference server device 12, the media conference control unit 803 determines the connection destination of the connection request source audit terminal device from the streaming server device 11 to the media conference server device. Connection processing is performed so as to change to 12 (step S96). Thereby, the above-described listening terminal device becomes a speaking terminal device capable of speaking in a media conference.

  On the other hand, when it is determined that the connection request source audit terminal device cannot be connected to the media conference server device 12, the media conference control unit 803 connects the connection request source audit terminal device to the media conference server device 12. The media conference information management unit 802 is instructed to store in the media conference information storage unit 801 as a possible connection reservation terminal device (step S99).

  If it is determined in step S96 that the received media conference information is a connection request to the media conference transmitted from a terminal device that has not participated in the media conference (non-participating terminal device), the media conference control unit 803 Determines whether or not the terminal device for requesting connection to the media conference is connectable to the media conference server device 12 (step S100). This determination is the same as the determination in step S95. When it is determined that the non-participating terminal device of the connection request source is connectable to the media conference server device 12, the media conference control unit 803 connects to the received media conference via the media conference transmission unit 807. As a response to the SIP INVITE method indicating a request, a SIP response “200 OK” indicating connection permission is transmitted to the non-participating terminal device of the connection request source (step S96). Accordingly, the media conference information management unit 802 writes information related to the connection requesting non-participating terminal device in the media conference information storage unit 801. The SIP call flow in this series of connection processing is the same as that shown in FIG. 2 in which the telephone device 15A is replaced with a terminal device and the telephone device 15B is replaced with the media conference server device 12.

  When it is determined in step S100 that the connection requesting non-participating terminal device cannot be connected to the media conference server device 12, that is, the number of speech terminal devices currently connected to the media conference server device 12 is the media conference server device 12. If the number is equal to or greater than the maximum number of connectable terminal devices, the media conference control unit 803 establishes communication between the media conference server device 12 and the streaming server device 11 via the media conference transmission unit 807 (step S101). . Then, the media conference control unit 803 transmits the media information of the media conference to the streaming server device 11, and in response to the received request for connection to the media conference, the connection to the streaming server device 11 is made to the connection request source terminal device. Instruct. Then, a transfer process for transferring a response corresponding to the connection request to the non-participating terminal device that is the connection request source via the streaming server device 11 is performed. Thus, the connection request source terminal device can connect to the streaming server device 11 and receive the media information of the media conference distributed from the streaming server device 11. That is, the non-participating terminal device that is the connection request source becomes an auditing terminal device that listens to the contents of the media conference. Details of processing for establishing communication between the media conference server device 12 and the streaming server device 11 and transmitting a connection request to the streaming server device 11 will be described later. In addition, the media conference control unit 803 transmits the terminal device as the attendance terminal device to the media conference information storage unit 801 as a connection reservation terminal device that can be a speech terminal device connected to the media conference server device 12. The media conference information management unit 802 is instructed to store the information.

  If it is determined in step S94 that there is a speech terminal device that can be changed to the listening terminal device, the media conference control unit 803 determines the connection destination of the speech terminal device specified as the speech terminal device that can be changed to the attendance terminal device. A connection change process for changing from the media conference server device 12 to the streaming server device 11 is performed (step S102). This connection change process will be described later.

  Next, referring to FIG. 7 and FIG. 8 for the process of establishing communication between the media conference server device 12 and the streaming server device 11 and transmitting the media information of the media conference to the streaming server device 11 in step S101. While explaining. 7 is a sequence diagram showing a procedure until communication is established between the media conference server device 12 and the streaming server device 11 by call control of the SIP server in step S101 of FIG. 6, and FIG. 8 is a procedure of FIG. It is a figure which shows the content of SDP (Session Description Protocol) Offer and SDPAnswer exchanged. Here, in FIG. 7, the SIP server 13 is omitted and some provisional responses are omitted in order to simplify the description and clarify the processing contents.

  As shown in FIG. 7, the media conference server device 12 transmits a SIP INVITE method to the streaming server device 11 as a call request to the streaming server device 11 (S1001). In this INVITE method, as shown in FIG. 8, SDP, that is, SDPOffer, which is information necessary for establishing a session, such as the IP address of the media conference server device 12, the port number used in the call, and the usable codec, include. In addition, this SDPOffer has a description of “a = sendly”. With this description, in this session, audio transmission is performed only from the media conference server device 12 to the streaming server device 11, and the streaming server device 11 It has been shown that no voice transmission is necessary.

  When the streaming server apparatus 11 accepts the call request, it transmits a success response indicating that the call request is understood and accepted to the media conference server apparatus 12 (S1002). This success response is a response to the INVITE method (S1001), and the response code is 200. The success response includes SDP that is information necessary for establishing a session, such as the IP address of the streaming server device 11, the port number used in the call, and the codec that can be used, as shown in FIG. It is. Also, there is a description “a = recvonly”, and this description indicates that in this session, the audio is only received from the media conference server device 12 and there is no audio transmission from the streaming server device 11. It has been

  Upon receiving the success response (S1002), the media conference server apparatus 12 confirms that the call request to the streaming server apparatus 11 has been accepted, and sends an ACK method indicating that the response to the call request has been received to the streaming server apparatus 11. (S1003).

  Next, transfer processing for transferring the media information of the media conference to the connection request source terminal device via the streaming server device 11 in step S101 will be described with reference to FIG. 9 and FIG. FIG. 9 is a sequence diagram showing a procedure until communication is established between any of the IP telephone apparatuses 15A to 15G and the streaming server apparatus 11 by call control of the SIP server in step S99 of FIG. 6, and FIG. It is a figure which shows the content of SDP (Session Description Protocol) Offer and SDPAnswer exchanged in the procedure of. Here, in FIG. 9, in order to simplify the description and clarify the processing contents, the SIP server 13 is omitted and some provisional responses are omitted. The terminal device that wishes to participate in the media conference will be described as one of the IP telephone devices 15A to 15G.

  When any of the IP telephone apparatuses 15A to 15G (hereinafter referred to as IP telephone apparatus 15) that desires to participate in the media conference calls the media conference server apparatus 12, the IP telephone apparatus 15 transmits to the media conference server apparatus 12. The SIP INVITE method is transmitted to the media conference server apparatus 12 as a call request (S1201). This INVITE method includes SDP, which is information necessary for establishing a session, such as the IP address of the IP telephone device 15, the port number used in the call, and the usable codec, as shown in FIG. Yes. Here, if the number of speaking terminal devices has already reached the maximum number of connectable terminal devices of the media conference server device 12, and the IP telephone device 15 and the media conference server device 12 cannot be connected, the media conference server device 12 Accepts a call request from the IP telephone device 15 for transfer processing to the streaming server device 11. Then, the media conference server device 12 transmits a success response indicating that the call request is understood and accepted to the IP telephone device 15 (S1202). This success response is a response to the INVITE method from the IP telephone apparatus 15, and the response code is 200.

  The IP telephone device 15 that has received the success response (S1202) confirms that the call request to the media conference server device 12 has been accepted, and displays an ACK method indicating that the response to the call request has been received. 12 (S1203). Upon receiving the ACK method, the media conference server apparatus 12 transmits an SIP INVITE method indicating that the call is to be put on hold to the IP telephone apparatus 15 in order to perform a transfer process to the streaming server apparatus 11 (S1204). The IP telephone device 15 transmits a success response indicating that the hold request has been accepted to the media conference server device 12 (S1205). This success response is a response to the INVITE method (S1204), and the response code is 200.

  Upon receiving the success response (S1205), the media conference server device 12 confirms that the hold has been accepted, and transmits an ACK method indicating that the response to the hold request has been received to the IP telephone device 15 (S1206). Next, in order to instruct the IP telephone apparatus 15 to connect to the streaming server apparatus 11, the media conference server apparatus 12 transmits a SIP REFER method to the IP telephone apparatus 15 as a transfer instruction (S1207). For example, when the SIP-URI of the streaming server device 11 is “sip: streaming @ XXXXXXX”, “REFER sip: streaming @ XXXXXXX” is transmitted to the IP telephone device 15.

  The IP telephone apparatus 15 that has received the transfer instruction transmits a success response indicating that the transfer instruction has been accepted to the media conference server apparatus 12 (S1208) and is in the process of connecting to the streaming server apparatus 11 based on the transfer instruction. A SIP NOTIFY method is transmitted to the media conference server apparatus 12 as a notification indicating the presence (S1209). This success response is a response to the REFER method, and its response code is 202.

  Upon receiving the notification (S1209), the media conference server device 12 confirms that the IP telephone device 15 accepts the transfer instruction and is performing the transfer process, and transmits a response to the notification to the IP telephone device 15 (S1210). .

  The IP telephone apparatus 15 that has received the response from the media conference server apparatus 12 transmits an SIP INVITE method to the streaming server apparatus 11 as a call request to the streaming server apparatus 11 based on the transfer instruction (S1211). This INVITE method includes SDP, that is, SDPoffer, which is information necessary for establishing a session, such as the IP address of the IP telephone device 15, the port number used in the call, and the usable codec, as shown in FIG. It is.

  The streaming server apparatus 11 accepts the call request, and transmits a success response indicating that the call request is understood and accepted to the IP telephone apparatus 15 (step S1212). This success response is a response to the INVITE method from the IP telephone apparatus 15, and the response code is 200. This successful response includes SDP, that is, SDP Answer, which is information necessary for establishing a session, such as the IP address of the streaming server device 11, the port number used in the call, and the usable codec. In addition, there is a description of “a = sendly”, and according to this description, in this session, the audio is only transmitted from the streaming server device, and the audio transmission from the IP telephone device 15 is not necessary. Is shown.

  The IP telephone apparatus 15 that has received the success response confirms that the call request to the streaming server apparatus 11 has been accepted, and transmits an ACK method indicating that the response to the call request has been received to the streaming server apparatus 11 ( S1213). Also, the IP telephone device 15 notifies the media conference server device 12 that the call request to the streaming server device 11 has been accepted by the SIP NOTIFY method (S1214). Upon receiving this notification, the media conference server device 12 confirms that the transfer instruction has been completed and the IP telephone device 15 is connected to the streaming server device 11, and transmits a response to the notification to the IP telephone device 15 (S1215). .

  Through the above processing, a session is established, and media data (voice data in this example) is transmitted from the streaming server apparatus 11 to the IP telephone apparatus 15 in one direction by RTP. You can listen to the contents of the media conference.

  In addition, after the transfer process, the media conference control unit 803 instructs the media conference information management unit 802 to store the IP telephone terminal device that has instructed the transfer process in the media conference information storage unit 801 as a connection reservation terminal. .

  In the present embodiment, a media conference using audio has been described. However, media information is not limited to audio, and may be media information such as video, text chat, and application sharing.

  A speech terminal device (speaking user) capable of transmitting and receiving media information to and from a number of terminal devices (users) participating in the media conference by the media conference server device 12 and the streaming server device 11 having the functions as described above. And an audience terminal device (auditing user) that can only receive media data in one direction and an opportunity to speak on the media conference to the audience terminal device (auditing user). Can be given.

  In this embodiment, the communication network is configured by a LAN and SIP is used for call control. However, the present invention is not limited to this. It is also possible to use something other than SIP.

  In addition, an object of the present invention is to supply a storage medium storing software program codes for realizing the functions of the embodiments to a system or apparatus, and a computer (or CPU, MPU, etc.) of the system or apparatus as a storage medium. This can also be achieved by reading and executing the stored program code.

  In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiment, and the program code and the storage medium storing the program code constitute the present invention.

  Examples of the storage medium for supplying the program code include a floppy (registered trademark) disk, a hard disk, a magneto-optical disk, a CD-ROM, a CD-R, a CD-RW, a DVD-ROM, a DVD-RAM, and a DVD. -RW, DVD + RW, magnetic tape, nonvolatile memory card, ROM, etc. can be used. Alternatively, the program code may be downloaded via a network.

  Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (operating system) running on the computer based on the instruction of the program code. A case where part or all of the actual processing is performed and the functions of the above-described embodiments are realized by the processing is also included.

  Further, after the program code read from the storage medium is written to a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion is performed based on the instruction of the program code. This includes the case where the CPU or the like provided in the board or the function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

It is a block diagram which shows typically the structure of the media conference system which concerns on one embodiment of this invention. FIG. 11 is a sequence diagram showing a procedure until a call is established between IP telephone devices by call control of a SIP server when a call is made from a certain IP telephone device to another IP telephone device. It is a figure which shows the content of SDP (Session Description Protocol) Offer and SDPAnswer. It is a block diagram which shows the structure of the media conference server apparatus 12 of FIG. It is a block diagram which shows the function structure of the media conference server apparatus 12 of FIG. It is a flowchart which shows the procedure of the connection control by the media conference control part 803 of the media conference server apparatus 12 of FIG. FIG. 7 is a sequence diagram showing a procedure until communication is established between the media conference server device 12 and the streaming server device 11 by call control of the SIP server in step S99 of FIG. 6. It is a figure which shows the content of SDP (Session Description Protocol) Offer and SDPAnswer exchanged in the procedure of FIG. FIG. 7 is a sequence diagram showing a procedure until communication is established between any of the IP telephone devices 15A to 15G and the streaming server device 11 by call control of the SIP server in step S99 of FIG. It is a figure which shows the content of SDP (Session Description Protocol) Offer and SDPAnswer exchanged in the procedure of FIG. It is a figure which shows an example of the time series display of the message exchanged on an electronic bulletin board. It is a figure which shows an example of the thread display of the message exchanged on an electronic bulletin board. It is a block diagram which shows the structure of the conventional telephone conference system.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Streaming server apparatus 12 Media conference server apparatus 13 SIP server 14 Gateway apparatus 15A-15G IP telephone apparatus 17A, 17B Telephone apparatus 18 Mobile telephone apparatus 19 Portable radio | wireless IP telephone apparatus 71 CPU
72 RAM
73 ROM
110, 111A, 111B Mobile terminal device 801 Media conference information storage unit 802 Media conference information management unit 803 Media conference control unit 804 Media conference identification unit 805 Media conference mixing unit 806 Media conference reception unit 807 Media conference transmission unit

Claims (13)

A media conference server device in which a plurality of speech terminal devices are connected via a communication line, and media information can be transmitted and received between the speech terminal devices;
Streaming that is connected to the media conference server device via the communication line and that is connected to one or more listening terminal devices, and distributes media information transmitted and received between the speaking terminal devices to the listening terminal device. A server device,
When the media conference server device receives a connection request to the speech terminal device from any of the listening terminal devices via the communication line, according to the number of speech terminal devices to which the media conference server device is currently connected, A media conferencing system that selects whether to register a connection request source listening terminal device as a connection reservation terminal device or to connect to a media conference server device as a speech terminal device.   The media conference server device registers the listening terminal device of the connection request source as the connection reservation terminal device when the number of speech terminal devices to which the media conference server device is currently connected has reached the maximum number of connectable terminal devices. The media conference system according to claim 1, wherein:   When the number of speech terminal devices connected to the media conference server device is less than the maximum number of connectable terminal devices, the media conference server device determines the connection destination of the connection request source listening terminal device from the streaming server device. 3. The connection to the media conference server apparatus is instructed to the listening terminal apparatus of the connection request source via the communication line so as to change to a media conference server apparatus. Media conferencing system.   4. The connection request according to claim 1, wherein the connection request is a DTMF signal transmitted from the connection request source listening terminal apparatus to the media conference server apparatus via the communication line. 5. Media conferencing system.   The said connection request is media information transmitted to the said media conference server apparatus via the said communication line from the auditory terminal apparatus of the said connection request origin, The any one of Claim 1 thru | or 3 characterized by the above-mentioned. Media conferencing system.   When the media conference server device receives a connection request from a terminal device that does not participate in the media conference via the communication line, the media conference server device connects the connection according to the number of speech terminal devices to which the media conference server device is currently connected. 2. The media conference system according to claim 1, wherein the requesting terminal device is selected as a listening terminal device or a speech terminal device. The media conference server device sends the received connection request via the communication line to the streaming server when the number of speech terminal devices to which the media conference server device is currently connected reaches the maximum number of connectable terminal devices. And transferring the connection request source terminal device as a connection reservation terminal device,
7. The media conference system according to claim 6, wherein the streaming server device connects the connection request source terminal device as a listening terminal device based on the transferred connection request.   The media conference server device connects the connection request source terminal device as a speech terminal device when the number of speech terminal devices to which the media conference server device is currently connected is less than the maximum connectable terminal device number. The media conference system according to claim 6 or 7.   The media conference server device has a listening terminal device in which the number of the speech terminal devices to which the media conference server device is currently connected has reached the maximum number of connectable terminal devices and is registered as the connection reservation terminal device. In addition, it is possible to identify a speech terminal device that does not transmit media information for a predetermined time or has a small number of media information transmissions based on the transmission status of each media information of the speech terminal device, and to connect the identified speech terminal device Instructing the specified utterance terminal device to connect to the media conference server device via the communication line so as to change the media conference server device to the streaming server device. The media conference system according to claim 1.   When the connection destination of the specified speech terminal device is changed from the media conference server device to the streaming server device, the media conference server device connects to the listening terminal device registered as the connection reservation terminal device Instructs the listening terminal device registered as the connection reservation terminal device to connect to the media conference server device via the communication line so as to change the streaming server device to the media conference server device. The media conferencing system according to claim 9. A plurality of speaking terminal devices are connected via a communication line, a media conference server device enabling media information to be transmitted and received between the speaking terminal devices, and connected to the media conference server device via the communication line. A method of controlling a media conference system comprising: a streaming server device that is connected to one or more attendance terminal devices and delivers media information transmitted and received between the speech terminal devices to the attendance terminal device;
When the media conference server device receives a connection request to the speech terminal device from any of the listening terminal devices via the communication line, according to the number of speech terminal devices to which the media conference server device is currently connected, A method for controlling a media conference system, comprising: selecting whether the connection request source listening terminal device is registered as a connection reservation terminal device or connecting to a media conference server device as a speech terminal device. A plurality of speaking terminal devices are connected via a communication line, a media conference server device enabling media information to be transmitted and received between the speaking terminal devices, and connected to the media conference server device via the communication line. In addition, the media conference is controlled to control a media conference system that includes one or more attending terminal devices and a streaming server device that distributes media information transmitted and received between the speaking terminal devices to the attending terminal devices. A program executed by a server device,
Upon receiving a connection request from any one of the listening terminal devices to the speaking terminal device via the communication line, the listening terminal of the connection request source according to the number of speaking terminal devices to which the media conference server device is currently connected A program comprising a module for selecting whether to register a device as a connection reservation terminal device or to connect to a media conference server device as a speech terminal device.   A storage medium storing the program according to claim 12 in a computer-readable manner.

Images