HK1220059B - Method of video layout and processing in mcu stacking - Google Patents

Description

Video layout and processing methods in MCU stacking

Technical Field

The present invention relates to multipoint conferencing technology, and more particularly to video layout in a conference conducted by stacking two or more Multipoint Control Units (MCUs).

Background Art

Companies and organizations are increasingly using audio/video conferencing and multipoint conferencing technologies to improve communication and efficiency. Large organizations have numerous multimedia endpoints distributed throughout the organization. Typically, one or more multipoint control units (MCUs) serve the internal multipoint multimedia conferencing needs of these endpoints.

A multimedia endpoint is a networked terminal that can provide real-time, two-way audiovisual communication with other terminals or an MCU (e.g., a VSX8000). An endpoint may also include an MCU. An MCU is a conference control entity located at a network node or in a terminal and can receive multiple media channels from an endpoint via an access port. The MCU processes audiovisual and data signals and distributes the processed signals to connected channels according to specific standards. A more detailed description of endpoints and MCUs can be found in International Telecommunication Union ("ITU") standards, such as, but not limited to, H.320, H.324, and H.323.

Several technologies have been developed to improve the use and efficiency of multipoint communication systems. Some of these technologies improve the process of establishing communication sessions by enabling walk-up calls, ad-hoc calls, virtual conference rooms, and the like. Some of these technologies are disclosed in U.S. Patent Nos. 7,085,243, 7,310,320, and 7,830,824, each of which is incorporated herein by reference.

Other techniques improve control of multipoint communication networks by providing a web server that monitors and controls multiple MCUs. An example of a web control technique using rapid updates is disclosed in U.S. Patent No. 6,760,750, which is incorporated herein by reference. Additional techniques have been disclosed for utilizing the resources of one or more MCUs, for cascading one or more MCUs, and generally for improving resource utilization of one or more MCUs. See, for example, U.S. Patent Nos. 8,542,266, 7,800,642, 7,492,730, 7,174,365, 7,113,992, and U.S. Patent Publication No. 2012-0236111 A1, each of which is incorporated herein by reference.

Referring to Figure 1 , MCU 116 typically has limited hardware and/or software resources. For example, MCU 116 may only be capable of encoding/decoding video signals from a certain number of endpoints 114. This situation may arise when MCU 116 may have limited processing power (hardware or software) to support encoding or decoding of video signals. In another example, MCU 116 may have a limited number of video input ports. In either case, a cascaded conference using cascaded MCUs (116a, 116b) can be used to overcome resource limitations.

In a cascaded conference, conference participants can be divided into two or more groups across two or more networks, each associated with a different MCU 116. One MCU 116 is designated as the conference master MCU 116a (MMCU), while the other MCU 116 is designated as the conference slave MCU 116b (SMCU). This technology can also be used in conferences between participants located at different sites, each with its own MCU 116. Each participant can use their own local MCU 116, and the entire conference can be conducted by cascading different local MCUs 116.

FIG1 is a block diagram of a multipoint conferencing communication system 100. For example, system 100 may be used by a company with multiple regional networks 110 or by a global conferencing service provider with multiple regional networks 110. Regional networks 110 may correspond to individual conferencing sites and may communicate with each other via packet-switched networks 120 and/or circuit-switched networks 130. While regional networks 110 may correspond to specific packet-based network segments or domains, this is not required and regional networks 110 may span multiple network segments or domains. Each regional network 110 may have multiple multimedia endpoints 114 and one or more local or site-specific MCUs 116.

One or more control servers 112 (CS) can be used in each regional network 110. Within each regional network 110, each of the local endpoints 114 can communicate with its associated local MCU 116 via a packet-switched network and/or a circuit-switched network (not shown). In one example of communication system 100, control server 112 is a web server that can communicate with each local MCU 116 via network 120 using the Internet Protocol (IP). Communication with MCU 116 can be accomplished via an application program interface (API) module (not shown), which can be part of MCU 116. Control server 112 can be a dedicated server for performing cascaded multi-site conferencing, but can also be embedded in MCU 116 or include applications sharing a single web server. For example, as disclosed in the patents and patent application references incorporated above, a single web server can execute additional conferencing technology applications and can be used to manage connections, calls, virtual conference rooms, monitor and control MCU 116, and so on.

In an alternative example, in addition to communicating with MCU 116, control server 112 may communicate with a management server (not shown) (e.g., a server at a company or a global service provider). The management server may include a management database (not shown) of potential endpoint users, such as company employees or customers of a global service provider. The management database may include information such as name, different types of addresses (e.g., email, phone, etc.), ID numbers (e.g., employee ID number, customer billing number, or client ID number), authentication numbers, and conference room numbers. In another embodiment, control server 112 and/or MCU 116 may include the management server.

Depending on the type of conference in progress, the associated conference videos are organized. One type of conference is called a video-switched conference, in which each participant sees a selected participant (the source of the video from one endpoint 114). The selected participant can remain constant during the conference or change based on the dynamics of the conference. For example, the currently active speaker can be shown as conference video to all participants (i.e., at all endpoints 114). If the active speaker changes, a new active speaker can be shown.

Another type of conference is a continuous presence conference (CP), in which video from one or more selected endpoints 114 can be shown continuously throughout the conference. In a continuous presence cascaded conference, SMCU 116b can compose a continuous presence video from the video signals of selected participants from its associated area network 110, according to the conference layout associated with SMCU 116b. The mixed audio and video of the associated conference is delivered to MMCU 116a in a manner similar to the video and audio of a single participant. A common continuous presence process involves scaling the video data from various source user terminals to change the frame resolution so that it can be later incorporated into the continuous presence layout and video mix.

The MMCU 116a can mix the audio and video received from one or more SMCUs 116b with the audio and video of the selected participant from the participant group associated with the MMCU 116a. The result is the mixed audio and video of the cascade conference. The MMCU 116a can deliver the mixed audio and video of the cascade conference to each of the endpoints 114a associated with itself then, and deliver it to the one or more SMCUs 116b that are connected. Each of the one or more SMCUs 116b can distribute the mixed audio and video of the cascade conference to the endpoints 114b associated therewith.

One challenge in managing cascaded conferences is that each MCU 116 (MMCU 116a and SMCU 116b) selects participants (endpoints 114) from its associated group to be mixed and displayed, regardless of how the selected participants relate to participants in other associated groups. In addition, the size of the image of the participant associated with MMCU 116a is often different from the image of the participant associated with SMCU 116b.

As shown in Figure 2, a continuous presentation layout 200 is used as a layout for a conference between four endpoints 114a1-114a4 associated with MMCU 116a and four endpoints 114b1-114b4 associated with SMCU 116b. Participants AM, BM, CM, and DM may be associated with endpoints 114a1-114a4, respectively, while participants AS, BS, CS, and DS may be associated with endpoints 114b1-114b4, respectively. The most active speakers may be participants AM, BM, DM, and AS, and it is desirable to select and display video from the most active participant in the 2x2 continuous presentation layout. However, in a cascaded conference, video from SMCU 116b is treated as video from any other endpoint 114a. Layout 200 illustrates how the composed video from SMCU 116b (AS, BS, CS, and DS) replaces the tile position of a single participant in the cascaded layout. As a result, each of the participants associated with SMCU 116b is given a smaller screen area (e.g., one-quarter of the space typically allocated to a participant in a 2x2 layout).

One way to correct for the difference in image size between the participants associated with MMCU 116a and the participants associated with SMCU 116b is to force SMCU 116b to use a video switching layout and deliver the video of a single selected participant. The image of the single selected participant is placed in the layout of the cascaded conference. Layout 220 illustrates a snapshot of a 2x2 continuous presentation cascaded conference in which SMCU 116b is forced to operate in switching mode. In switching mode, SMCU 116b delivers the image of the single selected participant that covers the entire frame. Therefore, when MMCU 116a scales the image down to place it in the continuous presentation layout of the cascaded video, the scaled-down image has the same size as the image of the participant associated with MMCU 116a.

Using this approach corrects the size issue, but prevents viewing of other participants associated with SMCU 116b, even if their audio energy is higher than that of AM and/or BM and/or DM. Therefore, there is a need for a system and method for composing video for a cascaded conference in which each of the participants is evaluated under the same criteria for display in the layout, regardless of whether the participant is associated with an MMCU or an SMCU.

Summary of the Invention

A system and method for providing a continuous presence layout in a stacked conference is disclosed. According to one embodiment, two or more MCUs are used to create a continuous presence layout in an MCU stacked video conference with one or more MCU connection controller (MCC) modules. During a continuous presence stacked conference, the MCC determines which participant endpoints are presented in the continuous presence layout and in which window the participant should appear. This selection can change dynamically during the conference. The MCC receives information such as the audio power of each participant in the conference. This information can be automatically delivered periodically by each MCU associated with the conference or can be retrieved by the MCU from each MCC. Based on this information, the MCC determines which endpoints will be connected to which MCU and which endpoint video will be displayed in each window of the continuous presence layout for each endpoint. This decision is communicated to each MCU and each endpoint involved in the stacked conference.

BRIEF DESCRIPTION OF THE DRAWINGS

The exemplary embodiments will be more readily understood by reading the following description and referring to the accompanying drawings, in which:

FIG1 illustrates a basic block diagram of a cascade conference in the prior art;

FIG2 illustrates a layout for displaying continuous presentation participants according to the prior art;

FIG3 is a block diagram of a stacked master MCU with an MCU connection controller (MCC);

FIG4 is a block diagram of an MCC; and

5 is a flow chart illustrating a process for managing a continuous presentation stack layout using MCC.

DETAILED DESCRIPTION

3 is a simplified block diagram of a stack master MMCU 116 a that can be used to conduct a multimedia stack conference with one or more continuous presentation layouts according to the present disclosure. The MMCU 116 a can include a network interface 420, an audio module 430, a control module 440, and a video module 450. The network interface 420 can receive and process communications from the terminal 114 ( FIG. 1 ) via the associated network communication area 110 ( FIG. 1 ) according to a variety of communication standards.

The network interface 420 may be used to receive control and data information from other MCUs 116b and/or one or more control servers 112 ( FIG. 1 ), and to transmit control and data information to other MCUs 116b and/or one or more control servers 112 ( FIG. 1 ). Further information regarding communication between endpoints and/or MCUs over various networks and information describing signaling, control, compression, and how to set up and deliver video calls may be found, for example, in International Telecommunication Union standards H.320, H.321, H.323, H.261, H.263, and H.264.

The video module 450 receives compressed video from multiple endpoints 114 associated with the MCU 116a via the network interface 420. Furthermore, the video module 450 can receive one or more associated continuous presentation layouts from other MCUs 116b involved in the stacked conference and/or can create one or more associated continuous presentation layouts for other MCUs 116b involved in the stacked conference. The video module 450 processes, structures, and encodes the video input. The video module 450 can have multiple input modules 452, output modules 456, and a common interface. For example, four input modules 452a-452d and five output modules 456a-456e are shown. Some of the input modules 452 and output modules 456 can be associated with endpoints 114, and some of the input modules 452 and output modules 456 can be associated with other MCUs 116b. Each input module 452 may include a video decoder, and each output module 456 may include a video encoder. During a stacking conference, the I/O group (input module 452 and output module 456) may be associated with other MCUs 116d involved in the stacking conference.

The number of groups associated with an MCU 116 depends on the conference architecture. For example, if the conference is a master/slave architecture, one of the MCUs 116 is designated as an MMCU 116a, while one or more MCUs 116 are designated as SMCUs 116b. In one embodiment, the control module 440 of the MMCU 116a includes a control server 112a, which includes an MCU connection controller (MCC) 460 (described in more detail below) for controlling the connection of endpoints 114 to the MCU 116. Although shown as integrated here, the MCC 460 can be separate from the MMCU 116a.

In the exemplary stacked MCU architecture, MMCU 116a has at least one output module 456 to distribute the bitstream to each of SMCUs 116b. The at least one output from MMCU 116a to SMCU 116b can be encoded with the highest fidelity and can include at least one more endpoint video stream than will be displayed in the CP layout. This allows the input module 452 of SMCU 116b to decode the bitstream received from conference master MMCU 116a, mix the audio and video for each endpoint (e.g., minus the audio and/or video belonging to receiving endpoint 114), and re-encode it for delivery to endpoints 114 associated with conference slave SMCU 116b.

The general functionality of the various components of video module 450 is known in the art and will not be exhaustively described in detail herein. However, a more detailed description of such details can be found by consulting U.S. Patent Publication No. 2002/0188731 and U.S. Patent No. 6,300,973, the contents of which are incorporated herein by reference. This disclosure describes the operation of video module 450 when preparing a stacked continuous presentation layout, as described below in conjunction with FIG. 4 .

The audio module 430 can receive compressed audio streams from multiple endpoints 114 or SMCU 116b via the network interface 420. The audio module 430 can process the compressed audio streams (including mixing related audio streams) and send the compressed mixed signals back to the endpoints 114 and SMCU 116b via the network interface 420. In an embodiment, the audio streams sent to each of the endpoints 114 or SMCU 116b can be different. For example, the audio streams sent to different endpoints in the endpoints 114 can be formatted according to different communication standards or according to the needs of the individual endpoints 114. As another example, the audio stream may not include the voice of the user associated with the endpoint 114 to which the audio stream is sent, but the voice can be included in all other audio streams.

Audio module 430 may be adapted to analyze received audio signals from endpoints 114 and determine the audio signal energy for each endpoint 114. Information regarding the signal energy may be passed to control module 440. The energy level may be used as a selection parameter for selecting an appropriate one or more endpoints 114 as the source of audio and/or video for the mixed conference, which are sometimes referred to as "presentation endpoints."

The MMCU 116a and the control server 112a can communicate with each SMCU 116b over an IP network using an API. The control server 112a can execute conferencing technology applications and can be used to manage scheduled calls, ad hoc calls, call forwarding, virtual conference rooms, monitor and control MCUs (e.g., "web commanders"), etc., as disclosed in the previously incorporated patents and patent application references.

Control server 112a may include an MCC 460 for managing the composition of different windows in one or more continuous presentation layouts of a stacked conference by managing which endpoints 114 are connected to which MCU 116. In one embodiment, MCC 460 resides within control server 112. In other embodiments, MCC 460 may be a standalone device/module and/or associated with more than one MCU 116, with only one MCC 460 active during a stacked conference. MCC 460 will be discussed in more detail with reference to Figures 4 and 5.

The control module 440 may be a logic unit that controls the operation of the MMCU 116a. In addition to the general operations of a typical MCU 116, the MMCU 116a can perform additional operations due to the control module 440. Specifically, the control module 440 may include a logic module (not shown) for controlling the formation of a continuous presentation layout. The MCC 460 may process information from other MCUs 116b involved in a stacked conference and information from endpoints 114 associated with the MCU 116a. By selecting which endpoints 114 will be connected to the network area 110 of the MMCU 116a, this information may be used to determine which endpoint 114 will be selected to be displayed in the continuous presentation layout.

FIG4 shows a block diagram illustrating various components of MCC 460. MCC 460 may include, among other elements, a communication module 462, a stacked conference parameter logger 464, a decision module 466, and a stacked architecture database 468. MCC 460 communicates with each MCU 116 involved in the stacked conference via communication module 462 and, if present, control server 112a ( FIG3 ). MCC 460 may receive information (parameters) associated with the configuration of one or more stacked continuous presentation layouts. This information may be collected from different MCUs 116. Parameters may include the audio power of different participants associated with a particular MCU 116 and the window position of each participant in the associated configured continuous presentation video generated by the MCU 116. Information from different MCUs 116 may be periodically collected by MCC 460. For example, in an alternative embodiment, the transmission of this information may be automatic and may be initiated by an MCU 116 upon detecting a change in one of the parameters.

MCC 460 can send instructions to each SMCU 116b involved in the stacked conference session via communication module 462. The instructions can include selection instructions regarding which endpoints 114 should be transferred to SMCU 116b or which endpoint 114 should be transferred from MMCU 116a. Further information detailing the transfer of endpoints 114 can be found in ITU standard H.450.2, which is incorporated herein by reference. Communication module 462 can receive instructions from control server 112a ( FIG. 3 ) regarding the architecture, MCUs 116b, and endpoints 114 involved in the conference and send status information for the stacked conference to control server 112a.

Conference parameter recorder 464 is a module in which dynamic parameters of a stacked conference and its associated endpoints 114 and MCUs 116 can be stored. Conference parameter recorder 464 can be a cyclic data structure organized in sampling periods and can include a final "T" period, where "T" is, for example, in the range of one to several dozen periods. Each periodic portion can include information about the audio energy of each participant, whether their image is included in the current one or more associated continuous presentation layouts, and so on. For example, the sampling period can range from tens of milliseconds to several seconds. In one embodiment, sampling parameters from different MCUs 116 can be managed by conference parameter recorder 464 and run in parallel with the activities of MCC 460. In an alternative embodiment, conference parameter recorder 464 can be managed by decision module 466.

To eliminate frequent changes in endpoint connectivity and changes in the video image of the presented endpoint 114, parameter recorder 464 may collect parameters from "J" sampling periods stored in conference parameter recorder 464 and select endpoints 114 where specific participants frequently speak louder than others. Although those participants were not the loudest speakers in the last one or two sampling periods, they may be designated as the currently presented endpoint. The value of "J" may be less than or equal to "T." Embodiments of the present invention may also utilize other methods for selecting the presented endpoint.

The architecture database 468 is a database that stores information related to each endpoint 114 and MCU 116 in a stacked conference. This information may include associations between endpoints 114 and MCUs 116, as well as associations between different MCUs 116 used for the current stacked conference. For example, information related to different stacking layouts that can be requested by an endpoint 114, as well as selection rules for selecting an appropriate endpoint 114, may also be stored in the architecture database 468. Additionally, address and alias information related to each endpoint 114, such as the IP address for each endpoint 114 and MCU 116, may also be stored. Information about the architecture may also be received from the control server 112a. Furthermore, the architecture database 468 may include dynamic information about the current layout, such as, but not limited to, which endpoints 114 are currently connected and to which MCU 116 they are connected, when new endpoints join or leave the conference, and so on. This information may be delivered from the relevant MCU 116.

Decision module 466 manages the composition of different stacked conference layouts involved in the current stacked continuous presence conference by managing the connections of different MCUs 116. Based on the conference parameters stored in conference parameter recorder 464 and the layout requirements stored in architecture database 468, decision module 466 can determine which endpoints 114 should be connected to MMCU 116a, and therefore which endpoints 114 should be mixed in each stacked conference layout.

FIG5 is a flow chart illustrating a process 600 for establishing and controlling a stacked continuous presence video conference according to the disclosed embodiment. For clarity and simplicity, process 600 is disclosed as being implemented by the active MCC 460 as part of the control server 112a in the control module 440 (as shown in FIG3 ). Process 600 can be initiated (602) by the control server 112a from the stack master MCU 116 at an appropriate start time (e.g., at the start time of a scheduled conference or when initiating an ad hoc call stacked continuous presence conference). The stack master MCU 116 manages all resources in the stacked conference and can be any of the MCUs 116 connected to the stacked conference. For clarity and simplicity, it is assumed that the stack master MCU 116 is also the conference master MMCU 116a, although this is not required.

At initiation, the stack master MCU 116 can poll the other MCUs 116 in the conference to determine the amount of resources (e.g., the number of video encoders and decoders) each MCU 116 has. Alternatively, after startup, the MCU 116 will report the amount of resources available to the stack master MCU 116.

The stack master MCU 116 then selects (604) a conference master MMCU 116a that may have the basic resources (i.e., the minimum number of encoders and decoders) to support the number of endpoints 114 to be displayed in the continuous presentation (CP) layout. For example, if a 2x2 CP layout is desired, the number of endpoint video streams to be presented ("N") is four. And ("K") may be the total number of endpoints to participate in the conference. In one embodiment, the MCU 116 with the largest number of resources is selected as the conference master MMCU 116a. In another embodiment, the first MCU 116 with the smallest number of resources is selected. The smallest number of resources may be defined, for example, as N decoders and N+1 encoders ("basic resources"). If no MCU 116 is determined to have at least the basic resources, the second MCU 116 will be selected as the master conference slave and configured as a stacked or cascaded SMCU 116b.

The stack master MMCU 116a may then poll each SMCU 116 for endpoint information for each SMCU 116. This information (which may include the IP address for each endpoint) is stored (606) in the stack architecture database 468 discussed above with reference to FIG4. Thus, the stack master keeps track of each endpoint 114 and where each endpoint 114 is connected. The endpoints 114 may be ordered as E(1) to E(K).

The MCC 460 then directs each SMCU 116b to transfer endpoint connections according to the desired continuous presentation layout (608). For example, if two endpoints 114 E(1) and E(2) are initially connected to the MMCU 116a, the MCC 460 can direct the MCU 116b controlling endpoints 114 E(3) and E(4) to transfer their connections to the MMCU 116a so that the MMCU 116a has at least N endpoints connected to its network area 110. The endpoints 114 that are initially selected to be displayed in the CP layout can be selected based on predetermined parameters. In this regard, certain endpoints 114 can be designated as the endpoints 114 that will be initially shown in the CP layout. In one embodiment, this is achieved by seeding a value for the audio energy associated with the endpoints 114.

The loop begins at step 612 and runs as long as the stacked continuous presentation conference is active and terminates when the stacked continuous presentation conference ends. In step 612, information about each endpoint 114 is retrieved from different MCUs 116. The information about each endpoint 114 can include the current audio energy and its window position in the relevant layout (if any). The information can be stored in the conference parameter recorder 464 (Figure 4). In an embodiment where the conference parameter recorder 464 is a circular buffer, a set of information belonging to the earliest sampling period may be deleted. The information stored in the conference parameter recorder 464 can be retrieved and processed (616) by the decision module 466 (Figure 4) to determine whether the layout and/or the connection of the endpoint 114 need to be changed.

In step 616, after selecting the presentation endpoints to display, the current mix of each stacked conference layout is examined and a determination is made as to whether a change to the mix of one or more MCUs 116b is required. If not, the decision module 466 may wait for the next iteration. If a change to the video mix is required, the decision module 466 may determine which MCUs 116 are involved in the current change and in which window of the CP layout the video images of the new endpoint or endpoints should be placed. Appropriate instructions are sent to the relevant MCUs 116, i.e., those connected to the relevant endpoints. Once the required changes have been made, the decision module 466 may wait for the next iteration.

Different methods can be used to determine whether a change is needed. For example, one method might rank the participants from highest to lowest audio energy (the number of participants depends on the number of possible windows in the CP layout, N in this example) and select the endpoint with the higher audio energy in the first "t0" sampling periods as the presented endpoint ("t0" can be any number from one to the maximum number of sampling periods stored in the conference parameter recorder 464). For example, if the audio energy of endpoints E(1) to E(N) connected to the MMCU 116a is greater than the audio energy from endpoints E(N+1) to E(K), then no change is needed. Other determinations are possible; another method might select the most frequent and loudest N speakers over the entire sampling period stored in the conference parameter recorder 464; another method might add a new participant to replace the weakest speaker selected in the last period. In an alternative embodiment, the removed presented endpoint can be a less frequent speaker, etc. The process 600 can then compare the newly selected participant with the currently displayed participants to determine (620) whether changes are required in the current layout. If, at step 620, no changes are required to the current CP layout, the process 600 can return to step 612 and wait for the next iteration.

If, at step 620, a change in the CP layout (either in the layout or in the selection of one or more participants) is required, and the endpoint to be proposed (newly displayed) is not connected to the MMCU 116a, the decision module 466 may determine (624) the required changes in each MCU 116. The changes may include instructions for transferring the newly proposed endpoint 114 to the network area 110a of the MMCU 116a, and transferring the endpoint E(N) (e.g., the endpoint 114 among the endpoints E(1)--E(N) with the least amount of audio energy). The new organization may be communicated to each MCU 116 via the signaling and control connections. Upon receiving the new settings, each MCU 116 updates (626) its internal resources accordingly to provide the required new audio and video mix. The process 600 then returns to step 612 and performs the next iteration.

The conference master MCU 116a encodes the CP layout using an additional encoder and sends the encoded audio and video streams to all conference slave MCUs 116b. The encoder uses the highest parameters of the conference for encoding. Slave MCUs 116b can then use a decoder assigned to an endpoint 114 in that slave MCU 116b to decode the stream received from the master MCU 116a. Slave MCUs 116b then re-encode the stream using the encoder assigned to each endpoint 114 associated with that slave MCU 116b to generate a stream suitable for that endpoint 114.

Throughout this disclosure, the terms "unit," "element," and "module" are used interchangeably. Anything designated as a unit or module can be a standalone unit, or a dedicated or integrated module. A unit or module can be modular or have modular aspects that allow it to be easily removed and replaced with another similar unit or module. Each unit or module can be any one or any combination of software, hardware, and/or firmware.

Throughout the description and claims of this disclosure, the words "comprise," "include," "have," and their conjugations are used to indicate that the object of the verb is one or more components, not necessarily a complete list of components, parts, elements, or portions of one or more entities. Those skilled in the art will appreciate that the subject matter of this disclosure can be implemented in software for MCU 116, in additional hardware added to MCU 116, or in additional software/hardware distributed among MCU 116.

It will be appreciated that the above-described apparatus, systems, and methods may be varied in a variety of ways, including by changing the order of steps and the exact implementation scheme used. The described embodiments include different features, not all of which are required in all embodiments of the present disclosure. Furthermore, some embodiments of the present disclosure utilize only some of the features or possible combinations of features. Different combinations of the features indicated in the described embodiments will be readily apparent to those skilled in the art. Furthermore, some embodiments of the present disclosure may be implemented by combining the features and elements described in connection with different embodiments of the present disclosure.

Claims (18)

1. A method for preparing a video layout for video conferencing, comprising: Select the first multipoint control unit among multiple multipoint control units as the master multipoint control unit; Receive information associated with multiple endpoints from the second multipoint control unit among the plurality of multipoint control units; Assign the first endpoint of the plurality of endpoints to the first multipoint control unit; The second endpoint of the plurality of endpoints is assigned to the second multipoint control unit; The first multi-point control unit receives parameters regarding the plurality of endpoints, the parameters being audio parameters and associated with the composition of the video layout; and In response to the change in the parameters, the first endpoint is reassigned to the second multipoint control unit, and the second endpoint is reassigned to the first multipoint control unit, thereby changing the video layout. 2. The method according to claim 1, further comprising: Analyze the parameters and determine the order of audio energy from largest to smallest for the first and second endpoints; and Determine the changes in the audio energy from high to low. 3. The method according to claim 1, further comprising: In response to reassigning the second endpoint to the first multipoint control unit, instructions for the mixed audio and video of the first multipoint control unit are determined. 4. The method according to claim 1, further comprising: In response to reassigning the first endpoint to the second multipoint control unit, instructions for mixing audio and video for the second multipoint control unit are determined; and The instruction to mix audio and video is sent from the first multipoint control unit to the second multipoint control unit. 5. The method of claim 1, wherein, in response to a change in the parameters, reassigning the first endpoint to the second multipoint control unit and reassigning the second endpoint to the first multipoint control unit, comprises: The instruction to transmit endpoint connection is sent from the first multipoint control unit to the second multipoint control unit. 6. The method of claim 1, wherein, in response to a change in the parameters, reassigning the first endpoint to the second multipoint control unit and reassigning the second endpoint to the first multipoint control unit, comprises: Instructions for mixing audio and video are sent from the first multipoint control unit to the second multipoint control unit. 7. A multipoint control unit for video conferencing, comprising: A means for identifying the multipoint control unit as the master multipoint control unit of a video conference; A means for receiving information associated with multiple endpoints from a second multipoint control unit identified as being from a multipoint control unit; A means for assigning a first endpoint of the plurality of endpoints to the master multipoint control unit; A means for assigning a second endpoint of the plurality of endpoints to the multipoint control unit; A means for receiving parameters about the plurality of endpoints by the main multipoint control unit, the parameters being audio parameters and associated with the composition of the video layout; and A means for changing the video layout of the video conference by reassigning the first endpoint to the slave multipoint control unit and the second endpoint to the master multipoint control unit in response to a change in the parameters. 8. The multi-point control unit according to claim 7, further comprising: A means for analyzing the parameters and determining the order of audio energy from largest to smallest at the first and second endpoints; and A device for determining changes in the order of the audio energy from high to low. 9. The multi-point control unit according to claim 7, further comprising: A means for determining instructions for the mixed audio and video of the main multipoint control unit in response to reassigning the second endpoint to the main multipoint control unit. 10. The multi-point control unit according to claim 7, further comprising: Means for determining instructions for the mixed audio and video of the multipoint control unit in response to reassigning the first endpoint to the multipoint control unit; and A means for sending instructions for mixing audio and video from the master multipoint control unit to the slave multipoint control unit. 11. The multipoint control unit of claim 7, wherein the means for changing the video layout of the video conference by reassigning the first endpoint to the slave multipoint control unit and the second endpoint to the master multipoint control unit in response to a change in the parameters is further configured to: The instruction to transmit endpoint connection is sent from the master multipoint control unit to the slave multipoint control unit. 12. The multipoint control unit of claim 7, wherein the means for changing the video layout of the video conference by reassigning the first endpoint to the slave multipoint control unit and the second endpoint to the master multipoint control unit in response to a change in the parameters is further configured to: Instructions for mixing audio and video are sent from the master multipoint control unit to the slave multipoint control unit. 13. A multipoint control unit connection controller for video conferencing technology, comprising: The communication module is configured as follows: Receive instructions associated with the video conference and with the endpoints and multipoint control units involved in the video conference; and Transmit status information about the video conference; The decision module is configured as follows: The first endpoint among the multiple endpoints in the video conference is assigned to the master multipoint control unit among the multiple multipoint control units of the video conference. Assign the second endpoint among the plurality of endpoints to the second multipoint control unit among the plurality of multipoint control units, and In response to a change in parameters received by the master multipoint control unit from the plurality of endpoints, the first endpoint is reassigned to the second multipoint control unit, and the second endpoint is reassigned to the master multipoint control unit, thereby changing the video layout of the video conference, wherein the parameters are audio parameters and are associated with the composition of the video layout; Stacked conference parameter recorder, which is configured as Collect the parameters of the video conference; and Store the parameters; and Stacked meeting architecture database, which is configured as Store information related to the endpoints and multipoint control units involved in the video conference. 14. The multipoint control unit connection controller of claim 13, wherein the stacked conference parameter recorder is further configured to: Conference parameters are sampled within a sampling period. 15. The multipoint control unit connection controller of claim 14, wherein the sampled conference parameters include the audio energy of each participant in the video conference. 16. The multipoint control unit connection controller of claim 13, wherein the stacked conference parameter recorder is further configured to: The endpoints of participants associated with the video conference are selected to be louder than those of other participants. 17. The multipoint control unit connection controller of claim 16, wherein the selected endpoint is selected even if the selected endpoint was not a louder speaker in the most recent sampling period. 18. The multipoint control unit connection controller of claim 13, wherein the stacked conference architecture database comprises: Information defining the association between the endpoints and the multipoint control unit of the video conference; and Information defining the associations between the multipoint control units of the video conference.