Mobile video consultation system suitable for commanding and scheduling service
Technical Field
The invention relates to the field of video conference, in particular to a mobile video conference system suitable for commanding and scheduling services.
Background
In a real-time audio and video communication system with participation of multiple parties, three main ways exist for the functions of the system:
group voice talkback: in the real-time intercom system, any party who wants to obtain the speaking right must press the PTT key to request and can speak after being authorized. If one person in the current group is speaking, the person wants to speak, and either waits for the other party to finish speaking or preempts the speaking right of the other party. Either speaking or listening to the participating party can only be in one state, a behavior known as half-duplex speech.
Voice conference: it is common for a terminal to dial into a virtual conference room to participate in a conference. Each terminal uploads local sound to the virtual meeting room, and the local sound is mixed in the virtual meeting room and then distributed to other terminals. In a voice conference, a participant terminal can talk at any time, and can also listen to the talk of other terminals, which is called full-duplex voice call.
Video conference: usually, each party participating in the video conference distributes own video to other parties and displays the video on the screen of the other party. Therefore, it is possible for the receiving party to see videos from other parties and hear sounds in the videos. In the whole video conference process, video and voice communication adopts video stream channel transmission and is full duplex. Namely: any participant can talk and send video at any time, and can also play the talk and video of other participants.
The problems faced are:
when the terminal mobility gradually becomes a mainstream use scene, if the video conference mode is continuously adopted to realize the audio and video communication between the terminals, a plurality of problems are faced:
1) under the mobile environment, the network conditions of all parties participating in the conference are different, some signals are good, some signals are poor, and if a full-duplex audio and video conference mode is still adopted, the terminal with poor network signals can influence the whole conference, so that the conference communication is difficult and possibly cannot be carried out.
2) Under the condition of poor terminal signals, if the audio and video are transmitted by adopting a channel of video stream like the traditional video conference, the audio and the video are delayed at the same time because the occupied bandwidth of the video is large, so that the voice communication is not smooth.
3) At a mobile terminal, if receiving and displaying multiple paths of videos sent by other terminals, the problem of large network traffic usage, easy congestion and poor real-time performance can be caused, and if network signals are not good, the problem can be more serious.
4) Under the use scene of command dispatch, because under the speech mode of full duplex, anyone can talk at any time, cause the pronunciation too noisy after the audio mixing easily, the topic dispersion scheduling problem of discussion needs to have the allocation of technical means to guarantee voice communication's speech right.
In summary, in the process of implementing the technical solution of the present invention, the inventors of the present application find that the above-mentioned technology has at least the following technical problems:
in the prior art, the existing video conference mode has the technical problems that the conference effect cannot be guaranteed when the network is poor, and audio mixing is easy to occur in the video conference.
Disclosure of Invention
The invention provides a mobile video conference system suitable for commanding and scheduling services, which solves the technical problems that the conference effect cannot be guaranteed when the network is poor and the video conference is easy to generate audio mixing in the existing video conference mode, realizes that different protocol channels are adopted for video data and audio data, preferentially ensures audio data transmission when the network is poor, ensures normal operation of the audio conference, only one terminal in a group sends the video data and plays the video data at other terminals, and avoids the technical effect of generating multi-channel videos occupied by high bandwidth.
In order to achieve the purpose, the application provides a mobile video consultation system suitable for commanding and scheduling services, and the video consultation function of 'vertical to end' of bidirectional audio and video command communication is achieved. The system comprises:
the system comprises a plurality of servers, a plurality of dispatching desks and a plurality of terminals; the server keeps the signaling and data connection with the dispatching desk and the terminal by using the IP connectivity with the dispatching desk and the terminal, ensures that various signaling and management commands can be correctly sent and received between the dispatching desk and the terminal, ensures that positioning information, picture information, file information and audio and video data generated in the system can be correctly stored and forwarded, and supports the retrieval and playback after the fact; the terminal is used for acquiring real-time video information through the first protocol channel, transmitting the real-time video information to the video conference group where the terminal is located, displaying the real-time video information, and receiving and displaying the real-time video information from the video conference group where the terminal is located through the first protocol channel; the terminal is also used for collecting real-time audio information through the second protocol channel, transmitting the real-time audio information to the video conference group where the terminal is located, playing the real-time audio information, and receiving and playing the real-time audio information from the video conference group where the terminal is located through the second protocol channel; when the system needs group video consultation, the terminal establishes a video consultation group, and the video consultation group comprises a plurality of terminal members: a terminal 1 to a terminal N, wherein N is a positive integer greater than or equal to 2; when the system carries out group video consultation, real-time video information of a certain terminal is displayed in a video consultation group, and real-time voice information of the same terminal or another terminal is played; the dispatching stations and the terminals respectively have IP connectivity with the server, and data connection is established between each dispatching station and each terminal and the server for exchanging real-time audio data, real-time video data, signaling and management information; the dispatching stations are respectively provided with corresponding grades, the dispatching stations are provided with corresponding authority administration relations according to the grades, and the high-grade dispatching station can manage the low-grade dispatching station; each dispatching desk has administration right to the corresponding terminal and is used for managing the establishment of a video consultation group, video acquisition and playing, audio acquisition and playing of the terminal, wherein when the system performs group video consultation, when the network state is detected to be poor, audio data transmission is preferentially performed, and when necessary, video transmission is closed, namely when the network state does not meet the preset requirement, the system preferentially adopts a second protocol channel to perform data transmission, and when the network state meets the preset requirement, the first protocol channel and the second protocol channel are simultaneously adopted to perform data transmission.
Furthermore, the terminals have corresponding priorities, and when the system performs group video consultation and one or more terminals simultaneously request voice operation, the system performs a priority judgment operation on the talk right according to the priorities of the terminals from high to low.
Further, when the system performs group video consultation, the video consultation group allows to display real-time video information of one terminal, and the video consultation group allows to play real-time voice information of one terminal.
Further, when the system performs group video consultation, the video consultation group is also used for displaying real-time position information of each terminal, and the terminals send picture or text or file information to the video consultation group, wherein the terminals send and receive the picture or text or file information by adopting a third protocol channel.
Further, when the system performs group video consultation, the video consultation groups can add new video consultation group members and delete the video consultation group members, and the dispatching desk can join any video consultation group in the authority of the dispatching desk to perform video consultation.
Furthermore, when the system carries out group video consultation, the group video consultation initiating terminal and the dispatching desk with the corresponding authority both have the consultation control right, the video source can be switched in the consultation, and when the video source of the consultation is switched, the selected new video source terminal can be informed of ringing; and the terminal or the dispatching desk with the conference control right currently can grant the conference control right to other group members in the video conference group or transfer the conference control right of the terminal.
Furthermore, when the system performs group video consultation, the group video consultation initiating terminal and the dispatching desk with corresponding authority both have the consultation control right, and the video watching authority of other group members in the video consultation group can be controlled.
Furthermore, the dispatching desk is provided with a camera for collecting video data; the dispatching desk is also provided with a display for playing video data; the dispatching desk is also provided with a hand microphone, the hand microphone comprises a microphone and an external loudspeaker box, and the hand microphone is used for collecting and playing audio data; the dispatching desk microphone supports the acquisition and playing requirements of half-duplex audio and full-duplex audio; the terminal is provided with a camera, a touch screen, a loudspeaker and a microphone, the camera is used for collecting video data, the touch screen is used for displaying the video data and simulating a PTT key, the loudspeaker is used for playing audio data, the microphone is used for collecting the audio data, the terminal is provided with the PTT key or simulates the PTT key through a UI interface and is used for realizing audio collection control, when the PTT key is pressed down, a voice call request is sent, the voice call request is carried out, and when the PTT key is popped up, the voice call is ended.
The mobile video conference system is provided with a full-duplex call mode, when the full-duplex call mode is started, the corresponding terminals can carry out full-duplex voice call in a video conference group, and the system can support the full-duplex call mode when needed.
Further, the dispatching desk and the terminal are provided with an address book, and the address of the address book comprises: three kinds of dispatching desk address, terminal address and group address;
the address information stored in the address book of the dispatching station comprises: the dispatcher station address stores other dispatcher stations, namely: the information of the dispatching desk directly or indirectly administered by the dispatching desk is stored at the same time; the information of all terminals directly administered by the dispatching desk; the dispatching desk directly manages all groups in which the terminals join; when the dispatching desk commands and supervises the dispatching desk managed by the dispatching desk, the three addresses in the address book of the dispatching desk can be synchronized;
the address information stored in the terminal address book comprises: directly managing the information of a dispatching desk of the terminal; information of other terminal members in all groups in which the terminal participates; group information in which the terminal participates.
Further, the terminal when establishing a group video conference includes:
the method comprises the steps that a terminal establishes a group, a terminal a1 initiates a group Grp1 establishment, a terminal a2 and a terminal a3 of an address book are selected to join, and the newly established Grp1 is automatically notified to a dispatching desk A, a terminal a2 and a terminal a 3;
a terminal a1 initiates a video conference in Grp1, real-time video of the terminal a1 is automatically uploaded to a server S and is distributed to other member terminals a2 and a3, and any one of all members of Grp1 can carry out real-time voice while watching the video of the terminal a1 through the terminals a2 and a 3;
in the Grp1 consultation process, the dispatching desk A can enter Grp1 to participate in the consultation, the consultation can be informed when the dispatching desk A enters, the real-time video uploaded by a1 can be seen by the participation of the consultation, and the dispatching desk A can also carry out real-time voice operation in Grp 1;
after the dispatching desk A participates in Grp1 consultation, a dispatching desk B can be added to enter the consultation, and the dispatching desk B obtains a ringing prompt and selects whether to join or not while being added to the consultation;
after the dispatching station B joins the conference, other dispatching stations or terminals in the own address book can be continuously added to enter Grp1, and the added dispatching stations or terminals obtain a ringing prompt and can be selectively added or rejected; and the multilevel dispatching desks are added into a group for video consultation, so that the requirements of cooperative command and flat command are met.
In the process of video consultation, a consultation initiator or a dispatching desk can switch video sources; namely: stopping video transmission of the current video source, selecting other terminals or dispatching stations participating in the conference as new video sources, selecting the terminal or dispatching station as the video source, obtaining a ringing notification, and allowing the terminal or dispatching station to select to accept video transmission or refuse;
in the process of consultation, the dispatching desk or the terminal can send pictures, characters or files to other participants in a group;
the conference initiator or the participating dispatcher station can close the current conference.
Further, in the mobile video conference system, the server S can establish a connection with other trunking intercom networks through the gateway, and establish a binding relationship between a video conference group and a certain group of the intercom networks.
One or more technical solutions provided by the present application have at least the following technical effects or advantages:
the technical effects that different protocol channels are adopted for video data and audio data, audio data transmission is preferentially ensured when a network is not good, normal running of an audio conference is guaranteed, video data of only one terminal in a group is played, and generation of multi-channel videos occupied by high bandwidth is avoided are achieved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the embodiments of the invention and are incorporated in and constitute a part of this application, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention;
FIG. 1 is a schematic diagram of the composition and direct jurisdictions of a mobile video conference system in accordance with the present application;
FIG. 2 is a schematic diagram of a server connected to other trunked talkback networks through a gateway;
FIG. 3 is a schematic diagram of the composition and direct, brief introduction jurisdiction of a mobile video conference system;
fig. 4 is a schematic diagram of a cross-department, cross-hierarchy, and flattened temporal conductor set of the mobile video conference system.
Detailed Description
In order that the above objects, features and advantages of the present invention can be more clearly understood, a more particular description of the invention will be rendered by reference to the appended drawings. It should be noted that the embodiments and features of the embodiments of the present application may be combined with each other without conflicting with each other.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced in other ways than those specifically described and thus the scope of the present invention is not limited by the specific embodiments disclosed below.
Referring to fig. 1-4, the present invention combines the techniques of intercom and one-way video sharing in the same group, and combines the management and command levels of the multi-level dispatching desk, so as to implement a new mobile video conference system suitable for commanding and dispatching services. The system shares the group talkback voice and the single-channel video and respectively adopts different protocol channels for transmission, so that the time delay of voice transmission can be ensured as much as possible even under the condition of unsmooth video transmission. Thus highlighting the performance advantages of single-channel video conferences and half-duplex voice, and can be used in a wired network in reverse.
Referring to fig. 1, a schematic diagram of system components and administration relations is shown, in which a video conference system includes: one or more servers; one or more dispatching desks; a plurality of mobile terminals.
The dispatching desk has administration relation to the mobile terminal and other dispatching desks, as shown by solid line in fig. 1; the dispatching desk with jurisdiction relation has three levels in the figure, and more levels can be actually defined.
The dispatching stations and the mobile terminals have IP connectivity with the server respectively, and each dispatching station and the terminal have a plurality of data connections with the server for exchanging real-time audio and video data, signaling and management information.
When the dispatching desk does not exist in the system, the terminal can also form a mobile video conference system with the server S. All parts related to the dispatcher station, whether the address book or the conference scenario, are empty, as described below.
Hardware characteristics:
a dispatching desk: the microphone is provided with a hand microphone (comprising a microphone and an external sound box), a camera and a display; a terminal: having a camera, a touch screen, a speaker, a microphone, and a PTT button (or simulating a PTT button through a UI interface); a server: AMD64CPU, LINUX server.
Address book and its address definition:
the dispatching desk and the terminal realize an address book, and the address book is divided into three parts: 1) a dispatcher station address; 2) a terminal address; 3) a group address;
address information stored in the address book of the dispatching station: 1) the dispatching station address stores the information of all other dispatching stations directly or indirectly administered by the dispatching station, and also stores the information of all the dispatching stations directly or indirectly administered by the dispatching station; 2) the information of all terminals directly managed by the user; 3) group information of all terminals under the direct control of the user;
when the dispatching desk commands and supervises the dispatching desk managed by the dispatching desk, the address book of the dispatching desk directly or indirectly managed by the dispatching desk can be synchronized, namely, the address book address of the dispatching desk at the moment is added with all the addresses of the address book of the dispatching desk commanded and supervised.
Address information stored in the terminal address book: 1) directly polices own dispatching desk information; 2) information of other terminal members in all groups in which the terminal participates; 3) the group information that the terminal itself participates in;
group (2):
the system can predefine or dynamically establish a group, the group is a basic consultation unit, the group only consists of terminals, and the dispatching desk can forcibly enter the group to participate in the consultation as long as the dispatching desk governs any terminal in the group.
And (3) a consultation scene:
scene 1: the terminal establishes a temporary mobile video conference, and needs the following steps:
step 1: the terminals establish a group first, for example, a1 in the figure initiates the establishment of group Grp1, selects a2 and a3 of the address book to join, and then the newly-established Grp1 will be automatically notified to the dispatcher station a and the terminals a2 and a 3.
Step 2: a1 initiates a video conference in Grp1, the real-time video of a1 is automatically uploaded to a server S, and other members a2, a3 are distributed, while the video of a1 is watched by a2 and a3, all members of Grp1 can simultaneously carry out voice one-to-many communication (the voice only supports half-duplex talkback voice), so that the effect of conference while watching the video is achieved. In the process of meeting, the a1, a2 and a3 mutually share own position information to each other, and the personnel of each participant can see the positions of other participants.
Wherein: the main characteristics of the consultation: a) the consultation units are groups, whether predefined groups or temporarily established groups; b) voice talkback, one-to-many half-duplex real-time talkback (full-duplex conversation can also be supported under the condition of independent transmission of audio data); c) and sharing 1 path of video in the group, and realizing 1-to-many sharing and forwarding. d) Different data channels are used for the one-to-many audio stream of voice talkback and the one-to-many video stream of video distribution, so that the two streams are not influenced mutually, and when an air interface signal is not good, the smoothness of a voice channel can be preferentially ensured; e) the video can collect audio information, and sound information in the video is played by default at a player, but as long as any person in the group speaks, the video sound is automatically muted; f) the dispatching stations in different levels can forcibly enter the group of the meetings to participate in the meetings.
And step 3: in the Grp1 consultation process, the dispatching desk A can forcedly enter the Grp1 to participate in the consultation, the consultation can be informed when the consultation enters the Grp1, the participant can see the real-time video uploaded by the a1, and the dispatching desk A can also keep voice one-to-many communication with other group members like the group members.
And 4, step 4: after the dispatching station A participates in the Grp1 consultation, a dispatching station B (or a dispatching station C) can be added to enter the consultation, and the dispatching station B is prompted and can select whether to join or not while being joined.
And 5: after the dispatcher station B joins the conference, other dispatcher stations or terminals in the own address book can be added to enter Grp 1. The added dispatcher station or terminal is prompted by the ringing, and can choose to join or reject.
Step 6: in the process of video conference, a conference initiator or a dispatching desk has the control right of the conference and can switch video sources, namely: and forcibly stopping the current video sender from sending the video, and selecting other terminals or dispatching desks participating in the conference to send the video. The selected terminal and the dispatching desk can obtain the ringing notification, and can choose to accept the video transmission or reject the video transmission.
And 7: during the course of the consultation, any participating dispatching desk or terminal can send pictures, texts and other files to other participants in groups.
And 8: the conference initiator or the participating dispatching desk has the conference control right, and the current conference can be closed.
Scene 2: the terminal initiates a mobile video conference in the existing group, and only the steps 2 to 8 of the scene 1 need to be followed.
Scene 3: the dispatching desk initiates a temporary video consultation, and needs to add a step 9 besides all steps of following the scene 1: after the video conference is closed, all members automatically quit the group, and the group is deleted.
Scene 4: the dispatcher initiates a consultation in the existing group, following the steps of scenario 2.
In the implementation process of the invention, the group voice talkback and the 1-channel video are shared and combined in the group, so that the participating parties can see the video of the 1-channel video source in the group while participating in the voice talkback, the participating parties of the conference also automatically share and see the real-time positions of other members, share the information of characters, files and pictures in the group, and the receiving end can view the information.
In the implementation process of the invention, voice and video are processed separately, and when the video can not be transmitted smoothly, the system automatically improves the priority of a voice channel and ensures the transmission of the voice; the talkback communication supports different members to set different priorities, and after the talkback key is pressed down, the system judges the speaking right, supports the group preemption and ensures that the terminal with high priority preferentially acquires the speaking right.
In the implementation process of the invention, the user who is on-line after the conference starts actively inquires the video conference group related to the system and prompts the user to join the conference, so that the user can join the conference. Even if the dispatching desk is not taken as a member of the group, the dispatching desk can be taken as a policeman of the group member and can be forcibly joined into the group for mobile video consultation; in the group, both the sponsor of the conference and the dispatching desk have the control right of the conference, the video source can be switched in the conference, and when the video source of the conference is switched, the selected new video source terminal can be informed by ringing. Meanwhile, the current controller can grant control to other group members or transfer own control.
In the implementation process of the invention, when a conference is carried out, both the sponsor of the conference and the dispatching desk can control the members to which the video source is played. On one hand, when some terminals do not need to watch, the traffic can be saved, and on the other hand, the terminals with poor network signals can only concentrate on group voice talkback without watching videos. In the process of the conference, the initiator and the dispatching station can add other dispatching stations and terminals to the group to participate in the conference. The address lists of the dispatching stations and the terminals define lists of other dispatching stations and terminals which can be selected to join in the group, and the lists are highly consistent with service scenes of users in many industries;
in the implementation process of the invention, when the video is collected in the consultation process, the video and the audio are collected together, but the audio information contained in the video stream cannot be played in real time at the video playing end. The video stream can be automatically stored as a video file in the system, and after the conference is finished, the audio and video information in the video file can be synchronously played back. The consultation can be carried out in any group, particularly cross-department consultation under a specific scene, and is particularly convenient to consult in a temporarily established group. The consultation terminal presses the PTT button to participate in talkback, and can also press the soft PTT button on the screen to participate in talkback.
Referring to fig. 2, fig. 2 is a schematic diagram illustrating that a server is connected to other trunking talkback networks through a gateway, and in the mobile video conference system, the server S may be connected to other trunking talkback networks through a gateway to bind a mobile video conference group to a certain group of the trunking talkback networks. Referring to fig. 2, the group of conference G11 is bound with the group (channel) of the conventional trunking communication system to form a larger group across the network, so as to realize the intercommunication between the audio data of the mobile video conference and the conventional trunking intercom system. The basis for realizing the functions is as follows: the audio and video data of the conference are transmitted through different data channels. Therefore, to realize the intercommunication with the traditional cluster network, only the audio data channel needs to be intercommunicated with the traditional cluster network.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.