CN111131751B - Information display method, device and system for video network conference - Google Patents

Abstract

The embodiment of the invention provides an information display method and system for a video networking conference, which comprises the following steps: the video network server respectively acquires video stream data from each video network meeting place terminal; the method comprises the steps that a video network server receives a scheduling request sent by a video network scheduling terminal; the video networking server determines a target video networking conference terminal from a plurality of video networking conference terminals according to the scheduling request; the video network server acquires target video stream data from video stream data of a target video network meeting place terminal and sends the target video stream data to a video network meeting analysis terminal which has accessed into a meeting in advance; the video network server receives participant information which is returned by the video network conference analysis terminal and generated according to target video stream data; and the video network server sends the information of the participants to a video network scheduling terminal. Therefore, the conference dispatcher can know the information of the participants participating in the conference in each meeting place in real time.

Description

Information display method, device and system for video network conference

technical field

The present invention relates to the technical field of the Internet of Video, in particular to an information display method of the Internet of Video conference and an information display system of the Internet of Video conference.

Background technique

The Internet of View is a real-time network that can realize real-time transmission of high-definition video across the network, push many Internet applications to high-definition video, and realize high-definition face-to-face.

Internet-of-Vision video conferencing refers to two or more individuals or groups in different places, through transmission lines and Internet-of-Vision equipment, audio, video, and document data are exchanged to achieve real-time interactive communication.

There are deficiencies in the current method of obtaining and displaying personnel information in the video conference. For example, the existing method of counting the number of participants in the video conference is to manually input the number of participants to count the number of participants when the Pamir interface is used to indicate the suspension of the meeting. number of people.

Contents of the invention

In view of the above problems, the embodiments of the present invention are proposed to provide an information display method for a video conference and a corresponding information display system for a video conference that overcome the above problems or at least partially solve the above problems.

In order to solve the above problems, the embodiment of the present invention discloses a method for displaying information of a video networking conference, which is applied to a video networking conference system. The video networking conference system includes a video networking server, a video networking analysis terminal, a video networking scheduling terminal, and more a video network site terminal; the method includes:

The video-networking server obtains video stream data from each video-networking venue terminal respectively; the video stream data is the venue environment data collected by the video-networking venue terminal;

The video networking server receives the scheduling request sent by the video networking scheduling terminal;

According to the scheduling request, the Internet-of-Vision server determines a target Internet-of-Vision venue terminal from among the plurality of Internet-of-Vision venue terminals;

The video networking server acquires target video stream data from the video stream data of the target video networking site terminal, and sends the target video stream data to the video networking conference analysis terminal that has joined the video conference in advance;

The video networking server receives the participant information generated according to the target video stream data returned by the video networking conference analysis terminal;

The video networking server sends the participant information to the video networking scheduling terminal.

Optionally, the scheduling request includes sampling duration and identification information of the Internet-of-Vision site terminal; the Internet-of-Vision server obtains the target video stream data from the video stream data of the target Internet-of-Vision site terminal, and sends the The target video stream data is sent to the pre-joined video conference analysis terminal, including:

The video networking server uses the identification information to determine the sampling order of the target video networking site terminals;

According to the sampling order and the sampling duration, the video-network server sequentially acquires target video stream data from the video stream data of the target video-network site terminal, and sends the target video stream data to the video-network Conference analysis terminal.

The embodiment of the present invention also discloses an information display system for a networked video conference, including: a networked video server, a networked video conference analysis terminal, a networked video scheduling terminal, and a plurality of networked video conference site terminals;

The video network scheduling terminal is used to send a scheduling request to the video network server, and display the participant information of each video network site terminal sent by the video network server;

The video-networking server is used to obtain video stream data from each video-networking venue terminal respectively; the video stream data is the venue environment data collected by the video-networking venue terminal. When requesting, determine the target Internet-of-Vision site terminal from the multiple Internet-of-Vision site terminals, and obtain the video stream data of the target Internet-of-Vision site terminal and send it to the Internet-of-Vision conference analysis terminal that has joined the conference in advance; and The conference participant information generated according to the target video stream data returned by the video-network conference analysis terminal is sent to the video-network scheduling terminal.

Optionally, the Internet of Video conference analysis terminal is pre-assigned with a virtual number of the Internet of Video;

The video networking scheduling terminal is used to add the video networking virtual number of the video networking conference analysis terminal to the conference list, so as to pull the video networking conference analysis terminal into the meeting.

Optionally, a conference analysis system is deployed on the video networking analysis terminal; the video networking conference analysis terminal is specifically configured to, when receiving the target video stream data sent by the video networking server, pass the conference analysis system The target video stream data is converted into picture information, face detection is performed on the picture information, and participant information is determined.

Optionally, the analysis terminal of the Internet-of-Vision conference is further configured to bind the information of the participants to the identification information of the corresponding Internet-of-Vision conference site terminal after determining the participant information, and bind the bound The participant information and the identification information are sent to the Internet of View server.

Optionally, a meeting list is maintained in the video-network scheduling terminal, and the meeting list includes meeting analysis options;

The video network scheduling terminal is also used to judge whether the trigger information for the conference analysis option is received when the conference site analysis service is started;

If yes, obtain and display participant information of the video conference corresponding to the conference analysis option.

The embodiment of the present invention also discloses an information display device for a networked video conference, which is applied to a networked video conference system. The networked video conference system includes a networked video server, a networked video analysis terminal, a networked video scheduling terminal, and a plurality of networked video conference sites a terminal; said means comprising:

The video stream data acquisition module is used for the video network server to obtain video stream data from each video network site terminal respectively; the video stream data is the site environment data collected by the video network site terminal;

A scheduling request receiving module, configured for the video networking server to receive the scheduling request sent by the video networking scheduling terminal;

A target video-network site terminal determination module, configured for the video-network server to determine a target video-network site terminal from the plurality of video-network site terminals for the scheduling request;

The target video stream data acquisition module is used for the video network server to obtain target video stream data from the video stream data of the target video network site terminal, and send the target video stream data to the pre-joined conference Video conference analysis terminal;

The participant information receiving module is used for the video networking server to receive the participant information generated according to the target video stream data returned by the video networking conference analysis terminal;

The participant information sending module is used for the video networking server to send the participant information to the video networking scheduling terminal.

Optionally, the scheduling request includes the sampling duration and the identification information of the Internet-of-Vision site terminal; the target video stream data acquisition module includes:

The sampling sequence determination submodule is used for the video networking server to determine the sampling sequence of the target video networking site terminal by using the identification information;

The target video stream data acquisition sub-module is used for the video network server to obtain target video stream data from the video stream data of the target video network site terminal in sequence according to the sampling order and the sampling duration, and the The target video stream data is sent to the video conference analysis terminal that has joined the conference in advance.

The embodiment of the present invention also discloses an information display device for a video networking conference, including:

one or more processors; and

One or more machine-readable media having instructions stored thereon, when executed by the one or more processors, causes the device to perform one or more methods for presenting information in a video conference as described above.

The embodiment of the present invention also discloses a computer-readable storage medium, the computer program stored in the medium enables the processor to execute the information presentation method for the video conference according to any one of the above items.

Embodiments of the present invention include the following advantages:

In the embodiment of the present invention, the analysis terminal is introduced into the video network conference to perform face detection on the video streams from the terminals of each venue, so as to determine the participant information of each branch venue, and send the participant information to the scheduling terminal for display , so that the conference scheduler can know the information of the participants participating in the conference at each conference site in real time.

Description of drawings

Fig. 1 is a flow chart of the steps of an embodiment of an information presentation method for a video networking conference in the present invention;

Fig. 2 is a structural block diagram of an embodiment of an information display system for a video networking conference of the present invention;

Fig. 3 is a structural block diagram of a kind of visual networking conference system of the present invention;

Fig. 4 is a structural block diagram of an embodiment of an information display device for a video networking conference in the present invention;

Fig. 5 is a schematic diagram of networking of a video network according to the present invention;

Fig. 6 is a schematic diagram of the hardware structure of a node server of the present invention;

FIG. 7 is a schematic diagram of a hardware structure of an access switch of the present invention;

FIG. 8 is a schematic diagram of a hardware structure of an Ethernet protocol conversion gateway according to the present invention.

Detailed ways

In order to make the above objects, features and advantages of the present invention more comprehensible, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

The core idea of the embodiment of the present invention is to introduce a networked video conference analysis terminal into the networked video conference system, and perform face detection on the venue environment data collected by the networked video conference terminal through the conference analysis system deployed on the video network conference analysis terminal, In order to obtain the information of the participants in the venue, and send it to the dispatching terminal of the Internet of View for display.

Referring to FIG. 1 , it shows a flow chart of the steps of an embodiment of an information display method for a video networking conference according to the present invention. The method can be applied to a video networking conference system. The video networking conference system includes a video networking server, and a video networking analysis A terminal, an Internet-of-Vision dispatching terminal, and a plurality of Internet-of-Vision site terminals; specifically, the following steps may be included:

Step 101, the video network server respectively obtains video stream data from each video network site terminal; the video stream data is the site environment data collected by the video network site terminal;

In the embodiment of the present invention, the video network server will receive the video stream data uploaded by the video network site terminal in real time. The video stream data is environmental data of the venue collected in real time by the terminal of the internet-of-view venue.

Step 102, the video networking server receives the scheduling request sent by the video networking scheduling terminal;

The Internet-of-Vision scheduling terminal is used to host the Internet-of-Vision conference. By inputting corresponding instructions on the Pamir interface of the Internet-of-Vision dispatching terminal, the progress of the conference can be controlled. Specifically, it can include managing the participating terminals and controlling the speaking object.

The Internet of Vision scheduling terminal can send a scheduling request to the Internet of Vision server according to actual needs, so as to control the interaction between the Internet of Vision server and other devices.

Step 103, the Internet of Vision server determines a target Internet of Vision terminal from the plurality of Internet of Vision site terminals for the scheduling request;

When the conference analysis mode is turned on on the Pamir interface of the Internet-of-Vision scheduling terminal, all or part of the conference sites can be selected as required, and the video stream data collected by the Internet-of-Vision conference site terminals of the selected conference sites can be analyzed.

Step 104, the video network server obtains target video stream data from the video stream data of the target video network site terminal, and sends the target video stream data to the video network conference analysis terminal that has joined the conference in advance;

The video-network conference analysis terminal is a virtual terminal with a corresponding video-network terminal number, which can be connected to the video-network conference system to analyze the data transmitted from the video-network server.

In an example, the virtual Internet of Vision number of the Internet of Vision analysis terminal can be added to the conference list through the Pamir interface, and the Internet of Vision analysis terminal can be marked.

In the embodiment of the present invention, the video networking server may acquire target video stream data from the video stream data according to the scheduling request, and send the target video stream data to the video networking conference analysis terminal for data analysis.

In the embodiment of the present invention, the scheduling request may also include the sampling duration and the identification information of the Internet-of-Vision site terminal, and the Internet-of-Vision server obtains the target video stream data from the video stream data for the scheduling request, and The step of sending the target video stream data to the video network conference analysis terminal may include:

S11. The video networking server determines the sampling sequence of the video networking site terminals by using the identification information;

In practical applications, the Internet-of-Vision scheduling terminal can select a venue that needs to count personnel information, add the identification information (such as the number of the Internet-of-Vision terminal) of the Internet-of-Vision venue terminal in the venue to the scheduling request, and send it to the Internet-of-Vision server. After receiving the scheduling request, the Internet-of-Vision server can analyze the scheduling request to obtain the identification information of the Internet-of-Vision terminals, and determine the sampling order of the Internet-of-Vision venue terminals according to the identification information of the Internet-of-Vision venue terminals, so as to sequentially start from the Internet-of-Vision terminals according to the sampling order. The site terminal acquires the video stream data.

S12. According to the sampling sequence and the sampling duration, the video networking server sequentially acquires target video streams from the video stream data, and sends the target video streams to the video networking conference analysis terminal.

In the case of many participating sites, the total amount of data uploaded by all sites to the SVNET server will be large. In this case, if SVNET processes all the data, it will bring a heavy load to the SVNET server . Therefore, in the embodiment of the present invention, the sampling duration can be set, and the sampling duration can be set according to the actual situation (such as 2s, 3s, 5s, 8s), which is not specifically limited in the present invention. According to the sampling duration, part of the video stream data is intercepted in real time from the video stream data for processing.

Step 105, the video networking server receives the participant information generated according to the target video stream data returned by the video networking conference analysis terminal;

In the embodiment of the present invention, the participant information may include information on the number of participants and identity information of the participants.

In the embodiment of the present invention, the video network conference analysis terminal can convert the received target video stream data into picture information through the conference analysis system, and perform face detection on the obtained picture information through the conference analysis system to determine the participants information, and send the participant information to the Internet of Things server.

In the specific implementation, after receiving the video stream data, the video conference analysis terminal will first decode the video stream data through the conference analysis system, decode the video stream data in H264 format into YUV420P format; then decode the data in YUV420P format Encoded into JPG format, this conversion is to compress the image, which can reduce the data of the image to one tenth of the original. After converting the video stream data into JPG pictures, the conference analysis system can send the JPG pictures to the face detection system (such as NCNN neural network, NCNN is an extremely optimized high-performance neural network forward computing framework) for inference detection, and obtains The result of face detection; and according to the result of face detection, the information of the participants is counted, and the information of the participants is sent to the server of the Internet of Things.

In one example, since there may be multiple conference sites, in order to avoid errors in correspondence between the conference site and the corresponding participant information, the video-network conference analysis terminal can combine the participant information of each conference site with the terminal of the video-network conference site terminal used by the conference site The number is bound, and the bound participant information and the terminal number of the venue are sent to the Internet of Things server.

Step 106, the Internet of Vision server sends the information of the participants to the Internet of Vision scheduling terminal.

After receiving the participant information, the video network server can send the participant information to the video network dispatching terminal, which is used to display the participant information, so that the conference dispatcher can understand the situation of the conference participants in each venue.

In the embodiment of the present invention, by introducing an analysis terminal into a networked video conference, face detection is performed on the video streams from terminals in each venue to determine the information of participants in each venue, and send the participants to the scheduling terminal for display, thereby This enables the conference scheduler to know the information of the participants participating in the conference at each conference site in real time.

Referring to Fig. 2, it shows a structural block diagram of an information display system embodiment of a video networking conference of the present invention; including: a video networking server 201, a video networking conference analysis terminal 202, a video networking scheduling terminal 203, and a plurality of video networking conference sites Terminal 204;

In the embodiment of the present invention, the video networking scheduling terminal 203 is used to send a scheduling request to the video networking server 201 and display the participant information of each video networking site terminal 204 sent by the video networking server 203 .

The video network server 201 is used to obtain video stream data from each video network conference site terminal 204 respectively; the video stream data is the site environment data collected by the video network conference site terminal 204, when receiving the scheduling request sent by the video network scheduling terminal 203, The video network server 201 can determine the target video network site terminal from a plurality of video network site terminals 204, and after obtaining the video stream data of the target video network site terminal, send it to the video network conference analysis terminal 202 that has joined the conference in advance; and The video network conference analysis terminal 202 returns the participant information generated according to the target video stream data and sends it to the video network scheduling terminal 203 .

In one embodiment of the present invention, the video networking conference analysis terminal 202 is pre-allocated with a video networking virtual number;

The video networking scheduling terminal 203 is also used to add the video networking virtual number of the video networking conference analysis terminal 202 to the conference list, so as to pull the video networking conference analysis terminal 202 into the meeting.

In the embodiment of the present invention, a conference analysis system is deployed on the video networking conference analysis terminal 202; the video networking conference analysis terminal 202 is specifically used to receive the target video stream data sent by the video networking server 201 The conference analysis system converts the target video stream data into picture information, performs face detection on the picture information, and determines conference participant information.

In the embodiment of the present invention, when the video network conference analysis terminal 202 receives the target video stream sent by the video network server 201, the target video stream can be converted into picture information through the conference analysis system.

In an example, the conference analysis system may first decode the target video stream in H264 format into YUV420 format, and then encode the data in YUV420 format into a picture in JPG picture format.

After obtaining the picture information, the Internet-based video conference analysis terminal 202 can perform face detection on the picture information to determine the information of the participants.

In an example, the meeting analysis system can input the JPG picture into the NCNN neural network for inference detection, obtain the face detection result, and determine the participant information from the face detection result.

After determining the participant information, the participant information can be sent to the Internet of View server 201, and the Internet of View server 201 sends the participant information to the Internet of View scheduling terminal 203 for display.

In the embodiment of the present invention, the video networking conference analysis terminal 202 is also used to bind the participant information with the identification information of the corresponding video networking site terminal 203 after determining the participant information, and bind the The subsequent participant information and the identification information are sent to the Internet of View server 201.

In the embodiment of the present invention, since there may be multiple conference sites, in order to avoid errors in correspondence between the conference site and the corresponding participant information, the video network conference analysis terminal 202 can combine the participant information of each conference site with the video network conference site used by the conference site The terminal number of the terminal 203 is bound, and the bound participant information is sent to the video network server 201 together with the terminal number of the venue.

In an example, the Internet of Vision conference analysis terminal 202 can also send the face detection time, meeting ID, and the detected current picture path to the Internet of Vision server for storage, and display it through the Internet of Vision scheduling terminal 203 .

Referring to FIG. 3 , it shows a structural block diagram of a video networking conference system of the present invention.

As shown in the figure, the Internet of Vision conference system includes an Internet of Vision server, a Pamir terminal (Internet of Vision scheduling terminal) communicating with the Internet of Vision server, at least one Internet of Vision site terminal and a conference analysis system (which can be a face detection system) deployed. The video conference analysis terminal.

Each venue has a venue terminal, which can use image acquisition equipment to collect the environmental data of the venue, convert the collected environmental data into video streams and send them to the internet-of-vision server in the data exchange center.

The Pamir terminals (including mobile phones, iPads, and notebooks) in the data display center can display the information of the participants at the terminal in real time.

The conference analysis system in the algorithm processing center can analyze the video stream collected by the terminals in the conference venue to obtain the information of the participants.

The video network server in the data exchange center (the video network server may include 2K servers, autonomous servers, and meeting management servers) is connected to the Pamir dispatching terminal, and the conference analysis system deployed on the virtual video network terminal (through the terminal number of the virtual video network terminal) Connect with the Internet of View server) and the venue terminal to realize the data exchange between the above-mentioned terminals.

In one example, during the meeting, the venue terminal will collect the environmental data of the venue in real time, and send it to the Internet of Vision server in the form of a video stream. When the Pamir scheduling terminal sends a scheduling request to the Internet of Vision, the server of the Internet of Vision A video stream with a preset duration can be selected from the acquired video streams according to the scheduling request, and the video stream with a preset duration can be sent to the conference analysis system for analysis. The conference analysis system will obtain the participant information from the video stream, and send it to the Pamir terminal for display via the Internet of Vision server.

In the embodiment of the present invention, the analysis terminal is introduced into the video network conference to perform face detection on the video streams from the terminals of each venue, so as to determine the participant information of each branch venue, and send the participant information to the scheduling terminal for display , so that the conference scheduler can know the information of the participants participating in the conference at each conference site in real time.

It should be noted that, for the method embodiment, for the sake of simple description, it is expressed as a series of action combinations, but those skilled in the art should know that the embodiment of the present invention is not limited by the described action sequence, because According to the embodiment of the present invention, certain steps may be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification belong to preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

Referring to FIG. 4 , it shows a structural block diagram of an embodiment of an information display device for a video networking conference of the present invention, which is applied to a video networking conference system. The video networking conference system includes a video networking server, a video networking analysis terminal, and a video networking conference system. Scheduling terminal, a plurality of Internet-connected conference site terminals; specifically, the following modules can be included:

The video stream data acquisition module 401 is used for the video network server to obtain video stream data from each video network site terminal respectively; the video stream data is the site environment data collected by the video network site terminal;

A scheduling request receiving module 402, configured for the video networking server to receive the scheduling request sent by the video networking scheduling terminal;

The target Internet-of-Vision site terminal determination module 403 is used for the Internet-of-Vision server to determine the target Internet-of-Vision site terminal from the plurality of Internet-of-Vision site terminals for the scheduling request;

The target video stream data sending module 404 is used for the video network server to obtain target video stream data from the video stream data of the target video network site terminal, and send the target video stream data to all pre-joined conference The video network conference analysis terminal;

The participant information receiving module 405 is used for the video networking server to receive the participant information generated according to the target video stream data returned by the video networking conference analysis terminal;

The participant information sending module module 406 is used for the video networking server to send the participant information to the video networking scheduling terminal.

In the embodiment of the present invention, the scheduling request includes the sampling duration and the identification information of the video network site terminal; the video stream data acquisition module includes:

The sampling sequence determination submodule is used for the video networking server to determine the sampling sequence of the target video networking site terminal by using the identification information;

The target video stream data acquisition sub-module is used for the video network server to obtain the target video stream from the video stream data of the target video network site terminal in sequence according to the sampling order and the sampling duration, and store the target The video stream is sent to the Internet of Vision analysis terminal.

As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for related parts, please refer to the part of the description of the method embodiment.

The Internet of Vision is an important milestone in the development of the network. It is a real-time network that can realize real-time transmission of high-definition video, and push many Internet applications to high-definition video, high-definition face-to-face.

The Internet of View adopts real-time high-definition video exchange technology, which can provide required services on one network platform, such as high-definition video conferencing, video surveillance, intelligent monitoring and analysis, emergency command, digital broadcast TV, time-lapse TV, online teaching, live broadcast , VOD on demand, TV mail, personalized recording (PVR), intranet (self-managed) channel, intelligent video broadcast control, information release and other dozens of video, voice, picture, text, communication, data and other services are all integrated in one System platform, realize high-definition quality video playback through TV or computer.

In order to enable those skilled in the art to better understand the embodiments of the present invention, the Internet of Things is introduced as follows:

Some of the technologies applied in the Internet of Things are as follows:

Network Technology

The network technology innovation of the Internet of View has improved the traditional Ethernet (Ethernet) to face the potentially huge video traffic on the network. Different from pure network packet switching (Packet Switching) or network circuit switching (Circuit Switching), video networking technology uses Packet Switching to meet Streaming requirements. The Internet of Vision technology has the flexibility, simplicity and low price of packet switching, and at the same time has the quality and security guarantee of circuit switching, realizing the seamless connection of switched virtual circuits and data formats throughout the network.

Switching Technology

Video networking adopts the two advantages of Ethernet asynchronous and packet switching, eliminates the defects of Ethernet under the premise of full compatibility, has end-to-end seamless connection of the whole network, directly connects to user terminals, and directly carries IP data packets. User data does not require any format conversion within the entire network. Video networking is a more advanced form of Ethernet. It is a real-time switching platform, which can realize the real-time transmission of large-scale high-definition video in the whole network that cannot be realized by the Internet at present, and push many network video applications to high-definition and unification.

Server Technology

The server technology on the Internet of View and unified video platform is different from the server in the traditional sense. Its streaming media transmission is based on connection-oriented, and its data processing capability has nothing to do with traffic and communication time. A single network layer can contain information command and data transmission. For voice and video services, the complexity of video streaming and unified video platform streaming media processing is much simpler than data processing, and the efficiency is greatly improved by more than 100 times compared with traditional servers.

Storage Technology

The ultra-high-speed storage technology of the unified video platform adopts the most advanced real-time operating system in order to adapt to the super-large capacity and super-large-flow media content, and maps the program information in the server command to the specific hard disk space, and the media content no longer passes through the server. It is delivered directly to the user terminal in an instant, and the user generally waits for less than 0.2 seconds. The optimized sector distribution greatly reduces the mechanical movement of the hard disk head seeking. The resource consumption is only 20% of the IP Internet of the same level, but the concurrent traffic generated is 3 times larger than that of the traditional hard disk array, and the overall efficiency is increased by more than 10 times.

Network Security Technology

The structural design of the Internet of View completely eradicates the network security problems that plague the Internet through the individual licensing system for each service, complete isolation of equipment and user data, and generally does not require anti-virus programs and firewalls, preventing hackers and virus attacks , to provide users with a structured worry-free security network.

Service Innovation Technology

The unified video platform integrates business and transmission together, whether it is a single user, a private network user or the sum of a network, it is just an automatic connection. User terminals, set-top boxes or PCs are directly connected to the unified video platform to obtain rich and colorful multimedia video services in various forms. The unified video platform adopts the "recipe-style" table matching mode to replace the traditional complex application programming. It can realize complex applications with very little code and realize "unlimited" new business innovations.

The networking of the Internet of View is as follows:

Vision networking is a network structure with centralized control. The network can be a tree network, star network, ring network, etc., but on this basis, a centralized control node is required in the network to control the entire network.

As shown in Figure 5, the Internet of View is divided into two parts: the access network and the metropolitan area network.

The equipment in the access network part can be mainly divided into three categories: node server, access switch, terminal (including various set-top boxes, encoding boards, storage, etc.). The node server is connected with the access switch, and the access switch can be connected with multiple terminals and can be connected with Ethernet.

Wherein, the node server is a node with centralized control function in the access network, which can control the access switches and terminals. The node server can be directly connected to the access switch, and can also be directly connected to the terminal.

Similarly, the devices in the MAN part can also be divided into three categories: MAN servers, node switches, and node servers. The metro server is connected to the node switch, and the node switch can be connected to multiple node servers.

Wherein, the node server is the node server of the access network part, that is, the node server belongs to both the access network part and the metropolitan area network part.

The metropolitan area server is a node with a centralized control function in the metropolitan area network, which can control node switches and node servers. The metro server can be directly connected to the node switch, or directly connected to the node server.

It can be seen that the entire Vision Network is a layered centralized control network structure, while the network controlled by the node server and the metro server can be in various structures such as tree, star, and ring.

Vividly speaking, the access network part can form a unified video platform (the part in the dotted circle), and multiple unified video platforms can form a video network; each unified video platform can be interconnected through the metropolitan area and the wide area video network.

Classification of Internet of Things devices

1.1 The devices in the Internet of View in the embodiment of the present invention can be mainly divided into three categories: servers, switches (including Ethernet gateways), terminals (including various set-top boxes, encoding boards, memory, etc.). As a whole, the Internet of Things can be divided into a metropolitan area network (or a national network, a global network, etc.) and an access network.

1.2 The equipment in the access network can be mainly divided into three categories: node servers, access switches (including Ethernet gateways), terminals (including various set-top boxes, encoding boards, storage, etc.).

The specific hardware structure of each access network device is:

Node server:

As shown in Figure 6, mainly comprise network interface module 601, switching engine module 602, CPU module 603, disk array module 604;

Wherein, network interface module 601, the bag that CPU module 603, disk array module 604 come in all enters switching engine module 602; The guiding information of packet stores this packet into the queue of corresponding packet cache 606; If the queue of packet cache 606 is close to full, then discards; Switching engine module 602 polls all packet cache queues, and forwards if the following conditions are met: 1) The sending buffer of the port is not full; 2) The queue packet counter is greater than zero. The disk array module 604 mainly realizes the control to the hard disk, including operations such as initialization, reading and writing to the hard disk; configuration (including downlink protocol packet address table, uplink protocol packet address table, and data packet address table), and configuration of the disk array module 604 .

Access switch:

As shown in Figure 7, mainly include network interface module (downlink network interface module 701, uplink network interface module 702), switching engine module 703 and CPU module 704;

Wherein, the packet (upstream data) that the downstream network interface module 701 comes in enters the packet detection module 705; Whether the destination address (DA), source address (SA), data packet type and packet length of the packet detection module 705 detection packet meet the requirements, if Meet, then distribute corresponding stream identifier (stream-id), and enter switching engine module 703, otherwise discard; Uplink network interface module 702 pack (downstream data) that comes in enters switching engine module 703; CPU module 704 The data packet that comes in Enter switching engine module 703; Switching engine module 1103 carries out the operation of looking into address table 706 to the bag that comes in, thereby obtains the guide information of bag; If the bag that enters switching engine module 703 is that downlink network interface goes to uplink network interface, then combines Flow identifier (stream-id) This packet is stored in the queue of corresponding packet buffer 707; If the queue of this packet buffer 707 is close to full, then discard; If the packet that enters switching engine module 703 is not downlink network interface, goes up If the data packet goes to the network interface, the data packet is stored in the queue of the corresponding packet buffer 707 according to the guiding information of the packet; if the queue of the packet buffer 707 is nearly full, it is discarded.

Switching engine module 703 polls all packet buffer queues, divides two kinds of situations in the embodiment of the present invention:

If the queue goes from the downlink network interface to the uplink network interface, the following conditions are met for forwarding: 1) the port sending buffer is not full; 2) the queue packet counter is greater than zero; 3) the token generated by the code rate control module is obtained ;

If the queue does not go from the downlink network interface to the uplink network interface, the following conditions are met for forwarding: 1) the sending buffer of the port is not full; 2) the packet counter of the queue is greater than zero.

The code rate control module 708 is configured by the CPU module 704, and generates tokens for all packet buffer queues going from the downlink network interface to the uplink network interface within a programmable interval to control the uplink forwarding code rate.

The CPU module 704 is mainly responsible for the protocol processing with the node server, the configuration of the address table 706, and the configuration of the code rate control module 708.

Ethernet protocol conversion gateway :

As shown in Figure 8, it mainly includes a network interface module (downlink network interface module 801, uplink network interface module 802), switching engine module 803, CPU module 804, packet detection module 805, code rate control module 808, address table 806, packet Buffer 807, MAC adding module 809, and MAC deleting module 810.

Wherein, the data packet coming in from the downlink network interface module 801 enters the packet detection module 805; the packet detection module 805 detects the Ethernet MAC DA, Ethernet MAC SA, Ethernet length or frame type, video network destination address DA, video network Source address SA, depending on whether the network data packet type and the packet length meet the requirements, if it is met, the corresponding flow identifier (stream-id) is allocated; then, the MAC deletion module 810 subtracts MAC DA, MAC SA, length or frametype ( 2byte), and enter the corresponding receive buffer, otherwise discard;

The downlink network interface module 801 detects the sending buffer of the port, if there is a packet, the Ethernet MAC DA of the corresponding terminal is known according to the visual network destination address DA of the packet, and the Ethernet MAC DA of the terminal, the MACSA of the Ethernet protocol conversion gateway, and the MACSA of the Ethernet protocol conversion gateway are added. Ethernet length or frame type, and send.

The functions of other modules in the Ethernet protocol conversion gateway are similar to those of the access switch.

terminal:

It mainly includes a network interface module, a business processing module and a CPU module; for example, a set-top box mainly includes a network interface module, an video and audio codec engine module, and a CPU module; an encoding board mainly includes a network interface module, an video and audio encoding engine module, and a CPU module; It mainly includes network interface module, CPU module and disk array module.

1.3 The equipment of the metropolitan area network can be mainly divided into two categories: node server, node switch, and metropolitan area server. Among them, the node switch mainly includes a network interface module, a switching engine module and a CPU module; the metro server mainly includes a network interface module, a switching engine module and a CPU module.

2. Definition of Internet of Vision data packets

2.1 Definition of access network data packet

The data packet of the access network mainly includes the following parts: destination address (DA), source address (SA), reserved bytes, payload (PDU), and CRC.

As shown in the table below, the data packets of the access network mainly include the following parts:

DA SA Reserved Payload CRC

in:

The destination address (DA) consists of 8 bytes (byte), the first byte indicates the type of data packet (such as various protocol packets, multicast data packets, unicast data packets, etc.), there are up to 256 possibilities, The second byte to the sixth byte is the address of the metropolitan area network, and the seventh and eighth bytes are the address of the access network;

The source address (SA) is also composed of 8 bytes (byte), and the definition is the same as that of the destination address (DA);

Reserved bytes consist of 2 bytes;

The payload part has different lengths according to different datagram types. If it is a variety of protocol packets, it is 64 bytes. If it is a single multicast data packet, it is 32+1024=1056 bytes. Of course, it is not limited to Above 2 types;

CRC consists of 4 bytes, and its calculation method follows the standard Ethernet CRC algorithm.

2.2 MAN packet definition

The topology of the metropolitan area network is a graph. There may be two or even more than two types of connections between two devices, that is, there may be more than 2 connections between node switches and node servers, node switches and node switches, and node switches and node servers. kind of connection. However, the MAN address of the MAN device is unique. In order to accurately describe the connection relationship between MAN devices, a parameter: label is introduced in the embodiment of the present invention to uniquely describe a MAN device.

The definition of labels in this manual is similar to the definition of labels in MPLS (Multi-Protocol Label Switch, Multi-Protocol Label Switching). Assuming that there are two connections between device A and device B, then there are 2 labels, the packet from device B to device A also has 2 labels. The label is divided into an incoming label and an outgoing label. Assuming that the label (incoming label) of the data packet entering device A is 0x0000, the label (outgoing label) of the data packet when it leaves device A may become 0x0001. The network access process of the metropolitan area network is a network access process under centralized control, which means that the address allocation and label allocation of the metropolitan area network are all dominated by the metropolitan area server, and the node switches and node servers are all passively executed. Different from MPLS label allocation, MPLS label allocation is the result of mutual negotiation between switches and servers.

As shown in the table below, the data packet of the MAN mainly includes the following parts:

DA SA Reserved Label Payload CRC

Namely destination address (DA), source address (SA), reserved byte (Reserved), label, payload (PDU), CRC. Among them, the format of the label can refer to the following definition: the label is 32bit, of which the high 16bit is reserved, and only the low 16bit is used, and its position is between the reserved byte of the data packet and the payload.

Each embodiment in this specification is described in a progressive manner, each embodiment focuses on the difference from other embodiments, and the same and similar parts of each embodiment can be referred to each other.

Those skilled in the art should understand that the embodiments of the present invention may be provided as methods, devices, or computer program products. Accordingly, embodiments of the invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, embodiments of the invention may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

Embodiments of the present invention are described with reference to flowcharts and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor or processor of other programmable data processing terminal equipment to produce a machine such that instructions executed by the computer or processor of other programmable data processing terminal equipment Produce means for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the The instruction means implements the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions can also be loaded into a computer or other programmable data processing terminal equipment, so that a series of operational steps are performed on the computer or other programmable terminal equipment to produce computer-implemented processing, thereby The instructions executed above provide steps for implementing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

Having described preferred embodiments of embodiments of the present invention, additional changes and modifications to these embodiments can be made by those skilled in the art once the basic inventive concept is appreciated. Therefore, the appended claims are intended to be construed to cover the preferred embodiment and all changes and modifications which fall within the scope of the embodiments of the present invention.

Finally, it should also be noted that in this text, relational terms such as first and second etc. are only used to distinguish one entity or operation from another, and do not necessarily require or imply that these entities or operations, any such actual relationship or order exists. Furthermore, the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article, or terminal equipment comprising a set of elements includes not only those elements, but also includes elements not expressly listed. other elements identified, or also include elements inherent in such a process, method, article, or end-equipment. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article or terminal device comprising said element.

The information display method of a video networking conference and the information display system of a video networking conference provided by the present invention have been introduced in detail above. In this paper, specific examples are used to illustrate the principle and implementation of the present invention. The above The description of the embodiment is only used to help understand the method of the present invention and its core idea; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation and scope of application. As mentioned above, the contents of this specification should not be construed as limiting the present invention.

Claims (8)

1. A method for displaying information of a video networking conference, characterized in that it is applied to a video networking conference system, and the video networking conference system includes a video networking server, a video networking conference analysis terminal, a video networking scheduling terminal, and a plurality of video networking meeting places A terminal; a conference analysis system is deployed on the video network conference analysis terminal, and the method includes: The video-networking server obtains video stream data from each video-networking venue terminal respectively; the video stream data is the venue environment data collected by the video-networking venue terminal; The video networking server receives the scheduling request sent by the video networking scheduling terminal, and the scheduling request includes the sampling duration and the identification information of the video networking site terminal; According to the scheduling request, the Internet-of-Vision server determines a target Internet-of-Vision venue terminal from among the plurality of Internet-of-Vision venue terminals; The video networking server uses the identification information to determine the sampling order of the target video networking site terminals; According to the sampling order and the sampling duration, the video-network server sequentially acquires target video stream data from the video stream data of the target video-network site terminal, and sends the target video stream data to the video-network Conference analysis terminal; The video networking server receives the participant information of the target video networking site terminal corresponding to the conference site generated according to the target video stream data returned by the video networking conference analysis terminal; the video networking conference analysis terminal is used when When receiving the target video stream data sent by the Internet-of-Vision server, the conference analysis system converts the target video stream data into picture information, performs face detection on the picture information, and determines the information of the participants. The above-mentioned participant information includes the number of participants; The video networking server sends the participant information to the video networking scheduling terminal. 2. An information display system for a networked video conference, characterized in that it includes: a networked video server, a networked video conference analysis terminal, a networked video scheduling terminal, and a plurality of networked video conference site terminals; the networked video conference analysis terminal is deployed with Conference analysis system; The Internet-of-Vision scheduling terminal is used to send a dispatch request to the Internet-of-Vision server, and display the participant information of each Internet-of-Vision site terminal sent by the Internet-of-Vision server, and the dispatch request includes sampling duration and all Describe the identification information of the Internet-connected site terminal; The video-networking server is used to obtain video stream data from each video-networking venue terminal respectively; the video stream data is the venue environment data collected by the video-networking venue terminal. When requesting, determine the target video-networking site terminal from the plurality of video-networking site terminals, use the identification information to determine the sampling sequence of the target video-networking site terminal, and sequentially from Acquiring target video stream data from the video stream data of the target Internet-of-Vision conference site terminal, and sending the target video-stream data to the Internet-of-Vision conference analysis terminal; The target video stream data generated by the target video network site terminal corresponding to the site participants information sent to the video network scheduling terminal, the participants information includes the number of participants information; The video network conference analysis terminal is configured to convert the target video stream data into picture information through the conference analysis system when receiving the target video stream data sent by the video network server, and process the picture information Face detection to determine the information of the participants. 3. The system according to claim 2, wherein the video networking conference analysis terminal is pre-allocated with a video networking virtual number; The video networking scheduling terminal is also used to add the video networking virtual number of the video networking conference analysis terminal to the conference list, so as to pull the video networking conference analysis terminal into the conference. 4. The system of claim 2, wherein: The video networking conference analysis terminal is also used to bind the participant information with the identification information of the corresponding video networking site terminal after determining the participant information, and bind the bound participant information and send the identification information to the Internet of View server. 5. The system according to claim 2, wherein a meeting list is maintained in the dispatching terminal of the Internet of View, and the meeting list includes meeting analysis options; The video network scheduling terminal is also used to judge whether the trigger information for the conference analysis option is received when the conference site analysis service is started; If yes, obtain and display participant information of the video conference corresponding to the conference analysis option. 6. An information display device for a networked video conference, characterized in that it is applied to a networked video conference system, and the networked video conference system includes a networked video server, a networked video analysis terminal, a networked video scheduling terminal, and multiple networked video conference site terminals ; The video network conference analysis terminal is deployed with a conference analysis system, and the device includes: The video stream data acquisition module is used for the video network server to obtain video stream data from each video network site terminal respectively; the video stream data is the site environment data collected by the video network site terminal; The scheduling request receiving module is used for the video networking server to receive the scheduling request sent by the video networking scheduling terminal, and the scheduling request includes the sampling duration and the identification information of the video networking site terminal; A target video-network site terminal determination module, configured for the video-network server to determine a target video-network site terminal from the plurality of video-network site terminals for the scheduling request; The target video stream data acquisition module is used for the video networking server to determine the sampling sequence of the target video networking site terminal by using the identification information; the video networking server sequentially selects from the sampling sequence and the sampling duration according to the sampling sequence and the sampling duration Obtain the target video stream data from the video stream data of the target video network meeting site terminal, and send the target video stream data to the video network conference analysis terminal; The participant information receiving module is used for the video networking server to receive the participant information of the target video networking venue terminal corresponding to the venue that is returned by the video networking conference analysis terminal and generated according to the target video stream data; The video network conference analysis terminal is used to convert the target video stream data into picture information through the conference analysis system when receiving the target video stream data sent by the video network server, and perform face recognition on the picture information. Detecting and determining the information of the participants, the information of the participants includes the number of participants; The participant information sending module is used for the video networking server to send the participant information to the video networking scheduling terminal. 7. An information display device for a video conference, characterized in that it comprises: one or more processors; and One or more machine-readable media having instructions stored thereon, when executed by the one or more processors, causes the device to perform the information presentation method for a video conference as claimed in claim 1 . 8. A computer-readable storage medium, characterized in that the computer program stored therein enables a processor to execute the information display method for a video networking conference as claimed in claim 1.

Images