CN119728911A

CN119728911A - Network online conference method based on video monitoring

Info

Publication number: CN119728911A
Application number: CN202411857328.5A
Authority: CN
Inventors: 肖萍
Original assignee: Jiangsu Xinyuan Digital Technology Co ltd
Current assignee: Jiangsu Xinyuan Digital Technology Co ltd
Priority date: 2024-12-17
Filing date: 2024-12-17
Publication date: 2025-03-28

Abstract

The invention discloses a network online conference method based on video monitoring, which relates to the technical field of network conferences and comprises the following steps of pre-configuring a conference terminal to be connected with a conference server for receiving and displaying audio and video data in a conference, wherein the conference terminal comprises a camera for collecting video images, a microphone for collecting sound and a loudspeaker for playing audio in the conference, the conference server is used for managing the conference and processing, distributing and storing the audio and video data of the conference terminal, and the conference server is used for detecting the video fluency of the conference terminal and performing self-adaptive adjustment. The self-adaptive adjustment can preferentially ensure the transmission of audio and video data, ensures that a participant can clearly hear speaking contents, reduces manual intervention by automatic adjustment, automatically adjusts according to video fluency, and reduces the workload of meeting organizers or manually adjusting video parameters.

Description

Network online conference method based on video monitoring

Technical Field

The invention relates to the technical field of network conferences, in particular to a network online conference method based on video monitoring.

Background

The network conference refers to a conference form which enables people located in different geographic positions to communicate, cooperate and exchange in real time through the internet technology.

Video conferencing this is one of the most intuitive forms in web conferencing. The participants can see the non-language information such as facial expression, limb language and the like of each other, and the reality of communication is enhanced. For example, in a remote interview scenario, interviewees can better observe the status of the recruiter through a video conference, and the recruiter can also more directly demonstrate his own image and communication capabilities. Video conferencing requires relatively high network bandwidth and device performance because multiple video streams need to be transmitted and processed simultaneously. To ensure video quality, video encoding techniques (e.g., h.264, h.265) are typically used to compress and transmit video.

Currently, a certain bandwidth is required to transmit video data during the transmission process in a video conference. If the bandwidth is insufficient, video data transmission is hindered as if a narrow road cannot accommodate enough vehicles. For example, high definition video conferencing typically requires at least 1-2Mbps of bandwidth to guarantee substantial fluency. If multiple high-resolution video streams are started at the same time, or if other devices in the network occupy a large amount of bandwidth (such as downloading files, watching high-definition video, etc.), the available bandwidth of the video conference is insufficient, and thus, a clip occurs. Meanwhile, delay occurs due to the excessively long time required for data transmission from the transmitting end to the receiving end. Network congestion, server distance, etc. may all cause delays. When the delay is too high, the video data cannot reach the receiving end in time, and a jam is caused. For example, in a transnational video conference, data may need to be transmitted through a plurality of network nodes and submarine optical cables, and a delay is increased due to a long distance, so that pictures and sounds may be asynchronous, and a jam occurs, thereby affecting the quality of the online conference.

For the problems in the related art, no effective solution has been proposed at present.

Disclosure of Invention

Aiming at the problems in the related art, the invention provides a network online conference method based on video monitoring, which aims to overcome the technical problems existing in the prior related art.

The technical scheme of the invention is realized as follows:

A network online conference method based on video monitoring comprises the following steps:

The pre-configured conference terminal is connected with a conference server and is used for receiving and displaying audio and video data in a conference, wherein the conference terminal comprises a camera for collecting video images, a microphone for collecting sound and a loudspeaker for playing audio in the conference;

The conference server detects the video fluency of the participant terminal and carries out self-adaptive adjustment, wherein the method comprises the step of carrying out video quality adjustment or frame rate adjustment according to the video dynamic degree of the current participant terminal.

Further, the conference server detects the video fluency of the participant terminal, and the method comprises the following steps:

The method for obtaining the video dynamic degree of the current client video comprises the following steps:

Calibrating the resolution of the video frame to be N multiplied by M, setting the pixel value of the current frame to be I _t (x, y) and the pixel value of the previous frame to be I _t-1 (x, y) for each pixel position (x, y), and then representing the inter-frame difference as D (x, y) = |I _t(x,y)-I_t-1 (x, y) |;

Obtaining an inter-frame difference sum

And setting a dynamic threshold T, when S > T, judging that the video dynamic degree is high, and when S is less than or equal to T, judging that the video dynamic degree is low.

Further, the adaptive adjustment includes the following steps:

if the current video dynamic degree is high, adjusting the video quality;

and if the current video dynamic degree is low, adjusting the video frame rate.

Further, the video quality adjustment includes performing resolution adjustment, including the steps of:

presetting a network bandwidth interval threshold value and setting a corresponding resolution level;

when detecting video jamming of a network online conference, acquiring a current client network bandwidth value, and acquiring a network bandwidth interval threshold corresponding to the current network bandwidth value;

According to the network bandwidth interval threshold, adjusting the current client video resolution to accord with the resolution level;

further, the adjusting the resolution of the video of the current client accords with the resolution level includes the following steps:

presetting a jamming degree threshold value and a time threshold value;

And periodically detecting the jamming degree of the video resolution of the client of the current resolution level, and adjusting the resolution level according to the jamming degree.

Further, the adjusting the resolution level according to the degree of clamping includes the following steps:

In the adjustment period, if the video resolution of the current client is free from jamming, the primary resolution level is improved, and the jamming degree is detected in the next adjustment period;

in the adjustment period, if the current client video resolution is occasionally stuck and the stuck time is smaller than the time threshold, not processing;

in the adjustment period, if the video resolution of the current client is frequently stuck and the stuck time is greater than the time threshold, the first-level resolution level is reduced, and the stuck degree is detected in the next adjustment period;

Further, the video frame rate adjustment includes the following steps:

Presetting a video frame rate adjustment interval threshold;

and acquiring the current video frame rate, and adjusting the current video frame rate to reduce the video frame rate by one according to the video frame rate adjustment interval threshold.

The invention has the beneficial effects that:

The conference server is used for managing the conference and processing, distributing and storing the audio and video data of the conference terminal, and carrying out self-adaptive adjustment on the video smoothness of the conference terminal, so that the network online conference based on video monitoring is realized, the self-adaptive adjustment can preferentially ensure the transmission of the audio and video data, ensure that a participant can clearly hear speaking content, ensure that the participant can keep up with conference rhythm even if the video quality is reduced due to network problems, understand conference points, improve the quality of the conference, automatically adjust and reduce manual intervention, automatically adjust according to the video smoothness, and reduce the workload of a conference organizer or manually adjust video parameters.

In addition, the adaptive adjustment strategy based on video smoothness detection can enable the network conference system to quickly react no matter in a wired network or a wireless network environment, or even in the case of high, low or large fluctuation of network bandwidth, so that the video data volume of each participant can be reduced, the conference can keep a relatively stable running state in the congested network environment, and the risk of conference interruption is reduced.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic flow chart of a network online conference method based on video monitoring according to an embodiment of the invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which are derived by a person skilled in the art based on the embodiments of the invention, fall within the scope of protection of the invention.

According to the embodiment of the invention, a network online conference method based on video monitoring is provided.

As shown in fig. 1, a network online conference method based on video monitoring according to an embodiment of the present invention includes the following steps:

The method comprises the steps of S1, pre-configuring a conference terminal to be connected with a conference server and used for receiving and displaying audio and video data in a conference, wherein the conference terminal comprises a camera for collecting video images, a microphone for collecting sound and a loudspeaker for playing audio in the conference;

And S2, the conference server detects the video fluency of the participant terminal and carries out self-adaptive adjustment, wherein the method comprises the step of carrying out video quality adjustment or frame rate adjustment according to the video dynamic degree of the current participant terminal.

The conference server detects the video fluency of the participant terminal and comprises the following steps:

D (x, y) of the whole video frame is accumulated to obtain an inter-frame differential sum

If the current video dynamic degree is high, the video quality adjustment is performed, including resolution adjustment, specifically as follows:

The technical scheme is that when the network bandwidth is lower than 1Mbps, the video resolution is 720×480, and when the bandwidth is between 1-2Mbps, the resolution of 1280×720 is adopted.

In particular, in application, for example, resolution is reduced from high definition (1920×1080) to standard definition (1280×720) or even lower.

In addition, the method for adjusting the current client video resolution to be in accordance with the resolution level further comprises the following steps:

a jamming degree threshold and a time threshold are preset, wherein the jamming degree threshold comprises occasional and frequent jamming degree threshold values which are set according to the jamming times in a period;

the method comprises the steps of periodically detecting the jamming degree of the video resolution of a client of the current resolution level, and adjusting the resolution level according to the jamming degree, wherein the method comprises the following steps:

Specifically, when the method is applied, if the video resolution of the client of the current resolution level is blocked only occasionally and for a short time, the video resolution is not processed, and if the video resolution is blocked frequently and for a long time, the first-level resolution is reduced. For example, if the first time the stuck is detected for less than 5 seconds, the process is not performed, and if the second time the stuck is detected for more than 5 seconds, the process is performed from 1080p to 720p, and if the stuck is not improved later, the stuck rate (stuck frame number/total frame number) exceeds 30% in the next 10 seconds, the process is performed again to 480p.

If the current video dynamic degree is low, the video frame rate is adjusted, which comprises the following steps:

Presetting a video frame rate adjustment interval threshold;

Acquiring a current video frame rate, and adjusting the current video frame rate to reduce one video frame rate according to a video frame rate adjustment interval threshold;

According to the technical scheme, the frame rate determines the fluency of the video. Normally, the video frame rate may be 30fps or 60fps. The system may reduce the frame rate to 15fps or less when signs of stuck network failure occur. This is because the number of video frames that need to be transmitted per second is reduced after the frame rate is reduced, thereby reducing the network transmission burden.

By means of the technical scheme, for scenes with more dynamic pictures (such as frequent walking of people and more demonstration actions in a conference), a relatively high frame rate is kept for video quality adjustment, and for scenes with more static pictures (such as mainly document display and PPT explanation), more frame rates are reduced. For example, in the document presentation stage, if a clip is detected, the frame rate is reduced from 30fps to 10fps, while the video quality is guaranteed.

In summary, by means of the above technical solution of the present invention, the following effects can be achieved:

The foregoing is merely a preferred embodiment of the present application and is not intended to limit the present application, and other embodiments of the present disclosure will be readily apparent to those skilled in the art after considering the disclosure herein in the specification and examples. This application is intended to cover any adaptations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It is to be understood that the present disclosure is not limited to the precise arrangements and instrumentalities shown in the drawings, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A network online conference method based on video surveillance, characterized in that it includes the following steps:

Pre-configure the conference terminal to connect to the conference server for receiving and displaying audio and video data in the conference, wherein the conference terminal includes: a camera for capturing video images, a microphone for capturing sound, and a speaker for playing audio in the conference; the conference server is used to manage the conference and process, distribute, and store the audio and video data of the conference terminal;

The conference server detects the video fluency of the participating terminals and performs adaptive adjustments, including: adjusting the video quality or frame rate according to the video dynamics of the current participating terminals.

2. The network online conference method based on video monitoring according to claim 1 is characterized in that the conference server detects the video fluency of the participating terminals, comprising the following steps:

Obtaining the video dynamics of the current client video includes the following steps:

The resolution of the calibrated video frame is N×M. For each pixel position (x, y), let the current frame pixel value be _It (x, y) and the previous frame pixel value be It _-1 (x, y). Then the inter-frame difference is expressed as: D(x, y) = | _It (x, y)-It _-1 (x, y)|;

Get the sum of inter-frame differences

A dynamic threshold T is set. When S＞T, the video is judged to have a high dynamic level. When S≤T, the video is judged to have a low dynamic level.

3. The network online conference method based on video surveillance according to claim 2, characterized in that the adaptive adjustment comprises the following steps:

If the current video has high dynamic level, adjust the video quality;

If the current video has low dynamic level, adjust the video frame rate.

4. The network online conference method based on video surveillance according to claim 3 is characterized in that the video quality adjustment comprises: adjusting the resolution, comprising the following steps:

Pre-set the network bandwidth interval threshold and set the corresponding resolution level;

When detecting that the online conference video is stuck, obtain the current client network bandwidth value, and obtain the network bandwidth interval threshold corresponding to the current network bandwidth value;

According to the network bandwidth range threshold, adjust the current client video resolution to meet the resolution level.

5. The network online conference method based on video surveillance according to claim 4 is characterized in that the step of adjusting the current client video resolution to conform to the resolution level comprises the following steps:

Pre-set the jam level threshold and time threshold;

The video resolution of the client at the current resolution level is periodically tested for the degree of freeze, and the resolution level is adjusted according to the degree of freeze.

6. The online conference method based on video surveillance according to claim 5 is characterized in that the step of adjusting the resolution level according to the degree of freeze comprises the following steps:

During the adjustment period, if the current client video resolution has no lag, the resolution level will be increased by one level, and the lag level will be tested during the next adjustment period.

During the adjustment period, if the current client video resolution is occasionally stuck and the stuck time is less than the time threshold, no processing will be done;

During the adjustment period, if the current client video resolution is frequently stuck and the stuck time is greater than the time threshold, the resolution level is reduced by one level, and the stuck degree is detected in the next adjustment period.

7. The network online conference method based on video surveillance according to claim 3, characterized in that the video frame rate adjustment comprises the following steps:

Pre-set the video frame rate adjustment interval threshold;

Get the current video frame rate, and adjust the interval threshold according to the video frame rate to adjust the current video frame rate to lower one video frame rate.