CN109698928B - Method and device for adjusting video stream in video conference system - Google Patents
Method and device for adjusting video stream in video conference system Download PDFInfo
- Publication number
- CN109698928B CN109698928B CN201811358922.4A CN201811358922A CN109698928B CN 109698928 B CN109698928 B CN 109698928B CN 201811358922 A CN201811358922 A CN 201811358922A CN 109698928 B CN109698928 B CN 109698928B
- Authority
- CN
- China
- Prior art keywords
- downlink
- video stream
- code rate
- uplink
- network state
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 51
- 238000001514 detection method Methods 0.000 claims abstract description 119
- 238000011144 upstream manufacturing Methods 0.000 claims description 32
- 230000001105 regulatory effect Effects 0.000 claims 1
- 230000008569 process Effects 0.000 description 12
- 230000005540 biological transmission Effects 0.000 description 4
- 230000003044 adaptive effect Effects 0.000 description 3
- 238000004590 computer program Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000006978 adaptation Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 239000000523 sample Substances 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/15—Conference systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/266—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
- H04N21/2662—Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/647—Control signaling between network components and server or clients; Network processes for video distribution between server and clients, e.g. controlling the quality of the video stream, by dropping packets, protecting content from unauthorised alteration within the network, monitoring of network load, bridging between two different networks, e.g. between IP and wireless
- H04N21/64723—Monitoring of network processes or resources, e.g. monitoring of network load
- H04N21/64738—Monitoring network characteristics, e.g. bandwidth, congestion level
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/647—Control signaling between network components and server or clients; Network processes for video distribution between server and clients, e.g. controlling the quality of the video stream, by dropping packets, protecting content from unauthorised alteration within the network, monitoring of network load, bridging between two different networks, e.g. between IP and wireless
- H04N21/64784—Data processing by the network
- H04N21/64792—Controlling the complexity of the content stream, e.g. by dropping packets
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computer Security & Cryptography (AREA)
- Databases & Information Systems (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention discloses a method and a device for adjusting video streams in a video conference system, which are used for recoding an uplink video stream to obtain a downlink video stream set comprising a plurality of different code rates, sending a detection packet and the downlink video stream to a downlink user end, obtaining a downlink network state according to a detection report, and judging whether to switch the downlink video stream according to a downlink network. The invention adopts the strategy of separately controlling the uplink video stream and the downlink video stream, the code rate of the uplink video stream is adjusted according to the uplink bandwidth from the uplink user terminal to the video conference system, the code rate of the downlink video stream is specifically adjusted according to the downlink bandwidth from the video conference system to each downlink user terminal, and the downlink user terminal with high bandwidth can obtain the video stream with higher quality, so that the downlink user terminals can not mutually influence.
Description
Technical Field
The present application relates to the field of video conferencing technologies, and in particular, to a method and an apparatus for adjusting a video stream in a video conferencing system.
Background
The remote medical online platform built based on the internet gradually becomes the development trend of the medical industry of China, and becomes an important means for relieving the medical current situation of China and improving the livelihood of people. The video conference system is used as an important communication tool and widely applied to scenes such as doctor-patient communication, medical teaching, large and small seminars or remote medical teaching. The above-mentioned scenes have relatively high requirements for the quality of the video played by the video conference system, and require high definition, voice quality and stability of the videos of both parties.
In the existing video conference system, subscription is realized by a data transparent transmission mode, but in the data transparent transmission mode, the code rates of an uplink video stream and a downlink video stream are consistent, if the uplink bandwidth from an uplink user end to the video conference system is high, but the downlink bandwidth from the video conference system to a downlink user is low, the bandwidth of the whole 'channel' from the uplink user end to the downlink user end is low, and the code rate of the transmitted video stream is also low. That is, when a plurality of downstream clients subscribe to the video of the same upstream client, if the downstream bandwidth of each downstream client is different, the method for the video conference system to push the downstream video stream to the downstream client adjusts the bitrate of the upstream video stream according to the minimum downstream bandwidth of the downstream clients and the upstream bandwidth of the upstream client, so that the downstream bandwidth of each downstream client affects the bitrate of the upstream video stream. This results in that even if the upstream bandwidth of the upstream ue is high, the downstream bandwidth of most downstream ues is also high, and only if the downstream bandwidth of individual downstream ue is low, the bitrate of the video stream is still low and the video quality is still poor.
Disclosure of Invention
The application provides a method and a device for adjusting video streams in a video conference system, which are used for solving the problem that downlink clients with low downlink bandwidth influence the video quality of downlink clients with high downlink bandwidth due to mutual influence among a plurality of downlink clients in the existing video conference system.
In a first aspect, to achieve the above object, the present application provides a method for adjusting a bitrate of a video stream in a video conference system, where the method includes:
acquiring an uplink video stream sent by an uplink user side;
recoding the uplink video stream to obtain a downlink video stream set comprising a plurality of different code rates, wherein the downlink video stream set comprises a first downlink video stream;
sending a downlink detection packet and the first downlink video stream to a downlink user side;
receiving a first downlink detection report and acquiring a first downlink network state;
and judging whether to switch the first downlink video stream according to the first downlink network state.
With reference to the first aspect, in a first possible implementation manner of the first aspect, the first downlink video stream is a video stream of a bitrate corresponding to a middle value of the multiple different bitrates.
With reference to the first aspect, in a second possible implementation manner of the first aspect, the determining whether to switch the first downlink video stream according to the first downlink network state includes:
if the first downlink network state is not overloaded, setting a first detection target code rate, wherein the first detection target code rate is greater than the code rate of the first downlink video stream;
sending a first code rate filling packet to a downlink user end according to the first detection target code rate, wherein the code rate of the first code rate filling packet is the difference value between the first detection target code rate and the code rate of the first downlink video stream;
receiving a second downlink detection report and acquiring a second downlink network state;
and if the second downlink network state is overload, stopping sending the first code rate filling packet, and determining not to switch the first downlink video stream.
With reference to the second possible implementation manner of the first aspect, in a third possible implementation manner of the first aspect, after the receiving the second probe report and acquiring the second downlink network status, the method further includes:
if the second downlink network state is not overloaded, switching the first downlink video stream into a second downlink video stream according to a first detection target code rate, wherein the code rate of the second downlink video stream is greater than that of the first downlink video stream and is less than or equal to the first detection target code rate;
setting a second detection target code rate, wherein the second detection target code rate is greater than the first detection target code rate;
sending a second code rate filling packet to a downlink user end according to the second detection target code rate, wherein the code rate of the second code rate filling packet is the difference value between the second detection target code rate and the code rate of the second downlink video stream;
receiving a third downlink detection report and acquiring a third downlink network state;
if the third downlink network state is not overloaded, switching the second downlink video stream into a third downlink video stream according to the second detection target code rate, wherein the code rate of the third downlink video stream is greater than that of the second downlink video stream and is less than or equal to the second detection target code rate;
if the third downlink network state is overload, stopping sending the second code rate filling packet;
with reference to the first aspect, in a fourth possible implementation manner of the first aspect, the determining whether to switch the first downlink video stream according to the first downlink network state further includes:
and if the first downlink network state is overload, switching the first downlink video stream into a fourth downlink video stream, wherein the code rate of the fourth downlink video stream is smaller than that of the first downlink video stream.
With reference to the fourth possible implementation manner of the first aspect, in a fifth possible implementation manner of the first aspect, before switching the first downlink video stream to the fourth downlink video stream, the method further includes:
and detecting whether the code rate filling packet is sent or not, and if so, stopping sending the code rate filling packet.
With reference to the fourth possible implementation manner of the first aspect, in a sixth possible implementation manner of the first aspect, after the switching the first downlink video stream to the fourth downlink video stream, the method further includes:
receiving a fourth downlink detection report and acquiring a fourth downlink network state;
and if the fourth downlink network state is overload, switching the fourth downlink video stream into a fifth downlink video stream, wherein the code rate of the fifth downlink video stream is smaller than that of the fourth downlink video stream.
With reference to the first aspect, in a seventh possible implementation manner of the first aspect, the adjusting, by the uplink client, the bitrate of the uplink video stream according to the uplink bandwidth specifically includes:
sending an uplink detection packet and a first uplink video stream to a video conference system;
receiving an uplink detection report and acquiring an uplink network state;
and judging whether to adjust the code rate of the first uplink video stream according to the uplink network state.
With reference to the first aspect, in an eighth possible implementation manner of the first aspect, before the acquiring the upstream video stream sent by the upstream user end, the method further includes:
the uplink user terminal carries out layered coding on the video stream, and obtains the base layer uplink video stream with the code rate smaller than a preset threshold value and the high-definition layer uplink video stream with the code rate larger than the preset threshold value.
In a second aspect, the present application further provides an apparatus for adjusting video streams in a video conference system, the apparatus comprising:
the uplink video stream acquisition module is used for acquiring an uplink video stream sent by an uplink user side;
the recoding module is used for recoding the uplink video stream to obtain a downlink video stream set comprising a plurality of different code rates, wherein the downlink video stream set comprises a first downlink video stream;
a sending module, configured to send a downlink detection packet and the first downlink video stream to a downlink client;
a detection report receiving module, configured to receive a first downlink detection report and obtain a first downlink network state;
and the judging module is used for judging whether to switch the first downlink video stream according to the first downlink network state.
In a third aspect, to achieve the above object, the present application provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements the steps of any one of the first aspect of the method for adjusting video streams in a video conference system.
In a fourth aspect, to achieve the above object, the present application provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the steps of any one of the methods for adjusting video streams in a video conference system according to the first aspect.
According to the technical scheme, the method and the device for adjusting the video stream code rate in the video conference system provided by the embodiment of the application adjust the uplink video stream adaptive to the uplink bandwidth through the uplink client; the video conference system recodes the uplink video stream to obtain a downlink video stream set comprising a plurality of different code rates, sends a downlink detection packet and a first downlink video stream to a downlink client, obtains the state of a downlink network according to a detection report, and judges whether to switch the first downlink video stream according to the downlink network; if the downlink network state is overload, gradually switching to a downlink video stream with low code rate adaptation; if the downlink network state is not overloaded, gradually detecting a higher downlink bandwidth lower limit by sending a code rate filling packet, and simultaneously switching to a downlink video stream matched with the highest downlink bandwidth lower limit according to the detected downlink bandwidth lower limit.
The method and the device adopt a strategy of separately controlling the uplink video stream and the downlink video stream, the code rate of the uplink video stream is adjusted according to the uplink bandwidth from the uplink user terminal to the video conference system, and the code rate of the downlink video stream is specifically adjusted according to the downlink bandwidth from the video conference system to each downlink user terminal. The video conference system obtains the video streams with different code rates by recoding the uplink video stream, and respectively pushes the video streams suitable for the bandwidths of the downlink user ends to each downlink user end, and the downlink user end with high bandwidth can obtain the video stream with high quality, so that the downlink user ends cannot mutually influence each other.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
In order to more clearly describe the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive exercise.
Fig. 1 is a schematic flowchart of a method for adjusting a video stream code rate in a video conference system according to an embodiment of the present application;
fig. 2 is a schematic flowchart of a process of adjusting an uplink video stream code rate by an uplink user end according to an embodiment of the present application;
fig. 3 is a schematic flowchart of a process of determining if the first downlink network status is not overloaded according to an embodiment of the present application;
fig. 4 is another schematic flow chart of the present application if the first downlink network status is not overloaded according to an embodiment of the present application;
fig. 5 is a schematic flowchart of a process of determining that the first downlink network status is overloaded according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of an apparatus for adjusting a video stream code rate in a video conference system according to an embodiment of the present application;
fig. 7 is a schematic diagram of a connection relationship between an upstream user end, a video conference system, and a downstream user end according to an embodiment of the present application.
Detailed Description
The features and advantages of the present application will become more apparent and appreciated from the following detailed description of the application.
The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. While the various aspects of the embodiments are presented in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.
In the description of the present application, it should be noted that the terms "upper", "lower", "inner", "outer", "front", "rear", "left" and "right" and the like indicate orientations or positional relationships based on operational states of the present application, and are only used for convenience of description and simplification of description, but do not indicate or imply that the referred device or element must have a specific orientation, be constructed in a specific orientation and be operated, and thus, should not be construed as limiting the present application. Furthermore, the terms "first," "second," "third," and "fourth" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
Specific embodiments of the present application will be described in detail below with reference to the accompanying drawings.
Fig. 1 is a schematic flowchart of a method for adjusting a video stream in a video conference system according to an embodiment of the present disclosure, and as shown in fig. 1, the method includes steps S100 to S500.
Step S100: and acquiring an uplink video stream sent by an uplink user side.
The inventor finds that the downlink user side subscribes the video of the uplink user on the video conference system, and the video conference system transmits the video stream in a data transmission mode. However, in the process of transmitting the transparent transmission data, the video stream is not changed in any form, i.e. is uninterrupted, grouped, encoded, encrypted or not confused, and the like, and the video stream is transmitted from the uplink user end to the downlink user end without any change, and the bit rate of the video stream is influenced and the video quality is not high due to different bandwidths between the downlink user ends.
The method adopts a strategy of separately controlling the uplink video stream and the downlink video stream, the code rate of the uplink video stream is adjusted according to the uplink bandwidth from the uplink user terminal to the video conference system, and the code rate of the downlink video stream is specifically adjusted according to the downlink bandwidth from the video conference system to each downlink user terminal. Therefore, the bandwidth of the downlink user end can not influence the quality of the uplink video stream and can not influence each other.
In this embodiment, the bitrate of the upstream video stream is adaptively adjusted by an upstream user according to an upstream bandwidth from the upstream user to a video conference system, and fig. 2 is a schematic flow chart of adjusting the bitrate of the upstream video stream by the upstream user according to the embodiment of the present application, where as shown in fig. 2, the adjusting method specifically includes:
an uplink user end sends an uplink detection packet and a first uplink video stream to a video conference system, wherein the first uplink video stream can be a video stream with a medium code rate, the video conference system periodically generates an uplink detection report from information such as delay and packet loss rate of the received uplink detection packet and feeds the uplink detection report back to the uplink user end, the uplink user end judges whether the current uplink network state is under load or overload according to the uplink detection report, if the current uplink network state is under load, the code rate of the uplink video stream is gradually increased, and the phenomenon that overload is caused by overlarge once lifting is avoided; and if the overload happens, the code rate of the upstream video stream is reduced moderately.
Since the bitrate of the upstream video stream is controlled by adjusting the corresponding parameters of the encoder, and can be continuously changed, the bitrate of the upstream video stream can be gradually adjusted. In the adjusting process, the amplitude of each increase is smaller, the amplitude of each decrease is larger, the amplitude can be preset, and a reasonable value can be estimated according to the detection packet report through a mathematical model.
Optionally, the uplink user end may further perform spatial layered coding on the uplink video stream to obtain two video streams: the video stream of the base layer with lower code rate, resolution and frame rate and the video stream of the high-definition layer with higher code rate, resolution and frame rate. Even if the video conference system only re-encodes the high-definition layer video stream all the way, the three-file video stream can be obtained for being selected by a downlink user side. Therefore, hardware resources of the uplink user end can be fully utilized to recode the video conference system to reduce the pressure.
Step S200: and recoding the uplink video stream to obtain a plurality of downlink video streams with different code rates, wherein the downlink video streams comprise a first downlink video stream.
Preferably, the first downlink video stream is a video stream with a bitrate corresponding to a middle value of a plurality of different bitrates, that is, a video stream with a medium bitrate.
Specifically, the video conference system performs re-encoding on the upstream video stream sent by the upstream client, that is, after the upstream video stream is decoded, the upstream video stream is encoded multiple times by using multiple different encoding parameters to obtain multiple downstream video streams with different code rates, so as to push the matched downstream video stream according to the downstream bandwidth of the downstream client, thereby ensuring the quality of the video watched by each downstream client.
Step S300: and sending the downlink detection packet and the first downlink video stream to a downlink user side.
After a plurality of downlink video streams with different code rates are obtained, the downlink video stream with the adaptive code rate needs to be pushed according to the downlink bandwidth of each downlink user end, and therefore the downlink bandwidth needs to be known to carry out targeted pushing, so whether the first downlink video stream with a medium code rate is sent to test is appropriate or not can be preferably adopted, if the medium code rate exceeds the downlink bandwidth of the downlink user end, the downlink video stream with the lower code rate can be immediately switched to through a detection packet report, and if the downlink bandwidth is enough, the downlink video stream with the higher code rate can be switched to subsequently. The downstream video stream with the lowest bit rate is not preferred to be used for probing, so that the problem that the watching effect is influenced because the downstream user side always plays the video with the low bit rate, namely the low quality, in the process of probing the downstream bandwidth is avoided. And the downlink video with medium code rate is adopted for probing, the detection of the downlink bandwidth and the time consumption of the switching process are both very short, and the influence is small. The bitrate of the first downstream video stream can be selected according to specific needs, and is not further limited herein.
Step S400: and receiving the first downlink detection report and acquiring the first downlink network state.
The video conference system can always send downlink detection packets, a downlink user side periodically generates downlink detection reports according to information such as delay, packet loss rate and the like of the received downlink detection packets and feeds the downlink detection reports back to the video conference system, the video conference system obtains a current downlink network state according to the downlink detection reports, the downlink network state comprises overload or no overload, and then whether the code rate of a currently played first downlink video stream is matched with a downlink bandwidth or not is judged according to the downlink network state, whether adjustment and switching are needed or not is judged, and the video quality of a downlink user is ensured.
Step S500: and judging whether to switch the first downlink video stream according to the first downlink network state.
In one embodiment, if the first downlink network status is not overloaded, the following steps are performed:
s501: setting a first detection target code rate, wherein the first detection target code rate is greater than the code rate of the first downlink video stream.
S502: and sending a first code rate filling packet to a downlink user end according to the first detection target code rate, wherein the code rate of the first code rate filling packet is the difference value between the first detection target code rate and the code rate of the first downlink video stream.
S503: and receiving a second downlink detection report and acquiring a second downlink network state.
S504: and if the second downlink network state is overload, stopping sending the first code rate filling packet, and determining not to switch the first downlink video stream.
As shown in fig. 3, if the first downlink network state is not overloaded, the bitrate of the first downlink video stream may be adapted to the downlink bandwidth, and may also be in an underloaded state, which requires further confirmation, so that a bitrate padding packet needs to be sent to ascertain a higher lower limit of the downlink bandwidth. And the code rate of the code rate filling packet is the difference value between the detection target code rate and the code rate of the currently played downlink video stream.
For example, the bitrate of the first downlink video stream is 600kbps, and if the first downlink network status is not overloaded, the bitrate of the first detection target is set to 650kbps, because the bitrate of the first downlink video stream being played is 600kbps, the target is required to send a first bitrate padding packet of 50kbps to the downlink ue. And if the received second downlink detection report shows that the second downlink detection report is overloaded, stopping sending the first code rate filling packet, and showing that the played first downlink video stream is matched with the bandwidth of the downlink user side, and the video is continuously played without switching.
In another embodiment, if the first downlink network status is not overloaded, the following steps are performed, as shown in fig. 4:
s501: setting a first detection target code rate, wherein the first detection target code rate is greater than the code rate of the first downlink video stream;
s502: sending a first code rate filling packet to a downlink user end according to the first detection target code rate, wherein the code rate of the first code rate filling packet is the difference value between the first detection target code rate and the code rate of the first downlink video stream;
s503: receiving a second downlink detection report and acquiring a second downlink network state;
s505: and if the second downlink network state is not overloaded, switching the first downlink video stream into a second downlink video stream according to a first detection target code rate, wherein the code rate of the second downlink video stream is greater than that of the first downlink video stream and is less than or equal to the first detection target code rate.
In this step, a first downlink bandwidth lower limit, that is, a first detection target code rate, is obtained, whether a second downlink video stream with a higher code rate that meets the first downlink bandwidth lower limit exists in the downlink video stream set or not is searched according to the first downlink bandwidth lower limit, and if the second downlink video stream exists, the second downlink video stream is switched to. The principle of selecting the code rate of the second downlink video stream is mainly to ensure that the switched downlink video stream with a higher code rate does not exceed the minimum value of the currently known downlink bandwidth possibility, so that the video playing after switching can be ensured not to be blocked.
S506: and setting a second detection target code rate, wherein the second detection target code rate is greater than the first detection target code rate.
S507: sending a second code rate filling packet to a downlink user end according to the second detection target code rate, wherein the code rate of the second code rate filling packet is the difference value between the second detection target code rate and the code rate of the second downlink video stream;
s508: and receiving a third downlink detection report and acquiring a third downlink network state.
S509: and if the third downlink network state is not overloaded, switching the second downlink video stream into a third downlink video stream according to the second detection target code rate, wherein the code rate of the third downlink video stream is greater than that of the second downlink video stream and is less than or equal to the second detection target code rate.
In this step, if not overloaded, a second downlink bandwidth lower limit, that is, a second detection target code rate is obtained, whether a third downlink video stream with a higher code rate which meets the requirement exists in the downlink video stream set or not is searched according to the second downlink bandwidth lower limit, and if yes, the third downlink video stream is switched to. Gradually increasing the detection target code rate, sending a code rate filling packet corresponding to the detection target code rate, and circularly executing the step S505 to the step S508 until the detection report shows that the current downlink network state is overload or the video stream is switched to the video stream with the highest code rate.
S510: and if the third downlink network state is overload, stopping sending the second code rate filling packet.
For example, the bitrate of the first downlink video stream is 600kbps, and if the first downlink network status is not overloaded, the bitrate of the first detection target is set to 650kbps, because the bitrate of the first downlink video stream being played is 600kbps, the target is required to send a first bitrate padding packet of 50kbps to the downlink ue. If the received second probing report indicates that the second downlink network status is not overloaded, 650kbps is set as the first bandwidth lower limit, and video streams which are not greater than 600kbps but less than or equal to 650kbps are searched for switching, and if so, switching is performed. Then setting a second detection target code rate to 750kbps, sending a second code rate filling packet to a downlink user end, wherein the code rate of the second code rate filling packet is the difference between the code rate of the currently sent video stream and the second detection target code rate, and if the video stream which is larger than 600kbps and smaller than or equal to 650kbps is switched in the previous step, and the code rate is assumed to be 630kbps, the code rate of the second code rate filling packet is 120kbps (750kbps-630 kbps); if there is no handover in the previous step, the second rate pad packet has a code rate of 150kbps (750kbps-600 kbps). And if the third detection report shows that the third network state is overload, stopping sending the second code rate filling packet and stopping the bandwidth detection process. If the third probing report shows that the third network status is still not overloaded, 750kbps is updated to the second lower bandwidth limit, and whether there is a video stream greater than 650kbps and less than or equal to 750kbps is found for switching, if so, switching is performed. And then, continuously increasing the detection target code rate to 800kbps, sending a third code rate filling packet to a downlink user end, and continuously detecting a higher downlink bandwidth lower limit until a detection report shows that the downlink network state is overloaded or the video stream with the highest code rate is switched to.
The embodiment is mainly applied to how to gradually detect a higher downlink bandwidth lower limit when the downlink network state is under load, and match a downlink video stream with a higher code rate according to the detected downlink bandwidth lower limit, so as to ensure the video quality watched by the downlink user side.
In one embodiment, if the first downlink network status is overload, the following steps are performed, as shown in fig. 5:
s511: and if the first downlink network state is overload, switching the first downlink video stream into a fourth downlink video stream, wherein the code rate of the fourth downlink video stream is smaller than that of the first downlink video stream.
S512: receiving a fourth downlink detection report and acquiring a fourth downlink network state;
and if the fourth downlink network state is overload, switching the fourth downlink video stream into a fifth downlink video stream, wherein the code rate of the fifth downlink video stream is smaller than that of the fourth downlink video stream.
S513: if the fourth downlink network status is not overloaded, the process returns to step S501 to restart the bandwidth detection process.
For example, the bitrate of the first downlink video stream is 600kbps, if the first downlink detection report shows that the first downlink network state is overloaded, the first downlink video stream of 600kbps is switched to a fourth downlink video stream of 500kbps, and then, if the received fourth downlink detection report shows that the fourth downlink network state is still overloaded, the currently played downlink video stream is continuously switched to a downlink video stream with a lower bitrate.
In the process of gradually reducing the code rate of the downlink video stream, the situation that the code rate of the switched downlink video stream is reduced by a large extent and underload occurs inevitably occurs, and at this time, the step S501 is skipped to restart the bandwidth detection process.
Preferably, in this embodiment, after acquiring that the downlink network status is overload, the method further includes: detecting whether a code rate filling packet is sent or not, if so, stopping sending the code rate filling packet, and waiting for a detection report of the next period; if not, switching the downstream video stream. The code rate filling packet occupies a certain bandwidth, and the overload may be caused by the occupation of the code rate filling packet, so that after the first downlink network state is obtained as the overload, whether the code rate filling packet is sent or not is detected, if so, the sending of the code rate filling packet is stopped, the total code rate is reduced, and the detection period can be reduced. .
The embodiment is mainly applied to how to gradually reduce the code rate of the downlink video stream when the downlink network state is overload, and ensure the video quality pushed to the downlink user side.
Corresponding to the embodiment of the method for adjusting the video stream code rate in the video conference system, the invention also provides an embodiment of a device for adjusting the video stream code rate in the video conference system. Referring to fig. 6, a schematic structural diagram of an apparatus for adjusting a bitrate of a video stream in a video conference system according to an embodiment of the present application is provided, where the apparatus includes:
an uplink video stream acquiring module 100, configured to acquire an uplink video stream sent by an uplink user end;
a re-encoding module 200, configured to re-encode the uplink video stream to obtain a downlink video stream set including multiple different code rates, where the downlink video stream set includes a first downlink video stream;
a sending module 300, configured to send a probe packet and the first downlink video stream to a downlink client;
a detection report receiving module 400, configured to receive a first downlink detection report and obtain a first downlink network status; the downlink network state acquisition unit is also used for receiving a second downlink detection report and acquiring a second downlink network state; the system is also used for receiving a third detection report and acquiring a third downlink network state; the system is also used for receiving a fourth detection report and acquiring a fourth downlink network state;
a determining module 500, configured to determine whether to switch the first downlink video stream according to the first downlink network state.
According to the method for adjusting the video stream code rate in the video conference system, the uplink video stream adaptive to the uplink bandwidth is adjusted through the uplink client; the video conference system performs recoding on the uplink video stream to obtain downlink video streams with different code rates, sends a downlink detection packet and the downlink video stream with the medium code rate to a downlink user end, obtains a downlink network state according to a detection report, and judges whether to switch the downlink video stream with the medium code rate according to the downlink network; if the downlink network state is overload, gradually reducing and switching to the downlink video stream with low code rate adaptation; if the downlink network state is not overloaded, gradually detecting a higher downlink bandwidth lower limit by sending a code rate filling packet, and simultaneously switching to a downlink video stream matched with the highest downlink bandwidth lower limit according to the detected downlink bandwidth lower limit.
As shown in fig. 7, the present application adopts a strategy of separately controlling an upstream video stream and a downstream video stream, where a bitrate of the upstream video stream is adjusted according to an upstream bandwidth from an upstream user end to a video conference system, and a bitrate of the downstream video stream is specifically adjusted according to a downstream bandwidth from the video conference system to each downstream user end. The video conference system obtains the video streams with different code rates by recoding the uplink video stream, and respectively pushes the video streams suitable for the bandwidths of the downlink user ends to each downlink user end, and the downlink user end with high bandwidth can obtain the video stream with higher quality, so that the downlink user ends cannot mutually influence each other.
According to the method for adjusting the video stream code rate in the video conference system, the embodiment of the invention also provides a readable storage medium and computer equipment. The readable storage medium stores an executable program, and the program when executed by the processor implements the steps of the method for adjusting the video stream bitrate in the video conference system; the computer device comprises a memory, a processor and an executable program stored on the memory and capable of running on the processor, and the processor executes the program to realize the steps of the method for adjusting the video stream code rate in the video conference system.
The invention has been described in detail with reference to specific embodiments and illustrative examples, but the description is not intended to be construed in a limiting sense. Those skilled in the art will appreciate that various equivalent substitutions, modifications or improvements may be made to the technical solution of the present invention and its embodiments without departing from the spirit and scope of the present invention, which fall within the scope of the present invention. The scope of the invention is defined by the appended claims.
Claims (10)
1. A method for adjusting video streams in a video conferencing system, the method comprising:
acquiring an uplink video stream sent by an uplink user side;
recoding the uplink video stream to obtain a downlink video stream set comprising a plurality of different code rates, wherein the downlink video stream set comprises a first downlink video stream;
sending a downlink detection packet and the first downlink video stream to a downlink user side;
receiving a first downlink detection report and acquiring a first downlink network state;
judging whether to switch the first downlink video stream according to the first downlink network state;
the determining whether to switch the first downlink video stream according to the first downlink network state includes:
if the first downlink network state is not overloaded, setting a first detection target code rate, wherein the first detection target code rate is greater than the code rate of the first downlink video stream;
and sending a first code rate filling packet to a downlink user end according to the first detection target code rate, wherein the code rate of the first code rate filling packet is the difference value between the first detection target code rate and the code rate of the first downlink video stream.
2. The method of claim 1, wherein the first downstream video stream is a video stream with a bitrate corresponding to a middle value of the plurality of different bitrates.
3. The method of claim 1, wherein the determining whether to switch the first downstream video stream according to the first downstream network status further comprises:
receiving a second downlink detection report and acquiring a second downlink network state;
and if the second downlink network state is overload, stopping sending the first code rate filling packet, and determining not to switch the first downlink video stream.
4. The method of claim 3, wherein after receiving the second probing report and obtaining the second downlink network status, the method further comprises:
if the second downlink network state is not overloaded, switching the first downlink video stream into a second downlink video stream according to a first detection target code rate, wherein the code rate of the second downlink video stream is greater than that of the first downlink video stream and is less than or equal to the first detection target code rate;
setting a second detection target code rate, wherein the second detection target code rate is greater than the first detection target code rate;
sending a second code rate filling packet to a downlink user end according to the second detection target code rate, wherein the code rate of the second code rate filling packet is the difference value between the second detection target code rate and the code rate of the second downlink video stream;
receiving a third downlink detection report and acquiring a third downlink network state;
if the third downlink network state is not overloaded, switching the second downlink video stream into a third downlink video stream according to the second detection target code rate, wherein the code rate of the third downlink video stream is greater than that of the second downlink video stream and is less than or equal to the second detection target code rate;
and if the third downlink network state is overload, stopping sending the second code rate filling packet.
5. The method of claim 1, wherein determining whether to switch the first downstream video stream according to the first downstream network status further comprises:
and if the first downlink network state is overload, switching the first downlink video stream into a fourth downlink video stream, wherein the code rate of the fourth downlink video stream is smaller than that of the first downlink video stream.
6. The method of adjusting video streams in a video conferencing system according to claim 5, wherein before switching the first downstream video stream to the fourth downstream video stream, further comprising:
and detecting whether the code rate filling packet is sent or not, and if so, stopping sending the code rate filling packet.
7. The method of adjusting video streams in a video conferencing system according to claim 5, wherein after switching the first downstream video stream to the fourth downstream video stream, further comprising:
receiving a fourth downlink detection report and acquiring a fourth downlink network state;
and if the fourth downlink network state is overload, switching the fourth downlink video stream into a fifth downlink video stream, wherein the code rate of the fifth downlink video stream is smaller than that of the fourth downlink video stream.
8. The method of claim 1, wherein the bitrate of the upstream video stream is adjusted by an upstream client according to an upstream bandwidth, and the adjusting specifically comprises:
sending an uplink detection packet and a first uplink video stream to a video conference system;
receiving an uplink detection report and acquiring an uplink network state;
and judging whether to adjust the code rate of the first uplink video stream according to the uplink network state.
9. The method of claim 1, wherein before acquiring the upstream video stream sent by the upstream user end, the method further comprises:
the uplink user terminal carries out layered coding on the video stream, and obtains the base layer uplink video stream with the code rate smaller than a preset threshold value and the high-definition layer uplink video stream with the code rate larger than the preset threshold value.
10. An apparatus for regulating video streams in a video conferencing system, the apparatus comprising:
the uplink video stream acquisition module is used for acquiring an uplink video stream sent by an uplink user side;
the recoding module is used for recoding the uplink video stream to obtain a downlink video stream set comprising a plurality of different code rates, wherein the downlink video stream set comprises a first downlink video stream;
a sending module, configured to send a downlink detection packet and the first downlink video stream to a downlink client;
a detection report receiving module, configured to receive a first downlink detection report and obtain a first downlink network state;
the judging module is used for judging whether to switch the first downlink video stream according to the first downlink network state;
the determining module is further configured to set a first detection target code rate when the first downlink network state is not overloaded, where the first detection target code rate is greater than the code rate of the first downlink video stream; and sending a first code rate filling packet to a downlink user end according to the first detection target code rate, wherein the code rate of the first code rate filling packet is the difference value between the first detection target code rate and the code rate of the first downlink video stream.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811358922.4A CN109698928B (en) | 2018-11-15 | 2018-11-15 | Method and device for adjusting video stream in video conference system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811358922.4A CN109698928B (en) | 2018-11-15 | 2018-11-15 | Method and device for adjusting video stream in video conference system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109698928A CN109698928A (en) | 2019-04-30 |
| CN109698928B true CN109698928B (en) | 2021-04-13 |
Family
ID=66229835
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811358922.4A Active CN109698928B (en) | 2018-11-15 | 2018-11-15 | Method and device for adjusting video stream in video conference system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109698928B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111064960B (en) * | 2019-12-31 | 2023-06-02 | 广州市百果园网络科技有限公司 | Video coding method, device, server and storage medium |
| CN114640818A (en) * | 2022-03-04 | 2022-06-17 | 京东科技信息技术有限公司 | Video processing method and device, electronic equipment and storage medium |
| CN115278152A (en) * | 2022-07-20 | 2022-11-01 | 阿里巴巴（中国）有限公司 | A transmission method, electronic device and storage medium for video call data |
Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102256101A (en) * | 2011-07-26 | 2011-11-23 | 中兴通讯股份有限公司 | Method and system for converting media capability in video conference, and application server |
| CN102413307A (en) * | 2011-11-30 | 2012-04-11 | 苏州希图视鼎微电子有限公司 | Adaptive code rate adjusting method and device for television video chat |
| CN102957672A (en) * | 2011-08-25 | 2013-03-06 | 中国电信股份有限公司 | Method, client side and system for self-adaptive playing of FLV (flash video) media streams |
| CN103957428A (en) * | 2014-05-16 | 2014-07-30 | 中国科学院上海高等研究院 | Streaming media server with multi-code-rate transcoding function based on many-core processor |
| CN104270646A (en) * | 2014-09-22 | 2015-01-07 | 何震宇 | Self-adaption transmission method and system based on mobile streaming media |
| CN104349219A (en) * | 2013-07-29 | 2015-02-11 | 中国科学院大学 | Strict increase loose decrease equal step congestion control algorithm based on mobile communication network |
| CN105227894A (en) * | 2014-07-01 | 2016-01-06 | 三亚中兴软件有限责任公司 | A kind of implementation method of video conference and multipoint control unit |
| CN105610869A (en) * | 2016-03-03 | 2016-05-25 | 腾讯科技（深圳）有限公司 | Method and device for scheduling streaming media |
| CN105812717A (en) * | 2016-04-21 | 2016-07-27 | 邦彦技术股份有限公司 | Multimedia conference control method and server |
| CN106412478A (en) * | 2015-07-30 | 2017-02-15 | 北京华纬讯电信技术有限公司 | Video coding and transmission method adaptive to unknown network environment |
| CN106454395A (en) * | 2016-09-20 | 2017-02-22 | 北京百度网讯科技有限公司 | Method and device for providing multi-code rate streaming media self-adaptively in server |
| CN108024125A (en) * | 2017-11-03 | 2018-05-11 | 青岛海信电器股份有限公司 | A kind of audio, video data playback method and server and client side |
| CN108093257A (en) * | 2017-12-05 | 2018-05-29 | 北京小米移动软件有限公司 | Bit rate control method, electronic equipment and the storage medium of Video coding |
| CN108289310A (en) * | 2017-01-09 | 2018-07-17 | 中兴通讯股份有限公司 | Network quality detection method and device |
| WO2018176494A1 (en) * | 2017-04-01 | 2018-10-04 | SZ DJI Technology Co., Ltd. | Method and system for video transmission |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107181968B (en) * | 2016-03-11 | 2019-11-19 | 腾讯科技（深圳）有限公司 | A kind of redundancy control method and device of video data |
-
2018
- 2018-11-15 CN CN201811358922.4A patent/CN109698928B/en active Active
Patent Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102256101A (en) * | 2011-07-26 | 2011-11-23 | 中兴通讯股份有限公司 | Method and system for converting media capability in video conference, and application server |
| CN102957672A (en) * | 2011-08-25 | 2013-03-06 | 中国电信股份有限公司 | Method, client side and system for self-adaptive playing of FLV (flash video) media streams |
| CN102413307A (en) * | 2011-11-30 | 2012-04-11 | 苏州希图视鼎微电子有限公司 | Adaptive code rate adjusting method and device for television video chat |
| CN104349219A (en) * | 2013-07-29 | 2015-02-11 | 中国科学院大学 | Strict increase loose decrease equal step congestion control algorithm based on mobile communication network |
| CN103957428A (en) * | 2014-05-16 | 2014-07-30 | 中国科学院上海高等研究院 | Streaming media server with multi-code-rate transcoding function based on many-core processor |
| CN105227894A (en) * | 2014-07-01 | 2016-01-06 | 三亚中兴软件有限责任公司 | A kind of implementation method of video conference and multipoint control unit |
| CN104270646A (en) * | 2014-09-22 | 2015-01-07 | 何震宇 | Self-adaption transmission method and system based on mobile streaming media |
| CN106412478A (en) * | 2015-07-30 | 2017-02-15 | 北京华纬讯电信技术有限公司 | Video coding and transmission method adaptive to unknown network environment |
| CN105610869A (en) * | 2016-03-03 | 2016-05-25 | 腾讯科技（深圳）有限公司 | Method and device for scheduling streaming media |
| CN105812717A (en) * | 2016-04-21 | 2016-07-27 | 邦彦技术股份有限公司 | Multimedia conference control method and server |
| CN106454395A (en) * | 2016-09-20 | 2017-02-22 | 北京百度网讯科技有限公司 | Method and device for providing multi-code rate streaming media self-adaptively in server |
| CN108289310A (en) * | 2017-01-09 | 2018-07-17 | 中兴通讯股份有限公司 | Network quality detection method and device |
| WO2018176494A1 (en) * | 2017-04-01 | 2018-10-04 | SZ DJI Technology Co., Ltd. | Method and system for video transmission |
| CN108024125A (en) * | 2017-11-03 | 2018-05-11 | 青岛海信电器股份有限公司 | A kind of audio, video data playback method and server and client side |
| CN108093257A (en) * | 2017-12-05 | 2018-05-29 | 北京小米移动软件有限公司 | Bit rate control method, electronic equipment and the storage medium of Video coding |
Non-Patent Citations (2)
| Title |
|---|
| 基于ACK包对的带宽估计算法研究;袁鹏飞;《中国优秀硕士学位论文全文数据库》;20140830;I139-29 * |
| 有限带宽网络视频流传输速率控制策略研究;梅鲁海;《电视技术》;20110731;第101-104页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109698928A (en) | 2019-04-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10334289B2 (en) | Efficient approach to dynamic frame size and frame rate adaptation | |
| JP6420006B2 (en) | Reducing latency in video phones | |
| US8527649B2 (en) | Multi-stream bit rate adaptation | |
| US8218657B2 (en) | System and method for automatic adjustment of streaming video bit rate | |
| US9609040B2 (en) | Efficient bitrate adaptation in video communications over IP networks | |
| CN104270649B (en) | Image coding device and video encoding method | |
| CN109698928B (en) | Method and device for adjusting video stream in video conference system | |
| KR20100106383A (en) | Apparatus and method for simulcast over a variable bandwidth channel | |
| JP2017069849A (en) | VIDEO CONTROL DEVICE, VIDEO DISTRIBUTION SYSTEM, AND VIDEO CONTROL METHOD | |
| CN101656887A (en) | Method and device for selecting rate control algorithm | |
| JP5434570B2 (en) | Stream distribution device | |
| KR20160044819A (en) | Apparatus and method for http adaptive streaming in wireless network environment | |
| AU2019201095A1 (en) | System and method for automatic encoder adjustment based on transport data | |
| Singh et al. | Optimising QoE for scalable video multicast over WLAN | |
| EP3744022B1 (en) | Abr control | |
| WO2002035844A2 (en) | Dynamic quality adaptation using personal quality profiles and composite performance metric | |
| KR20130123417A (en) | Method and system for controlling data packet transmissions over lossy protocols | |
| KR20140140364A (en) | Apparatus and method for providing contents | |
| Kumar et al. | Machine learning approach for quality adaptation of streaming video through 4G wireless network over HTTP | |
| CN117834947A (en) | Layered coding method, device, equipment and computer readable medium for media stream |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20190430 Assignee: Guiyang Longma Video Technology Co.,Ltd. Assignor: GUIYANG LONGMASTER INFORMATION & TECHNOLOGY Co.,Ltd. Contract record no.: X2022520000003 Denomination of invention: A method and device for adjusting video stream in video conference system Granted publication date: 20210413 License type: Common License Record date: 20220302 Application publication date: 20190430 Assignee: GUIYANG SANJIU INTERNET MEDICAL Co.,Ltd. Assignor: GUIYANG LONGMASTER INFORMATION & TECHNOLOGY Co.,Ltd. Contract record no.: X2022520000002 Denomination of invention: A method and device for adjusting video stream in video conference system Granted publication date: 20210413 License type: Common License Record date: 20220301 |
|
| EE01 | Entry into force of recordation of patent licensing contract |