CN112087635A - Image coding control method, device, equipment and computer readable storage medium - Google Patents

Abstract

The present application provides an image encoding control method, apparatus, device, and computer-readable storage medium, which relate to the technical field of image processing and can solve the problem of long image encoding delay. The specific technical scheme is as follows: count the network delay of each frame of images in a preset number of frame images; calculate the average value of the network delay of the preset number of frame images according to the network delay of each frame of image; determine the average network delay Whether the value is greater than the preset threshold; when the average value of the network delay is greater than the preset threshold, the number M of dropped frames is determined according to the preset rule; the consecutive M frames of images that are not currently encoded are discarded. The number of lost frames can be determined before encoding, and a corresponding number of images can be discarded before encoding, which can relieve network pressure, shorten image encoding delay, and reduce CPU encoding load.

Description

Image coding control method, apparatus, device, and computer-readable storage medium

technical field

The present application relates to the field of image processing, and in particular, to an image coding control method, apparatus, device, and computer-readable storage medium.

Background technique

At present, in the process of encoding and decoding a graphic image, image acquisition is usually performed at a fixed frame rate, and after the acquired image is encoded, it waits for transmission in a buffer. During the transmission process, it will be judged whether the encoded image in the cache needs to be discarded according to the network delay.

For all images, since it is determined whether or not to discard the images after the encoding is completed, the image encoding delay is increased, and the encoding resources of a CPU (Central Processing Unit, central processing unit) are also wasted.

SUMMARY OF THE INVENTION

Embodiments of the present application provide an image encoding control method, apparatus, device, and computer-readable storage medium, which can solve the problem of long image encoding delay. The technical solution is as follows:

According to a first aspect of the embodiments of the present application, an image coding control method is provided, the method comprising:

Count the network delay of each frame in the preset number of frames;

Calculate the average value of the network delay of the preset number of frame images according to the network delay of each frame of image;

judging whether the average value of the network delay is greater than a preset threshold;

When the average value of the network delay is greater than the preset threshold, determine the number of dropped frames M according to a preset rule, where M is an integer;

Discard the consecutive M frames of images that are not currently encoded.

In one embodiment, the determining the number M of dropped frames according to a preset rule includes:

The number M of dropped frames is determined according to the difference between the average value of the network delay and the preset threshold.

In one embodiment, the determining the number M of dropped frames according to the difference between the average value of the network delay and the preset threshold includes:

The number of dropped frames M is determined according to the first formula, and the first formula is: M=[(T _average -A)/(t _acquisition +t _encoding )];

Wherein, the T _average is the average value of the network delay, the A is a preset threshold, the t _acquisition is the average time required to collect a frame of image, and the t _code is the average time required to encode a frame of image.

In one embodiment, the determining the number M of dropped frames according to the difference between the average value of the network delay and the preset threshold includes:

The number of dropped frames M is determined according to the second formula, and the second formula is:

M=[(T _average -A)/(t _acquisition +t _encoding )]+1;

Wherein, the T _average is the average value of the network delay, the A is a preset threshold, the t _acquisition is the average time required to collect a frame of image, and the t _code is the average time required to encode a frame of image.

In one embodiment, the method further includes:

Detecting a set operation for setting a preset number;

The preset number is determined according to the setting operation.

In the image coding control method provided by the embodiment of the present application, the network delay of each frame of images in a preset number of frame images is counted in real time during the image transmission process, and the average value of the network delay of the preset number of frame images is calculated. When the average value is greater than the preset threshold, the number M of dropped frames is determined according to the preset rule, and the consecutive M frames of images that are not currently encoded are discarded. The number of lost frames can be determined before encoding, and a corresponding number of images can be discarded before encoding, which can relieve network pressure, shorten image encoding delay, and reduce CPU encoding load.

According to a second aspect of the embodiments of the present application, an image coding control device is provided, including:

The statistics module is used to count the network delay of each frame of images in the preset number of frame images;

a calculation module, configured to calculate the average value of the network delay of the preset number of frame images according to the network delay of each frame of image;

A first determination module, configured to determine whether the average value of the network delay is greater than a preset threshold, and when the average value of the network delay is greater than the preset threshold, determine the number of lost frames M according to a preset rule, Said M is an integer;

The frame discarding module is used for discarding consecutive M frames of images that are not currently encoded.

In one embodiment, the first determining module is specifically configured to determine the number M of dropped frames according to the difference between the average value of the network delay and the preset threshold.

In one embodiment, the first determining module is specifically configured to determine the number M of dropped frames according to a first formula, and the first formula is: M=[(T _average -A)/(t _acquisition +t _encoding )] ;

Wherein, the T _average is the average value of the network delay, the A is a preset threshold, the t _acquisition is the average time required to collect a frame of image, and the t _code is the average time required to encode a frame of image.

In one embodiment, the first determining module is specifically configured to determine the number of dropped frames M according to a second formula, and the second formula is:

M=[(T _average -A)/(t _acquisition +t _encoding )]+1;

Wherein, the T _average is the average value of the network delay, the A is a preset threshold, the t _acquisition is the average time required to collect a frame of image, and the t _code is the average time required to encode a frame of image.

In one embodiment, the apparatus further includes:

a detection module for detecting a setting operation for setting a preset number;

A second determining module, configured to determine the preset number according to the setting operation.

In the image coding control device provided by the embodiment of the present application, the statistics module counts the network delay of each frame of images in the preset number of frame images, the calculation module calculates the average value of the network delay of the preset number of frame images, and the first determines When judging that the average value is greater than the preset threshold, the module determines the number of dropped frames M according to the preset rule, and the frame discarding module discards consecutive M frames of images that are not currently encoded. The number of lost frames can be determined before encoding, and a corresponding number of images can be discarded before encoding, which can relieve network pressure, shorten image encoding delay, and reduce CPU encoding load.

According to a third aspect of the embodiments of the present application, an image encoding control device is provided, the image encoding control device includes a processor and a memory, the memory stores at least one computer instruction, the instruction is loaded by the processor and Executed to realize the steps executed in the image coding control method as described above.

According to a fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided, where at least one computer instruction is stored in the storage medium, and the instruction is loaded and executed by a processor to implement the above-mentioned image encoding control method steps performed in .

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not limiting of the present application.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description serve to explain the principles of the application.

1 is a flowchart of an image coding control method provided by an embodiment of the present application;

2 is a flowchart of a specific image coding control method provided by an embodiment of the present application;

FIG. 3 is a schematic structural diagram of an image coding control apparatus provided by an embodiment of the present application;

4 is a schematic structural diagram of another image coding control apparatus provided by an embodiment of the present application;

FIG. 5 is a schematic structural diagram of an image coding control device provided by an embodiment of the present application.

Detailed ways

Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. Where the following description refers to the drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the illustrative examples below are not intended to represent all implementations consistent with this application. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present application as recited in the appended claims.

An embodiment of the present application provides an image coding control method. As shown in FIG. 1 , the image coding control method may include the following steps:

Step 101: Count network delays of a preset number of frames of images.

During the image transmission process, the network delay of each frame of images in the preset number of frames of images that have been recently transmitted is counted in real time. Specifically, the network delay of one frame of image is equal to the difference between the time stamp when the image is received by the receiver and the time stamp when the image is sent by the sender.

Step 102: Calculate the average value of the network delay of the preset number of frame images.

After the network delay of each frame of images in the preset number of frame images that have been recently transmitted is obtained by statistics, the average value of the network delay of the preset number of frame images is calculated according to the network delay of each frame image obtained by statistics. For example, if the preset number is N, the network delay of each frame of the N frames of images is continuously counted, and then the average value of the network delay of the N frames of images is calculated.

Step 103: Determine whether the average value is greater than a preset threshold.

After calculating the average value of the network delay of the preset number of frame images, it is judged whether the average value of the network delay is greater than the preset threshold.

Step 104: Determine the number M of dropped frames according to a preset rule.

When the average value of the network delay of the preset number of frame images is greater than the preset threshold, the number M of dropped frames is determined according to the preset rule, where M is an integer. For example, the number M of dropped frames may be determined according to the difference between the average value of the network delay and the preset threshold. In one embodiment, the number M of dropped frames can be obtained by using the difference and the ratio of the average duration required to acquire one frame of image and the sum of the average duration required to encode one frame of image.

Step 105: Discard the consecutive M frames of images that are not currently encoded.

After the number M of lost frames is determined, the consecutive M frames of images that are not currently encoded are directly discarded, and then, go to step 101 to continue the statistics of the network delay. In this step, images that have already been transmitted or encoded are not discarded, and subsequent processing is still performed normally, and only consecutive M frames of images that have not been encoded are discarded.

In the image coding control method provided by the embodiment of the present application, the network delay of each frame of images in a preset number of frame images is counted in real time during the image transmission process, and the average value of the network delay of the preset number of frame images is calculated. When the average value is greater than the preset threshold, the number M of dropped frames is determined according to the preset rule, and the consecutive M frames of images that are not currently encoded are discarded. The number of lost frames can be determined before encoding, and a corresponding number of images can be discarded before encoding, which can relieve network pressure, shorten image encoding delay, and reduce CPU encoding load.

Based on the image coding control method provided by the above-mentioned embodiment corresponding to FIG. 1 , another embodiment of the present application provides a specific image coding control method, which can be applied to a video image coding end device. For example, the video image coding end device may It is a terminal device or a network server, etc. Referring to FIG. 2 , the image coding control method provided by this embodiment may include the following steps:

Step 201: Count the network delay of a preset number of frames of images.

During the image transmission process, the average network delay of each frame of images in the preset number of frames of images that have been recently transmitted is counted in real time. The preset number can be set according to actual needs, for example, it can be set to 10 frames, 20 frames, 30 frames and so on. The network delay of a frame of image refers to the difference between the time stamp when the image is received by the receiver and the time stamp when the image is sent by the sender.

In one embodiment, the function of setting a preset number may be provided, and specifically, it may be implemented by the following methods: detecting a setting operation for setting the preset number; and determining the preset number according to the setting operation. For example, a setting interface for setting a preset number can be provided for the user, and the user can input the preset number according to the interface prompts, such as setting 10 frames, 20 frames, 30 frames, etc.; or, through the shortcut of the preset number key to set the preset number.

Step 202: Calculate the average value of the network delay of the preset number of frame images.

After the network delay of each frame of images of the preset number of frame images that have been recently transmitted is obtained by statistics, the average value of the network delay of the preset number of frame images is calculated according to the network delay of each frame of image. For example, if the preset number is N, the network delay of each frame of the N frames of images is continuously counted, and then the average value of the network delay of the N frames of images is calculated.

Specifically, the average network delay T _average of consecutive N frames of images can be calculated by the following formula:

其中，Rt_x表示第x帧图像传输到接收端的时间戳，St_x表示第x帧图像在发送端发送时的时间戳。Among them, ▽t _x is the network delay of the xth frame image, x is an integer greater than 0 and less than or equal to N,

Among them, Rt _x represents the time stamp of the xth frame image transmitted to the receiving end, and St _x represents the time stamp of the xth frame image sent by the sending end.

Step 203: Determine whether the average value is greater than a preset threshold.

It is judged whether the average value of the network delay of the preset number of frame images obtained by statistics is greater than the preset threshold, and if it is greater than the preset threshold, go to step 204, otherwise go to step 201 to continue the statistics.

In this step, the calculated average value of the network delay of the preset number of frame images is compared with the preset threshold, and it is determined whether the average value is greater than the preset threshold. The preset threshold is a preset value, which can be set according to experience, for example, can be set to 100ms, 90ms, 80ms, etc., or can be set to other values. The higher the preset threshold value is, the higher the tolerance for the delay is, therefore, the setting of the preset threshold value can be determined according to the usage scenario.

In one embodiment, a function of setting a preset threshold may be provided, and specifically, it may be implemented by the following methods: detecting a setting operation for setting the preset threshold; and determining the preset threshold according to the setting operation. For example, a setting interface for setting a preset threshold value may be provided for the user, and the user may input the preset threshold value according to the interface prompts, such as setting to 100ms, 90ms, 80ms, etc.; Set threshold settings.

Step 204: Determine the number M of dropped frames according to the difference between the average value of the network delay and the preset threshold.

Assuming that the average value of the network delay obtained by statistics is T _average , the number M of dropped frames can be calculated by the following first formula, and can also be calculated by the following second formula.

Wherein, the first formula is: M=[(T _average -A)/(t _acquisition +t _encoding )];

The second formula is: M=[(T _average -A)/(t _acquisition +t _encoding )]+1;

In the above formula, T _average represents the average value of the network delay, A represents the preset threshold, t _acquisition is the average time required to collect a frame of image, t _code is the average time required to encode a frame of image, t _acquisition and The t _code can be set according to the empirical value. where the symbol [ ] represents the rounding symbol, that is, [(T _average - A)/(t _acquisition + t _encoding )] is equal to the integer part of the result obtained by (T _average - A)/(t _acquisition + t _encoding ).

Step 205: Discard the consecutive M frames of images that are not currently encoded.

After the number M of lost frames is determined, the consecutive M frames of images that are not currently encoded are directly discarded, and then the process goes to step 101 to continue the statistics of the network delay.

In this step, the continuous M frames of images that have not been encoded at present are directly discarded, and at the same time, the images that have been transmitted or encoded will not be discarded, and subsequent processing is still performed normally, and the continuous M frames that have not been encoded are discarded. image.

In the image coding control method provided by the embodiment of the present application, the network delay of each frame of images in a preset number of frame images is counted in real time during the image transmission process, and the average value of the network delay of the preset number of frame images is calculated. When the average value is greater than the preset threshold, the number M of dropped frames is determined according to the difference between the average value of the network delay and the preset threshold, and then M consecutive frames of images that are not currently encoded are discarded. The number of lost frames can be determined before encoding, and a corresponding number of images can be discarded before encoding, which can relieve network pressure, shorten image encoding delay, and reduce CPU encoding load.

Based on the image coding control methods described in the embodiments corresponding to FIG. 1 and FIG. 2 , the following are apparatus embodiments of the present application, which may be used to execute the method embodiments of the present application.

An embodiment of the present application provides an image coding control apparatus. As shown in FIG. 3 , the image coding control apparatus 30 includes: a statistics module 301 , a calculation module 302 , a first determination module 303 , and a frame discarding module 304 . in,

The statistics module 301 is used to count the network delay of each frame of images in the preset number of frame images;

The calculation module 302 is configured to calculate the average value of the network delay of a preset number of frame images according to the network delay of each frame of images obtained by the statistics module 301;

The first determination module 303 is configured to determine whether the average value of the network delay calculated by the calculation module 302 is greater than a preset threshold, and when the average value of the network delay is greater than the preset threshold, determine the number of dropped frames M according to a preset rule, where M is an integer;

The frame discarding module 304 is used for discarding consecutive M frames of images that are not currently encoded.

In one embodiment, the first determining module 303 is specifically configured to determine the number M of dropped frames according to the difference between the average value of the network delay and the preset threshold.

In one embodiment, the first determining module 303 is specifically configured to determine the number M of dropped frames according to a first formula, where the first formula is: M=[(T _average -A)/(t _acquisition +t _encoding )];

Among them, T _average is the average value of network delay, A is a preset threshold, t _acquisition is the average time required to collect a frame of image, and t _code is the average time required to encode one frame of image.

In one embodiment, the first determining module 303 is specifically configured to determine the number of dropped frames M according to a second formula, where the second formula is:

M=[(T _average -A)/(t _acquisition +t _encoding )]+1;

Among them, T _average is the average value of network delay, A is a preset threshold, t _acquisition is the average time required to collect a frame of image, and t _code is the average time required to encode one frame of image.

In one embodiment, as shown in FIG. 4 , the image coding control apparatus further includes a detection module 305 and a second determination module 306 . in,

The detection module 305 is used to detect the setting operation for setting the preset quantity;

The second determination module 306 is configured to determine the preset number according to the setting operation detected by the detection module 305 .

In the image encoding control device provided by the embodiment of the present application, the statistics module calculates the network delay of each frame of images in a preset number of frame images in real time during the image transmission process, and the calculation module calculates the network delay of the preset number of frame images. The first determining module determines the number M of dropped frames according to a preset rule when judging that the average is greater than the preset threshold, and the frame discarding module discards M consecutive frames of images that are not currently encoded. The number of lost frames can be determined before encoding, and a corresponding number of images can be discarded before encoding, which can relieve network pressure, shorten image encoding delay, and reduce CPU encoding load.

Referring to FIG. 5 , an embodiment of the present application further provides an image coding control device 50 . The image coding control device 50 includes a memory 501 and a processor 502 . The memory 501 stores at least one computer instruction, and the processor 502 is used for Load and execute at least one computer instruction to implement the image coding control method described in the embodiment corresponding to FIG. 1 or FIG. 2 .

Based on the image coding control methods described in the embodiments corresponding to FIG. 1 and FIG. 2, the embodiments of the present application further provide a computer-readable storage medium, for example, the non-transitory computer-readable storage medium may be a read-only memory ( English: Read Only Memory, ROM), random access memory (English: Random Access Memory, RAM), CD-ROM, magnetic tape, floppy disk and optical data storage device, etc. The storage medium stores computer instructions for executing the image coding control methods described in the embodiments corresponding to FIG. 1 and FIG. 2 , and details are not described herein again.

Other embodiments of the present application will readily occur to those skilled in the art upon consideration of the specification and practice of what is disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of this application and include common knowledge or practice in the art not disclosed in this application means. The specification and examples are to be regarded as exemplary only, with the true scope and spirit of the application being indicated by the appended claims.

It is to be understood that the present application is not limited to the precise structures described above and shown in the accompanying drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims (10)

1. an image coding control method, is characterized in that, described method comprises: Count the network delay of each frame in the preset number of frames; Calculate the average value of the network delay of the preset number of frame images according to the network delay of each frame of image; judging whether the average value of the network delay is greater than a preset threshold; When the average value of the network delay is greater than the preset threshold, determine the number of dropped frames M according to a preset rule, where M is an integer; Discard the consecutive M frames of images that are not currently encoded. 2. The method according to claim 1, wherein the determining the number of dropped frames M according to a preset rule comprises: The number M of dropped frames is determined according to the difference between the average value of the network delay and the preset threshold. 3. The method according to claim 2, wherein the determining the number M of dropped frames according to the difference between the average value of the network delay and the preset threshold comprises: The number of dropped frames M is determined according to the first formula, and the first formula is: M=[(T _average -A)/(t _acquisition +t _encoding )]; Wherein, the T _average is the average value of the network delay, the A is a preset threshold, the t _acquisition is the average time required to collect a frame of image, and the t _code is the average time required to encode a frame of image. 4. The method according to claim 2, wherein the determining the number M of dropped frames according to the difference between the average value of the network delay and the preset threshold comprises: The number of dropped frames M is determined according to the second formula, and the second formula is: M=[(T _average -A)/(t _acquisition +t _encoding )]+1; Wherein, the T _average is the average value of the network delay, the A is a preset threshold, the t _acquisition is the average time required to collect a frame of image, and the t _code is the average time required to encode a frame of image. 5. The method according to claim 1, wherein the method further comprises: Detecting a set operation for setting a preset number; The preset number is determined according to the setting operation. 6. An image coding control device, comprising: The statistics module is used to count the network delay of each frame of images in the preset number of frame images; a calculation module, configured to calculate the average value of the network delay of the preset number of frame images according to the network delay of each frame of image; A first determination module, configured to determine whether the average value of the network delay is greater than a preset threshold, and when the average value of the network delay is greater than the preset threshold, determine the number of lost frames M according to a preset rule, Said M is an integer; The frame discarding module is used for discarding consecutive M frames of images that are not currently encoded. 7 . The apparatus according to claim 6 , wherein the first determining module is specifically configured to determine the number M of dropped frames according to the difference between the average value of the network delay and the preset threshold. 8 . 8. The apparatus of claim 6, wherein the apparatus further comprises: a detection module for detecting a setting operation for setting a preset number; A second determining module, configured to determine the preset number according to the setting operation. 9. An image coding control device, characterized in that the image coding control device comprises a processor and a memory, wherein the memory stores at least one computer instruction, and the instruction is loaded and executed by the processor to realize the rights Steps performed in the image coding control method according to any one of claims 1 to 5. 10. A computer-readable storage medium, wherein at least one computer instruction is stored in the storage medium, and the instruction is loaded and executed by a processor to implement any one of claims 1 to 5. The steps performed in the image coding control method.

Images