[go: up one dir, main page]

CN114584834B - Video quality optimization method, device, equipment and storage medium - Google Patents

Video quality optimization method, device, equipment and storage medium Download PDF

Info

Publication number
CN114584834B
CN114584834B CN202210100824.0A CN202210100824A CN114584834B CN 114584834 B CN114584834 B CN 114584834B CN 202210100824 A CN202210100824 A CN 202210100824A CN 114584834 B CN114584834 B CN 114584834B
Authority
CN
China
Prior art keywords
image frame
image
parameter
quantization parameter
scene mode
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210100824.0A
Other languages
Chinese (zh)
Other versions
CN114584834A (en
Inventor
刘曼曼
崔同兵
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Bigo Technology Pte Ltd
Original Assignee
Bigo Technology Pte Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Bigo Technology Pte Ltd filed Critical Bigo Technology Pte Ltd
Priority to CN202210100824.0A priority Critical patent/CN114584834B/en
Publication of CN114584834A publication Critical patent/CN114584834A/en
Application granted granted Critical
Publication of CN114584834B publication Critical patent/CN114584834B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440245Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124Quantisation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136Incoming video signal characteristics or properties
    • H04N19/137Motion inside a coding unit, e.g. average field, frame or block difference
    • H04N19/139Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/142Detection of scene cut or scene change
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440218Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

The embodiment of the invention discloses a video quality optimization method, a device, equipment and a storage medium, wherein the method comprises the following steps: in the video coding process, determining a first coding parameter of a first image frame and a second coding parameter of a second image frame, wherein the first image frame comprises a key frame in a video image, and the second image frame comprises a prediction frame in the video image; determining a scene mode of the first image frame, the scene mode including a static scene mode and a dynamic scene mode; and if the scene mode is a static scene mode, and the first coding parameter is determined to meet the adjustment condition according to the first coding parameter and the second coding parameter, the first coding parameter is adjusted so as to carry out image quality compensation on the first image frame. According to the scheme, the stability of video image quality is obviously improved, meanwhile, excessive compensation of the image quality is avoided, and the random access response speed in the live broadcast process can be met.

Description

Video quality optimization method, device, equipment and storage medium
Technical Field
The embodiment of the application relates to the technical field of video processing, in particular to a video quality optimization method, a device, equipment and a storage medium.
Background
In recent years, with the popularization of intelligent terminals and mobile internet, the time length of people on the mobile internet per day is greatly increased, and the video applications, especially the live broadcast applications, have the largest increase, such as game live broadcast, education live broadcast, e-commerce live broadcast, news live broadcast and the like, have obvious increase of application services. However, due to limitations of hardware capabilities such as self power consumption, mobile bandwidth stability and the like of the intelligent terminal, challenges of live broadcasting of terminal equipment are larger than those of traditional equipment or computer live broadcasting. Because the processing capacity of the intelligent terminal is limited, the video coding compression efficiency is limited, and meanwhile, for random access response is timely, the frame interval of key frame images of video pictures in general live broadcasting is set to be about 2 seconds, and the compression rate which is not high is further reduced due to frequent insertion of key frames, so that the live broadcasting picture quality is difficult to ensure under the extreme condition of limited bandwidth, and for static scenes with more stable background, the breathing effect of unstable picture quality is easy to occur, and the breathing effect is usually represented as negligence and negligence of the picture quality periodicity.
In the prior art, in order to solve the problem of unstable video image quality, time domain filtering denoising is generally adopted before encoding to enhance the correlation between the front and rear adjacent frame images, or an intra-frame refreshing technology is used to achieve the purpose of reducing respiratory effect, but in a mobile live broadcast scene of an intelligent terminal, the time domain filtering before encoding consumes additional calculation amount, and the power consumption of the intelligent terminal can be obviously increased; the intra-frame refresh technique divides the whole IDR (Instantaneous Decoding Refresh ) frame into intra-frame encoded stripes in a plurality of forward predicted frames, which does not really solve the problem of flash in refresh zone, and in addition, the reduction of IDR frames also affects the random access response speed in live broadcast, so that the method adopting intra-frame refresh cannot efficiently solve the problem of unstable video image quality in the live broadcast process.
Disclosure of Invention
The embodiment of the invention provides a video quality optimization method, a device, equipment and a storage medium, which solve the problem of unstable video picture quality in the prior art, especially in the live broadcast process of an intelligent terminal, remarkably improve the stability of video picture quality, avoid excessive compensation of picture quality and meet the random access response speed in the live broadcast process.
In a first aspect, an embodiment of the present invention provides a video quality optimization method, including:
in the video coding process, determining a first coding parameter of a first image frame and a second coding parameter of a second image frame, wherein the first image frame comprises a key frame in a video image, the second image frame comprises a predicted frame in the video image, and the first image frame and the second image frame are adjacent image frames;
determining a scene mode of the first image frame, the scene mode including a static scene mode and a dynamic scene mode;
and if the scene mode is a static scene mode, and the first coding parameter is determined to meet the adjustment condition according to the first coding parameter and the second coding parameter, the first coding parameter is adjusted so as to carry out image quality compensation on the first image frame.
In a second aspect, an embodiment of the present invention further provides a video quality optimization apparatus, including:
the coding parameter determining module is used for determining a first coding parameter of a first image frame and a second coding parameter of a second image frame in the video coding process, wherein the first image frame comprises a key frame in a video image, the second image frame comprises a prediction frame in the video image, and the first image frame and the second image frame are adjacent image frames;
a scene mode determining module, configured to determine a scene mode of the first image frame, where the scene mode includes a static scene mode and a dynamic scene mode;
and the coding parameter adjustment module is used for adjusting the first coding parameter to carry out image quality compensation on the first image frame if the scene mode is a static scene mode and the first coding parameter meets the adjustment condition according to the first coding parameter and the second coding parameter.
In a third aspect, an embodiment of the present invention further provides a video quality optimization apparatus, including:
one or more processors;
storage means for storing one or more programs,
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the video quality optimization method described in the embodiments of the present invention.
In a fourth aspect, embodiments of the present invention also provide a storage medium storing computer-executable instructions that, when executed by a computer processor, are configured to perform the video quality optimization method of embodiments of the present invention.
In the embodiment provided by the invention, the first coding parameter of the first image frame and the second coding parameter of the second image frame are determined in the video coding process, wherein the first image frame comprises a key frame in a video image, the second image frame comprises a predicted frame in the video image, the first image frame and the second image frame are adjacent image frames, the scene mode of the first image frame is determined, and if the scene mode is a static scene mode, and the first coding parameter is determined according to the first coding parameter and the second coding parameter to meet the adjustment condition, the first coding parameter is adjusted to perform image quality compensation on the first image frame.
Drawings
Fig. 1 is a flowchart of a video quality optimization method according to an embodiment of the present invention;
FIG. 2 is a method for determining image quantization parameters according to an embodiment of the present invention;
FIG. 3 is a flowchart of a method for determining an image scene mode according to an embodiment of the present invention;
FIG. 4 is a flowchart of a method for determining an average quantization parameter offset according to an embodiment of the present invention;
FIG. 5 is a flowchart of another method for optimizing video quality according to an embodiment of the present invention;
FIG. 6 is a flowchart of another method for optimizing video quality according to an embodiment of the present invention;
fig. 7 is a block diagram of a video quality optimization device according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of a video quality optimization device according to an embodiment of the present invention.
Detailed Description
Embodiments of the present invention will be described in further detail below with reference to the drawings and examples. It should be understood that the particular embodiments described herein are illustrative only and are not limiting of embodiments of the invention. It should be further noted that, for convenience of description, only some, but not all of the structures related to the embodiments of the present invention are shown in the drawings.
The terms first, second and the like in the description and in the claims, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged, as appropriate, such that embodiments of the present application may be implemented in sequences other than those illustrated or described herein, and that the objects identified by "first," "second," etc. are generally of a type and not limited to the number of objects, e.g., the first object may be one or more. Furthermore, in the description and claims, "and/or" means at least one of the connected objects, and the character "/", generally means that the associated object is an "or" relationship.
Fig. 1 is a flowchart of a video quality optimization method provided by an embodiment of the present invention, for improving stability of video image quality, where the method may be executed by a computing device, such as a smart phone, a tablet computer, etc., and specifically includes the following steps:
step S101, in the video encoding process, determining a first encoding parameter of a first image frame and a second encoding parameter of a second image frame.
Wherein video coding refers to the process of coding and compressing images in the original video. Due to the persistence of vision effect of the human eye, when a sequence of video frames is played at a certain rate, a video is presented that is continuous in motion. Because of the high similarity between successive frames, to facilitate storage and transmission, the original video image is typically encoded and compressed to remove redundancy in the spatial and temporal dimensions.
In one embodiment, a first encoding parameter for a first image frame and a second encoding parameter for a second image frame are determined during video encoding. The first coding parameter characterizes the coding compression condition of the first image frame, namely characterizes the image quality of the first image frame, and the distortion condition is relatively obvious when the compression rate is higher, so that the image quality is lower; the lower the compression ratio, the higher its corresponding image quality. Accordingly, the second encoding parameter characterizes the encoding compression of the second image frame.
In one embodiment, the first image frame comprises a key frame (I frame) in the video image, the second image frame comprises a predicted frame (e.g., a forward predicted frame P frame) in the video image, and the video encoding process defines each frame image as a different frame type according to an encoding rule, such as an I frame, a B frame, and a P frame. Wherein the first image frame and the second image frame are adjacent image frames. The video image is divided into a plurality of continuous frame images during encoding, and the first image frame and the second image frame are two adjacent frames in the divided frame images. Optionally, the second frame image is a previous frame image to the first frame image.
In one embodiment, during the video encoding process, the above-mentioned process of determining the first encoding parameter and the second encoding parameter corresponding to the first image frame and the second image frame is triggered when the first image frame is encoded. Wherein different video compression algorithms correspond to different processes of determining the first encoding parameter and the second encoding parameter. Taking the widely used h.264 video compression algorithm as an example, the first encoding parameter is a first quantization parameter, and the second encoding parameter is a second quantization parameter. In the h.264 video compression process, the quantization parameter QP is used as the sequence number of the quantization step Qstep, and the image detail compression condition is represented by different values. For luma coding, the quantization step size Qstep has 52 values in total, the corresponding quantization parameter QP has values of 0 to 51, and for chroma coding, the QP has values of 0 to 39. The smaller the quantization parameter value, the finer the quantization, the higher the image quality, and the longer the resulting code stream, while most of the details of the image are preserved. Conversely, if the QP increases, details of the picture begin to be lost, the code rate decreases, the degree of picture distortion increases, and the picture quality decreases. Optionally, taking the first encoding parameter as a first quantization parameter and the second encoding parameter as a second quantization parameter as an example, a process of determining the first encoding parameter of the first image frame and the second encoding parameter of the second image frame is shown in fig. 2, and fig. 2 is a method for determining an image quantization parameter according to an embodiment of the present invention, which specifically includes:
step S1011, determining an average quantization parameter offset of the second image frame.
Wherein the quantization parameter offset may be generated by an encoder-integrated image detection unit. In one embodiment, the image detection unit may generate a corresponding quantization parameter offset for a plurality of macroblocks partitioned based on the image, and the scheme may directly acquire the quantization parameter offset, and determine an average quantization parameter offset of the second image frame based on the quantization parameter offset. Alternatively, the average quantization parameter offset of the second image frame may be obtained by obtaining a corresponding average value of the macroblock quantization parameter offsets of the respective macroblocks of the second image frame.
Step S1012, the initial quantization parameters of the first image frame and the second image frame are adjusted by the average quantization parameter offset to obtain the corresponding first quantization parameter and second quantization parameter.
When determining the first quantization parameter of the first image frame and the second quantization parameter of the second image frame, the average quantization parameter offset calculated in the step S1011 adjusts the initial quantization parameters of the first image frame and the second image frame to obtain the first quantization parameter and the second quantization parameter, respectively. The initial quantization parameter of the first image frame and the initial quantization parameter of the second image frame are original quantization parameters generated when the encoder encodes, which can be understood that the first quantization parameter and the second quantization parameter in the scheme are quantization parameters obtained by updating the quantization parameters of the original image frame through the calculated average quantization parameter offset, that is, the first quantization parameter and the second quantization parameter.
Specifically, the calculation formula of the first quantization parameter may be:
qp_pred_I=qp_I-qp_offset_avg
wherein qp_pred_i is the first quantization parameter obtained after adjustment, qp_i is the initial quantization parameter corresponding to the first image, and qp_offset_avg is the determined average quantization parameter offset.
The calculation formula of the second quantization parameter may be:
qp_pred_P=qp_P-qp_offset_avg
wherein qp_pred_p is the second quantization parameter obtained after adjustment, and qp_p is the initial quantization parameter corresponding to the second image.
Step S102, determining a scene mode of the first image frame, where the scene mode includes a static scene mode and a dynamic scene mode.
Wherein different image frames may correspond to different scene modes. In one embodiment, the scene modes are divided into a static scene mode and a dynamic scene mode. For the first image frame, when determining whether to adjust the first image frame, a corresponding scene mode is first determined. In the determination of the scene mode, the scene mode of the first image frame may be determined by determining an average motion vector of the second image frame. Wherein, as mentioned above, the second image frame is a predicted frame adjacent to the first image frame, and optionally, a frame before the first image frame. Of course, if a plurality of predicted frames are included before the first image frame, the average value may be obtained after determining the average motion vector of each of the plurality of predicted frames adjacent to the first image frame, so that the scene mode of the first image frame may be determined based on the average value. Optionally, fig. 3 is a flowchart of a method for determining an image scene mode according to an embodiment of the present invention, as shown in fig. 3, specifically including:
step S1021, obtaining the macro block motion vector of each macro block of the second image frame, and calculating the average motion vector of the second image frame according to the macro block motion vector of each macro block.
In the image coding process, an image is generally divided into a plurality of macro blocks, and a motion vector of each macro block is calculated separately, wherein the motion vector represents the relative displacement between a current coding block and a best matching block in a reference image of the current coding block. The macroblock motion vector of each macroblock can be obtained from the data output from the macroblock coding analysis unit in the existing encoder, and the absolute values of the macroblock motion vectors of each macroblock are summed and averaged to obtain an average motion vector.
Optionally, in an embodiment, when determining the average motion vector of the second image frame, selecting a macroblock corresponding to the image key area, that is, an image key area macroblock, and calculating the average motion vector of the second image frame by using the macroblock motion vector corresponding to the image key area macroblock. Specifically, when the macro block motion vector of each macro block output by the macro block coding analysis unit is obtained, whether the macro block is in the region of the ROI (Region of interest ) is judged, if so, the motion vectors of the macro block are accumulated until the motion vector of the last macro block of the second frame image frame is completely determined, and the average motion vector is obtained by calculation. When determining whether the macro block is in the ROI area, determining whether the macro block is the macro block corresponding to the element smaller than 0 in the matrix by judging that the macro block is in each macro block matrix output by the encoder, if so, judging that the macro block is in the ROI area, otherwise, is outside the ROI area, and not participating in the operation of the average motion vector. The specific calculation formula is exemplified by:
wherein mv_avg is the calculated average motion vector, n is the number of macro blocks in the ROI area, the motion vector of each macro block is characterized by components in x and y directions, mv_mb_roi.x represents the motion vector of the x component in the macro block, mv_mb_roi.y represents the motion vector of the y component in the macro block, abs () is the absolute value.
Step S1022, comparing the average motion vector with a preset scene determination threshold to determine a scene mode of the first image frame.
In one embodiment, the scene mode of the first image frame is determined by comparing the resulting average motion vector to a scene decision threshold by setting the scene decision threshold. Alternatively, the scene determination threshold may be set to 150. Specific decision rules refer to the following formula:
wherein mv_threshold represents a scene determination threshold, mv_avg represents a determination result of a static scene mode, and mv_avg represents an average motion vector is_static of the first image determined in the foregoing step. As can be seen from the above formula, when the average motion vector is smaller than the scene determination threshold, the current scene motion degree of the first image frame is smaller, and the current scene motion degree is determined to be a static scene mode, otherwise, the current scene motion degree is determined to be a non-static scene mode.
Step S103, if the scene mode is a static scene mode, and it is determined that the first coding parameter meets an adjustment condition according to the first coding parameter and the second coding parameter, the first coding parameter is adjusted to perform image quality compensation on the first image frame.
In one embodiment, image quality compensation for the first image frame is accomplished by adjusting a first encoding parameter of the first image frame. Before the first image quality compensation, it is determined whether the determined first image is in a static scene mode, if so, when the first coding parameter is determined based on the first coding parameter and the second coding parameter determined in step S101 and the first coding parameter meets the adjustment condition, the first coding parameter is adjusted to compensate the picture quality of the first image frame. Illustratively, taking the h.264 video compression algorithm as an example, the first coding parameter is a first quantization parameter, and the picture quality of the first image frame is compensated by adjusting the first quantization parameter. As described above, the smaller the first quantization parameter, the more image details are retained, and the lower the image distortion degree, the parameter value of the first quantization parameter is correspondingly reduced, so as to perform image quality compensation on the first image frame.
It should be noted that, the above adjustment method is described by taking an h.264 video compression algorithm as an example, in other video compression algorithms, if the higher the quantization parameter is, the more image details remain, and the lower the image distortion degree is, the parameter value of the quantization parameter is correspondingly increased, so as to compensate the image quality of the image frame.
According to the scheme, in the video quality optimization process, when the key frame is judged to be in a static scene mode and the coding parameters between the key frame and the adjacent predicted frames meet certain conditions, the image quality compensation is performed, not every key frame image is compensated, the key frames needing compensation can be more effectively identified, the excessive compensation of the key frames is avoided, and meanwhile, the phenomenon of unbalanced image quality when the key frame images and the adjacent predicted frame images are played is well restrained. Furthermore, when judging the scene mode, the motion vector of the macro block in the key region of the image is calculated, so that the scene mode is determined more accurately and reasonably.
Fig. 4 is a flowchart of a method for determining an average quantization parameter offset according to an embodiment of the present invention, as shown in fig. 4, specifically:
step S201, in the video coding process, obtaining macro block quantization parameter offsets of all macro blocks of the second image frame, determining macro blocks of the image key area in all macro blocks, and calculating according to the macro block quantization parameter offsets corresponding to the macro blocks of the image key area to obtain average quantization parameter offsets of the second image frame.
In one embodiment, when the average quantization parameter offset calculation of the second image is performed, the average quantization parameter offset is calculated by using only the macroblock quantization parameter offset of the macroblock corresponding to the image key region (ROI region), that is, the macroblock of the key region. Specifically, the calculation formula may be:
wherein qp_offset_avg is the calculated average quantization parameter offset, n is the number of macroblocks in the ROI area, qp_offset_mb_roi is the element less than 0 in the matrix qp_offset_mb, and abs () is the absolute value. In this embodiment, the number of macro blocks in the ROI area is the number of elements smaller than 0 in the matrix qp_offset_mb, and the macro blocks in the ROI area are characterized by a value smaller than 0 in the matrix qp_offset_mb, otherwise, the macro blocks outside the ROI area do not participate in the calculation of the average quantization parameter offset.
The number of rows of the matrix qp_offset_mb is denoted as mb_row, the number of columns is denoted as mb_col, specific row values and column values are calculated according to the image size and the size of the divided macro block, taking the macro block size as 16×16 as an example, the value of mb_row is (height+15) > >4, and the value of mb_col is (width+15) > >4, where height is the height of the image and width of the image.
Step S202, initial quantization parameters of the first image frame and the second image frame are adjusted through average quantization parameter offset to obtain corresponding first quantization parameters and second quantization parameters.
Step S203, determining a scene mode of the first image frame, where the scene mode includes a static scene mode and a dynamic scene mode.
Step S204, if the scene mode is a static scene mode, and the first quantization parameter is determined to meet the adjustment condition according to the first quantization parameter and the second quantization parameter, the first quantization parameter is adjusted to perform image quality compensation on the first image frame.
According to the scheme, when the average quantization parameter offset of the second image frame is determined, the image key region is selected as a calculation object, so that the determined average quantization parameter offset is stronger in pertinence, the calculated value is more reasonable, and the influence of the non-image key region on the optimization of the video quality is filtered.
In another embodiment, the first encoding parameter and the second encoding parameter may be obtained by calculating an average quantization parameter offset of the first image frame or a third image frame subsequent to the first image frame, and adjusting initial quantization parameter offsets of the first image frame and the second image frame based on the average quantization parameter offset. The average quantization parameter offset for the first image frame and the average quantization parameter offset for the third image frame are identical to the average quantization parameter offset for the second image frame, which is not described herein.
Fig. 5 is a flowchart of another video quality optimization method provided by an embodiment of the present invention, which provides specific conditions that the first coding parameter and the second coding parameter meet, so as to further implement image quality compensation, as shown in fig. 5, and specifically includes:
step S301, in the video encoding process, determining a first encoding parameter of a first image frame and a second encoding parameter of a second image frame.
Step S302, determining a scene mode of the first image frame, where the scene mode includes a static scene mode and a dynamic scene mode.
Step S303, if the scene mode is a static scene mode and it is determined that the difference between the first quantization parameter and the second quantization parameter is greater than the preset quantization parameter threshold, the first encoding parameter is adjusted to perform image quality compensation on the first image frame.
In one embodiment, a difference between the first quantization parameter and the second quantization parameter is determined, and if the difference is greater than a preset quantization parameter threshold, the difference is large, taking an h.264 video compression algorithm as an example, the preset quantization parameter threshold may be set to 1, that is, if the difference is greater than 1, the image quality compensation is performed on the first image frame.
In a mobile live broadcast scene, a phenomenon of negligence and damage of pictures is easy to occur by frequently inserting a key frame, particularly, in order to avoid play jamming in live broadcast, the size of the key frame is usually limited, and the compression efficiency of the key frame is lower than that of a forward predicted frame and a bidirectional predicted frame, so that when the code rate of the key frame is insufficient, the picture quality of the key frame is lower than that of the predicted frame before the key frame, and a breathing effect of blurring of a periodical key frame is easy to occur in a relatively stable live broadcast environment, and when the condition of key frame quality compensation is determined, the key frame quality compensation is performed to avoid the occurrence of the breathing effect when the quantization parameter difference between the current key frame and the previous predicted frame is determined to be too large.
Fig. 6 is a flowchart of another video quality optimization method according to an embodiment of the present invention, which provides a specific method for adjusting coding parameters, as shown in fig. 6, and specifically includes:
in the video encoding process, a first encoding parameter of a first image frame and a second encoding parameter of a second image frame are determined in step S401.
Step S402, determining a scene mode of the first image frame, where the scene mode includes a static scene mode and a dynamic scene mode.
Step S403, if the scene mode is a static scene mode, and the first coding parameter meets the adjustment condition according to the first coding parameter and the second coding parameter, obtaining the quantization parameter proportion value of the first image frame and the second image frame, and adjusting the first coding parameter according to the quantization parameter proportion value.
In one embodiment, taking an h.264 video compression algorithm as an example, the first encoding parameter is a first quantization parameter, the second encoding parameter is a second quantization parameter, the first quantization parameter and the second quantization parameter are respectively adjusted based on initial quantization parameters of the first image frame and the second image frame, and a quantization parameter ratio value is denoted as an ipratio, where the ipratio represents a ratio value of the initial quantization parameters of the first image frame and the second image frame, and the first encoding parameter, that is, the first quantization parameter, is adjusted based on the ratio value. The specific adjustment formula may be: qp_i/(ipratio+0.4).
According to the scheme, when the picture quality is compensated, the quantization parameter of the image to be compensated is adjusted based on the quantization parameter proportion value of the two frames of images, so that the image quality between the compensated image and the adjacent image is kept consistent dynamically, the respiratory effect during video playing is reduced, and the picture quality compensation mode is more reasonable and is not simply and widely used for improving the picture quality.
Fig. 7 is a block diagram of a video quality optimization device according to an embodiment of the present invention, where the device is configured to execute the video quality optimization method according to the foregoing embodiment, and the device has functional modules and beneficial effects corresponding to the execution method. As shown in fig. 7, the apparatus specifically includes: an encoding parameter determination module 101, a scene mode determination module 102, and an encoding parameter adjustment module 103, wherein,
a coding parameter determining module 101, configured to determine, during video coding, a first coding parameter of a first image frame and a second coding parameter of a second image frame, where the first image frame includes a key frame in a video image, the second image frame includes a predicted frame in the video image, and the first image frame and the second image frame are adjacent image frames;
a scene mode determining module 102, configured to determine a scene mode of the first image frame, where the scene mode includes a static scene mode and a dynamic scene mode;
and the encoding parameter adjustment module 103 is configured to adjust the first encoding parameter to perform image quality compensation on the first image frame if the scene mode is a static scene mode and the first encoding parameter meets an adjustment condition according to the first encoding parameter and the second encoding parameter.
According to the scheme, when the first coding parameter of the first image frame and the second coding parameter of the second image frame are determined in the video coding process, wherein the first image frame comprises a key frame in a video image, the second image frame comprises a predicted frame in the video image, the first image frame and the second image frame are adjacent image frames, a scene mode of the first image frame is determined, and if the scene mode is a static scene mode, and the first coding parameter is determined according to the first coding parameter and the second coding parameter to meet the adjustment condition, the first coding parameter is adjusted so as to perform image quality compensation on the first image frame, the scheme can obviously improve the stability of video image quality, avoid overcompensation of image quality and meet the random access response speed in the live broadcast process.
In one possible embodiment, the first encoding parameter includes a first quantization parameter, the second encoding parameter includes a second quantization parameter, and the determining the first encoding parameter of the first image frame and the second encoding parameter of the second image frame includes:
determining an average quantization parameter offset for the second image frame;
and adjusting the initial quantization parameters of the first image frame and the second image frame through the average quantization parameter offset to obtain corresponding first quantization parameters and second quantization parameters.
In one possible embodiment, the determining the average quantization parameter offset of the second image frame includes:
and acquiring macro block quantization parameter offsets of all macro blocks of the second image frame, and calculating average quantization parameter offsets of the second image frame according to the macro block quantization parameter offsets of all macro blocks.
In one possible embodiment, the calculating the average quantization parameter offset of the second image frame according to the macroblock quantization parameter offset of each macroblock includes:
determining a macro block of a key area of the image in each macro block;
and calculating the average quantization parameter offset of the second image frame according to the macro block quantization parameter offset corresponding to the macro block of the image key area.
In one possible embodiment, the determining the scene mode of the first image frame, the scene mode including a static scene mode and a dynamic scene mode includes:
and determining an average motion vector of the second image frame, and determining a scene mode of the first image frame according to the average motion vector.
In one possible embodiment, the determining the average motion vector of the second image frame, and determining the scene mode of the first image frame according to the average motion vector, includes:
obtaining macro block motion vectors of all macro blocks of the second image frame, and calculating to obtain an average motion vector of the second image frame according to the macro block motion vectors of all macro blocks;
and comparing the average motion vector with a preset scene judgment threshold value to determine a scene mode of the first image frame.
In one possible embodiment, the calculating the average motion vector of the second image frame according to the macroblock motion vectors of the respective macroblocks includes:
determining a macro block of a key area of the image in each macro block;
and calculating an average motion vector of the second image frame according to the macro block motion vector corresponding to the macro block of the image key area.
In one possible embodiment, the first coding parameter includes a first quantization parameter, the second coding parameter includes a second quantization parameter, and before determining that the first coding parameter satisfies an adjustment condition according to the first coding parameter and the second coding parameter, the method further includes:
determining a difference between the first quantization parameter and the second quantization parameter;
and if the quantization parameter difference is larger than a preset quantization parameter threshold value, determining that the first quantization parameter meets an adjustment condition.
In a possible embodiment, said adjusting said first coding parameter comprises:
acquiring quantization parameter proportion values of the first image frame and the second image frame;
and adjusting the first coding parameter according to the quantization parameter proportion value.
Fig. 8 is a schematic structural diagram of a video quality optimization apparatus according to an embodiment of the present invention, where, as shown in fig. 8, the apparatus includes a processor 201, a memory 202, an input device 203, and an output device 204; the number of processors 201 in the device may be one or more, one processor 201 being taken as an example in fig. 8; the processor 201, memory 202, input devices 203, and output devices 204 in the apparatus may be connected by a bus or other means, for example in fig. 8. The memory 202 is a computer readable storage medium, and may be used to store software programs, computer executable programs, and modules, such as program instructions/modules corresponding to the video quality optimization method in the embodiment of the present invention. The processor 201 executes various functional applications of the device and data processing, i.e., implements the video quality optimization method described above, by running software programs, instructions, and modules stored in the memory 202. The input means 203 may be used to receive entered numeric or character information and to generate key signal inputs related to user settings and function control of the device. The output device 204 may include a display device such as a display screen.
The embodiment of the present invention also provides a storage medium containing computer executable instructions, which when executed by a computer processor, are configured to perform a video quality optimization method described in the above embodiment, specifically including:
in the video coding process, determining a first coding parameter of a first image frame and a second coding parameter of a second image frame, wherein the first image frame comprises a key frame in a video image, the second image frame comprises a predicted frame in the video image, and the first image frame and the second image frame are adjacent image frames;
determining a scene mode of the first image frame, the scene mode including a static scene mode and a dynamic scene mode;
and if the scene mode is a static scene mode, and the first coding parameter is determined to meet the adjustment condition according to the first coding parameter and the second coding parameter, the first coding parameter is adjusted so as to carry out image quality compensation on the first image frame.
It should be noted that, in the embodiment of the video quality optimization apparatus, each unit and module included are only divided according to the functional logic, but not limited to the above division, so long as the corresponding functions can be implemented; in addition, the specific names of the functional units are also only for distinguishing from each other, and are not used to limit the protection scope of the embodiments of the present invention.
Note that the above is only a preferred embodiment of the present invention and the technical principle applied. It will be understood by those skilled in the art that the embodiments of the present invention are not limited to the particular embodiments described herein, but are capable of numerous obvious changes, rearrangements and substitutions without departing from the scope of the embodiments of the present invention. Therefore, while the embodiments of the present invention have been described in connection with the above embodiments, the embodiments of the present invention are not limited to the above embodiments, but may include many other equivalent embodiments without departing from the spirit of the embodiments of the present invention, and the scope of the embodiments of the present invention is determined by the scope of the appended claims.

Claims (11)

1. A method for optimizing video quality, comprising:
in the video coding process, determining a first coding parameter of a first image frame and a second coding parameter of a second image frame, wherein the first image frame comprises a key frame in a video image, the second image frame comprises a predicted frame in the video image, and the first image frame and the second image frame are adjacent image frames;
determining a scene mode of the first image frame, the scene mode including a static scene mode and a dynamic scene mode;
if the scene mode is a static scene mode, and the first coding parameter meets the adjustment condition according to the first coding parameter and the second coding parameter, adjusting the first coding parameter so as to carry out image quality compensation on the first image frame;
wherein when the first encoding parameter is a first quantization parameter and the second encoding parameter is a second quantization parameter, determining the first encoding parameter of the first image frame and the second encoding parameter of the second image frame includes: and determining the average quantization parameter offset of the second image frame, and adjusting the initial quantization parameters of the first image frame and the second image frame through the average quantization parameter offset to obtain a first quantization parameter corresponding to the first image frame and a second quantization parameter corresponding to the second image frame.
2. The method of video quality optimization of claim 1, wherein said determining an average quantization parameter offset for the second image frame comprises:
and acquiring macro block quantization parameter offsets of all macro blocks of the second image frame, and calculating average quantization parameter offsets of the second image frame according to the macro block quantization parameter offsets of all macro blocks.
3. The method according to claim 2, wherein said calculating an average quantization parameter offset for the second image frame from the macroblock quantization parameter offsets for the respective macroblocks comprises:
determining a macro block of a key area of the image in each macro block;
and calculating the average quantization parameter offset of the second image frame according to the macro block quantization parameter offset corresponding to the macro block of the image key area.
4. The method of video quality optimization of claim 1, wherein said determining a scene mode of the first image frame comprises:
and determining an average motion vector of the second image frame, and determining a scene mode of the first image frame according to the average motion vector.
5. The method of video quality optimization of claim 4, wherein said determining an average motion vector for the second image frame, and determining a scene mode for the first image frame based on the average motion vector, comprises:
obtaining macro block motion vectors of all macro blocks of the second image frame, and calculating to obtain an average motion vector of the second image frame according to the macro block motion vectors of all macro blocks;
and comparing the average motion vector with a preset scene judgment threshold value to determine a scene mode of the first image frame.
6. The method according to claim 5, wherein said calculating an average motion vector of the second image frame from the macroblock motion vectors of the respective macroblocks, comprises:
determining a macro block of a key area of the image in each macro block;
and calculating an average motion vector of the second image frame according to the macro block motion vector corresponding to the macro block of the image key area.
7. The method according to any one of claims 1 to 6, wherein determining that the first coding parameter satisfies an adjustment condition according to a first coding parameter and the second coding parameter comprises:
determining a difference between the first quantization parameter and the second quantization parameter;
and if the quantization parameter difference is larger than a preset quantization parameter threshold value, determining that the first quantization parameter meets an adjustment condition.
8. The method according to any one of claims 1-6, wherein said adjusting the first coding parameter comprises:
acquiring quantization parameter proportion values of the first image frame and the second image frame;
and adjusting the first coding parameter according to the quantization parameter proportion value.
9. Video quality optimizing apparatus, characterized by comprising:
the coding parameter determining module is used for determining a first coding parameter of a first image frame and a second coding parameter of a second image frame in a video coding process, wherein when the first coding parameter is a first quantization parameter and the second coding parameter is a second quantization parameter, the coding parameter determining module is specifically used for determining average quantization parameter offset of the second image frame, and adjusting initial quantization parameters of the first image frame and the second image frame through the average quantization parameter offset to obtain the first quantization parameter corresponding to the first image frame and the second quantization parameter corresponding to the second image frame, wherein the first image frame comprises a key frame in a video image, the second image frame comprises a predicted frame in the video image, and the first image frame and the second image frame are adjacent image frames;
a scene mode determining module, configured to determine a scene mode of the first image frame, where the scene mode includes a static scene mode and a dynamic scene mode;
and the coding parameter adjustment module is used for adjusting the first coding parameter to carry out image quality compensation on the first image frame if the scene mode is a static scene mode and the first coding parameter meets the adjustment condition according to the first coding parameter and the second coding parameter.
10. A video quality optimization apparatus, the video quality optimization apparatus comprising: one or more processors; storage means for storing one or more programs that, when executed by the one or more processors, cause the one or more processors to implement the video quality optimization method of any of claims 1-8.
11. A storage medium storing computer executable instructions which, when executed by a computer processor, are for performing the video quality optimization method of any one of claims 1-8.
CN202210100824.0A 2022-01-27 2022-01-27 Video quality optimization method, device, equipment and storage medium Active CN114584834B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210100824.0A CN114584834B (en) 2022-01-27 2022-01-27 Video quality optimization method, device, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210100824.0A CN114584834B (en) 2022-01-27 2022-01-27 Video quality optimization method, device, equipment and storage medium

Publications (2)

Publication Number Publication Date
CN114584834A CN114584834A (en) 2022-06-03
CN114584834B true CN114584834B (en) 2024-02-13

Family

ID=81771263

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210100824.0A Active CN114584834B (en) 2022-01-27 2022-01-27 Video quality optimization method, device, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN114584834B (en)

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103179394A (en) * 2013-01-21 2013-06-26 北京航空航天大学 An I-frame Bit Rate Control Method Based on Regional Video Quality Stability
CN103731673A (en) * 2012-10-10 2014-04-16 浙江大华技术股份有限公司 Method and device for encoding video
CN109068137A (en) * 2013-07-10 2018-12-21 微软技术许可有限责任公司 The Video coding of interest region perception
CN110365983A (en) * 2019-09-02 2019-10-22 珠海亿智电子科技有限公司 A kind of macro-block level bit rate control method and device based on human visual system
CN110996102A (en) * 2020-03-03 2020-04-10 眸芯科技(上海)有限公司 Video coding method and device for inhibiting intra-frame block respiration effect in P/B frame
CN111164980A (en) * 2017-11-30 2020-05-15 深圳市大疆创新科技有限公司 System and method for controlling video encoding within image frames
CN111314700A (en) * 2020-03-03 2020-06-19 眸芯科技(上海)有限公司 Video coding method and device for inhibiting respiratory effect
CN111866504A (en) * 2020-07-17 2020-10-30 Oppo广东移动通信有限公司 An encoding method, encoder and computer-readable storage medium
WO2020243906A1 (en) * 2019-06-04 2020-12-10 SZ DJI Technology Co., Ltd. Method, device, and storage medium for encoding video data base on regions of interests

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8582645B2 (en) * 2009-06-11 2013-11-12 Texas Instruments Incorporated Reducing flicker in the display of video streams
US9591302B2 (en) * 2012-07-02 2017-03-07 Microsoft Technology Licensing, Llc Use of chroma quantization parameter offsets in deblocking
US9661329B2 (en) * 2014-04-30 2017-05-23 Intel Corporation Constant quality video coding
CN110049321B (en) * 2018-01-16 2022-09-06 腾讯科技(深圳)有限公司 Video coding method, device, equipment and storage medium

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103731673A (en) * 2012-10-10 2014-04-16 浙江大华技术股份有限公司 Method and device for encoding video
CN103179394A (en) * 2013-01-21 2013-06-26 北京航空航天大学 An I-frame Bit Rate Control Method Based on Regional Video Quality Stability
CN109068137A (en) * 2013-07-10 2018-12-21 微软技术许可有限责任公司 The Video coding of interest region perception
CN111164980A (en) * 2017-11-30 2020-05-15 深圳市大疆创新科技有限公司 System and method for controlling video encoding within image frames
WO2020243906A1 (en) * 2019-06-04 2020-12-10 SZ DJI Technology Co., Ltd. Method, device, and storage medium for encoding video data base on regions of interests
CN110365983A (en) * 2019-09-02 2019-10-22 珠海亿智电子科技有限公司 A kind of macro-block level bit rate control method and device based on human visual system
CN110996102A (en) * 2020-03-03 2020-04-10 眸芯科技(上海)有限公司 Video coding method and device for inhibiting intra-frame block respiration effect in P/B frame
CN111314700A (en) * 2020-03-03 2020-06-19 眸芯科技(上海)有限公司 Video coding method and device for inhibiting respiratory effect
CN111866504A (en) * 2020-07-17 2020-10-30 Oppo广东移动通信有限公司 An encoding method, encoder and computer-readable storage medium

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
Inter-frame Correlation Based Quantization Parameter Offset Optimization for Screen Content Video Coding;Juanting Fan等;《2015 IEEE International Conference on Multimedia Big Data》;全文 *
X264的平均比特率控制算法优化;田一姝等;《计算机应用》(第3期);全文 *
一种H.264帧层码率控制的改进算法;段厚勇等;《计算机应用》;第29卷(第4期);全文 *

Also Published As

Publication number Publication date
CN114584834A (en) 2022-06-03

Similar Documents

Publication Publication Date Title
WO2021244341A1 (en) Picture coding method and apparatus, electronic device and computer readable storage medium
US12125171B2 (en) Video denoising method and apparatus, and storage medium
US11902567B2 (en) Method, system and apparatus for intra-refresh in video signal processing
US8571106B2 (en) Digital video compression acceleration based on motion vectors produced by cameras
US8139883B2 (en) System and method for image and video encoding artifacts reduction and quality improvement
CN105472205B (en) Real-time video noise reduction method and device in encoding process
CN112672149B (en) Video processing method and device, storage medium and server
US20120002723A1 (en) Stereo video coding apparatus and stereo video coding method
CN114466189B (en) Code rate control method, electronic equipment and storage medium
US20200120342A1 (en) Motion adaptive encoding of video
Paul et al. Video coding with dynamic background
CN117176955A (en) Video encoding method, video decoding method, computer device, and storage medium
CN112752106B (en) Video image frame coding control method and device and computer storage medium
US10856002B2 (en) Method, device and system for method of encoding a sequence of frames in a video stream
CN114584834B (en) Video quality optimization method, device, equipment and storage medium
CN111385571A (en) Method and device for controlling code rate of ultra-long image group
US9635359B2 (en) Method and apparatus for determining deblocking filter intensity
US20080080618A1 (en) Video decoding apparatus and method of the same
JP2007228519A (en) Image encoding device and image encoding method
US20200045324A1 (en) Image encoding method and encoder
US10051270B2 (en) Video encoding method using at least two encoding methods, device and computer program
US9542611B1 (en) Logo detection for macroblock-based video processing
JP3800435B2 (en) Video signal processing device
CN115834906A (en) Video encoding and decoding method and device, electronic equipment and medium
CN114449281A (en) Intelligent code rate control optimization method based on scene change

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant