CN117459793B - Video noise optimization processing method - Google Patents
Video noise optimization processing method Download PDFInfo
- Publication number
- CN117459793B CN117459793B CN202311787634.1A CN202311787634A CN117459793B CN 117459793 B CN117459793 B CN 117459793B CN 202311787634 A CN202311787634 A CN 202311787634A CN 117459793 B CN117459793 B CN 117459793B
- Authority
- CN
- China
- Prior art keywords
- coefficient
- pixel
- pixel point
- edge
- pixel points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440227—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by decomposing into layers, e.g. base layer and one or more enhancement layers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440254—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering signal-to-noise parameters, e.g. requantization
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440263—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the spatial resolution, e.g. for displaying on a connected PDA
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440281—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the temporal resolution, e.g. by frame skipping
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/14—Picture signal circuitry for video frequency region
- H04N5/20—Circuitry for controlling amplitude response
- H04N5/202—Gamma control
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Processing (AREA)
Abstract
The invention relates to the technical field of video noise reduction optimization, in particular to a video noise optimization processing method, which comprises the steps of performing surface conversion on all frame images in a video to obtain ST coefficients of pixel points of each frame image and sub-bands of each decomposition layer in each direction; obtaining the noise disturbance suspected degree of each pixel point of each frame image according to the distribution condition of each edge of each frame image and the distribution difference of the gray scale of each pixel point on the time sequence; obtaining ST coefficient aggregation indexes of the pixel points according to ST coefficient distribution differences of the pixel points in the pixel point space window; and obtaining ST correction threshold coefficients of the pixel points according to ST coefficient aggregation indexes, noise disturbance suspected degrees, ST coefficients and high-frequency coefficients of sub-bands in the direction of each decomposition layer of the pixel points, and performing Surfacelet inverse transformation to complete denoising treatment of video noise. The invention reserves the details and the edge information in the video and effectively filters the noise information in the news simulcast video.
Description
Technical Field
The application relates to the technical field of video noise reduction optimization, in particular to a video noise optimization processing method.
Background
Broadcast video refers to video content transmitted and played through media such as a broadcast television station, a network platform and the like, plays an important role in society, and is an important path for leisure and entertainment of audiences and an important platform for cultural inheritance and communication. However, the video is easily interfered by shooting environment and electronic equipment in the process of acquisition and transmission, so that noise appears in video pictures, and the definition of the video is affected, therefore, the video needs to be subjected to denoising processing, and the quality and the visual experience of the broadcast film and television video are improved.
The surface threshold denoising algorithm is a method for processing multidimensional signals based on surface changes, wherein the surface changes can effectively capture and represent curved surface singular in the multidimensional signals, the method is very suitable for video processing, and the denoising processing is realized by converting a video sequence into a surface domain and thresholding surface coefficients. However, the conventional Surfacelet threshold denoising algorithm does not consider the context information in the video image and the correlation of each coefficient in the Surfacelet, which may cause problems of detail texture and edge blurring and flickering of the video image.
Disclosure of Invention
In order to solve the technical problems, the invention provides a video noise optimization processing method for solving the existing problems.
The invention discloses a video noise optimization processing method, which adopts the following technical scheme:
one embodiment of the present invention provides a video noise optimization processing method, which includes the following steps:
performing surface transformation on all frame images in the video to obtain ST coefficients of pixel points of each frame image and sub-bands of each decomposition layer in each direction;
obtaining each edge of each frame of image by adopting an edge detection algorithm, and obtaining the noise disturbance suspected degree of each pixel point of each frame of image according to the distribution condition of each edge of each frame of image and the distribution difference of the gray scale of each pixel point on the time sequence;
setting a space window for each pixel point of each frame of image, and obtaining the ST coefficient density of the pixel points according to the distribution of ST coefficients of the pixel points in the space window of the pixel points; obtaining ST coefficient dense similarity of the pixel points according to ST coefficient dense difference of the pixel points in the pixel point space window; obtaining ST coefficient aggregation indexes of the pixel points according to the ST coefficient density and the ST coefficient density similarity of the pixel points; obtaining ST correction coefficients of the pixel points according to ST coefficient aggregation indexes, noise disturbance suspected degrees and ST coefficients of the pixel points;
high frequency coefficient based on each decomposition level direction sub-bandThe threshold value is used for obtaining ST correction coefficient threshold values of all decomposition layers; and obtaining ST correction threshold coefficients of each pixel point of each decomposition layer according to the ST correction coefficient threshold values of each decomposition layer, the ST coefficients of each pixel point and the ST correction coefficients, and carrying out Surfacelet inverse transformation on the ST correction threshold coefficients subjected to threshold processing to obtain denoised videos.
Preferably, the obtaining the noise disturbance suspected level of each pixel point of each frame image according to the distribution condition of each edge of each frame image and the distribution difference of the gray scale of each pixel point on the time sequence between each frame image includes:
obtaining the edge dynamic matching degree of each edge of each frame image according to the similarity degree between the edge contour histograms of each edge of each frame image, taking the edge dynamic matching degree of each pixel point of each frame image as the edge dynamic matching degree of each pixel point, and setting the edge dynamic matching degree of each pixel point which is not on the edge in each frame image as 1;
constructing gray time distribution straight lines and gray time change sequences of all pixel points according to gray value distribution of all pixel points at the same positions in all frame images;
for each pixel point of each frame of image, obtaining a gray time variation difference coefficient of the pixel point according to the difference between gray time variation sequences of the pixel points and the gray time distribution straight line;
taking Euclidean distance between the pixel point and the gray time distribution straight line as gray time deviation degree of the pixel point;
and taking the ratio of the product of the gray time variation difference coefficient of the pixel point and the gray time deviation degree and the edge dynamic matching degree of the pixel point as the noise disturbance suspected degree of the pixel point.
Preferably, the obtaining the edge dynamic matching degree of each edge of each frame image according to the similarity degree between the edge contour histograms of each edge of each frame image includes:
and (3) for the edge contour histogram of each edge of each frame image, carrying out the pasteurization distance calculation on the edge contour histogram of the edge and the edge contour histograms of all edges of all frame images except the edge, obtaining the minimum pasteurization distance, and taking the reciprocal of the minimum pasteurization distance as the edge dynamic matching degree of the edge of the image where the edge contour histogram is located.
Preferably, the construction of the gray scale time distribution straight line and the gray scale time variation sequence of each pixel point according to the gray scale value distribution of each pixel point at the same position in all frame images includes:
for each pixel point, sequencing the gray values of the pixel points in all frame images according to a time sequence to form a gray time distribution sequence of the pixel points;
performing straight line fitting on the gray time distribution sequence of the pixel points to obtain gray time distribution straight lines of the pixel points; and performing first-order difference processing on the gray level time distribution sequence of the pixel points to obtain a gray level time change sequence of the pixel points.
Preferably, the obtaining the gray time variation difference coefficient of the pixel according to the difference between the gray time variation sequences of the pixel and the gray time distribution line includes:
calculating dtw distance average values of the gray level time change sequences of the pixel points and the gray level time change sequences of all the pixel points except the pixel points;
calculating the slope of a gray time distribution straight line of a pixel point, and obtaining an exponential function by taking the slope as an index and taking a natural constant as a base;
taking the product of the dtw distance mean value and the calculation result of the exponential function as a gray scale time variation difference coefficient of the pixel point.
Preferably, the obtaining the ST coefficient density of the pixel according to the ST coefficient distribution of the pixel in the pixel space window includes:
calculating the square of the difference value of ST coefficients of the pixel points and the residual pixel points for each residual pixel point except the pixel points in the pixel point space window, and calculating the Euclidean distance between the pixel points and the residual pixel points;
and calculating the average value of the inverse of the product of the square difference value and the Euclidean distance of all the rest pixel points in the pixel point space window as the ST coefficient concentration of the pixel points.
Preferably, the obtaining the dense similarity of the ST coefficients of the pixel points according to the difference of the densities of the ST coefficients of the pixel points in the spatial window of the pixel points includes:
and for each residual pixel point except the pixel point in the pixel point space window, calculating the difference square of the ST coefficient density of the pixel point and the residual pixel point, and taking the average value of the inverse of the difference square of all the residual pixel points in the pixel point space window as the ST coefficient density similarity of the pixel point.
Preferably, the obtaining the ST coefficient aggregation index of the pixel according to the ST coefficient density and the ST coefficient dense similarity of the pixel includes:
and taking the product of the ST coefficient density and the ST coefficient density similarity of the pixel points as an ST coefficient aggregation index.
Preferably, the obtaining the ST correction coefficient of the pixel according to the ST coefficient aggregation index, the noise disturbance suspected level and the ST coefficient of the pixel includes:
and calculating the product of the ST coefficient of the pixel point and the ST coefficient aggregation index, and taking the normalized value of the ratio of the product to the noise disturbance suspected degree of the pixel point as the ST correction coefficient of the pixel point.
Preferably, the obtaining the ST correction threshold coefficient of each pixel point of each decomposition layer after the thresholding according to the ST correction coefficient threshold value of each decomposition layer, the ST coefficient of each pixel point of each decomposition layer, and the ST correction coefficient includes:
regarding each pixel point of each decomposition layer, when the absolute value of the ST correction coefficient of the pixel point is larger than or equal to the ST correction coefficient threshold value of the decomposition layer, taking the ST coefficient of the pixel point as the ST correction threshold value coefficient of the pixel point;
when the absolute value of the ST correction coefficient of the pixel point is smaller than the ST correction coefficient threshold value of the decomposition layer, 0 is taken as the ST correction threshold value coefficient of the pixel point.
The invention has at least the following beneficial effects:
aiming at the problems that the background information in a video image and the correlation of each coefficient in the surface are not considered in the traditional surface threshold denoising algorithm, and detail textures, edge blurring and flickering exist when the video is denoised, the invention provides a video noise optimization processing method, which is used for obtaining the edge dynamic matching degree of each edge of each image according to the Pasteur distance between each edge of each image and the corresponding nearby edge of other frame images in a news simulcast video, and is used for representing whether the edge is the edge of a background area or the edge of a unchanged human body area and accurately analyzing the noise condition from a local angle;
further analyzing the association condition between noise and non-noise on the time sequence from the distribution condition of the gray time change sequence of each pixel point, simultaneously combining the local angles to more comprehensively analyze the distribution of noise in the video, considering the context information of the news simulcast video, and improving the retaining capability of non-noise edge information in the news simulcast video;
according to the correlation among ST coefficients of the news simulcast video in the surface area, an ST coefficient aggregation index is constructed, and the analysis is carried out from the two layers of the concentration degree and the similarity of the ST coefficients, so that the details in the news simulcast video can be better reserved;
and combining the distribution characteristics of the ST coefficient generated by noise in the Surfacelet domain to obtain the ST correction coefficient, and denoising the news simulcast video by using an ST threshold denoising algorithm based on the obtained ST correction coefficient, so that the detail and the edge information in the news simulcast video can be better kept, and the noise information in the news simulcast video can be effectively filtered.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions and advantages of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are only some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a video noise optimization processing method provided by the invention;
fig. 2 is a schematic diagram of the index construction of the ST correction coefficient for each pixel.
Detailed Description
In order to further describe the technical means and effects adopted by the present invention to achieve the preset purpose, the following detailed description refers to specific implementation, structure, features and effects of a video noise optimization processing method according to the present invention with reference to the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" means that the embodiments are not necessarily the same. Furthermore, the particular features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
The following specifically describes a specific scheme of the video noise optimization processing method provided by the invention with reference to the accompanying drawings.
The embodiment of the invention provides a video noise optimization processing method.
Specifically, referring to fig. 1, the following method for optimizing video noise is provided, and the method includes the following steps:
step S001, preprocessing the video to obtain ST coefficients of each pixel point of each frame of image and sub-bands of each decomposition layer in each direction.
In this embodiment, taking a news simulcast video without subtitle addition as an example, a continuous news simulcast video is obtainedConverting frame image into gray level image, recording as news simulcast gray level image, and time-sequentially recording +.>The frame news simulcast gray level images are arranged to obtain a news simulcast three-dimensional video sequence +.>Which is provided withMiddle->Three-dimensional video sequence for news simulcast>In (a)Gray value of dot, i.e. +.>The +.f in the frame news simulcast gray level image>Line->Gray values corresponding to the column pixels, wherein、/>、/>，/>、/>The length and width of the news simulcast greyscale image are shown, respectively.
For the obtained news simulcast three-dimensional video sequencePerforming surface transformation to obtain ST coefficients of each pixel point of each frame image and corresponding direction sub-bands of each decomposition layer, and forming ST coefficients of all pixel points of all frame images into ST coefficient three-dimensional sequence->Wherein->For the three-dimensional sequence of ST coefficients->Is->ST coefficients corresponding to points. The number of decomposition layers and the number of direction sub-bands divided by each decomposition layer in the Surfacelet transform are respectively recorded as +.>、/>. Wherein video frame number->Width of video image->And length->Number of decomposition layers->Number of directional subbands per layer +.>Is defined by the practitioner himself, in this embodiment set to +.>、/>、/>、/>The number of direction subbands corresponding to each decomposition level is set to 192, 48, 12, respectively. Conversion to the well known form of SurfaceletThe technology is not described in detail in this embodiment.
Step S002, according to the distribution characteristics of noise data in each frame image in time and space and the correlation between ST coefficients, ST correction coefficients of each pixel point of each frame image are obtained.
Because the content in the news simulcast video generally does not have larger motion deviation, namely the background information in the video is generally fixed, and the position of the host in the video generally does not move, only slight limb motion or facial change occurs, the gray values of the pixels in the normal region in the news simulcast gray image between different frames are approximate, namely the gray values of the pixels in the same position in the news simulcast gray image do not change or change slightly with time.
Based on the above analysis, canny edge detection algorithm was used for allRespectively processing the frame news simulcast gray level images to obtain +.>The Canny edge detection algorithm is a known technique and will not be described in detail. And respectively carrying out connected domain analysis on each binarized image, wherein each connected domain respectively represents one edge in the news simulcast gray scale image. Calculating gradient direction angles of all pixel points on each edge by using Sobel operator to obtain an edge profile histogram corresponding to each edge>Wherein the horizontal axis in the edge contour histogram represents each different value of the gradient direction angle, and the vertical axis represents the frequency of occurrence of each gradient direction angle in the edge corresponding to the edge contour histogram.
Obtaining the first based on the edge contour histogramThe +.f in the news simulcast gray image of the frame>Edge dynamic matching degree of individual edges->For indicating->Maximum similarity of each edge to all edges in the news simulcast grey scale image of its neighboring frame, reflecting +.>Whether the edge is the edge of the background area or the human body area in the news simulcast gray image>The +.f in the news simulcast gray image of the frame>Edge dynamic matching degree of individual edges->The method comprises the following steps:
in the above formula:indicate->The +.f in the frame news simulcast gray level image>The degree of edge dynamic matching of the individual edges,、/>indicate->Frame and->The +.f in the news simulcast gray image of the frame>Suo and->Edge contour histograms corresponding to the edges; />Indicate->Number of edge contour histograms in news simulcast gray-scale image of frame,/-for each frame>Representing a video frame number; />The barking distance function represents the calculation of the barking distance between the two histograms in the bracket, wherein the barking distance is a known technique, and the description of the embodiment is omitted. />The minimum value is represented as a minimum value function; />Represents the tuning constant, wherein +.>The empirical value was taken to be 1.
The smaller the value of (2) is, the>、/>The smaller the difference between the two edge profile histograms, i.e. the more similar the two edges corresponding to the two edge profile histograms, the description +.>The +.f in the news simulcast gray image of the frame>The more likely the edges of the individual edges are caused by changes in the human body, and +.>Description of the->The edges do not change in the image of their neighboring frame, i.e. +.>The edges are suspected to be the edges of the background area or the edges of the unchanged human body area.
Further, acquiring gray time distribution sequences of pixel points at each position in the news simulcast gray imageWherein, pixel point in news simulcast gray level image +.>Corresponding grey time distribution sequence->Refers to having the sameAnd->The gray values of all points of a position are in a sequence of ascending chronological order. For all grey time distribution sequences +.>Respectively performing straight line fitting and first-order difference processing to obtain gray level time distribution straight line and gray level time variation sequence corresponding to each gray level time distribution sequence>The abscissa of the gray time distribution line is time, the ordinate is gray value, and the straight line fitting and the first-order difference are known techniques, which are not described in detail in this embodiment. Get->The +.f in the frame news simulcast gray level image>Line->Noise disturbance plausibility of column pixel points>：
In the above formula:representing +.f in news simulcast gray image>Line->Gray time variation difference coefficient of column pixel points; />、/>Respectively representing +.>Line->Column pixel and +.>Line->A gray time variation sequence corresponding to the column pixel points; />Is->Function, representing the calculation of +.using dtw algorithm>、Dtw distance between the two, wherein stw algorithm is a known technique, and the description of this embodiment is omitted; />、/>Respectively representing the length and the width of the news simulcast gray level image; />Representing +.f in news simulcast gray image>Line->Slope of gray time distribution straight line corresponding to column pixel point, +.>An exponential function based on a natural constant e is represented.
Indicate->The +.f in the frame news simulcast gray level image>Line->Gray scale time deviation of column pixel points;indicate->The +.f in the frame news simulcast gray level image>Line->The Euclidean distance between the column pixel points and the corresponding gray time distribution straight line; />Representing parameter tuning constants for preventing +.>When the value of (2) is 0, different +.>With the sameWherein->The empirical value was taken to be 1.
Indicate->The +.f in the frame news simulcast gray level image>Line->Noise disturbance suspicion of column pixel points,indicate->The +.f in the frame news simulcast gray level image>Line->The dynamic matching degree of the edge where the row pixel point is located, if the pixel point is a non-edge pixel point, the corresponding point is +.>Assigned a value of 1.
The gray values of most pixels in the news simulcast gray image are not changed along with the time under the influence of noise, namely the gray time distribution of the pixels is approximate, and the change of the gray values of the pixels in the local area caused by the change of limbs or faces of a host in the news simulcast video is approximate, namely the gray time distribution corresponding to the pixels is approximate, and the noise is causedThe change in the pixel gray value is random. ThenThe larger, i.e. pixel point in news simulcast grey scale image +.>The larger the difference between the gradation time-varying sequences corresponding to the remaining pixel points is, and +.>The larger the value of (i) is, the pixel point in the news simulcast gray image +.>The larger gray value change appears in the gray time distribution sequence, which shows that the more likely the gray time distribution sequence of the pixel is interfered by noise, the pixel point in the news simulcast gray image is->Three-dimensional video sequence for broadcasting news in a joint way>The more likely the corresponding points in (a) are noise points, i.e. noise disturbance plausibility +.>The greater the value of (2).
The larger the value of (2) is, the +.>The +.f in the frame news simulcast gray level image>Line->The gray value of the column pixel point deviates from the corresponding gray time distribution straight lineThe greater the degree, i.e.)>The larger the value of (2), the more likely the pixel point corresponding to the point is interfered by noise, and the greater the degree of interference by noise is, namely noise disturbance suspected degree is->The greater the value of (2).
The larger the value of (c) is, the more suspected the pixel point corresponding to the point is the pixel point in the body contour or the face contour of the presenter in the news simulcast gray level image, and the more likely the factor causing the abnormal change of the gray level value of the pixel point is caused by the change of the limb or the face of the presenter at the moment, the noise disturbance suspected degree (b) corresponding to the point is>The smaller the value of (2).
Next, in the ST-coefficient three-dimensional sequenceIn which the larger ST coefficients generated by noise are generally distributed in isolation, while the larger ST coefficients generated by image edges are generally distributed in concentration, thus in +.>The +.f in the frame news simulcast gray level image>Line->The column pixel point is set as the center and is provided with a +.>A spatial window of size, wherein->In this embodiment, the experimental value is 5, and the practitioner can set himself to obtain the point +.>ST coefficient aggregation index>The calculation method comprises the following steps:
in the above formula:indicate->The +.f in the frame news simulcast gray level image>Line->ST coefficient density of column pixel points; />Indicate->The +.f in the frame news simulcast gray level image>Line->Column pixel pointsST coefficient; />Indicate->The +.f in the frame news simulcast gray level image>Line->Except +.>Of the remaining points other thanST coefficients corresponding to the respective points; />Indicate->The +.f in the frame news simulcast gray level image>Line->The number of points in the spatial window in which the column pixel points are located; />Indicate->The +.f in the frame news simulcast gray level image>Line->Column pixel and the first +.in the space window where it is>Euclidean distance between points; />Represents the tuning constant, wherein +.>The empirical value was taken to be 1.
Point(s)The closer the value of the ST coefficient is to the rest of the points in the spatial window in which it lies, i.eThe smaller the value of (2), and the dot +.>The closer the distance from the rest of the points, i.eThe smaller the value of (2) is, the description is given of the three-dimensional sequence +.>Midpoint (at the middle point)>The denser the distribution of points with the ST coefficients close to the point in the local space, i.e. +.>The greater the value of (2).
Indicate->The +.f in the frame news simulcast gray level image>Line 1/>ST coefficient dense similarity of column pixel points;indicate->The +.f in the frame news simulcast gray level image>Line->Except +.>The remaining points outside +.>ST coefficient density corresponding to each point; />Represents the tuning constant, wherein +.>Taking experience value of 1%>Indicate->The +.f in the frame news simulcast gray level image>Line->ST coefficient aggregation index of column pixel points.
Point(s)The closer the value of the ST coefficient density is to the rest of the points in the spatial window in which it lies, i.eThe smaller the value of (2), the description point +.>The more similar the ST coefficient distribution between the remaining points in the spatial window in which it lies, i.e. +.>The greater the value of (2).
The larger the value of (2) is, the point +.>The more similar the ST coefficient distribution between the remaining points in the spatial window it is located, but +.>The larger the value of (2) is, the more ∈>The denser the distribution of points close to the ST coefficient of the point in the spatial window where it is located, the description point +.>The closer the ST coefficient value is to the points in the vicinity thereof, and the more similar the distribution of ST coefficients is, the point +.>The more unlikely it is an outlier, namely ST coefficient aggregation index +.>The greater the value of (2).
Further, since the pixel point at each position in the original image corresponds to the ST coefficient at the corresponding position generated after the Surfacelet conversion, according to the first embodimentThe +.f in the frame news simulcast gray level image>Line->ST coefficient aggregation index of column pixel points +.>And noise disturbance plausibility->Obtain->The +.f in the frame news simulcast gray level image>Line->ST correction coefficient of column pixel>The calculation method comprises the following steps:
in the above formula:indicate->The +.f in the frame news simulcast gray level image>Line->The ST correction coefficient of the column pixel point,indicate->The +.f in the frame news simulcast gray level image>Line->ST coefficient of column pixel, +.>Indicate->The +.f in the frame news simulcast gray level image>Line->ST coefficient aggregation index of column pixel, < ->Indicate->The +.f in the frame news simulcast gray level image>Line->Noise disturbance plausibility of column pixel, < ->Normalization function, which means normalizing the value in brackets; />Represents the tuning constant, wherein +.>The empirical value was taken to be 1.
The larger the value of (2) is, the +.>The +.f in the frame news simulcast gray level image>Line->The column pixel points and the points nearby have close ST coefficient values and similar ST coefficient distribution, and are +.>The larger, i.e. dot +.>The larger the value of the ST coefficient of (2) is, the more unlikely the point is to be a noise point, the ST correction coefficient corresponding to the point is +.>The larger should be; whileRepresenting the point in a news simulcast three-dimensional video sequence +.>The more the middle is suspected to be noise point, ">The larger the ST correction coefficient corresponding to this point is +.>The smaller.
The index construction diagram of the ST correction coefficient of each pixel is shown in fig. 2.
Step S003, denoising the video is completed according to the ST correction coefficient of each pixel point of each frame image.
Three-dimensional sequence of ST coefficient obtained in the last stepST correction coefficient +.>Three-dimensional sequence->ST coefficient +.>Obtaining the ST correction coefficient three-dimensional sequence +.>。
According to ST correction coefficient three-dimensional sequenceThree-dimensional sequence of ST coefficients using a Surfacelet threshold denoising algorithmAnd (3) carrying out subsequent processing to finish denoising the news simulcast video, and specifically:
the high frequency coefficients of the high frequency sub-bands among the ST coefficients of each decomposition layer are employedCalculating the threshold value to obtain ST correction coefficient threshold value of each decomposition layer>Wherein->The threshold is a known technology, and this embodiment is not described in detail.
Then correcting coefficient threshold according to the use threshold function ST of each decomposition layerThe ST coefficients of the pixels of each decomposition layer are thresholded, and in this embodiment, a hard threshold function is used, where the hard threshold function is:
in the above formula:indicate->ST correction threshold coefficients after threshold processing in the decomposition layers; />、/>Respectively represent +.>ST coefficients and ST correction coefficients in the respective decomposition layers, the two coefficients being corresponding; />Indicate->ST correction coefficient threshold values for the individual decomposition layers.
And carrying out Surfacelet inverse transformation on the ST correction threshold coefficient after the threshold processing, reconstructing the denoised video signal, and completing the denoising processing of the news simulcast video.
Aiming at the problems that the background information in a video image and the correlation of each coefficient in the surface are not considered in the traditional surface threshold denoising algorithm, and detail textures, edge blurring and flickering exist when the video is denoised, the embodiment of the invention provides a video noise optimization processing method, which is used for accurately analyzing noise conditions from a local angle according to the edge dynamic matching degree of each edge of each image obtained according to the Pasteur distance between each edge of each image in a news simulcast video and the corresponding nearby edge of other frame images, wherein the background information in the video image and the correlation of each coefficient in the surface are not considered;
further analyzing the association condition between noise and non-noise on the time sequence from the distribution condition of the gray time change sequence of each pixel point, simultaneously combining the local angles to more comprehensively analyze the distribution of noise in the video, considering the context information of the news simulcast video, and improving the retaining capability of non-noise edge information in the news simulcast video;
according to the correlation among ST coefficients of the news simulcast video in the surface area, an ST coefficient aggregation index is constructed, and the analysis is carried out from the two layers of the concentration degree and the similarity of the ST coefficients, so that the details in the news simulcast video can be better reserved;
and combining the distribution characteristics of the ST coefficient generated by noise in the surface area to obtain the ST correction coefficient, and denoising the news simulcast video by using a surface threshold denoising algorithm based on the obtained ST correction coefficient, so that the detail and the edge information in the news simulcast video can be better kept, and the noise information in the news simulcast video can be effectively filtered.
It should be noted that: the sequence of the embodiments of the present invention is only for description, and does not represent the advantages and disadvantages of the embodiments. And the foregoing description has been directed to specific embodiments of this specification. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
In this specification, each embodiment is described in a progressive manner, and the same or similar parts of each embodiment are referred to each other, and each embodiment mainly describes differences from other embodiments.
The above embodiments are only for illustrating the technical solution of the present application, and are not limiting; the technical solutions described in the foregoing embodiments are modified or some of the technical features are replaced equivalently, so that the essence of the corresponding technical solutions does not deviate from the scope of the technical solutions of the embodiments of the present application, and all the technical solutions are included in the protection scope of the present application.
Claims (2)
1. A method for optimizing video noise, the method comprising the steps of:
performing surface transformation on all frame images in the video to obtain ST coefficients of pixel points of each frame image and sub-bands of each decomposition layer in each direction;
obtaining each edge of each frame of image by adopting an edge detection algorithm, and obtaining the noise disturbance suspected degree of each pixel point of each frame of image according to the distribution condition of each edge of each frame of image and the distribution difference of the gray scale of each pixel point on the time sequence;
setting a space window for each pixel point of each frame of image, and obtaining the ST coefficient density of the pixel points according to the distribution of ST coefficients of the pixel points in the space window of the pixel points; obtaining ST coefficient dense similarity of the pixel points according to ST coefficient dense difference of the pixel points in the pixel point space window; obtaining ST coefficient aggregation indexes of the pixel points according to the ST coefficient density and the ST coefficient density similarity of the pixel points; obtaining ST correction coefficients of the pixel points according to ST coefficient aggregation indexes, noise disturbance suspected degrees and ST coefficients of the pixel points;
obtaining ST correction coefficient threshold values of all decomposition layers by adopting a 3 sigma threshold value according to high frequency coefficients of sub-bands in the direction of each decomposition layer; according to the ST correction coefficient threshold value of each decomposition layer, the ST coefficient of each pixel point and the ST correction coefficient, obtaining the ST correction threshold value coefficient of each pixel point of each decomposition layer, and carrying out Surfacelet inverse transformation on the ST correction threshold value coefficient subjected to threshold value processing to obtain a denoised video;
the obtaining the noise disturbance suspected degree of each pixel point of each frame image according to the distribution condition of each edge of each frame image and the distribution difference of the gray scale of each pixel point on the time sequence, comprises the following steps:
obtaining the edge dynamic matching degree of each edge of each frame image according to the similarity degree between the edge contour histograms of each edge of each frame image, taking the edge dynamic matching degree of each pixel point of each frame image as the edge dynamic matching degree of each pixel point, and setting the edge dynamic matching degree of each pixel point which is not on the edge in each frame image as 1;
constructing gray time distribution straight lines and gray time change sequences of all pixel points according to gray value distribution of all pixel points at the same positions in all frame images;
for each pixel point of each frame of image, obtaining a gray time variation difference coefficient of the pixel point according to the difference between gray time variation sequences of the pixel points and the gray time distribution straight line;
taking Euclidean distance between the pixel point and the gray time distribution straight line as gray time deviation degree of the pixel point;
taking the ratio of the product result of the gray time variation difference coefficient of the pixel point and the gray time deviation degree and the edge dynamic matching degree of the pixel point as the noise disturbance suspected degree of the pixel point;
the obtaining the edge dynamic matching degree of each edge of each frame image according to the similarity degree between the edge contour histograms of each edge of each frame image comprises the following steps:
for the edge contour histogram of each edge of each frame image, carrying out the pasteurization distance calculation on the edge contour histogram of the edge and the edge contour histograms of all edges of all frame images except the edge, obtaining the minimum pasteurization distance, and taking the reciprocal of the minimum pasteurization distance as the edge dynamic matching degree of the edge of the image where the edge contour histogram is located;
the construction of the gray scale time distribution straight line and the gray scale time change sequence of each pixel point according to the gray scale value distribution of the same position of each pixel point in all frame images comprises the following steps:
for each pixel point, sequencing the gray values of the pixel points in all frame images according to a time sequence to form a gray time distribution sequence of the pixel points;
performing straight line fitting on the gray time distribution sequence of the pixel points to obtain gray time distribution straight lines of the pixel points; performing first-order difference processing on the gray level time distribution sequence of the pixel points to obtain a gray level time change sequence of the pixel points;
the obtaining the gray time variation difference coefficient of the pixel point according to the difference between the gray time variation sequences of the pixel point and the gray time distribution straight line comprises the following steps:
calculating dtw distance average values of the gray level time change sequences of the pixel points and the gray level time change sequences of all the pixel points except the pixel points;
calculating the slope of a gray time distribution straight line of a pixel point, and obtaining an exponential function by taking the slope as an index and taking a natural constant as a base;
taking the product of the dtw distance mean value and the calculation result of the exponential function as a gray time variation difference coefficient of the pixel point;
the step of obtaining the ST coefficient density of the pixel points according to the ST coefficient distribution of the pixel points in the pixel point space window comprises the following steps:
calculating the square of the difference value of ST coefficients of the pixel points and the residual pixel points for each residual pixel point except the pixel points in the pixel point space window, and calculating the Euclidean distance between the pixel points and the residual pixel points;
calculating the average value of the inverse of the product of the square difference value and the Euclidean distance of all the rest pixel points in the pixel point space window as the ST coefficient concentration of the pixel points;
the step of obtaining the ST coefficient dense similarity of the pixel points according to the ST coefficient dense difference of the pixel points in the pixel point space window comprises the following steps:
for each residual pixel except the pixel in the pixel space window, calculating the square of the difference value of the ST coefficient density of the pixel and the residual pixel, and taking the average value of the inverse of the square of the difference value of all the residual pixels in the pixel space window as the ST coefficient density similarity of the pixel;
the obtaining the ST coefficient aggregation index of the pixel point according to the ST coefficient density and the ST coefficient density similarity of the pixel point comprises the following steps:
taking the product of the ST coefficient density and the ST coefficient density similarity of the pixel points as an ST coefficient aggregation index;
the step of obtaining the ST correction coefficient of the pixel point according to the ST coefficient aggregation index, the noise disturbance suspected degree and the ST coefficient of the pixel point comprises the following steps:
and calculating the product of the ST coefficient of the pixel point and the ST coefficient aggregation index, and taking the normalized value of the ratio of the product to the noise disturbance suspected degree of the pixel point as the ST correction coefficient of the pixel point.
2. The method for optimizing video noise according to claim 1, wherein the step of obtaining the ST correction threshold coefficient for each pixel of each decomposition layer after thresholding based on the ST correction coefficient threshold value for each decomposition layer, the ST coefficient for each pixel of each decomposition layer, and the ST correction coefficient comprises:
regarding each pixel point of each decomposition layer, when the absolute value of the ST correction coefficient of the pixel point is larger than or equal to the ST correction coefficient threshold value of the decomposition layer, taking the ST coefficient of the pixel point as the ST correction threshold value coefficient of the pixel point;
when the absolute value of the ST correction coefficient of the pixel point is smaller than the ST correction coefficient threshold value of the decomposition layer, 0 is taken as the ST correction threshold value coefficient of the pixel point.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311787634.1A CN117459793B (en) | 2023-12-25 | 2023-12-25 | Video noise optimization processing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311787634.1A CN117459793B (en) | 2023-12-25 | 2023-12-25 | Video noise optimization processing method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117459793A CN117459793A (en) | 2024-01-26 |
| CN117459793B true CN117459793B (en) | 2024-04-09 |
Family
ID=89591458
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311787634.1A Active CN117459793B (en) | 2023-12-25 | 2023-12-25 | Video noise optimization processing method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117459793B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN119151934B (en) * | 2024-11-19 | 2025-01-28 | 山东衡昊信息技术有限公司 | Extracardiac operation monitoring system and method based on image processing |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102158637A (en) * | 2011-04-01 | 2011-08-17 | 西安电子科技大学 | Method for denoising space self-adaption threshold video based on Surfacelet transform domain |
| CN103116761A (en) * | 2013-03-07 | 2013-05-22 | 武汉大学苏州研究院 | Dynamic texture recognition method on basis of image sequence |
-
2023
- 2023-12-25 CN CN202311787634.1A patent/CN117459793B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102158637A (en) * | 2011-04-01 | 2011-08-17 | 西安电子科技大学 | Method for denoising space self-adaption threshold video based on Surfacelet transform domain |
| CN103116761A (en) * | 2013-03-07 | 2013-05-22 | 武汉大学苏州研究院 | Dynamic texture recognition method on basis of image sequence |
Non-Patent Citations (3)
| Title |
|---|
| Surfacelet联合多向Cycle Spinning的图像去噪方法;曹家年;汤春瑞;刘丹丹;;哈尔滨工程大学学报;20090815(第08期);全文 * |
| 基于Surfacelet变换和SPIHT算法的视频压缩;王海燕;殷俊;潘显萌;;计算机科学;20161115(第S2期);全文 * |
| 基于Surfacelet变换的三维CT图像边缘检测;孙运慧;白培瑞;张晓洁;;扬州大学学报(自然科学版);20180828(第03期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117459793A (en) | 2024-01-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106355561B (en) | A Total Variational Image Denoising Method Based on Noise Prior Constraints | |
| CN105472204B (en) | Inter-frame Noise Reduction Method Based on Motion Detection | |
| EP4258204A1 (en) | Image processing method, apparatus, computer program and computer-readable data carrier | |
| US8208565B2 (en) | Pre-processing method and system for data reduction of video sequences and bit rate reduction of compressed video sequences using temporal filtering | |
| CN110223254A (en) | A kind of image de-noising method generating network based on confrontation | |
| CN108564597B (en) | Video foreground object extraction method fusing Gaussian mixture model and H-S optical flow method | |
| US20080284904A1 (en) | Pre-processing method and system for data reduction of video sequences and bit rate reduction of compressed video sequences using spatial filtering | |
| CN111489303A (en) | Maritime affairs image enhancement method under low-illumination environment | |
| CN107705260B (en) | The denoising system and calculation method of medical X-ray image | |
| CN108109123B (en) | Image denoising method | |
| CN117459793B (en) | Video noise optimization processing method | |
| CN109636766A (en) | Polarization differential and intensity image Multiscale Fusion method based on marginal information enhancing | |
| CN104881847A (en) | Match video image enhancement method based on wavelet analysis and pseudo-color processing | |
| CN113837974A (en) | NSST (non-subsampled contourlet transform) domain power equipment infrared image enhancement method based on improved BEEPS (Bayesian particle swarm optimization) filtering algorithm | |
| CN111311508B (en) | Noise reduction method for pavement crack image with noise | |
| CN104616259B (en) | A kind of adaptive non-local mean image de-noising method of noise intensity | |
| Kumar et al. | A new locally adaptive patch variation based CT image denoising | |
| CN112837238A (en) | Image processing method and device | |
| CN104754183B (en) | A kind of real-time monitor video adaptive filter method and its system | |
| CN110706186A (en) | Underwater information anti-interference method based on compressed sensing and Renyi entropy | |
| CN111429362A (en) | Blood vessel enhancement method of endoscope fluorescence image | |
| CN111147804B (en) | Video frame reconstruction method | |
| Xie et al. | Just noticeable visual redundancy forecasting: a deep multimodal-driven approach | |
| CN110490819A (en) | Image de-noising method based on NSST and NLM filtering and hard -threshold technology | |
| CN107798676B (en) | A reference-free image quality perception method based on multi-scale natural scene statistics |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |