CN116188897A

CN116188897A - Stripe noise detection model training method, stripe noise detection method and device

Info

Publication number: CN116188897A
Application number: CN202211696024.6A
Authority: CN
Inventors: 李滨伯; 崔东林
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2022-12-28
Filing date: 2022-12-28
Publication date: 2023-05-30

Abstract

The disclosure provides a stripe noise detection model training method, a stripe noise detection method and a stripe noise detection device. The disclosure relates to the technical field of artificial intelligence, in particular to the technical fields of computer vision, image processing, deep learning, video quality inspection, intelligent recommendation and the like. The specific implementation scheme is as follows: acquiring a first sample data set comprising a plurality of first sample images; inputting the first sample image into a model to be trained to obtain N predicted values corresponding to grids on the first sample image output by the model to be trained, wherein the N predicted values are at least used for predicting whether stripe noise exists on the grids of the first sample image, and the N predicted values are predicted values corresponding to N channels of the model to be trained; and training the model to be trained based on the N predicted values corresponding to each grid of the first sample image and the N true values corresponding to each grid of the first sample image to obtain the stripe noise detection model. The present disclosure can efficiently and accurately detect a stripe noise problem in an image.

Description

Stripe noise detection model training method, stripe noise detection method and device

Technical Field

The disclosure relates to the technical field of artificial intelligence, in particular to the technical fields of computer vision, image processing, deep learning, video quality inspection, intelligent recommendation and the like.

Background

With the rapid development of internet technology, massive video resources and picture resources are uploaded to a network, so that the difficulty of recommending search technology is greatly increased, and in order to effectively meet and promote search experience of different users, resources with higher quality are required to be provided for the users. However, the video uploaded by the user may be subject to power frequency interference, unbalanced voltage in the system, transmission problems and other external factors, or the image sensor calibration deviation, unbalanced internal current, system noise or vibration and other internal factors caused by the long-term exposure and uninterrupted working characteristics of the monitoring camera, so that various stripe noise problems exist in the video. Therefore, how to efficiently and accurately detect the stripe noise problem from a huge amount of video resources becomes a technical problem to be solved.

Disclosure of Invention

The disclosure provides a stripe noise detection model training method, a stripe noise detection method and a stripe noise detection device.

According to a first aspect of the present disclosure, there is provided a stripe noise detection model training method, comprising: obtaining a first sample data set comprising a plurality of first sample images, each first sample image comprising at least one stripe noise, each first sample image being divided into a plurality of grids; inputting a first sample image into a model to be trained to obtain N predicted values corresponding to grids on the first sample image output by the model to be trained, wherein the N predicted values are at least used for predicting whether stripe noise exists on the grids of the first sample image, the N predicted values are predicted values corresponding to N channels of the model to be trained, and N is a positive integer; and training the model to be trained based on the N predicted values corresponding to each grid of the first sample image and the N true values corresponding to each grid of the first sample image to obtain the stripe noise detection model.

According to a second aspect of the present disclosure, there is provided a stripe noise detection method comprising: acquiring an image to be detected; inputting the image to be detected into a stripe noise detection model to obtain a stripe noise detection result of the image to be detected, wherein the stripe noise detection model is trained by the method according to the first aspect, the stripe noise detection result comprises N predicted values corresponding to each grid on the image to be detected, the N predicted values are at least used for predicting whether stripe noise exists on the grids of the image to be detected, the N predicted values are predicted values corresponding to N channels of the stripe noise detection model, and N is a positive integer.

According to a third aspect of the present disclosure, there is provided a banding noise detection model training apparatus, comprising: a first acquisition module for acquiring a first sample data set comprising a plurality of first sample images, each first sample image comprising at least one stripe noise, each first sample image being divided into a plurality of grids; the second acquisition module is used for inputting the first sample image into the model to be trained to obtain N predicted values corresponding to grids on the first sample image output by the model to be trained, wherein the N predicted values are at least used for predicting whether stripe noise exists on the grids of the first sample image, the N predicted values are predicted values corresponding to N channels of the model to be trained, and N is a positive integer; the training module is used for training the model to be trained based on the N predicted values corresponding to each grid of the first sample image and the N true values corresponding to each grid of the first sample image to obtain the stripe noise detection model.

According to a fourth aspect of the present disclosure, there is provided a stripe noise detection device comprising: the third acquisition module is used for acquiring an image to be detected; the detection module is configured to input the image to be detected into a stripe noise detection model to obtain a stripe noise detection result of the image to be detected, where the stripe noise detection model is trained by the method described in the first aspect, the stripe noise detection result includes N predicted values corresponding to each grid on the image to be detected, the N predicted values are at least used for predicting whether stripe noise exists on the grid of the image to be detected, the N predicted values are predicted values corresponding to N channels of the stripe noise detection model, and N is a positive integer.

According to a fifth aspect of the present disclosure, there is provided an electronic device comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the stripe noise detection model training method provided in the first aspect and/or the stripe noise detection method provided in the second aspect.

According to a sixth aspect of the present disclosure, there is provided a non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the banding noise detection model training method provided in the first aspect and/or the banding noise detection method provided in the second aspect.

According to a seventh aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, performs the banding noise detection model training method provided in the first aspect and/or the banding noise detection method provided in the second aspect.

According to the technical scheme, the problem of stripe noise in the image can be efficiently and accurately detected.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.

Drawings

In the drawings, the same reference numerals refer to the same or similar parts or elements throughout the several views unless otherwise specified. The figures are not necessarily drawn to scale. It is appreciated that these drawings depict only some embodiments according to the disclosure and are not therefore to be considered limiting of its scope.

FIG. 1 is a flow diagram of a stripe noise detection model training method according to an embodiment of the present disclosure;

FIG. 2 is a schematic source diagram of a first sample dataset according to an embodiment of the present disclosure;

FIG. 3 is a flow diagram of determining an optimal stripe noise detection model according to an embodiment of the present disclosure;

FIG. 4 is a schematic diagram of a framework of a banding noise detection model according to an embodiment of the disclosure;

FIG. 5 is an overall architecture diagram of stripe noise detection according to an embodiment of the present disclosure;

FIG. 6 is a flow diagram of a stripe noise detection method according to an embodiment of the present disclosure;

FIG. 7 is a schematic diagram of a construction of a banding noise detection model training apparatus according to an embodiment of the disclosure;

fig. 8 is a schematic structural view of a stripe noise detection device according to an embodiment of the present disclosure;

FIG. 9 is a schematic diagram of a scenario of banding noise detection model training in accordance with an embodiment of the present disclosure;

FIG. 10 is a schematic diagram of a scenario of stripe noise detection according to an embodiment of the present disclosure;

fig. 11 is a schematic structural diagram of an electronic device used to implement the banding noise detection model training method and/or the banding noise detection method of the embodiments of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

The terms first, second, third and the like in the description and in the claims and in the above-described figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to cover a non-exclusive inclusion, such as a series of steps or elements. The method, system, article, or apparatus is not necessarily limited to those explicitly listed but may include other steps or elements not explicitly listed or inherent to such process, method, article, or apparatus.

In the related art, the stripe noise detection method can be roughly divided into two periods. The detection method in the first period is based on periodic characteristic attribute of stripes or abrupt change of brightness average value, standard deviation, gradient and other characteristics of adjacent positions in an image. However, when these shallow features are used for evaluation, it is difficult to solve irregular distribution of streak noise with small neighborhood variation. In the detection method in the second period, the deep features such as the structural features and the direction features of stripe noise are extracted, and comprehensive judgment is carried out by combining, so that good performance is achieved in the aspect of detection precision. However, on one hand, the calculation speed of extracting the manually defined features for judgment is slow, such as calculating the mean and variance of the whole image for subsequent processing; on the other hand, these features may not be suitable for noise detection in multi-sensor images. Thus, the above aspect makes it more difficult to accurately and rapidly detect streak noise. To further improve the video quality evaluation, the inventor proposes a new stripe noise detection algorithm based on deep learning.

In order to at least partially solve one or more of the above problems and other potential problems, the present disclosure proposes a stripe noise detection algorithm based on deep learning, which considers stripe noise as a linear target, and considers the randomness problem of stripe noise in angle and length, and performs model training by using a large amount of labeled and expanded sample data and selects an optimal parameter model, thereby improving the robustness of the model, effectively detecting the stripe noise problem existing in a video image, and having great reference value and meaning for applications such as downstream quality evaluation, product recommendation, and the like.

An embodiment of the present disclosure provides a stripe noise detection model training method, and fig. 1 is a schematic flow diagram of a stripe noise detection model training method according to an embodiment of the present disclosure, where the stripe noise detection model training method may be applied to a stripe noise detection model training device. The banding noise detection model training device is located in the electronic equipment. The electronic device includes, but is not limited to, a stationary device and/or a mobile device. For example, the fixed device includes, but is not limited to, a server, which may be a cloud server or a general server. For example, mobile devices include, but are not limited to: cell phone, tablet computer, vehicle terminal. In some possible implementations, the banding noise detection model training method can also be implemented by way of a processor invoking computer readable instructions stored in a memory. As shown in fig. 1, the training method of the stripe noise detection model includes:

S101: obtaining a first sample data set comprising a plurality of first sample images, each first sample image comprising at least one stripe noise, each first sample image being divided into a plurality of grids;

s102: inputting a first sample image into a model to be trained to obtain N predicted values corresponding to grids on the first sample image output by the model to be trained, wherein the N predicted values are at least used for predicting whether stripe noise exists on the grids of the first sample image, the N predicted values are predicted values corresponding to N channels of the model to be trained, and N is a positive integer;

s103: and training the model to be trained based on the N predicted values corresponding to each grid of the first sample image and the N true values corresponding to each grid of the first sample image to obtain the stripe noise detection model.

In the embodiment of the disclosure, the number of the first sample images included in the first sample data set may be increased or decreased according to the requirement. How to obtain the first sample data set will be described in detail later, and will not be described here again.

In an embodiment of the present disclosure, the first sample image is an image having at least one stripe noise. For example, the first sample image may be an online captured picture. For another example, the first sample image is an image obtained after framing a video. As another example, the first sample image is an image from an image database. The present disclosure does not limit the manner in which the first sample image is acquired.

In the embodiment of the present disclosure, the size of the first sample image is M, m=m×m, that is, the width and the height of the first sample image are equal. Each first sample image is divided into s×s grids, i.e., the first sample image is divided into S rows and S columns, thereby obtaining s×s grids. In this way, the banding noise included in the first sample image can be analyzed based on the respective grids, thereby contributing to an increase in the detection speed of the banding noise.

In the embodiment of the disclosure, a model to be trained is used for outputting N predicted values of each grid on an image according to an input image. Based on this, it can be appreciated that in embodiments of the present disclosure, the model to be trained may include at least one output for outputting at least N predicted values for each grid on the image. The stripe noise detection model is obtained by training a model to be trained through a predicted value and a true value, and therefore, the stripe noise detection model has the same model structure as the model to be trained, and is different in that model parameters are updated after training.

By adopting the method for training the stripe noise detection model, which is provided by the embodiment of the disclosure, a first sample image is input into a model to be trained, and N predicted values corresponding to each grid on the first sample image output by the model to be trained are obtained; and finally, training the model to be trained based on the N predicted values corresponding to each grid of the first sample image and the N true values corresponding to each grid of the first sample image to obtain the stripe noise detection model. In the method, each first sample image is divided into a plurality of grids, and whether stripe noise exists on the first sample image is predicted based on N predicted values corresponding to the grids on the first sample image, so that the accuracy of stripe prediction can be improved, and the accuracy of a stripe noise detection model obtained through training can be improved. Compared with a processing scheme for analyzing the sample image to obtain the image characteristics, according to the processing scheme for predicting whether stripe noise exists on the sample image according to the image characteristics, the accuracy of the predicted stripe noise can be improved, and the robustness of a stripe noise detection model can be improved, so that the stripe noise model is suitable for noise detection in a multi-sensor image.

In some embodiments, the N predictors include: the confidence coefficient of the predicted value of the coordinate offset of the central point and the predicted value of probability corresponding to the predicted value x, y, regression value h and 180 angles respectively.

Here, the center point coordinate offset prediction value includes: offset from the X-axis coordinate of the center point of the grid (denoted as X), offset from the Y-axis coordinate of the center point of the grid (denoted as Y).

Here, the confidence of the center point coordinate offset predictor may be denoted as confidence. Taking the current grid as the grid of Si rows and Sj columns on the first sample image as an example, if the confidence of the grid is greater than a preset threshold value, indicating that stripe noise exists on the grid; and if the confidence of the grid is smaller than or equal to a preset threshold value, indicating that the grid has no stripe noise.

In some embodiments, S103 may include:

s1031: constructing a first loss function based on the predicted value of the center point coordinate offset and the true value of the center point coordinate offset of each grid;

s1032: constructing a second loss function based on the first type confidence predicted value and the confidence value true value of the first type confidence predicted value of each grid, wherein the first type confidence predicted value is greater than a preset threshold value;

S1033: constructing a third loss function based on the second type confidence predicted value of each grid and the confidence value true value of the second type confidence predicted value, wherein the second type confidence predicted value is a confidence predicted value smaller than or equal to a preset threshold value;

s1034: constructing a fourth loss function based on the probability prediction value and the probability true value which correspond to the 180 angles of each grid respectively;

s1035: the model to be trained is trained based on the first, second, third, and fourth loss functions.

In the embodiment of the present disclosure, the execution order of S1031, S1032, S1033, and S1034 is not limited.

In practical application, the preset threshold value can be selected by evaluating the recall rate and the false detection rate of the model, and can also be set according to experience.

Here, training the model to be trained based on the first, second, third, and fourth loss functions includes: determining a total loss function based on the first loss function, the second loss function, the third loss function, and the fourth loss function; the model to be trained is trained based on the total loss function.

In practical applications, S103 may further include:

s1036: a fifth penalty function is constructed based on the stripe noise class prediction value and the stripe noise class truth value for each grid. Here, S1036 occurs before S1035.

Further, S1035 may be changed to: the model to be trained is trained based on the first, second, third, fourth, and fifth loss functions.

Here, the detected various types of band noise may be regarded as the same category.

The total loss function may be calculated according to the following formula:

wherein the first term on the right side of the equation represents a position coordinate error term, the second term and the third term both represent confidence error terms, the fourth term represents an angle prediction error term, and the fifth term represents a classification loss term.

Wherein the equation representation divides the feature map into s×s grids. Each grid generates 3 candidate detection lines. If the true detection object center point does not exist in the grid, only calculating a confidence error; otherwise, the position coordinate error must be calculated. The confidence error adopts a cross entropy loss function, and comprises two kinds of detected objects and non-detected objects. Adding a weighting coefficient lambda to the loss of no detected object _noobj The contribution weight of the loss value without object to the total loss can be reduced. The position coordinate error is obtained by modifying the mean square error loss function, where α1, α2, α3, and k are used to balance the weight coefficients bx, by, and bh (which represent the x, y offsets and target lengths under the corresponding grid, respectively, and k is the ratio of the input and output signature sizes). The classification loss uses a cross entropy loss function; when predicted objects in a grid are true And when the real object corresponds, calculating the classification loss. The angle prediction error term is used for solving the angle problem by using a cross entropy loss function and adopting a classification idea, and meanwhile, a round smooth label (Circular Smooth Label, CSL) is selected to be used for coding at the label, so that positive and negative samples are more balanced, and a training result is more robust.

Therefore, the model to be trained is trained based on the total loss function formed by the plurality of loss functions, and the robustness of the stripe noise model obtained through training can be enhanced.

In some embodiments, prior to S101, the banding noise detection model training method may further include:

s104: counting the lengths of all stripe noises included in the first sample data set;

s105: and clustering based on the lengths of all the stripe noises to obtain K priori length values, wherein the priori length values are the reference lengths of the stripe noises adopted by the stripe noise detection model, and K is an integer not less than 1.

Here, a K-means clustering algorithm (K-means) may be employed to cluster the lengths of all stripe noises. In the embodiments of the present disclosure, K may be 3, or 4, or 5, or other values. The above is illustrative only and is not intended to be limiting as to the full potential of K, but is not intended to be exhaustive.

In the embodiment of the disclosure, taking a rectangular image with k=3 and a size of 300×400 as an example, the diagonal line length of the rectangular image is 500, it is obvious that the prior length of such rectangular image should be greater than 0 and less than 500. For example, the a priori length may be 220, 320, 380. For example, the a priori length may be 220, 320, 380. For another example, the a priori length may be 200, 300, 350. For another example, the a priori length may be 180, 250, 310. The above is merely exemplary and is not intended to be an all-inclusive possible limitation on the a priori length, but is not intended to be exhaustive.

Therefore, by selecting K priori length values, a plurality of effective reference bases are provided for N predicted values of each grid on the output image of the stripe noise detection model, so that the robustness of the stripe noise detection model obtained by training is improved, and the accuracy of the detection result of the stripe noise detection model obtained by training is improved.

In some embodiments, the N channels of the model to be trained may be divided into K groups of channels, each group of channels comprising 185 channels. The 185 channels comprise 180 angle channels, 1 regression value channel, 1 confidence coefficient channel, 1 stripe noise category channel, 1X-axis offset channel, 1Y-axis offset channel, and 180 angle channels are used for predicting probability prediction values corresponding to the 180 angles respectively.

Here, the regression value channel is used to predict the regression value employed in calculating the stripe noise length.

Here, the confidence channel is used to output the confidence of the center point coordinate offset.

Here, an X-axis offset channel is used to predict the offset of the X-axis coordinates relative to the top left corner of the current grid.

Here, the Y-axis offset channel is used to predict the offset of the Y-axis coordinates relative to the upper left corner of the current grid.

Here, 180 angular channels include 1 ° channel, 2 ° channel, 3 ° channel, …,180 ° channel altogether. Wherein, the 1-degree channel is used for predicting the probability value of the stripe noise angle of 1 degree; the 2 ° channel is used to predict a probability value of 2 ° for the stripe noise angle, and the …,180 ° channel is used to predict a probability value of 180 ° for the stripe noise angle.

In the embodiment of the present disclosure, each class of stripe noise is regarded as one class.

If the classification of the stripe noise is classified into Q, 1 stripe noise classification channel may be changed into Q stripe noise classification channels.

Therefore, N channels of the model to be trained are divided into K groups of channels, and the K groups of channels correspond to the K stripe noise priori length values, so that the robustness of the stripe noise detection model obtained by training is improved, and the accuracy of the detection result of the stripe noise detection model obtained by training is improved.

In some embodiments, S101 may include:

s1011: acquiring an image with quality lower than a preset quality threshold;

s1012: and labeling the image according to a preset labeling rule to generate a first sample data set.

Here, the preset quality threshold may be set or adjusted according to the requirements of accuracy or speed, etc.

Here, the manner in which the image is acquired includes, but is not limited to: capturing images on line; the video is cut into frames to obtain images.

Here, labeling the image according to the preset labeling rule may include: and marking stripe noises appearing in the image by using lines, returning coordinates of the front end and the rear end of the lines, and taking all the stripe noises as a category.

In this way, the authenticity of the first sample data set can be improved through collecting and labeling the real images, so that the scene adaptability of the strip noise detection model obtained through training can be improved.

In some embodiments, S101 may further comprise:

s1013: synthesizing the inclined stripe noise in the image according to a preset synthesis mode;

s1014: a first sample data set is generated based on the synthesized oblique stripe noise.

Here, S1013 and S1014 are in parallel relation with S1011 and S1012.

Fig. 2 shows a schematic source diagram of a first sample data set, as shown in fig. 2, the source of which comprises: the real stripe noise data set is collected and marked, and the artificial synthesized simulation stripe noise data set is obtained.

Therefore, the existing image can be subjected to stripe noise simulation, the inclined stripe noise is synthesized, the number of samples can be greatly increased, and further the training effect of the stripe noise detection model is improved.

In some embodiments, synthesizing the oblique stripe noise in the image in a preset synthesis manner includes:

randomly taking w break points on the line direction central line of each image; wherein xi is a coordinate value of the discontinuity point on a line direction central line of the image, and w is an integer not less than 1;

for each discontinuity xi, calculating a row-by-row offset from top to bottom along the column direction of the image;

acquiring row-by-row offset points based on the offset;

and connecting the offset points row by row to form a straight line, wherein the break point is taken as the center point of the straight line.

For example, w=5 discontinuities are randomly taken on the line-direction center line of the image, and coordinate values of the 5 discontinuities on the line-direction center line of the image are set as x1, x2, x3, x4, x5, respectively. It will be appreciated that the value of w may be set or adjusted as desired.

Exemplary, in [0.75,1.25 ]]，[-5,5]，

W=5 groups of random numbers alpha 1, beta 1 and theta' 1 are taken in three ranges; α2, β2, θ'2; α3, β3, θ'3; α4, β4, θ'4; α5, β5, θ '5, and calculating the arctangent values of θ'1, θ '2, θ'3, θ '4, θ'5, which are θ 1, θ2, θ 3, θ 4, θ 5.

Therefore, the inclined stripe noise can be simulated based on the existing image, the sample size of the stripe noise is increased, and further the training effect of the stripe noise detection model is improved.

In some embodiments, acquiring the progressive offset point based on the size of the offset includes:

if the number of the break points is 1, multiplying all pixel values from a straight line taking the break point as a center to the tail end of the image in the row direction by a first random number alpha i and adding a second random number beta i, and ending;

if the number of the break points is greater than 1, taking two break points as a group according to a judgment basis, acquiring two straight lines by each group of break points, multiplying all pixel values in the middle range of the two straight lines with the two break points as the center by the first random number alpha i and adding the second random number beta i; for all pixel values from the line centered on the last discontinuity to the end of the line direction of the image, multiply by a third random number αm and add a fourth random number βm and end.

Wherein offsetxj= (m/2-Rowj) |tan θ|, m represents the image size, and Rowj represents the number of lines.

Therefore, the progressive offset points can be obtained rapidly, the efficiency of simulating the image stripe noise is improved, the efficiency of generating rich and various first sample data sets is improved, and the training effect of the stripe noise detection model is improved.

In some embodiments, based on the size of the offset, acquiring the progressive offset point may further include:

if the number of the break points is 1, calculating offset from row to row, and when (xi+offsetxj) <0, the abscissa of the offset point is 0; when (xi+offsetxj) > m, the offset point is not counted, wherein offsetxj represents the offset amount and m represents the image size;

if the number of the break points is greater than 1, taking two adjacent break points from left to right as a group, for the left break point, calculating offset from row to row, and when (xi+offsetxj) <0, the abscissa of the offset point is 0; when (xi+offsetxj) > m, the offset point is not counted; for the right break point, calculating offset from row to row, and when (xi+offsetxj) <0, not counting offset points; when (xi+offsetxj) > m, the abscissa of the offset point is m.

Here, m=m×m, that is, the width and the height of the image are equal.

For ease of understanding, the following is illustrative.

If the number of discontinuities is 1, there is and only one (x 1, m/2) discontinuity. In [0.75,1.25 ]]，[-5,5]，

Three random numbers α1, β1, θ '1 are taken over three ranges, and the arctangent value θ1 of θ'1 is calculated. The progressive offset offsetx is calculated from top to bottom along the image column direction with (x 1, m/2) as the center point. Taking (x 1, 0) and (x 1, 50) as examples: an offset of (x 1, 0) is equal to (m/2-0) tan θ1 and an offset point is determined if x1+ (m/2-0) tan θ1| <0, obtaining offset points (0, 0); if x1+ (m/2-0) |tan θ1|>m, not counting offset points; otherwise, an offset point (x1+ (m/2-0) is obtained. (x 1, 50) is equal to (m/2-50)* Tan θ1 and determining the offset point if x1+ (m/2-50) tan θ1|<0, obtaining offset points (0, 50); if x1+ (m/2-50) |tan θ1|>m, not counting offset points; otherwise, the offset point (x 1, (m/2-50) |tan θ1|) is obtained, and the offset points are connected to form a straight line with the center of (x 1, m/2). All pixel values to the right of the straight line in the row direction are multiplied by the random number α1 and added with the random number β1 and end.

If the number of discontinuities is 3, there are (x 1, m/2), (x 2, m/2), (x 3, m/2) three discontinuities. In [0.75,1.25 ]]，[-5,5]，

Three sets of random numbers alpha 1, beta 1 and theta' 1 are taken in three ranges; α2, β2, θ'2; α3, β3, θ'3; and, two sets of discontinuities ((x 1, m/2), (x 2, m/2)), ((x 2, m/2), (x 3, m/2)) are obtained. Taking the example of the first group ((x 1, m/2), (x 2, m/2)) break point, calculating the progressive offset offsetx from top to bottom along the image column direction, the offset for (x 1, 0), (x 2, 0) is equal to (m/2-0) tan θ1, if x1+ (m/2-0) tan θ1 for the left break point<0, obtaining offset points (0, 0); if x1+ (m/2-0) |tan θ1| >m, not counting offset points; otherwise, an offset point (x1+ (m/2-0) tan θ1, 0) is obtained. For the right break point, if x2+ (m/2-0) |tan θ1|<0, obtaining offset points (0, 0); if x2+ (m/2-0) |tan θ1|>m, not counting offset points; otherwise, obtaining an offset point (x2+ (m/2-0) tan theta 1|, 0); the offset for (x 1, 50), (x 2, 50) is equal to (m/2-50) |tan θ1|, for the left break point, if x1+ (m/2-50) |tan θ1|<0, obtaining offset points (0, 50); if x1+ (m/2-50) |tan θ1|>m, not counting offset points; otherwise, an offset point (x1+ (m/2-50) is obtained |tan θ1|, 50). For the right break point, if x2+ (m/2-50) |tan θ1|<0, obtaining offset points (0, 50); if x2+ (m/2-0) |tan θ1|>m, not counting offset points; otherwise, an offset point (x2+ (m/2-0) tan θ1|, 50) is obtained. And connecting the offset points obtained by the left and right break points respectively to form two straight lines taking (x 1, m/2) and (x 2, m/2) as central points. All pixel values in the middle range of two straight lines centered at two break points are multiplied by a random number alpha i and added with the random number beta i, and the straight line centered at the last break point reaches the end of the image line directionAnd multiplying all pixel values by the random number alpha m and adding the random number beta m to finish.

The tilt band noise simulation method based on the above can only realize tilt bands within (-pi/4, pi/4). Thus, when the image is generated, we rotate the image 90 ° clockwise with a probability of 0.5 for the image, so that simulated banding noise data for all possibilities (-pi/2, pi/2) can be obtained.

Therefore, the number of samples can be effectively increased, and a large amount of labor cost required by manual labeling is reduced. The sample number is abundant and various, so that the training effect of the stripe noise detection model, such as the robustness of the stripe noise detection model and the detection accuracy of the stripe noise detection model, can be improved.

In some embodiments, as shown in fig. 3, the banding noise detection model training method may further include: obtaining a second sample dataset comprising a plurality of second sample images; at least one second image sample of the plurality of second sample images does not include banding noise; obtaining a plurality of stripe noise detection models in response to the training, verifying the plurality of stripe noise detection models based on the second sample data set; and taking the stripe noise detection model with the optimal verification result as a final stripe noise detection model.

Here, the second sample data set may be independent of the first sample data set, or may have a partial intersection with the first sample data set.

Here, the optimum verification result can be understood as the fastest detection speed and the highest detection accuracy.

In practice, the data set may be randomly divided into a training set (first sample data set) and a validation set (second sample data set) in an equal ratio of 8:2 or 9:1. And performing random clipping, overturning, color space conversion and other picture amplification operations on the training set pictures, uniformly scaling the amplified pictures into 640-sized Red Green Blue (RGB) format images serving as the input of a model. The training set is trained by utilizing a video stripe noise detection algorithm, and the algorithm can randomly select 4 pictures from the training set based on a data enhancement method, and synthesize one picture in a random scaling, random cutting, random arrangement and other modes, so that the detection effect of a model on a small target is enhanced, and meanwhile, the use of calculation resources is saved to a certain extent. And then, continuously updating model weight parameters in a random gradient descending mode to enable the models to be converged, and selecting the optimal model on the verification set as a final model. Here, the data enhancement method includes, but is not limited to, the Mosaic data enhancement algorithm.

In this way, the effectiveness of the finally determined stripe noise detection model can be improved by performing verification processing on the plurality of stripe noise detection models obtained by training.

FIG. 4 shows a schematic diagram of a framework of a stripe noise detection model, which we refer to a backbone network (backbone) of a network model such as the YOLOv5 network model for the network model part and improve the output of the backbone network; meanwhile, a loss term of the angle of the stripe noise is added to the loss function. For a network model, based on the physical characteristics and visual characteristics of stripe noise, on the basis of a YOLOv5 network model, only one output end is reserved, so that the parameter quantity required by the model is reduced, and the model training and testing speed is improved. Meanwhile, only one output end is reserved in a single-head detection mode, and the method has a better effect on detection precision for intensive target detection. For the output end, a priori length value is selected by using a K-means clustering algorithm, CSL coding is carried out on rotation degree (namely angle) in a sample label, the angle is regarded as a classification problem, the model is easier to converge, and the accuracy requirement of linear target detection is far higher than that of common target detection. Here, general target detection includes, but is not limited to, face detection, license plate detection, and the like. Thus, the CSL encoded sample tag is more effective than the One-Hot encoded (One-Hot) sample tag.

In the embodiment of the disclosure, CSL encoding is adopted, the label of the correct angle value is set to be 1, and the label adjacent to the correct angle value is set to be a number greater than 0 but less than 1, so that the variety in the lost item can be increased, and the number of positive samples can be increased. By way of example, the stripe noise is classified into 180 angles (considered as 180 categories), if the true angle of the stripe noise is 80 degrees, the label at 80 ° is set to 1, the labels at 79 ° and 81 ° are set to 0.3, the farther from the true value, the smaller the label value, for example, the label values at 1 ° and 159 ° are set to 0.0001, forming a normal distribution. Thus, the sample size is further increased.

As shown in fig. 4, the lengths of all samples of the stripe noise are counted first, and the length values of the 3 clustering center points are obtained by using a K-means algorithm and used as the priori length values of the candidate detection lines. Based on the (x, y, h, confidence) 4 basic parameters on the candidate detection line on the detector, angle prediction is performed by taking the angle prediction value as an auxiliary parameter. And carrying out CSL coding on the prediction angle by using the probability prediction values of 180 channels, judging the inclination angle (namely the prediction angle) of the stripe noise, and outputting the channel dimension of the characteristic diagram as 3 (4+1+180) =555. According to b _h ＝4δ(t _h ) ² P _h Calculate the length, wherein δ (x) =1/(1+e) ^-x )，P _h For a priori length value, b _h Representing the length prediction value, t, of the model _h Is the regression value, t _h Compressing to 0-1. As shown in fig. 4, (1) and (2) are framed with dashed boxes, not the contents of the backbone network; (2) the hexagon in (a) is an output end, and a 555-dimensional feature map of M×M is output.

FIG. 5 shows an overall flow chart of banding noise detection, as shown in FIG. 5, of an image/video dataset that does not contain banding noise, via an analog banding noise algorithm, resulting in a dataset that contains banding noise; the data set and an image/video data set originally containing stripe noise generate an image/video target data set containing stripe noise; by adopting the stripe noise detection algorithm, training a model to be trained by utilizing the target data set to obtain an optimal stripe noise detection model; and detecting the stripe noise in the image by using the optimal stripe noise model, and performing visualization processing on the detected stripe noise.

Therefore, the sample volume expansion method can effectively increase the number of samples and reduce a large amount of labor cost required by manual labeling. The model can effectively detect various stripe noises on the video, has strong downstream application capability, and has strong expansibility in various fields such as quality evaluation, video recommendation and the like.

It should be understood that the schematic diagrams shown in fig. 2, 3, 4 and 5 are merely exemplary and not limiting, and that they are scalable, and that various obvious changes and/or substitutions may be made by one skilled in the art based on the examples of fig. 2, 3, 4 and 5, and the resulting technical solutions still fall within the scope of the disclosed embodiments of the present disclosure.

The embodiment of the disclosure provides a stripe noise detection method, which can be applied to electronic equipment. Hereinafter, a method for detecting stripe noise according to an embodiment of the present disclosure will be described with reference to a flowchart shown in fig. 6.

S601: acquiring an image to be detected;

s602: inputting the image to be detected into a stripe noise detection model to obtain a stripe noise detection result of the image to be detected, wherein the stripe noise detection result comprises N predicted values corresponding to each grid on the image to be detected, the N predicted values are at least used for predicting whether stripe noise exists on the grid of the image to be detected, the N predicted values are predicted values corresponding to N channels of the stripe noise detection model, and N is a positive integer.

The embodiments of the present disclosure do not limit the source of the image to be detected.

Here, the banding noise detection model is trained according to the training method of the banding noise detection model described above.

Therefore, whether the stripe noise exists on the image is detected through the stripe noise detection model, the accuracy of stripe noise detection can be improved, and therefore better data support is provided for downstream application capacity.

In some embodiments, the stripe noise detection method may further comprise:

s603: before inputting the image to be detected into the stripe noise detection model, scaling the image to be detected into an image with a target size, and carrying out normalization processing on the image with the target size to obtain a normalized image to be detected.

Here, inputting the image to be detected into the stripe noise detection model includes: and inputting the normalized image to be detected into a stripe noise detection model.

Here, S603 occurs before S601.

Illustratively, the target size is m×m. For example, m=640, that is, scaling the image to be detected into an RGB format image with 640×640 size, and normalizing;

here, normalization includes: the intensity values in the range of 0 to 255 are normalized to between 0 and 1.

Therefore, the speed of detecting the stripe noise can be improved by preprocessing the image to be detected, and the accuracy of detecting the stripe noise can be improved.

In some embodiments, the N predictors include: the confidence coefficient of the predicted value of the coordinate offset of the center point, the predicted value of the probability corresponding to the predicted value of the coordinate offset of the center point, the regression value and 180 angles respectively.

In some embodiments, the stripe noise detection method may further comprise: and under the condition that the confidence coefficient is larger than a preset threshold value, judging that stripe noise exists at the position where the target grid with the confidence coefficient larger than the preset threshold value is located.

Here, the target mesh refers to a mesh whose confidence is greater than a preset threshold.

Here, the preset threshold value may be set or adjusted according to the requirements of the speed, the accuracy, or the like.

Therefore, whether stripe noise exists on each grid of the image can be rapidly determined, and the detection speed of the stripe noise is improved.

In some embodiments, the stripe noise detection method may further comprise: and determining the position of the stripe noise based on the predicted value of the central point coordinate offset of the target grid, the regression value and the predicted probability values corresponding to the 180 angles respectively.

In some embodiments, a maximum probability prediction value is determined based on probability prediction values corresponding to 180 angles, respectively; determining an angle corresponding to the maximum probability predicted value as the angle of the stripe noise; and determining the position of the stripe noise by combining the angle of the stripe noise based on the predicted value and the regression value of the central point coordinates of the target grid.

In some implementations, a priori length values are obtained; determining the length of the stripe noise according to the regression value and the priori length value; and determining the position of the stripe noise based on the predicted value and the regression value of the central point coordinates of the target grid and the length of the stripe noise.

In this way, the precise location of banding noise on the image can be determined, thereby helping to provide accurate data support for downstream applications.

In some embodiments, determining the location of the banding noise based on the predicted value of the center point coordinate offset of the target grid, the regression value, and the predicted value of the probabilities corresponding to the 180 angles, respectively, includes: determining the angle of the stripe noise based on the probability prediction values corresponding to the 180 angles respectively; determining the length of the stripe noise according to the regression value and the priori length value; and determining the position of the stripe noise based on the predicted value of the central point coordinate offset of the target grid and combining the length and the angle.

In this way, the precise location of banding noise on the image can be quickly determined, thereby helping to provide accurate data support for downstream applications.

The invention provides a strip noise detection scheme based on deep learning, which utilizes a large amount of marked and expanded sample data to perform model training and select an optimal parameter model, can effectively and quickly detect the strip noise problem in pictures or videos, and has great reference value and significance for downstream quality evaluation, product recommendation and other applications.

The embodiment of the disclosure provides a stripe noise detection model training device, as shown in fig. 7, which may include: a first obtaining module 701, configured to obtain a first sample data set, where the first sample data set includes a plurality of first sample images, each first sample image includes at least one stripe noise, and the first sample image is divided into a plurality of grids; the second obtaining module 702 is configured to input the first sample image to a model to be trained, obtain N predicted values corresponding to each grid on the first sample image output by the model to be trained, where the N predicted values are at least used for predicting whether stripe noise exists on the grid of the first sample image, the N predicted values are predicted values corresponding to N channels of the model to be trained, and N is a positive integer; the training module 703 is configured to train the model to be trained based on the N predicted values corresponding to each grid of the first sample image and the N true values corresponding to each grid of the first sample image, to obtain a stripe noise detection model.

In an embodiment of the present disclosure, N predicted values in the stripe noise detection model training apparatus include: confidence of the predicted value of the center point coordinate offset, the predicted value of the probability corresponding to the regression value and 180 angles respectively.

In the embodiment of the present disclosure, the training module 703 includes: the first construction submodule is used for constructing a first loss function based on the predicted value of the central point coordinate offset of each grid and the true value of the central point coordinate offset; the second construction submodule is used for constructing a second loss function based on the first-class confidence predicted value of each grid and the confidence value true value of the first-class confidence predicted value, wherein the first-class confidence predicted value is greater than a preset threshold value; the third construction submodule is used for constructing a third loss function based on the second-class confidence predicted value of each grid and the confidence value true value of the second-class confidence predicted value, and the second-class confidence predicted value is the confidence predicted value smaller than or equal to a preset threshold value; the fourth construction submodule is used for constructing a fourth loss function based on the probability prediction value and the probability true value which correspond to the 180 angles of each grid respectively; and the training submodule is used for training the model to be trained based on the first loss function, the second loss function, the third loss function and the fourth loss function.

In an embodiment of the present disclosure, the stripe noise detection model training device further includes: a statistics module 704 (not shown in fig. 7) for counting the length of all stripe noises included in the first sample data set; a clustering module 705 (not shown in fig. 7) is configured to perform clustering based on the lengths of all stripe noises, to obtain K prior length values, where the prior length values are reference lengths of stripe noises used by the stripe noise detection model, and K is an integer not less than 1.

In the embodiment of the disclosure, in the stripe noise detection model training device, N channels of a model to be trained are divided into K groups of channels, and each group of channels comprises 185 channels; the 185 channels comprise 180 angle channels, 1 regression value channel, 1 confidence coefficient channel, 1 stripe noise class channel, 1 center point coordinate X-axis offset channel, 1 center point coordinate Y-axis offset channel, and 180 angle channels are used for predicting probability prediction values corresponding to the 180 angles respectively.

In an embodiment of the present disclosure, the first obtaining module 701 includes: the acquisition sub-module is used for acquiring images with quality lower than a preset quality threshold value; the first generation sub-module is used for marking the image according to a preset marking rule and generating a first sample data set.

In an embodiment of the present disclosure, the first obtaining module 701 includes: a synthesis submodule for synthesizing the inclined stripe noise in the image according to a preset synthesis mode; a second generation sub-module for generating the first sample data set based on the synthesized oblique stripe noise.

In the embodiment of the disclosure, the synthesis submodule is specifically configured to: randomly taking x break points on the line direction central line of each image; wherein xi is the coordinate value of the discontinuity point on the line-direction central line of the image; for each discontinuity xi, calculating a row-by-row offset from top to bottom along the column direction of the image; acquiring row-by-row offset points based on the offset; and connecting the offset points line by line to form a straight line, wherein the break point is the center point of the straight line.

In the embodiment of the disclosure, the synthesis submodule is further specifically configured to: if the number of the break points is 1, multiplying all pixel values from a straight line taking the break point as a center to the tail end of the image in the row direction by a first random number and adding a second random number; if the number of the break points is greater than 1, taking two break points as a group, acquiring two straight lines by each group of break points, multiplying all pixel values in the middle range of the two straight lines with the two break points as the center by a first random number and adding a second random number; for all pixel values from a straight line centered at the last discontinuity to the end of the line direction of the image, the third random number is multiplied and the fourth random number is added.

In the embodiment of the disclosure, the synthesis submodule is further specifically configured to: if the number of the break points is 1, calculating offset from row to row, and when (xi+offsetxj) <0, the abscissa of the offset point is 0; when (xi+offsetxj) > m, the offset point is not counted, wherein offsetxj represents the offset amount and m represents the image size; if the number of the break points is greater than 1, taking two adjacent break points from left to right as a group, for the left break point, calculating offset from row to row, and when (xi+offsetxj) <0, the abscissa of the offset point is 0; when (xi+offsetxj) > m, the offset point is not counted; for the right break point, calculating offset from row to row, and when (xi+offsetxj) <0, not counting offset points; when (xi+offsetxj) > m, the abscissa of the offset point is m.

In any one embodiment of the disclosed embodiment, the stripe noise detection model training device further includes: a second acquisition module 706 (not shown in fig. 7) for acquiring a second sample data set comprising a plurality of second sample images; at least one second image sample of the plurality of second sample images does not include banding noise; a verification module 707 (not shown in fig. 7) for verifying the plurality of stripe-noise detection models based on the second sample data set in response to training to obtain the plurality of stripe-noise detection models; a first determining module 708 (not shown in fig. 7) is configured to use the stripe noise detection model with the optimal verification result as a final stripe noise detection model.

It should be understood by those skilled in the art that the functions of each processing module in the stripe noise detection model training device according to the embodiments of the present disclosure may be understood with reference to the foregoing description of the stripe noise detection model training method, and each processing module in the stripe noise detection model training device according to the embodiments of the present disclosure may be implemented by an analog circuit implementing the functions of the embodiments of the present disclosure, or may be implemented by running software that performs the functions of the embodiments of the present disclosure on an electronic device.

The device for training the stripe noise detection model can improve the precision of the trained stripe noise detection model, so that the stripe noise problem in an image can be efficiently and rapidly detected.

The embodiment of the disclosure provides a stripe noise detection device, as shown in fig. 8, which includes: a third acquiring module 801, configured to acquire an image to be detected; the detection module 802 is configured to input an image to be detected into a stripe noise detection model to obtain a stripe noise detection result of the image to be detected, where the stripe noise detection model is trained by using the stripe noise detection model training device according to any one of the above claims, the stripe noise detection result includes N predicted values corresponding to each grid on the image to be detected, the N predicted values are at least used for predicting whether stripe noise exists on the grid of the image to be detected, the N predicted values are predicted values corresponding to N channels of the stripe noise detection model, and N is a positive integer.

In the embodiment of the disclosure, the stripe noise detection device further includes a processing module 803 (not shown in fig. 8) configured to scale the image to be detected into an image of a target size and normalize the image of the target size to obtain a normalized image to be detected before inputting the image to be detected into the stripe noise detection model. The detection module 802 includes: and inputting the normalized image to be detected into a stripe noise detection model.

In an embodiment of the present disclosure, the band noise detection apparatus, the N predicted values include: confidence of the predicted value of the center point coordinate offset, the predicted value of the probability corresponding to the regression value and 180 angles respectively.

In an embodiment of the disclosure, the stripe noise detection device further includes: a determining module 804 (not shown in fig. 8) is configured to determine that the stripe noise exists at the location of the target grid where the confidence is greater than the preset threshold if the confidence is greater than the preset threshold.

In an embodiment of the present disclosure, the stripe noise detection device may further include: a second determining module 805 (not shown in fig. 8) is configured to determine a location of the stripe noise based on the predicted value of the center point coordinate offset of the target grid, the regression value, and the predicted probability values corresponding to the 180 angles, respectively.

In an embodiment of the present disclosure, the second determining module 805 (not shown in fig. 8) includes: the first determining submodule is used for determining angles of the stripe noise based on probability prediction values corresponding to the 180 angles respectively; the second determining submodule is used for determining the length of the stripe noise according to the regression value and the priori length value; and the third determination submodule is used for determining the position of the stripe noise based on the central point coordinate offset predicted value of the target grid and combining the length and the angle.

It should be understood by those skilled in the art that the functions of each processing module in the stripe noise detection device according to the embodiments of the present disclosure may be understood by referring to the foregoing description of the stripe noise detection method, and each processing module in the stripe noise detection device according to the embodiments of the present disclosure may be implemented by using an analog circuit that implements the functions of the embodiments of the present disclosure, or may be implemented by running software that implements the functions of the embodiments of the present disclosure on an electronic device.

The stripe noise detection device disclosed by the embodiment of the invention can efficiently and accurately detect the stripe noise problem in the image.

The embodiment of the disclosure provides a scene diagram of a stripe noise detection model training, as shown in fig. 9.

As described above, the method for training the stripe noise detection model provided by the embodiment of the present disclosure is applied to an electronic device. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile apparatuses, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing apparatuses.

The electronic device performs the steps of: acquiring a first sample data set comprising a plurality of first sample images, each first sample image comprising at least one stripe noise, the first sample images being divided into a plurality of grids; inputting the first sample image into a model to be trained to obtain N predicted values corresponding to grids on the first sample image output by the model to be trained, wherein the N predicted values are at least used for predicting whether stripe noise exists on the grids of the first sample image, the N predicted values are predicted values corresponding to N channels of the model to be trained, and N is a positive integer; and training the model to be trained based on the N predicted values corresponding to each grid of the first sample image and the N true values corresponding to each grid of the first sample image to obtain the stripe noise detection model.

Wherein the first sample data set and the plurality of first sample images comprised by the first sample data set may be acquired from an image data source. The image data source may be various forms of data storage devices, such as a laptop computer, a desktop computer, a workstation, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The data sources may also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. Furthermore, the image data source and the user terminal may be the same device.

It should be understood that the scene diagram shown in fig. 9 is merely illustrative and not restrictive, and that various obvious changes and/or substitutions may be made by one skilled in the art based on the example of fig. 9, and the resulting technical solutions still fall within the scope of the disclosure of the embodiments of the present disclosure.

The embodiment of the disclosure also provides a scene diagram of stripe noise detection, as shown in fig. 10.

As described above, the stripe noise detection method provided by the embodiment of the present disclosure is applied to an electronic device. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile apparatuses, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing apparatuses.

The electronic device performs the steps of: acquiring an image to be detected; inputting an image to be detected into a stripe noise detection model to obtain a stripe noise detection result of the image to be detected, wherein the stripe noise detection model is trained by utilizing the stripe noise detection model training method according to any one of the stripe noise detection models, the stripe noise detection result comprises N predicted values corresponding to each grid on the image to be detected, the N predicted values are at least used for predicting whether stripe noise exists on the grids of the image to be detected, the N predicted values are predicted values corresponding to N channels of the stripe noise detection model, and N is a positive integer.

Wherein the image to be detected may be acquired from a data source. The image data source may be various forms of data storage devices, such as a laptop computer, a desktop computer, a workstation, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The data sources may also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. Furthermore, the image data source and the user terminal may be the same device.

It should be understood that the scene diagram shown in fig. 10 is merely illustrative and not restrictive, and that various obvious changes and/or substitutions may be made by one skilled in the art based on the example of fig. 10, and the resulting technical solutions still fall within the scope of the disclosure of the embodiments of the present disclosure.

In the technical scheme of the disclosure, the acquisition, storage, application and the like of the related user personal information all conform to the regulations of related laws and regulations, and the public sequence is not violated.

According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

Fig. 11 illustrates a schematic block diagram of an example electronic device 1100 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile apparatuses, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing apparatuses. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 11, the apparatus 1100 includes a computing unit 1101 that can perform various appropriate actions and processes according to a computer program stored in a Read-Only Memory (ROM) 1102 or a computer program loaded from a storage unit 1108 into a random access Memory (Random Access Memory, RAM) 1103. In the RAM1103, various programs and data required for the operation of the device 1100 can also be stored. The computing unit 1101, ROM1102, and RAM1103 are connected to each other by a bus 1104. An Input/Output (I/O) interface 1105 is also connected to bus 1104.

Various components in device 1100 are connected to I/O interface 1105, including: an input unit 1106 such as a keyboard, a mouse, etc.; an output unit 1107 such as various types of displays, speakers, and the like; a storage unit 1108, such as a magnetic disk, optical disk, etc.; and a communication unit 1109 such as a network card, modem, wireless communication transceiver, or the like. The communication unit 1109 allows the device 1100 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunication networks.

The computing unit 1101 may be a variety of general purpose and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 1101 include, but are not limited to, a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU), various dedicated artificial intelligence (Artificial Intelligence, AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (Digital Signal Processor, DSP), and any suitable processors, controllers, microcontrollers, etc. The calculation unit 1101 performs the respective methods and processes described above, such as the banding noise detection model training method/the banding noise detection method. For example, in some embodiments, the banding noise detection model training method/banding noise detection method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 1108. In some embodiments, some or all of the computer programs may be loaded and/or installed onto device 1100 via ROM1102 and/or communication unit 1109. When the computer program is loaded into the RAM1103 and executed by the computing unit 1101, one or more steps of the banding noise detection model training method/banding noise detection method described above may be performed. Alternatively, in other embodiments, the computing unit 1101 may be configured to perform the stripe noise detection model training method/stripe noise detection method by any other suitable means (e.g. by means of firmware).

Various implementations of the systems and techniques described here above can be implemented in digital electronic circuitry, integrated circuitry, field programmable gate arrays (Field Programmable Gate Array, FPGAs), application specific integrated circuits (Application Specific Integrated Circuit, ASICs), application-specific standard products (ASSPs), system On Chip (SOC), complex programmable logic devices (Complex Programmable Logic Device, CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable stripe noise detection model training apparatus, such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access Memory, a read-Only Memory, an erasable programmable read-Only Memory (EPROM), a flash Memory, an optical fiber, a portable compact disc read-Only Memory (Compact Disk Read Only Memory, CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., cathode Ray Tube (CRT) or liquid crystal display (Liquid Crystal Display, LCD) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local area network (Local Area Network, LAN), wide area network (Wide Area Network, WAN) and the internet.

The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server incorporating a blockchain.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel, sequentially, or in a different order, provided that the desired results of the disclosed aspects are achieved, and are not limited herein.

The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions, improvements, etc. that are within the principles of the present disclosure are intended to be included within the scope of the present disclosure.

Claims

1. A method of training a banding noise detection model, comprising:

obtaining a first sample data set comprising a plurality of first sample images, each of the first sample images comprising at least one stripe noise, the first sample images being divided into a plurality of grids;

inputting the first sample image into a model to be trained to obtain N predicted values corresponding to grids on the first sample image output by the model to be trained, wherein the N predicted values are at least used for predicting whether stripe noise exists on the grids of the first sample image, the N predicted values are predicted values corresponding to N channels of the model to be trained, and N is a positive integer;

And training the model to be trained based on the N predicted values corresponding to each grid of the first sample image and the N true values corresponding to each grid of the first sample image to obtain a stripe noise detection model.

2. The method of claim 1, wherein the N predictors include: the confidence coefficients of the predicted value of the coordinate offset of the center point, the predicted value of the probability corresponding to the regression value and 180 angles respectively;

the training the model to be trained based on the N predicted values corresponding to each grid of the first sample image and N true values corresponding to each grid of the first sample image includes:

constructing a first loss function based on the predicted value of the center point coordinate offset and the true value of the center point coordinate offset of each grid;

constructing a second loss function based on the first type confidence predicted value of each grid and the confidence value true value of the first type confidence predicted value, wherein the first type confidence predicted value is a confidence predicted value larger than a preset threshold value;

constructing a third loss function based on the second class confidence predicted value of each grid and the confidence value true value of the second class confidence predicted value, wherein the second class confidence predicted value is a confidence predicted value smaller than or equal to the preset threshold value;

Constructing a fourth loss function based on the probability prediction value and the probability true value which correspond to the 180 angles of each grid respectively;

the model to be trained is trained based on the first, second, third, and fourth loss functions.

3. The method of claim 1, further comprising:

counting the length of all stripe noises included in the first sample data set;

and clustering based on the lengths of all the stripe noises to obtain K priori length values, wherein the priori length values are the reference lengths of the stripe noises adopted by the stripe noise detection model, and K is an integer not less than 1.

4. A method according to claim 3, wherein the N channels of the model to be trained are divided into K groups of channels, each group comprising 185 channels; the 185 channels comprise 180 angle channels, 1 regression value channel, 1 confidence coefficient channel, 1 stripe noise class channel, 1 center point coordinate X-axis offset channel and 1 center point coordinate Y-axis offset channel, and the 180 angle channels are used for predicting probability prediction values corresponding to the 180 angles respectively.

5. The method of claim 1, wherein the acquiring the first sample data set comprises:

Acquiring an image with quality lower than a preset quality threshold;

and labeling the image according to a preset labeling rule to generate the first sample data set.

6. The method of claim 1, wherein the acquiring the first sample data set comprises:

synthesizing the inclined stripe noise in the image according to a preset synthesis mode;

the first sample data set is generated based on the synthesized oblique stripe noise.

7. The method of claim 6, wherein synthesizing the oblique stripe noise in the image in a preset synthesis manner, comprises:

randomly taking w break points on the central line of each image in the row direction; wherein xi is the coordinate value of the discontinuity point on the line-direction central line of the image;

for each discontinuity xi, calculating a row-by-row offset from top to bottom along a column direction of the image;

acquiring a progressive offset point based on the offset;

and connecting the progressive offset points to form a straight line, wherein the break point is the center point of the straight line.

8. The method of claim 7, wherein the obtaining a progressive offset point based on the size of the offset comprises:

if the number of the break points is 1, multiplying all pixel values from a straight line taking the break point as a center to the tail end of the image in the row direction by a first random number and adding a second random number;

If the number of the break points is greater than 1, taking two break points as a group, acquiring two straight lines by each group of break points, multiplying all pixel values in the middle range of the two straight lines with the two break points as the center by the first random number and adding the second random number; for all pixel values from a straight line centered at the last discontinuity to the end of the line direction of the image, the third random number is multiplied and the fourth random number is added.

9. The method of claim 7, wherein the obtaining a progressive offset point based on the size of the offset further comprises:

10. The method of any one of claims 1 to 9, further comprising:

obtaining a second sample dataset comprising a plurality of second sample images; at least one second image sample of the plurality of second sample images does not include banding noise;

obtaining a plurality of stripe-noise detection models in response to the training, validating the plurality of stripe-noise detection models based on the second sample dataset;

and taking the stripe noise detection model with the optimal verification result as a final stripe noise detection model.

11. A method of stripe noise detection, comprising:

acquiring an image to be detected;

inputting the image to be detected into a stripe noise detection model to obtain a stripe noise detection result of the image to be detected, wherein the stripe noise detection model is trained by the method according to any one of claims 1 to 10, the stripe noise detection result comprises N predicted values corresponding to each grid on the image to be detected, the N predicted values are at least used for predicting whether stripe noise exists on the grid of the image to be detected, the N predicted values are predicted values corresponding to N channels of the stripe noise detection model, and N is a positive integer.

12. The method of claim 11, further comprising:

before inputting the image to be detected into the stripe noise detection model, scaling the image to be detected into an image with a target size, and carrying out normalization processing on the image with the target size to obtain a normalized image to be detected;

wherein the inputting the image to be detected into the banding noise detection model includes:

and inputting the normalized image to be detected into the stripe noise detection model.

13. The method of claim 11, wherein the N predictors include:

the confidence coefficients of the predicted value of the coordinate offset of the center point, the predicted value of the probability corresponding to the regression value and 180 angles respectively;

the method further comprises the steps of:

and under the condition that the confidence coefficient is larger than a preset threshold value, judging that stripe noise exists at the position where the target grid with the confidence coefficient larger than the preset threshold value is located.

14. The method of claim 13, further comprising:

and determining the position of the stripe noise based on the predicted value of the coordinate offset of the central point of the target grid, the regression value and the predicted probability values corresponding to the 180 angles respectively.

15. The method of claim 14, wherein the determining the location of the banding noise based on the center point coordinate offset predictor, the regression value, and the probability predictor for each of the 180 angles of the target grid comprises:

determining the angle of the stripe noise based on the probability prediction values corresponding to the 180 angles respectively;

determining the length of the stripe noise according to the regression value and the priori length value;

and determining the position of the stripe noise based on the central point coordinate offset predicted value of the target grid by combining the angle of the stripe noise and the length of the stripe noise.

16. A banding noise detection model training device, comprising:

a first acquisition module for acquiring a first sample data set comprising a plurality of first sample images, each of the first sample images comprising at least one stripe noise, the first sample images being divided into a plurality of grids;

the second acquisition module is used for inputting the first sample image into a to-be-trained model to obtain N predicted values corresponding to grids on the first sample image output by the to-be-trained model, wherein the N predicted values are at least used for predicting whether stripe noise exists on the grids of the first sample image, the N predicted values are predicted values corresponding to N channels of the to-be-trained model, and N is a positive integer;

And the training module is used for training the model to be trained based on the N predicted values corresponding to each grid of the first sample image and the N true values corresponding to each grid of the first sample image to obtain a stripe noise detection model.

17. A stripe noise detection device comprising:

the third acquisition module is used for acquiring an image to be detected;

the detection module is configured to input the image to be detected into a stripe noise detection model to obtain a stripe noise detection result of the image to be detected, where the stripe noise detection model is trained by using the method according to any one of claims 1 to 10, the stripe noise detection result includes N predicted values corresponding to each grid on the image to be detected, the N predicted values are at least used for predicting whether stripe noise exists on the grid of the image to be detected, and the N predicted values are predicted values corresponding to N channels of the stripe noise detection model, and N is a positive integer.

18. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 15.

19. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1 to 15.

20. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 15.