CN105405109A - Dirty spot detection method based on zonal background modeling - Google Patents

Abstract

The invention discloses a dirty point detection method based on strip background modeling. This method adopts a background modeling algorithm different from the past, adds image denoising, image segmentation and mathematical morphology methods, and reduces the false detection rate. The accurate detection rate and its practicality are improved. The implementation process is as follows: firstly input the dirty point detection image collected from a certain camera module, and then preprocess the image through grayscale transformation, bilateral filter and image cropping to obtain the original image. Secondly, the background model of the original image is established by using the strip background modeling algorithm proposed by the present invention. Subtract the original image and the obtained background model to obtain the difference image, set an appropriate threshold to remove some error points of the difference image, and screen out potential dirty points. Finally, according to the characteristics of dirty point imaging, mathematical morphology technology is used to eliminate isolated and discrete noise points in the difference image, and finally detect dirty points.

Description

A Dirty Spot Detection Method Based on Strip Background Modeling

technical field

The invention belongs to the technical field of image recognition, relates to image denoising, image background modeling and mathematical morphology technology, and can be used in the field of target detection.

Background technique

In recent years, with the increasing popularity of digital products with camera functions such as smartphones and tablet computers, the demand for mobile phone camera modules is increasing. In the actual production process, the detection of dirty spots on the camera module is an important means to ensure product quality. However, with the gradual increase in the output of modules, the manual dirty spot detection method can no longer meet the demand. How to realize the automation of dirty spot detection , Improving the yield rate is of concern to people. The dirty point detection image is an important basis for dirty point detection. How to process the dirty point detection image according to the imaging principle and characteristics of the dirty point detection image and the characteristics of the image background light intensity distribution is particularly important for the accuracy of detection. Based on this, the banded background modeling dirty point detection algorithm is proposed, which is composed of image denoising, image background modeling, image algebraic operation and mathematical morphology technology; firstly, the dirty point detection image is preprocessed; The point detection image is modeled as a strip background, and potential dirty points are screened out. Finally, the mathematical morphology technology is used to detect the dirty points, so as to realize the automation of dirty point detection and improve the accuracy of detection.

Usually, the dirty point detection image is obtained by placing a white transparent shield in front of the camera module for shooting. Imaging smudges are essentially shadows caused by dust particles that fall in front of the imaging sensor. Based on the principle of dirty point imaging combined with the observation of a large number of dirty point imaging pictures, it can be concluded that the characteristics of dirty point imaging are: (1) causing attenuation of light intensity, causing shadows on the image, and the depth distribution of shadows is uneven. The smaller the brightness near the center of the dirty point, the greater the light intensity attenuation; (2) Most of them are circular, and the shape of the dirty point is similar to the shape of the aperture. In addition, because the distribution of external light is not absolutely uniform, the position of the white occluder is not absolutely parallel to the focal plane of the camera, resulting in the uneven distribution of background light intensity in the dirty point detection image. In fact, even if the external light is completely evenly distributed, because the camera inevitably has dark corners, it is still impossible to obtain a background image with uniform background light intensity. In this way, the above-mentioned imaging characteristics of the dirty point detection image and the uneven distribution of background light intensity have brought difficulties to the automatic detection of dirty spots. This problem can be described as: it is difficult to use a uniform brightness standard to judge whether there are dirty spots in the camera ;Because the size of the foreign matter cannot be determined, the size of the generated dirty spots is relatively random. Even for the same foreign matter, different cameras will cause differences in the size of the dirty spots due to the different aperture sizes. It is difficult to use a uniform shape standard to judge whether it is Dirty dots; both vignetting and dirty dots are indistinguishable from each other because they darken areas of the image. Therefore, to solve such problems, first of all, it is necessary to avoid the influence of the vignetting of the dirty point detection image on the detection effect, that is, to smooth the edge of the image and crop some pixels on the edge of the image. It is necessary to use a bilateral filter to denoise the image. The obtained image is used as the original image. Secondly, the polynomial surface fitting technology is used to fit the image to establish a background model, which reflects the characteristics of the image from a mathematical point of view. By subtracting the fitted image from the original image, the prediction error of each point is calculated, and the potential dirt is screened out. point to obtain the difference image. Finally, the mathematical morphology technology is used to detect potential dirty points, remove isolated and discrete noise points, detect dirty points, and obtain a binary image of dirty points. Therefore, sequentially using image denoising, polynomial surface fitting to establish a model, image algebraic operations, and mathematical morphology technology to process dirty point detection images is an important basis for realizing automatic detection of dirty points and improving the accuracy of dirty point detection. What kind of fitting method to model the background of the dirty point detection image and reduce the prediction error has become the key to solving this problem.

Image denoising is an important preprocessing method in image processing. In the process of image acquisition, transmission and storage, it is always unavoidable to be interfered by various noise sources. In order to obtain more accurate information from the image, selecting an appropriate image denoising preprocessing algorithm becomes the key to subsequent processing. As a basic feature of images, edges provide an important feature parameter for people to describe or recognize objects and interpret images. Therefore, the basic goal of image denoising is to suppress and remove noise while trying not to damage or lose image edges. The bilateral filter is a non-linear filtering technique that can well preserve details such as image edges while denoising. The bilateral filter is a Gaussian filter function based on spatial distribution. Compared with the Gaussian filter, a Gaussian kernel function based on the difference in pixel gray value is added, which ensures the preservation of pixel values near the edge. It not only considers the proximity of space but also the similarity of gray values. Only those with similar gray values in the neighborhood are averaged together, which is more in line with the visual habits of the human eye. Therefore, the present invention uses a bilateral filter to process the dirty point detection image to achieve the purpose of edge preservation and denoising to obtain a smooth image.

Background modeling is also called background estimation. Its main purpose is to transform the detection of moving objects in video frame images into a binary classification problem based on the current estimated background, and classify all pixels into background or moving foreground, and then classify The results are post-processed to obtain the final detection results. The basic idea of the background modeling algorithm is to model the background of the image. Once the background model is established, the current image is compared with the background model, and the foreground target is determined according to the comparison result. Some commonly used algorithms for background modeling in foreground detection include inter-frame difference method, complex background modeling based on Bayesian theory and background estimation based on time-axis filtering. The background modeling algorithm is mainly used in the detection of moving objects, and it is also used for static scenes such as static backgrounds and no moving objects. Since static images belong to static scenes, the present invention adopts the idea of background modeling to carry out background modeling on dirty point detection images. for modeling.

Surface fitting is an important research topic in Computer-Aided Geometric Design (CAGD), and it has a wide range of applications in computer graphics, reverse engineering, numerical calculation and so on. Fitting is usually implemented in two ways, interpolation and approximation. The basic idea of the polynomial surface fitting algorithm is to find the analytical formula between the function f(x, y) and the variables x and y according to the actual experimental test data, so that the determined surface passes or approximately passes the experimental test points. That is to say, all experimental data points can be approximately distributed on the space surface represented by the function f(x,y). Considering the characteristics of the dirty point detection image, directly fitting the entire detection image cannot fully reflect the relationship between the pixel values in the image. The strip background modeling algorithm proposed in the present invention is to first divide the original image into several Then set the best fitting parameters to fit the obtained strip image blocks. There are two main differences from the general background modeling algorithm. On the one hand, we do not directly model the background of the image, but first divide the image into several image blocks, and then model each image block. Then splicing the built image block background model into a whole block model. On the other hand, we model the background of the image through a polynomial surface fitting algorithm. The specific algorithm can be described as: by setting an image block of a certain size as the segmentation reference, the original image is divided into several image blocks whose size is equal to the segmentation reference, and then the polynomial surface fitting algorithm is used to sequentially fit each image block , splicing the image blocks according to the order of segmentation to obtain a fitted image, and the fitted image is the background model. Experimental results show that the banded background modeling algorithm proposed by the present invention has an excellent fitting effect, enables smooth transition of the curved surface, and can reduce the influence of points with large errors. Finally, according to the idea of the background modeling algorithm, compare the original image with the background model, that is, subtract the fitted image from the original image, so as to calculate the prediction error of each point in the image and obtain the difference image, so as to filter out potential dirty points in the future. .

Mathematical Morphology is a subject based on set theory, which is very suitable for the geometric shape analysis and description of signals. The basic idea is to use structural elements to "probe" the signal, retain the main shape, and delete the irrelevant shape (such as noise, glitch). As the structural element of the probe, it can directly carry knowledge, such as direction, size, chromaticity and other information, to detect and study the structural features containing the main information of the signal. Different structural elements can get different results. Morphological and difference operations, namely dilation and erosion, are the basis of mathematical morphology. Mathematical morphology first deals with binary images, which is called binary mathematical morphology (BinaryMorphology). Binary mathematical morphology is a processing procedure for sets. The essence of its morphological operator is to express the interaction between a collection of objects or shapes and structural elements, and the shape of the structural elements determines the shape information of the signal extracted by this operation. Morphological image processing is to move a structural element in the image, and then perform set operations such as intersection and union on the structural element and the binary image below. The process of first erosion and then expansion is called opening operation. It has the effect of eliminating small objects, separating objects in thin places and smoothing the boundaries of larger objects. The present invention uses an open operation to remove isolated and discrete noise points and improve the accuracy of dirty point detection.

Contents of the invention

The purpose of the present invention is to propose a stripped background modeling dirty point detection algorithm by improving the low versatility of the dirty point detection algorithm based on differential technology and the high misjudgment rate of the dirty point detection algorithm based on target modeling . The algorithm consists of image denoising, image segmentation, image background modeling and mathematical morphology techniques. Firstly, the bilateral filter is used to preprocess the dirty point detection image, which can keep the edge properties of the image unchanged and remove image noise, and then crop some pixels on the edge of the image, which helps to avoid dark corners or dark edges of the dirty point detection image. effect, and improve the accuracy of detection. Secondly, in the background modeling stage of the dirty point detection image, considering that the modeling effect of the dirty point detection image directly affects the subsequent screening of potential dirty points, we designed a banded background modeling algorithm, which is not for the whole Instead of performing background modeling on a single image, a model is built on image blocks that are divided into several bands. And different from the method adopted by the general background modeling, the polynomial surface fitting method is used to fit the segmented image blocks. Such a fine-tuned fitting of the dirty point detection image obtains a better fitting effect, which is beneficial to reduce the fitting The prediction error between the image and the original image achieves the purpose of filtering out dirty spots as accurately as possible. Finally, using mathematical morphology technology to remove isolated and discrete noise points can not only improve the accuracy of dirty point detection but also show the shape of dirty points more clearly. Therefore, using this algorithm, the misjudgment rate of dirty point detection can be reduced, and the efficiency of dirty point detection can be improved. Moreover, compared with the dirty point detection algorithm based on difference technology and the dirty point detection algorithm based on object modeling, this algorithm has higher practicability and universality.

The technical solution of the present invention is to firstly input the dirty point detection image collected from a certain camera module, transform the dirty point detection image into a grayscale image through grayscale transformation, and use a bilateral filter to process the grayscale image. Noise is removed from the edges of the image, making the image smoother. In order to avoid misjudgment caused by dark corners or dark edges of the grayscale image, some pixels on the edge of the smooth grayscale image are cut off, and the cropped grayscale image is used as the original image. In the stage of establishing the background model for the original image, considering that the light intensity distribution of the original image and the depth distribution of the dirty points are not uniform, if the entire original image is directly modeled, that is, the entire original image is directly fitted, it cannot The original surface is fully approximated, and there is a serious underfitting phenomenon. We designed a banded background modeling algorithm, which can be specifically described as: set an image block of an appropriate size as a segmentation template, and segment the image in turn according to the size of the template until the original image is inseparable, and obtain several banded image blocks; Set appropriate polynomial fitting parameters, and fit these image blocks one by one according to the order of segmentation. By segmenting the image first and then fitting, the fitting image can be refined and a better fitting effect can be obtained. After the above operations, the fitted image blocks are spliced into a fitting image with the same size as the original image in the order of segmentation, that is to say, the background model of the entire original image is established. Subtract the original image and the fitted image, that is, compare the original image with the background model to obtain the difference image, set an appropriate threshold to remove some error points of the difference image, and screen out potential dirty points. In order to screen out potential dirty points more accurately, according to the characteristics of dirty point imaging, the present invention uses mathematical morphology technology to eliminate isolated and discrete noise points in the difference image, and finally detects potential dirty points.

Specific steps are as follows:

1. Input a dirty point detection image collected from a camera module, transform the dirty point detection image into a grayscale image and process it into a smooth grayscale image through a bilateral filter. Crop the edge pixels of the grayscale image, and use the cropped grayscale image as the original image.

Dirty point detection images can be collected by different camera modules.

Wherein, the edge pixel values of the cropped grayscale image can be set according to the characteristics of the distribution of dirty points in the dirty point detection image.

Two, adopt the strip background modeling algorithm that the present invention proposes, set the image block of appropriate size as segmentation template, according to the size of this template, the above-mentioned original image that obtains is divided into several strip image blocks successively, until the size of image is less than The template cannot be subdivided so far. Fit the segmented image blocks one by one according to the sequence of segmentation, stitch the image blocks in turn to obtain the fitted image, and establish the background model of the entire original image.

3. The strip background modeling algorithm proposed by the present invention performs subtraction operation on the original image and the background model obtained above, calculates the prediction error of each point corresponding to the two images, and obtains the difference image.

4. By setting an appropriate threshold, perform threshold processing on the difference image to screen out potential dirty points.

The threshold can be set to a certain value according to the obtained prediction error value, and then the threshold is adjusted according to the detection effect. Through a large number of experiments, the threshold can be adjusted to an appropriate value.

Among them, the threshold processing can be described as: mark the points whose error value is negative and whose absolute value is greater than the threshold as potential dirty points, and assign the point a value of 1; points that do not meet the above conditions are regarded as non-potential dirty points, and These points are assigned a value of 0. The final difference image becomes a binary image.

5. The binary image obtained above is processed by mathematical morphology technology, the area of potential dirty points is detected, and the isolated and discrete noise points in the binary image are removed, and the remaining potential dirty points can be regarded as dirty points.

Aiming at the imaging principle, imaging characteristics and light intensity distribution characteristics of dirty point detection images, the present invention proposes a background modeling dirty point detection algorithm, which combines image denoising, polynomial surface fitting, algebraic operation of images and mathematical morphology In this way, the dirty point detection image is processed step by step, which avoids the influence of dark corners or dark edges of the dirty point detection image on the dirty point detection, and realizes automatic detection of dirty points. In order to further improve the accuracy of dirty point detection, based on the background modeling algorithm, a banded background modeling algorithm is proposed, which reduces the prediction error, screens out potential dirty points more accurately, and improves the efficiency of dirty point detection. .

Description of drawings

Fig. 1 is a flow chart of the dirty point detection method for strip background modeling in the present invention.

Fig. 2 is a flow chart of the modeling algorithm of the striped background in the present invention.

Figure 3 A dirty point detection image of a camera.

Fig. 4 is an effect diagram of the original image A obtained after the present invention preprocesses a dirty point detection image.

Fig. 5 The present invention adopts the designed strip background modeling algorithm to establish the background model of the original image A, that is, the effect diagram of the fitted image B.

Fig. 6 is an effect diagram of difference image C obtained after the original image A and original image B are processed by the present invention using the strip background modeling algorithm.

FIG. 7 is an effect diagram of obtaining a binary image E of dirty spots after processing the binary image D of latent dirty spots by using mathematical morphology in the present invention.

detailed description

Specific embodiments of the present invention will be further described in detail below.

As shown in Fig. 1, a kind of flow chart of the dirty point detection method of belt-shaped background modeling of the present invention, first input a dirty point detection image, the dirty point detection image is preprocessed, this process can be described as: through grayscale transformation The dirty point detection image is converted into a grayscale image, and the grayscale image is edge-preserved and denoised by a bilateral filter to obtain a smooth grayscale image, and then some pixels on the edge of the image are cut to obtain the original image.

The next step is the process of building a background model of the original image. As shown in Fig. 2 band background modeling algorithm flowchart of the present invention:

Input parameters: original image A;

Output result: Fitting image B.

Set an image block of an appropriate size as a segmentation template, that is, keep the height of the spline consistent with the height of image A, and only set its width to an appropriate pixel value c, then the template is an image block whose height is equal to the original image and whose width is c . The specific segmentation method is: keep the height of the image unchanged, and only divide the width of the image. That is, in the direction perpendicular to the image width, starting from the leftmost edge of the image, ensure that the image is sequentially segmented with the width c of each spline as the segmentation unit, until the image width is inseparable, and several strip-like image blocks are obtained.

(2) Set an appropriate polynomial fitting parameter f, perform polynomial surface fitting on each obtained image block according to the order of segmentation, and obtain several fitting image blocks.

(3) According to the order of fitting, the fitted image blocks are spliced into a fitted image B, thereby establishing the background model of the entire original image A.

The basic idea of the banded background modeling algorithm proposed by the present invention is: considering that the original image still has the characteristics of uneven light intensity distribution and dirty point depth distribution, directly establish a background model for the entire original image, that is, use a polynomial surface If the fitting algorithm directly fits the entire image, it cannot fully reflect the physical characteristics of the original image, and serious underfitting phenomena will occur, and the fitting effect cannot meet the needs of detection. We do not directly build a background model for the entire original image, but adopt a local modeling method, that is, first divide the original image into several strip-shaped image blocks, build models for these image blocks one by one, and finally stitch the background of the image blocks in turn The model is the background model of the entire original image. Different from the usual background modeling algorithm, we do not directly model the image, but divide the original image into several strip-shaped image blocks sequentially by setting a certain size of image block as a segmentation template. After the above segmentation stage, we adopt a method different from the general background modeling algorithm to model each image block, that is, set appropriate polynomial fitting parameters, and perform polynomial surface fitting on these image blocks. Such a refined fitting image can more fully reflect the physical characteristics of each area of the original image, reduce the possibility of underfitting, and achieve the desired fitting effect. Finally, the fitted images are spliced sequentially, that is, the background model of each image block is spliced to obtain the background model of the entire original image. Among them, how many blocks should the original image be divided into? How many polynomial fits are performed? The adjustment can be made according to the fitting effect of the experiment, and the present invention obtains appropriate empirical values as the values of these parameters through multiple experiments. Finally, the fitted image blocks are spliced into a fitted image according to the sequence of segmentation to ensure that the size of the fitted image is equal to that of the original image.

The next step is to compare the original image A with the fitted image B, that is, perform subtraction, calculate the prediction error value e of each pixel corresponding to the two images, and obtain the difference image C.

Secondly, the next step is to set the threshold d, and mark the points whose prediction error value is negative and whose absolute value is greater than the threshold d as potential dirty points, that is, the pixel value of the potential dirty points becomes 1; the points that do not meet the above conditions are marked as Non-potentially dirty points, that is, the pixel values of non-potentially dirty points become 0. In this way, the difference image becomes a latent dirty point binary image D from a grayscale image.

Since there are some non-dirty noise points in the potential dirty point binary image D, according to the characteristics of dirty points, the next step of the present invention is to use mathematical morphology algorithm to detect the marked potential dirty points, and remove isolated and discrete points by opening operation. Noise points and dirty points are detected. Among them, the present invention detects potential dirty points by using an open operation, which refers to a process of first expanding and then corroding. The specific operation method is as follows: firstly perform two dilation operations, and then perform two erosion operations to smooth the contours of potential dirty points, disconnect narrow connections and eliminate thin protrusions. The reason why the present invention adopts this kind of operation is that it has the function of eliminating small objects, separating objects at slender places and smoothing the boundaries of larger objects, which can meet the requirements of eliminating small noise points and detecting dirty points, and finally obtains dirty points Value image E.

The above content is the detailed steps and implementation method of the strip background modeling dirty point detection algorithm of the present invention. Any changes made by those skilled in the art without departing from the concept of the present invention fall within the protection scope of the present invention.

Claims (6)

1., based on a kind of dirty point detecting method of banded background modeling, the method comprises the following steps:

1) input a dirty some detected image, greyscale transformation is carried out to image and obtains gray level image, then by two-sided filter, gray level image is treated to level and smooth gray level image; And cut out the partial pixel point going to smooth grey image edge, using the gray level image of acquisition as original image;

2) image block arranging suitable size, as segmentation template, according to the size of this template, from the original image upper left corner, is divided into some band-like image blocks successively, until the size of image is less than this template, only can not be further divided into; According to the order of segmentation, employing many-sided curve fit models method one by one matching splits the image block of rear gained, splices the image block of matching successively, obtains fitted figure picture, set up the background model of view picture original image;

3) subtraction operation is carried out to the background model of original image and above-mentioned gained, the predicated error of the corresponding each point of both calculating, obtain error image;

4) by arranging suitable threshold value, threshold process being carried out to error image, obtaining the bianry image comprising potential dirty point;

5) adopt the above-mentioned bianry image comprising potential dirty point of Mathematical Morphology Method process, detect the region of potential dirty point, remove isolated, the discrete noise in bianry image, the potential dirty point finally retained can be considered to dirty point.

2. according to claim 1.1) described in method, by greyscale transformation and two-sided filter process dirty some detected image, and suitably cut out and go the partial pixel point of this image border to obtain original image.

3. according to claim 1.2) described in method, the same with original image height by rational height, width be a certain appropriate value image block as segmentation template; According to the size of this template, in the direction of vertical original image width, from original image high order end edge, original image is divided into the image block of some band shapes; By these image blocks of many-sided curve fit models method successively matching, and splice the image block after matching successively, obtain background model.

4. according to claim 1.3) described in method, the pixel value of original image each pixel corresponding to its background model carries out subtraction, the difference of the corresponding each point of both acquisitions, and these difference points constitute error image.

5. according to claim 1.4) described in method, compare the pixel value of each point and the size of threshold value in error image, pixel value be negative value and the point that its absolute value is greater than threshold value is labeled as potential dirty point, and be 1 by the pixel value assignment of this point; Otherwise, the pixel value of this point is considered as non-potential dirty point, and assignment is 0.

6. according to claim 1.5) described in method, adopt Mathematical Morphology technology to eliminate discrete, isolated noise, obtain the dirty some bianry image only comprising dirty point.