CN114354607A - A Photometric Stereo Defect Detection Method Based on Helical Phase Contrast Filtering Algorithm - Google Patents

Abstract

The invention discloses a luminosity three-dimensional flaw detection method based on a spiral phase contrast filtering algorithm, which comprises a data acquisition part and a data processing part, wherein the data acquisition part is used for acquiring luminosity three-dimensional flaw data; the data acquisition part comprises: obliquely emitting light emitted by the light source to the surface of a Lambert reflecting object to form diffuse reflection; the resolution required by object defects is calculated, and the proper multiplying power and the camera size of the required objective lens are calculated; shooting and storing a reflected light intensity image by the objective lens and the camera group; the light source is respectively placed on a plurality of directions of the object to obliquely irradiate the object, and a plurality of reflected light intensity graphs corresponding to the plurality of directions are acquired. The invention comprises the following steps: 1. spiral match filtering is added into the luminosity stereo algorithm, the flaw position edge is enhanced, the surface background of the object is removed, background interference is avoided, and the image contrast is enhanced. 2. Compared with the existing threshold segmentation background and flaw method, the method has more obvious background removing effect. 3. And gamma compression is introduced to process the image, so that the image is smoother and the human visual characteristics are compensated.

Description

A Photometric Stereo Defect Detection Method Based on Helical Phase Contrast Filtering Algorithm

technical field

The invention relates to the field of helical phase contrast filtering algorithms in the field of phase imaging and the field of photometric stereoscopic algorithms in machine vision, in particular to a photometric stereoscopic defect detection method based on the helical phase contrast filtering algorithm.

Background technique

In the last century, in order to be able to perform label-free imaging of objects, Zernike developed phase contrast microscopy imaging technology, Gabor proposed holography, researchers combined phase contrast microscopy and holography to develop a spiral phase contrast filtering imaging method, The image produced by this label-free technology contains the refractive index information of the object thickness and surface structure, enhances and highlights the edge of the surface structure of the object, removes and equalizes the surface background, and makes the surface structure edge and background have high contrast. The background interference caused by the strong background is also avoided.

In recent years, capturing the "look" of an object from an image has become increasingly important. By "appearance", we generally refer to a model that predicts how an object will look in all possible fields of view and lighting conditions. To adequately sample the appearance of an object, as it is indeed arbitrarily variable in shape and reflectivity, requires images from various views and light sources combined, which is impractical in most cases. Fortunately, real-world objects often exhibit regularities that can be exploited to drastically reduce the number of images required. Therefore, choosing effective (or near-effective) constraints, strong enough to be used in real systems, is crucial for appearance capture. So far, photometric stereo vision detection work has shown that it is feasible to recover an unambiguous appearance model from changing images under known lighting conditions, with camera viewing angle determined. This is an important special case for appearance capture, as it relies only on photometric cues and avoids solving correspondence problems. It is also important because such explicit appearance models have been shown to be useful for vision tasks such as flaw detection. However, in the field of defect detection, the image processed by the photometric stereo method requires that the background is eliminated and balanced and smooth, only the defect position is highlighted, and the background and the defect position form the sharpest contrast. The general practice for this purpose is to separate the background and the flaw by setting a threshold, but this method sometimes causes misjudgment due to the similar pixel values of the background and the flaw, and the background and the flaw cannot be completely separated. Therefore, there is an urgent need for a photometric stereoscopic defect detection method based on a helical phase contrast filtering algorithm to solve this problem in the prior art.

In order to solve the above technical problems, a new technical solution is proposed.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a photometric stereoscopic defect detection method based on a helical phase contrast filtering algorithm, so as to solve the problems raised in the above background art.

In order to achieve the above-mentioned purpose, the present invention provides the following technical solutions: a photometric stereoscopic defect detection method based on a helical phase contrast filtering algorithm, comprising a data acquisition part and a data processing part;

The data collection part includes the following steps:

Step a, obliquely incident the light emitted by the light source on the surface of the Lambertian reflection object;

Step c, the incident light forms diffuse reflection on the surface of the object;

Step c, count the required resolution of object defects, and calculate the required lens magnification and the number of camera pixels according to the required resolution;

Step d, the combination of the lens and the camera is placed directly above the object to form an orthogonal projection to collect the reflected light carrying the surface structure information of the object;

Step e, connect the camera to the computer, and shoot and save the intensity map of the reflected light through the camera software on the computer;

In step f, the light sources are respectively placed on multiple orientations of the object to irradiate the object obliquely, and steps a to e are repeated multiple times to collect multiple reflected light intensity maps corresponding to multiple orientations.

Preferably, the data processing part includes the following steps:

Step 1, light source direction calibration: first place a smooth ball in the orthogonal projection scene, and take the reflected light intensity map under 4 different light source direction conditions, put the 4 intensity maps into MATLAB to calculate and fit a smooth The circle boundary of the sphere, and locate the coordinates of the center of the circle (xc _i , yc _i ), where i is the index of the 4 images;

Step 2, respectively locate the bright spots on the surface of the sphere on the 4 pictures, the coordinates of the bright spots are expressed as ( _hxci , _hyci ), and the bright spots on the surface are used to reflect the direction of the light source;

Step 3, calculate the normal vector of the smooth sphere surface according to the following formula:

n:n _x =hxci _-xci , _{n y} = _{hyci -yci} ,

Step 4, estimate the light intensity: define a light parameter cost function with the following formula:

其中，N是每张强度图上的像素数，j是强度图上的像素索引，I是每张强度图上的强度值(像素值)，ρ是物体的反射率，

是第i个光源方向的单位矢量，λ_i是第i个光照强度，λ_i＝||e_i||；

where N is the number of pixels on each intensity map, j is the pixel index on the intensity map, I is the intensity value (pixel value) on each intensity map, ρ is the reflectivity of the object,

is the unit vector of the ith light source direction, λ _i is the ith light intensity, λ _i =||e _i ||;

Step 5: Execute the reflectivity ρ _j and normal vector n _j of the surface structure of the object obtained by the photometric stereo: ρ _j =||b _j ||,

Step 6, according to the characteristics of only needing a two-dimensional image to judge the defect type and its position in the image, the obtained normal vector takes its third channel data n(3);

Step 7, write a helical phase plate function

F表示傅里叶变换；Step 8, write a 4f system, place n(3) as the object at the object plane position of the 4f system, do spiral phase contrast imaging, and find the complex function U in the image plane:

F stands for Fourier transform;

Step 9, take its amplitude A for the image plane complex function U:

Step 10: Perform gamma compression on the obtained amplitude value A to obtain the final object defect image I _out : I _out =BA ^γ . B is a constant, γ is a correction parameter, and the fraction less than 1 is called the coding gamma value.

Preferably, the direction formula of the light source is: L=2n _z *nR, where R indicates that the camera direction is (0, 0, 1), and if viewed from the camera perspective, the coordinates are (0, 0, -1);

Preferably, the minimization formula of the cost function is:

b_j＝ρ_jn_j表示在j像素处的未知场景属性；

b _j =ρ _j n _j represents the unknown scene attribute at j pixel;

是一个非线性最小二乘问题，可以通过Levenberg-Marquardt算法来求解。Preferably,

is a nonlinear least squares problem that can be solved by the Levenberg-Marquardt algorithm.

Preferably, for all e _i , I _j =L ^T b _j , then b _j =(LL ^T ) ^-1 LI _j can be obtained according to the least squares method.

Preferably, the multiple orientations of the object in the step f refer to selecting multiple angles in the 360° direction of the object, taking four angles of 0°, 90°, 180°, and 270° as examples.

Preferably, the plurality of sheets in the step f refers to more than 3 sheets, wherein, preferably 4 sheets.

Preferably, the step f is specifically as follows: placing the light source on the object in four directions, front, back, left and right, and irradiating the object obliquely, repeating the above steps a to e four times, and collecting and obtaining four corresponding reflected light intensities in the four directions. picture.

Compared with the prior art, the beneficial effects of the present invention are:

1. The spiral proportional filter is added to the photometric stereo algorithm, the edge of the defect position is enhanced, the surface background of the object is removed, the background interference is avoided, and the image contrast is enhanced.

2. Compared with the existing threshold segmentation method for background and flaws, the present invention has a more obvious effect of removing background.

3. Gamma compression is introduced to process the image, the image is smoother, and the human visual characteristics are compensated.

Description of drawings

FIG. 1 is a schematic diagram of a photometric stereoscopic imaging optical assembly provided with a light source obliquely illuminated at a position of 0° above an object from a camera perspective.

FIG. 2 is a schematic diagram of a photometric stereoscopic imaging optical assembly provided with a light source obliquely illuminated at a position of 90° above an object from a camera perspective.

FIG. 3 is a schematic diagram of a photometric stereoscopic imaging optical assembly provided with a light source obliquely illuminated at a position of 180° above an object from a camera perspective.

FIG. 4 is a schematic diagram of a photometric stereoscopic imaging optical assembly provided by a light source obliquely illuminated at a position of 270° above an object from a camera perspective.

FIG. 5 is a schematic diagram of Lambertian reflection characteristics.

FIG. 6 is a schematic diagram of light source direction calibration.

FIG. 7 is a schematic diagram of a spiral phase plate.

Figure 8 is the intensity map of the light source irradiating the car door steel camera at the 0° position.

Figure 9 shows the intensity map of the light source irradiating the car door steel camera at a position of 90°.

Figure 10 shows the intensity map of the light source irradiating the car door steel camera at a position of 180°.

Figure 11 shows the intensity map of the light source irradiating the car door steel camera at a position of 270°.

FIG. 12 is an image of the four captured intensity maps of FIGS. 8 to 11 after being processed by a photometric stereo algorithm.

FIG. 13 is the image of FIG. 12 after being processed by the helical phase contrast filtering algorithm.

FIG. 14 is the gamma-compressed image of FIG. 13 .

Figure 15 is the intensity map of the light source irradiating the iron ring camera at the 0° position.

Figure 16 shows the intensity map of the light source irradiating the iron ring camera at a position of 90°.

Figure 17 shows the intensity map of the light source irradiating the iron ring camera at a position of 180°.

Figure 18 shows the intensity map of the light source irradiating the iron ring camera at a position of 270°.

FIG. 19 is an image of the four captured intensity maps of FIGS. 15 to 18 after being processed by a photometric stereo algorithm.

Fig. 20 is the image of Fig. 19 processed by the helical phase contrast filtering algorithm.

FIG. 21 is the image of FIG. 20 after gamma compression.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Referring to the accompanying drawings, the present invention provides a technical solution: a method for detecting photometric stereoscopic defects based on a helical phase contrast filtering algorithm, comprising the following steps:

1. Data collection part.

1. The light emitted by the light source is obliquely incident on the surface of the Lambertian reflecting object.

2. Incident light forms diffuse reflection on the surface of the object.

3. Calculate the required resolution of object defects, and calculate the required lens magnification and the number of camera pixels according to the required resolution.

4. The combination of the lens and the camera is placed directly above the object to form an orthogonal projection to collect the reflected light carrying the surface structure information of the object.

5. Connect the camera to the computer, shoot and save the intensity map of the reflected light through the camera software on the computer.

6. Place the light source on the object in four directions, front, back, left and right, and irradiate the object obliquely. Repeat the above steps 1 to 5 four times to collect the corresponding 4 reflected light intensity maps in the 4 directions (if necessary, also Multiple frames can be collected, but at least 3 frames should be collected, evenly distributed on the 360° azimuth of the object).

Second, the data processing part.

8. Light source direction calibration: first place a smooth sphere in the orthogonal projection scene, and capture reflected light intensity maps under 4 different light source directions, put the 4 intensity maps into MATLAB to calculate and fit a smooth sphere , and locate the coordinates of the center of the circle (xc _i , yc _i ), where i is the index of the 4 images.

9. Locate the highlights on the surface of the sphere on the 4 images respectively. The coordinates of the highlights are expressed as (hxc _i , _hyci ), and the surface highlights are used to reflect the direction of the light source.

10. Calculate the normal vector n of the smooth sphere surface: n _x = hxci _-xci , _{n y} = _{hyci -yci} ,

11. The light source direction L=2n _z *nR, R indicates that the camera direction is (0, 0, 1), and if viewed from the camera perspective, the coordinates are (0, 0, -1).

12. Estimate light intensity: define a light parameter cost function,

N是每张强度图上的像素数，j是强度图上的像素索引，I是每张强度图上的强度值(像素值)，ρ是物体的反射率，

是第i个光源方向的单位矢量，λ_i是第i个光照强度，λ_i＝||e_i||。

N is the number of pixels on each intensity map, j is the pixel index on the intensity map, I is the intensity value (pixel value) on each intensity map, ρ is the reflectivity of the object,

is the unit vector of the ith light source direction, λ _i is the ith light intensity, λ _i =||e _i ||.

13. The minimization of the cost function is

b_j＝ρ_jn_j表示在j像素处的未知场景属性。

b _j = ρ _j n _j represents the unknown scene attribute at j pixels.

这是一个非线性最小二乘问题，可以通过Levenberg-Marquardt算法来求解。14.

This is a nonlinear least squares problem that can be solved by the Levenberg-Marquardt algorithm.

15. For all e _i , I _j =L ^T b _j , then b _j =(LL ^T ) ^-1 LI _j can be obtained according to the least squares method.

16. The reflectivity ρ _j and the normal vector n _j of the object surface structure obtained by performing the photometric stereo: ρ _j =||b _j ||,

17. Since only a two-dimensional image is needed to determine the type of defect and its position in the image, the obtained normal vector takes its third channel data n(3).

18. Write a spiral phase plate function

F表示傅里叶变换。19. Write a 4f system, place n(3) as the object at the object plane position of the 4f system, do spiral phase contrast imaging, and find the complex function U in the image plane:

F stands for Fourier transform.

20. Take the amplitude A of the complex function U of the image plane:

21. Perform gamma compression on the obtained amplitude value A to obtain the final object defect image I _out : I _out =BA ^γ . B is a constant, γ is a correction parameter, and the fraction less than 1 is called the coding gamma value.

The imaging device of the present invention includes: four light sources with consistent light intensity, an object to be measured, an objective lens and an industrial camera group, a computer

The objective lens and the industrial camera are connected to form a combination, and the sensor of the industrial camera is located at the focal plane of the objective lens. The computer is connected with the industrial camera through a data cable, which is used to save the four reflection intensity maps captured by the camera to the computer, and perform algorithmic processing on the acquired images on the computer.

As shown in Figure 1, it includes the following steps:

S1. From the perspective of the camera, 4 light sources with the same light intensity are placed in four directions of 0°, 90°, 180°, and 270° of the object to be measured for illumination. The 4 light sources are turned on in sequence to illuminate the object to be tested, and only one light source is turned on at a time.

S2. The light emitted by the 4 light sources is incident on the surface of the object to be measured in turn to form diffuse reflection.

S3. The objective lens and the camera group are located directly above the object to be measured and sequentially collect the reflected light from the object, and form 4 signals on the camera in sequence.

S4. The 4 signals on the camera are sequentially transmitted to the computer through the data cable to form 4 images.

When in use, the present invention: 1. Adding helical proportional filtering to the photometric stereo algorithm, the edge of the defect position is enhanced, the surface background of the object is removed, the background interference is avoided, and the image contrast is enhanced. 2. Compared with the existing threshold segmentation method for background and flaws, the present invention has a more obvious effect of removing background. 3. Gamma compression is introduced to process the image, the image is smoother, and the human visual characteristics are compensated.

Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, and substitutions can be made in these embodiments without departing from the principle and spirit of the invention and modifications, the scope of the present invention is defined by the appended claims and their equivalents.

Claims (8)

1. a photometric stereoscopic flaw detection method based on helical phase contrast filtering algorithm, is characterized in that, comprises data acquisition part and data processing part; The data collection part includes the following steps: Step a, obliquely incident the light emitted by the light source on the surface of the Lambertian reflection object; Step b, the incident light forms diffuse reflection on the surface of the object; Step c, count the required resolution of object defects, and calculate the required lens magnification and the number of camera pixels according to the required resolution; Step d, the combination of the lens and the camera is placed directly above the object to form an orthogonal projection to collect the reflected light carrying the surface structure information of the object; Step e, connect the camera to the computer, and shoot and save the intensity map of the reflected light through the camera software on the computer; In step f, the light sources are respectively placed on multiple orientations of the object to irradiate the object obliquely, and steps a to e are repeated multiple times to collect multiple reflected light intensity maps corresponding to multiple orientations. The data processing part includes the following steps: Step 1, light source direction calibration: first place a smooth ball in the orthogonal projection scene, and take the reflected light intensity map under 4 different light source direction conditions, put the 4 intensity maps into MATLAB to calculate and fit a smooth The circle boundary of the sphere, and the coordinates of the center of the circle (xc _t , yc _t ) are located, and i is the index of the 4 images; Step 2, respectively locate the bright spots on the surface of the sphere on the 4 pictures, the coordinates of the bright spots are expressed as ( _hxci , _hyci ), and the bright spots on the surface are used to reflect the direction of the light source; Step 3, calculate the normal vector of the smooth sphere surface according to the following formula:

Step 4, estimate the light intensity: define a light parameter cost function with the following formula:

where N is the number of pixels on each intensity map, j is the pixel index on the intensity map, I is the intensity value (pixel value) on each intensity map, ρ is the reflectivity of the object,

is the unit vector of the ith light source direction, λ _i is the ith light intensity, λ _i =||e _i ||, Step 5: Execute the reflectivity ρ _j and normal vector n _j of the surface structure of the object obtained by the photometric stereo: ρ _j =||b _j ||,

Step 6, according to the characteristics of only needing a two-dimensional image to judge the defect type and its position in the image, the obtained normal vector takes its third channel data n(3); Step 7, write a helical phase plate function

Step 8, write a 4f system, place n(3) as the object at the object plane position of the 4f system, do spiral phase contrast imaging, and find the complex function U in the image plane:

F stands for Fourier transform; Step 9, take its amplitude A for the complex function U of the image plane:

Step 10: Perform gamma compression on the obtained amplitude value A to obtain the final object defect image I _out : I _out =BAγ. B is a constant, γ is a correction parameter, and the fraction less than 1 is called the coding gamma value. 2. The photometric stereoscopic defect detection method based on helical phase contrast filtering algorithm according to claim 1, characterized in that: the direction formula of the light source is: L=2n _z *nR, wherein, R indicates that the camera direction is (0 , 0, 1), if viewed from the camera perspective, the coordinates are (0, 0, -1). 3. the photometric stereoscopic flaw detection method based on helical phase contrast filtering algorithm according to claim 1, is characterized in that: the minimization formula of cost function is:

b _j = ρ _j n _j represents the unknown scene attribute at j pixels. 4. the photometric stereoscopic flaw detection method based on helical phase contrast filtering algorithm according to claim 3, is characterized in that:

is a nonlinear least squares problem that can be solved by the Levenberg-Marquardt algorithm. 5. The photometric stereoscopic flaw detection method based on helical phase contrast filtering algorithm according to claim 4, is characterized in that: for all e _i have I _j =L ^T b _j , then b _j can be obtained according to least squares method =(LL ^T ) ^-1 LI _j . 6. The photometric stereoscopic flaw detection method based on helical phase contrast filtering algorithm according to claim 1, wherein: the multiple orientations of the object in the step f refer to selecting multiple angles in the 360° direction of the object, Take four angles of 0°, 90°, 180°, and 270° as examples. 7 . The photometric stereoscopic defect detection method based on the helical phase contrast filtering algorithm according to claim 6 , wherein the plurality of frames in step f refers to more than 3 frames, and preferably 4 frames. 8 . 8. The photometric stereoscopic defect detection method based on a spiral phase contrast filtering algorithm according to claim 7, wherein the step f is specifically, placing the light source on the object in four directions, respectively, to obliquely irradiate the object, Repeat the above steps a to e four times, and collect four corresponding reflected light intensity maps in four directions.

Images