Disclosure of Invention
The application provides a defect detection method, a defect detection system and a defect detection device for a micro-display, which can improve the defect detection precision of the micro-display.
The first aspect of the present application provides a defect detection method for a micro display, including:
Acquiring a first initial image and a second initial image of the micro-display, wherein the first initial image is an image acquired by a camera in a fly shooting mode, and the second initial image is an image acquired by a positioning component;
Acquiring first initial coordinate parameters corresponding to a plurality of lamp beads according to the first initial image, and acquiring second initial coordinate parameters corresponding to a plurality of lamp beads according to the second initial image;
Acquiring a first effective coordinate parameter and an abnormal coordinate parameter in the first initial coordinate parameter according to standard coordinate parameters corresponding to a plurality of lamp beads in a preset standard image;
acquiring a second effective coordinate parameter corresponding to the abnormal coordinate parameter according to the first effective coordinate parameter;
Acquiring a first physical coordinate parameter of any one of the lamp beads according to the first effective coordinate parameter and the second effective coordinate parameter;
acquiring a second physical coordinate parameter of any lamp bead according to the second initial coordinate parameter and the standard coordinate parameter;
And calculating the coordinate offset of any lamp bead according to the first physical coordinate parameter and the second physical coordinate parameter.
Optionally, the step of obtaining the first valid coordinate parameter and the abnormal coordinate parameter in the first initial coordinate parameter according to standard coordinate parameters corresponding to the plurality of lamp beads in the preset standard image includes:
acquiring the standard image of the micro display;
obtaining the standard coordinate parameters corresponding to the lamp beads according to the standard image;
Acquiring a correlation coefficient of the standard coordinate parameter and the first initial coordinate parameter;
judging whether the correlation coefficient is in a preset range or not;
If the correlation coefficient is in the preset range, judging whether the first initial coordinate parameter of each lamp bead is matched with the corresponding standard coordinate parameter;
if the first initial coordinate parameter of the lamp bead is matched with the corresponding standard coordinate parameter, determining that the first initial coordinate parameter of the lamp bead is a first effective coordinate parameter, and if the first initial coordinate parameter of the lamp bead is not matched with the corresponding standard coordinate parameter, determining that the first initial coordinate parameter of the lamp bead is an abnormal coordinate parameter.
Optionally, the step of obtaining the second valid coordinate parameter corresponding to the abnormal coordinate parameter according to the first valid coordinate parameter includes:
Acquiring the first effective coordinate parameters corresponding to the two adjacent lamp beads corresponding to the abnormal coordinate parameters;
and calculating a second effective coordinate parameter corresponding to the abnormal coordinate parameter by adopting an interpolation method.
Optionally, the step of obtaining the first physical coordinate parameter of any one of the lamp beads according to the first effective coordinate parameter and the second effective coordinate parameter includes:
acquiring a minimum circumscribed rectangle corresponding to any lamp bead according to the first effective coordinate parameter and the second effective coordinate parameter;
Obtaining a peripheral rectangle positioned at the periphery of the minimum circumscribed rectangle according to a preset interval;
Determining a transition area where the boundary of the lamp bead is located from the peripheral rectangle to the direction of the minimum circumscribed rectangle;
Threshold segmentation is carried out on the transition region, and boundary pixel points meeting preset conditions are screened;
Fitting a target rectangle corresponding to any lamp bead according to the obtained plurality of boundary pixel points;
Acquiring a coordinate value of a central pixel point positioned in the target rectangle;
and acquiring a first physical coordinate parameter of any lamp bead according to the coordinate value of the central pixel point and a preset calibration coefficient.
Optionally, the step of obtaining the second physical coordinate parameter of any one of the lamp beads according to the second initial coordinate parameter and the standard coordinate parameter includes:
Calculating the difference value between the standard coordinate parameter of any lamp bead and the corresponding second initial coordinate parameter;
And calculating a second physical coordinate parameter of any lamp bead according to the second initial coordinate parameter, the difference value and the acquired frame number value corresponding to the second initial image.
Optionally, the defect detection method further includes:
Dividing each lamp bead on the first initial image into a sub-area according to the first effective coordinate parameter and the second effective coordinate parameter;
acquiring characteristic information of the lamp beads in any one of the subareas;
calculating the reduction coordinate parameters of the lamp beads according to the characteristic information;
and acquiring a restored image according to the restored coordinate parameters.
Optionally, after the step of acquiring the restored image according to the restored coordinate parameters, the defect detection method further includes:
Differentiating the restored image and the standard image to obtain a differential image;
performing image preprocessing on the differential image;
Performing binarization processing on the difference image after image pretreatment;
carrying out connected region marking on the binarized differential image;
acquiring characteristic information of each communication area;
Judging whether the lamp beads positioned in any one of the communication areas have defects or not according to the characteristic information.
In a second aspect, the present application provides a defect detection system for a microdisplay, comprising:
The acquisition unit is used for acquiring a first initial image and a second initial image of the micro-display, wherein the first initial image is an image acquired by a camera in a fly shooting mode, the second initial image is an image acquired by a positioning component, and the acquisition unit is used for acquiring first initial coordinate parameters corresponding to a plurality of lamp beads according to the first initial image and acquiring second initial coordinate parameters corresponding to the lamp beads according to the second initial image;
The calculating unit is used for obtaining a first effective coordinate parameter and an abnormal coordinate parameter in the first initial coordinate parameter according to standard coordinate parameters corresponding to a plurality of lamp beads in a preset standard image, obtaining a second effective coordinate parameter corresponding to the abnormal coordinate parameter according to the first effective coordinate parameter, obtaining a first physical coordinate parameter of any lamp bead according to the first effective coordinate parameter and the second effective coordinate parameter, and obtaining a second physical coordinate parameter of any lamp bead according to the second initial coordinate parameter and the standard coordinate parameter, and calculating a coordinate offset of any lamp bead according to the first physical coordinate parameter and the second physical coordinate parameter.
Optionally, the defect detection system further includes a matching module, where the matching module is configured to divide each of the light beads on the first initial image into a sub-area according to the first effective coordinate parameter and the second effective coordinate parameter;
the matching module is also used for acquiring characteristic information of the lamp beads in any one of the subareas;
the matching module is also used for calculating the restoration coordinate parameters of the lamp beads according to the characteristic information;
the matching module is also used for acquiring a restored image according to the restored coordinate parameters.
Optionally, the defect detection system further includes a detection module, where the detection module is configured to differentiate the restored image from the standard image to obtain a differential image;
the detection module is also used for carrying out image preprocessing on the differential image;
The detection module is also used for carrying out binarization processing on the difference image after the image pretreatment;
the detection module is also used for marking the communication area of the difference image after binarization processing;
The detection module is also used for acquiring the characteristic information of each communication area;
The detection module is also used for judging whether the lamp beads positioned in any one of the communication areas have defects or not according to the characteristic information.
A third aspect of the present application provides a defect detection apparatus for a micro display, comprising a defect detection system for a micro display as described in the second aspect and a stage for placing a micro display to be tested.
From the above technical scheme, the application has the following effects:
The method comprises the steps of firstly obtaining a first initial image and a second initial image of a micro-display, wherein the first initial image is an image collected by a camera in a fly shooting mode, the second initial image is an image collected by a positioning component, then obtaining first initial coordinate parameters corresponding to a plurality of lamp beads according to the first initial image, wherein the first initial coordinate parameters comprise first effective coordinate parameters and abnormal coordinate parameters, and obtaining second initial coordinate parameters corresponding to a plurality of lamp beads according to the second initial image, obtaining first effective coordinate parameters and abnormal coordinate parameters in the first initial coordinate parameters according to standard coordinate parameters corresponding to a plurality of lamp beads in a preset standard image, obtaining second effective coordinate parameters corresponding to the abnormal coordinate parameters according to the first effective coordinate parameters, further obtaining first physical coordinate parameters of any lamp bead according to the first effective coordinate parameters and the second effective coordinate parameters, further obtaining second physical coordinate parameters of any lamp bead according to the second effective coordinate parameters and the standard coordinate parameters, and finally calculating coordinate offset of any lamp bead according to the first physical coordinate parameters and the second physical coordinate parameters. Through the steps, the first initial image acquired by the micro-display in the fly shooting mode can be initially positioned, first initial coordinate parameters corresponding to the lamp beads are acquired according to the initial positioning result, then a first effective coordinate parameter in the first initial coordinate parameters and a second effective coordinate parameter corresponding to the abnormal coordinate parameters are determined by utilizing a preset standard image, and then the first initial image is further positioned according to the first effective coordinate parameter and the second effective coordinate parameter in the first initial coordinate parameters, so that a first physical coordinate parameter with accurate positioning of the lamp beads is acquired. And positioning the reference points of the plurality of lamp beads according to a second initial coordinate parameter corresponding to the second initial image and a standard coordinate parameter corresponding to the standard image acquired by the positioning component to obtain a second physical coordinate parameter corresponding to the plurality of lamp beads, and finally determining coordinate offset corresponding to the plurality of lamp beads according to the first physical coordinate parameter and the second physical coordinate parameter. Therefore, the condition that the lamp beads are positioned inaccurately due to image offset can be reduced through multiple positioning, and further the defect detection precision of the micro display can be improved.
Detailed Description
The application provides a defect detection method, a defect detection system and a defect detection device for a micro-display, which are used for improving the defect detection precision of the micro-display.
The defect detection method of the micro display described by the application is applied to a server, a system, a terminal or other devices with logic processing capability for implementation, and the application is not limited to the method. An embodiment of the present application will be described by taking a system with image analysis and processing capabilities as an example, referring to fig. 1, and an embodiment of a defect detection method of a micro display according to the present application includes:
101. the system acquires a first initial image and a second initial image of the micro-display, wherein the first initial image is an image acquired by the camera in a shooting mode, and the second initial image is an image acquired by the positioning component.
In this embodiment, the aperture focal length of the camera lens is fixed under the condition of meeting the high-speed fly-swatter mode, then the positional relationship between the detection platform and the camera is adjusted, the point light source is fixed on one side of the camera lens, and the camera parameters are stored, so as to complete the preparation work in the early stage. When the high-speed flying shooting mode is started, shooting action of the camera is triggered, image acquisition of the micro-display is completed, a first initial image of the micro-display is generated, and the first initial image is transmitted to the system. Before the first initial image is acquired by using the fly-swatting mode, the positioning component is required to acquire the image of the micro-display in the static state to obtain a second initial image.
It should be noted that, when the camera completes shooting in the fly-swatting mode, the first initial image is often degraded by interference and influence of various noises during the process of acquiring, transmitting and storing the first initial image. Therefore, in order to improve the image quality of the first initial image, it may be first subjected to noise reduction processing using a mean filtering algorithm or a gaussian filtering algorithm. When using the mean filtering algorithm, a3×3 mean filtering window may be used to filter the first initial image. When using a gaussian filtering algorithm, the signal may be weighted averaged using a gaussian function as a filter to smooth the signal and reduce noise, filtering the first initial image.
102. The system acquires first initial coordinate parameters corresponding to the plurality of lamp beads according to the first initial image, and acquires second initial coordinate parameters corresponding to the plurality of lamp beads according to the second initial image.
In this embodiment, the system performs bead feature extraction on all the beads on the first initial image, and determines a first initial coordinate parameter corresponding to each bead on the first initial image according to the extracted bead feature information. And similarly, the system extracts the lamp bead characteristics of all the lamp beads on the second initial image, and determines second initial coordinate parameters corresponding to all the lamp beads on the second initial image according to the extracted lamp bead characteristic information.
In this embodiment, the first initial coordinate parameter includes a coordinate value of a geometric center of the lamp bead and a distance from the geometric center of the lamp bead to a boundary of the lamp bead. For example, in the row direction, the distance from the geometric center of the beads to the boundary of the beads is L1, and in the column direction, the distance from the geometric center of the beads to the boundary of the beads is L2. Wherein L1 and L2 may be equal or unequal.
103. The system acquires a first effective coordinate parameter and an abnormal coordinate parameter in a first initial coordinate parameter according to standard coordinate parameters corresponding to a plurality of lamp beads in a preset standard image.
In this embodiment, because the first initial image acquired by the system in the fly-swatter mode may be offset, deformed, blurred, etc., the first initial coordinate parameter acquired by the system according to the first initial image is a rough coordinate parameter, and the first initial coordinate parameter includes a first effective coordinate parameter with higher positioning accuracy and an abnormal coordinate parameter with lower positioning accuracy. At this time, the system may select an image with good be decorated with lanterns beads, obvious features and clear background from the several finished microdisplay images as a standard image, and compare the standard image with the first initial image to determine a first effective coordinate parameter and an abnormal coordinate parameter in the first initial coordinate parameter.
104. And the system acquires a second effective coordinate parameter corresponding to the abnormal coordinate parameter according to the first effective coordinate parameter.
In this embodiment, after determining the first effective coordinate parameter and the abnormal coordinate parameter in the first initial coordinate parameter, the system needs to determine the second effective coordinate parameter corresponding to the abnormal coordinate parameter, and specifically may use an interpolation method to fit the second effective coordinate parameter corresponding to the abnormal coordinate parameter according to the known first effective coordinate parameter.
105. And the system acquires the first physical coordinate parameter of any lamp bead according to the first effective coordinate parameter and the second effective coordinate parameter.
In this embodiment, the system obtains the first effective coordinate parameter and the second effective coordinate parameter as effective coordinates of each bead on the first initial image on the image coordinate system, and the system needs to convert the effective coordinates into the first physical coordinate parameter by using a preset calibration coefficient, where the first physical coordinate parameter is a real physical coordinate of each bead on the first initial image, where the real physical coordinate is a point location of a flyswath. It can be understood that when the first effective coordinate parameter and the second effective coordinate parameter are converted into the first physical coordinate parameter, each bead can be further positioned, that is, a mode of searching an edge area by using a minimum circumscribed rectangle is used for determining a more accurate coordinate of each bead, and then the coordinate is converted into the first material coordinate parameter by using a preset calibration coefficient, so that a subsequent embodiment of a specific implementation process is described again.
106. And the system acquires the second physical coordinate parameter of any lamp bead according to the second initial coordinate parameter and the standard coordinate parameter.
In this embodiment, the system first performs a difference between the first initial coordinate parameter and the standard coordinate parameter, and then obtains a second physical coordinate parameter of each lamp bead according to the obtained difference, where the second physical coordinate parameter is a real physical coordinate of the standard point bit corresponding to each lamp bead relative to the flyswatter point.
107. And the system calculates the coordinate offset of any lamp bead according to the first physical coordinate parameter and the second physical coordinate parameter.
The system can obtain the coordinate offset corresponding to each lamp bead by differencing the calculated first physical coordinate parameter and the second physical coordinate parameter, and the coordinate offset is the real physical offset of the lamp bead.
In this embodiment, a first initial image and a second initial image of the micro-display are first acquired, where the first initial image is an image acquired by the camera in the fly-shooting mode, and the second initial image is an image acquired by the positioning component. And then acquiring first initial coordinate parameters corresponding to the plurality of lamp beads according to the first initial image, wherein the first initial coordinate parameters comprise first effective coordinate parameters and abnormal coordinate parameters, and acquiring second initial coordinate parameters corresponding to the plurality of lamp beads according to the second initial image. And acquiring a first effective coordinate parameter and an abnormal coordinate parameter in the first initial coordinate parameter according to standard coordinate parameters corresponding to a plurality of lamp beads in a preset standard image. And acquiring a second effective coordinate parameter corresponding to the abnormal coordinate parameter according to the first effective coordinate parameter. And further acquiring the first physical coordinate parameter of any lamp bead according to the first effective coordinate parameter and the second effective coordinate parameter. And further acquiring a second physical coordinate parameter of any lamp bead according to the second initial coordinate parameter and the standard coordinate parameter. And finally, calculating the coordinate offset of any lamp bead according to the first physical coordinate parameter and the second physical coordinate parameter. Through the method, the first initial image acquired by the micro-display in the fly shooting mode can be initially positioned, first initial coordinate parameters corresponding to the lamp beads are acquired according to the initial positioning result, then a first effective coordinate parameter in the first initial coordinate parameters and a second effective coordinate parameter corresponding to the abnormal coordinate parameters are determined by utilizing a preset standard image, then the first initial image is further positioned according to the first effective coordinate parameter and the second effective coordinate parameter in the first initial coordinate parameters, and the first physical coordinate parameters with accurate positioning of the lamp beads are acquired. And positioning the reference points of the plurality of lamp beads according to a second initial coordinate parameter corresponding to the second initial image and a standard coordinate parameter corresponding to the standard image acquired by the positioning component to obtain a second physical coordinate parameter corresponding to the plurality of lamp beads, and finally determining coordinate offset corresponding to the plurality of lamp beads according to the first physical coordinate parameter and the second physical coordinate parameter. Therefore, the condition that the lamp beads are positioned inaccurately due to image offset can be reduced through multiple positioning, and then the defect detection precision of the micro-display can be improved.
Referring to fig. 2-1, 2-2, 2-3 and 2-4, another embodiment of a defect detection method for a micro display according to the present application includes:
201. The system takes a first initial image and a second initial image of the micro-display, wherein the first initial image is an image acquired by the camera in a shooting mode, and the second initial image is an image acquired by the positioning component.
202. The system acquires first initial coordinate parameters corresponding to the plurality of lamp beads according to the first initial image, and acquires second initial coordinate parameters corresponding to the plurality of lamp beads according to the second initial image.
Steps 201 to 202 in this embodiment are similar to steps 101 to 102 in the embodiment shown in fig. 1, and will not be described here again.
203. The system acquires a standard image of the microdisplay.
204. And the system acquires standard coordinate parameters corresponding to the lamp beads according to the standard image.
205. The system obtains a correlation coefficient of the standard coordinate parameter and the first initial coordinate parameter.
206. The system determines whether the correlation coefficient is within a preset range, and if so, performs step 207.
207. The system determines whether the first initial coordinate parameter of each bead matches the corresponding standard coordinate parameter, if so, then step 208 is executed, and if not, then step 209 is executed.
208. The system determines a first initial coordinate parameter of the light bead as a first valid coordinate parameter.
209. The system determines the first initial coordinate parameter of the lamp bead as an abnormal coordinate parameter.
Optionally, in this embodiment, after the system acquires the standard image, the system extracts the image features of each bead on the standard image, and determines the standard coordinate coefficient corresponding to each bead according to the extracted image feature information. And sliding the standard image on the first initial image one by one along the possible position areas of each lamp bead, and calculating the correlation coefficient between the standard coordinate parameter and the first initial coordinate parameter of the current lamp bead in each sliding process. When calculating the correlation coefficient, zero-averaging is needed to be carried out on the standard coordinate parameter and the first initial coordinate parameter, and then the normalized cross-correlation between the standard coordinate parameter and the first initial coordinate parameter is calculated, specifically, the normalized cross-correlation can be calculated by adopting the following formula:
wherein (x ', y') represents a standard coordinate parameter of any one of the lamp beads on the standard image, (x, y) represents a first initial coordinate parameter of any one of the lamp beads on the first initial image, T (x+x ', y+y') represents a current pixel value of the standard image, I (x ', y') represents a current pixel value of the first initial image, Representing the average pixel value of the standard image,Representing the average pixel value of the first initial image, NCC (x, y) represents the correlation coefficient of the standard coordinate parameter with the first initial coordinate parameter. The numerator part of the formula is the covariance of the two signals and the denominator part is the product of their respective standard deviations for normalizing the cross-correlation value results to within the range of [ -1, 1]. If NCC (x, y) is close to 1, it means that the area of possible locations of the current lamp beads in the first initial image is highly similar to the standard image. If NCC (x, y) is close to-1, it indicates that the area of possible locations of the current lamp bead in the first initial image is highly opposite to the standard image. If NCC (x, y) is close to 0, it indicates that the possible location area of the current lamp bead in the first initial image has no obvious correlation with the standard image. Therefore, the standard image and all possible position areas of the lamp beads near a certain lamp bead A can be subjected to correlation coefficient calculation one by one, whether the correlation coefficient is in a preset range or not is judged, and if the correlation coefficient is in the preset range, whether the first initial coordinate parameter of the lamp bead A is matched with the corresponding standard coordinate parameter is continuously judged. If the first initial coordinate parameter of the lamp bead A is matched with the corresponding standard coordinate parameter, the first initial coordinate parameter of the lamp bead A is indicated to be a first effective coordinate parameter. If the first initial coordinate parameter of the lamp bead A is not matched with the corresponding standard coordinate parameter, the first initial coordinate parameter of the lamp bead A is an abnormal coordinate parameter.
210. The system obtains first effective coordinate parameters corresponding to two adjacent lamp beads corresponding to the abnormal coordinate parameters.
211. The system calculates a second effective coordinate parameter corresponding to the abnormal coordinate parameter by adopting an interpolation method.
Alternatively, in this embodiment, the system may first arrange all the first initial coordinate parameters according to the distribution of the lamp beads, and then select all the first initial coordinate parameters located in the same row as the abnormal coordinate parameters or all the first initial coordinate parameters located in the same column as the abnormal coordinate parameters. And selecting two first effective coordinate parameters adjacent to the abnormal coordinate parameter from all the selected first initial coordinate parameters, and finally carrying out coordinate fitting on the two first effective coordinate parameters by using an interpolation method to obtain a second effective coordinate parameter corresponding to the abnormal coordinate parameter.
212. And the system acquires the minimum circumscribed rectangle corresponding to any lamp bead according to the first effective coordinate parameter and the second effective coordinate parameter.
213. And the system acquires the peripheral rectangle positioned at the periphery of the minimum circumscribed rectangle according to the preset interval.
214. The system determines the transition area where the boundary of the lamp bead is located from the peripheral rectangle to the direction of the minimum circumscribed rectangle.
215. The system performs threshold segmentation on the transition region and screens boundary pixel points meeting preset conditions.
216. And fitting a target rectangle corresponding to any lamp bead according to the acquired plurality of boundary pixel points by the system.
217. The system acquires the coordinate value of the central pixel point positioned in the target rectangle.
218. The system acquires a first physical coordinate parameter of any lamp bead according to the coordinate value of the central pixel point and a preset calibration coefficient.
Alternatively, in this embodiment, the minimum circumscribed rectangle of the lamp beads refers to a minimum area rectangle that may contain the target lamp beads. After the system obtains the first effective coordinate parameters and the second effective coordinate parameters of all the lamp beads, the minimum circumscribed rectangle of the corresponding lamp beads can be solved according to the first effective coordinate parameters or the second effective coordinate parameters corresponding to each lamp bead, and the system can be realized by adopting a rotating shell clamping method, namely, firstly, a convex shell of a target lamp bead region is solved, then the minimum circumscribed rectangle of the convex shell is solved, and a specific minimum circumscribed rectangle solving algorithm is not limited. Each side of the minimum bounding rectangle is a vertical direction or a parallel direction, the vertical direction is a direction perpendicular to one coordinate axis in the image coordinate system of the first initial image, and the parallel direction is a direction parallel to the coordinate axis.
The minimum circumscribed rectangle is expanded by a preset interval to obtain a peripheral rectangle of the periphery, for example, 30 pixels are expanded, and specific expansion parameters are not limited here according to actual requirements. And then, taking the rectangular edge of the peripheral rectangle as a base point, and carrying out edge detection towards the direction of the minimum circumscribed rectangle so as to determine the transition area where the boundary of the lamp bead is positioned. And then, carrying out threshold segmentation on the transition region, screening out boundary pixel points meeting preset conditions, and fitting out a target rectangle corresponding to the lamp bead according to the boundary pixel points, wherein the target rectangle is the minimum circumscribed rectangle corresponding to the real outline of the lamp bead. And finally, acquiring the coordinate value of the central pixel point in the target rectangle, wherein the coordinate value of the central pixel point is the real coordinate value of the image coordinate system of any lamp bead on the first initial image. Setting the coordinate value of the central pixel point of any lamp bead asThe first physical coordinate parameter corresponding to the lamp bead is thatWherein the method comprises the steps of
219. The system calculates the difference between the standard coordinate parameter of any lamp bead and the corresponding second initial coordinate parameter.
220. And the system calculates a second physical coordinate parameter of any lamp bead according to the second initial coordinate parameter, the difference value and the frame number value corresponding to the acquired second initial image.
Optionally, in this embodiment, the second initial coordinate parameter of any one of the lamp beads is set asThe difference value between the standard coordinate parameter and the second initial coordinate parameter of the corresponding lamp bead isThe second physical coordinate parameters corresponding to the lamp beads areThe calculation formula of the second physical coordinate parameter is as follows:
wherein n is the number of frames corresponding to the current second initial image, AndThe difference between the standard coordinate parameter and the second initial coordinate parameter in the x-direction and the y-direction, respectively.
221. And the system calculates the coordinate offset of any lamp bead according to the first physical coordinate parameter and the second physical coordinate parameter.
In this embodiment, after the first physical coordinate parameter and the second physical coordinate parameter are obtained, the difference between the first physical coordinate parameter and the second physical coordinate parameter may be made, and the corresponding difference may be determined as the coordinate offset of the lamp bead. The coordinate offset of any one lamp bead is set as offset (offset x,offsety), and the corresponding calculation formula is as follows:
Wherein, An x-axis coordinate value that is a first physical coordinate parameter,A y-axis coordinate value that is a first physical coordinate parameter,An x-axis coordinate value that is a second physical coordinate parameter,Is the y-axis coordinate value of the second physical coordinate parameter.
222. And the system divides each lamp bead on the first initial image into a sub-area according to the first effective coordinate parameter and the second effective coordinate parameter.
223. The system acquires the characteristic information of the lamp beads in any sub-area.
224. And the system calculates the restoration coordinate parameters of the lamp beads according to the characteristic information.
225. And the system acquires a restored image according to the restored coordinate parameters.
Optionally, in this embodiment, in the high-speed fly-shooting mode, due to the high-speed motion of the detection platform or the industrial camera, the target image may be blurred, slightly deformed, distorted, and the like, so that the real defect on the surface of the lamp bead cannot be extracted. Therefore, the Lucas-Kanade algorithm can be adopted to correct the fine deformation of the lamp beads, so as to detect the real surface defects. The Lucas-Kanade algorithm is a method for finding the correspondence between the previous frame and the current frame by using the change of pixels in an image sequence in a time domain and the correlation between adjacent frames, so as to calculate the motion information of an object between the adjacent frames, and the basic idea is based on the following three assumptions:
1. The brightness is constant, namely, the brightness of the same point is not changed along with the change of time.
2. The motion is that the change of time does not cause the severe change of the position, and the gray level change caused by the unit position change between the front frame and the rear frame can be used for approximating the partial derivative of gray level to the position only under the condition of small motion.
3. Spatially consistent-adjacent points on a scene are projected onto the image as well as adjacent points, and the adjacent points are at a consistent speed. Since the optical flow method basic equation constraint is only one, and the velocity in the x and y directions is required, there are two unknown variables, so that n equations need to be solved continuously.
The basic constraint equation is:
Let a pixel I (x, y, t) be at the light intensity of the first frame (where t represents the time dimension in which it is located). It moves the distance (dx, dy) to the next frame, taking the dt time. Because of the same pixel, the light intensity of the pixel before and after the motion is assumed to be unchanged, namely:
I(x,y,t)=I(x+dx,y+dy,t+dt)
After the right side of the formula is unfolded by using the Taylor formula, the following steps are obtained:
let u, v be the velocity vectors of the x-axis and y-axis, respectively, namely:
Is provided with Then there are:
Ixdx+Iydy+Itdt=0
Ixu+Iyv=-It
The matrix is expressed as follows:
Finally, the least square method is utilized to obtain:
Wherein, And finally solving the optical flow.
The system cuts each lamp bead on the first initial image according to the first effective coordinate parameter or the second effective coordinate parameter corresponding to the lamp beads so as to divide each lamp bead into a sub-area. Then, the system performs feature extraction on the lamp beads in each sub-area to obtain corresponding feature information. And inputting the characteristic information into the basic constraint equation of the Lucas-Kanade algorithm, and determining the restoration coordinate parameters of the corresponding lamp beads according to the solved optical flow. And finally, mapping the restored coordinate parameters of all the lamp beads to the first initial image to obtain a restored image of the micro display.
226. And the system carries out difference on the restored image and the standard image to obtain a difference image.
227. The system performs image preprocessing on the differential image.
228. The system carries out binarization processing on the difference image after the image preprocessing.
229. And the system marks the connected region of the binarized differential image.
230. The system acquires the characteristic information of each communication area.
231. The system judges whether the lamp beads positioned in any communication area have defects or not according to the characteristic information.
Optionally, in this embodiment, the restored image and the standard image are first differentiated, and a differential image of the restored image and the standard image is obtained. Then, the obtained differential image is subjected to image preprocessing, and the image preprocessing process can comprise graying processing and denoising processing. The graying process is to convert the color image into gray image to simplify the subsequent process. The denoising process is to remove noise in the image using a filter (e.g., a gaussian filter). After the image preprocessing is completed, binarization processing is performed, and the differential image is converted into a black-and-white image, so that defects and a background can be clearly distinguished, and the binarization processing can be completed by using methods such as threshold segmentation or adaptive threshold segmentation, and the like, and the method is not limited in the specification. And then, carrying out connected region marking on the binarized differential image, identifying each connected region in the image, and finishing the connected region marking by using a connected component marking algorithm. And extracting the characteristic information of each marked connected region, such as the characteristic information of area, perimeter, shape and the like, and distinguishing the normal part from the defect part in the image through the characteristic information of the part. Finally, according to the extracted characteristic information, a proper rule or model is designed to judge whether the lamp beads of each communication area have defects or not, for example, the lamp beads can be judged according to the characteristics of the area, the perimeter and the like, and then the judgment result is analyzed and recorded to generate a lamp bead surface defect detection result of the micro-display. It is worth mentioning that after the detection of the surface defects of the lamp beads is completed, performance evaluation can be performed according to the defect detection process, wherein the performance evaluation comprises indexes such as accuracy, recall, false detection rate and the like, so as to verify the effectiveness and reliability of the lamp beads.
Referring to fig. 3, an embodiment of a defect detection system for a micro display according to the present application includes:
the acquiring unit 301 is configured to acquire a first initial image and a second initial image of the micro display, where the first initial image is an image acquired by the camera in a fly-by mode, the second initial image is an image acquired by the positioning component, and is configured to acquire a first initial coordinate parameter corresponding to the plurality of light beads according to the first initial image, and acquire a second initial coordinate parameter corresponding to the plurality of light beads according to the second initial image.
The calculating unit 302 is configured to obtain a first effective coordinate parameter and an abnormal coordinate parameter in a first initial coordinate parameter according to standard coordinate parameters corresponding to a plurality of lamp beads in a preset standard image, and obtain a second effective coordinate parameter corresponding to the abnormal coordinate parameter according to the first effective coordinate parameter, where the calculating unit is further configured to obtain a first physical coordinate parameter of any lamp bead according to the first effective coordinate parameter and the second effective coordinate parameter, and to obtain a second physical coordinate parameter of any lamp bead according to the second initial coordinate parameter and the standard coordinate parameter, and to calculate a coordinate offset of any lamp bead according to the first physical coordinate parameter and the second physical coordinate parameter.
In this embodiment, the acquiring unit 301 acquires a first initial image and a second initial image of the micro-display, where the first initial image is an image acquired by the camera in the fly-by mode, the second initial image is an image acquired by the positioning component, and is used for acquiring a first initial coordinate parameter corresponding to a plurality of beads according to the first initial image and a second initial coordinate parameter corresponding to a plurality of beads according to the second initial image, and the calculating unit 302 acquires a first effective coordinate parameter and an abnormal coordinate parameter in the first initial coordinate parameter according to standard coordinate parameters corresponding to a plurality of beads in a preset standard image, and acquires a second effective coordinate parameter corresponding to the abnormal coordinate parameter according to the first effective coordinate parameter and the second effective coordinate parameter, and is also used for acquiring a first physical coordinate parameter of any bead according to the first effective coordinate parameter and the second effective coordinate parameter, and is used for acquiring a second physical coordinate parameter of any bead according to the second physical coordinate parameter and the second physical coordinate parameter of any bead, and is used for calculating a coordinate offset of any bead according to the first physical coordinate parameter and the second physical coordinate parameter of the second physical coordinate parameter. Through the method, the first initial image acquired by the micro-display in the fly shooting mode can be initially positioned, first initial coordinate parameters corresponding to the lamp beads are acquired according to the initial positioning result, then a first effective coordinate parameter in the first initial coordinate parameters and a second effective coordinate parameter corresponding to the abnormal coordinate parameters are determined by utilizing a preset standard image, then the first initial image is further positioned according to the first effective coordinate parameter and the second effective coordinate parameter in the first initial coordinate parameters, and the first physical coordinate parameters with accurate positioning of the lamp beads are acquired. And positioning the reference points of the plurality of lamp beads according to a second initial coordinate parameter corresponding to the second initial image and a standard coordinate parameter corresponding to the standard image acquired by the positioning component to obtain a second physical coordinate parameter corresponding to the plurality of lamp beads, and finally determining coordinate offset corresponding to the plurality of lamp beads according to the first physical coordinate parameter and the second physical coordinate parameter. Therefore, the condition that the lamp beads are positioned inaccurately due to image offset can be reduced through multiple positioning, and then the defect detection precision of the micro-display can be improved.
Referring to fig. 4, another embodiment of a defect detection system of a micro display according to the present application includes:
the acquiring unit 401 is configured to acquire a first initial image and a second initial image of the micro display, where the first initial image is an image acquired by the camera in a fly-by mode, the second initial image is an image acquired by the positioning component, and is configured to acquire a first initial coordinate parameter corresponding to the plurality of light beads according to the first initial image, and acquire a second initial coordinate parameter corresponding to the plurality of light beads according to the second initial image.
The computing unit 402 is specifically configured to acquire a standard image of the micro display. And obtaining standard coordinate parameters corresponding to the lamp beads according to the standard image. And obtaining the correlation coefficient of the standard coordinate parameter and the first initial coordinate parameter. Judging whether the correlation coefficient is in a preset range. If the correlation coefficient is within the preset range, judging whether the first initial coordinate parameter of each lamp bead is matched with the corresponding standard coordinate parameter. If the first initial coordinate parameter of the lamp bead is matched with the corresponding standard coordinate parameter, determining the first initial coordinate parameter of the lamp bead as a first effective coordinate parameter, and if the first initial coordinate parameter of the lamp bead is not matched with the corresponding standard coordinate parameter, determining the first initial coordinate parameter of the lamp bead as an abnormal coordinate parameter. And obtaining a second effective coordinate parameter corresponding to the abnormal coordinate parameter according to the first effective coordinate parameter, and obtaining first effective coordinate parameters corresponding to two adjacent lamp beads corresponding to the abnormal coordinate parameter. And calculating a second effective coordinate parameter corresponding to the abnormal coordinate parameter by adopting an interpolation method. The computing unit is also specifically used for acquiring a minimum circumscribed rectangle corresponding to any lamp bead according to the first effective coordinate parameter and the second effective coordinate parameter. And acquiring a peripheral rectangle positioned at the periphery of the minimum circumscribed rectangle according to the preset interval. And determining the transition area where the boundary of the lamp bead is located from the peripheral rectangle to the direction of the minimum circumscribed rectangle. And carrying out threshold segmentation on the transition region, and screening boundary pixel points meeting preset conditions. Fitting a target rectangle corresponding to any lamp bead according to the obtained plurality of boundary pixel points. And acquiring coordinate values of a central pixel point positioned in the target rectangle. And acquiring a first physical coordinate parameter of any lamp bead according to the coordinate value of the central pixel point and a preset calibration coefficient. And the method is specifically used for calculating the difference value between the standard coordinate parameter of any lamp bead and the corresponding second initial coordinate parameter. And calculating a second physical coordinate parameter of any lamp bead according to the second initial coordinate parameter, the difference value and the frame number value corresponding to the acquired second initial image. And the coordinate offset of any lamp bead is calculated according to the first physical coordinate parameter and the second physical coordinate parameter.
The matching module 403 is configured to divide each bead on the first initial image into a sub-area according to the first effective coordinate parameter and the second effective coordinate parameter. The matching module is also used for acquiring the characteristic information of the lamp beads in any sub-area. The matching module is also used for calculating the restoration coordinate parameters of the lamp beads according to the characteristic information. The matching module is also used for acquiring a restored image according to the restored coordinate parameters.
The detection module 404 is configured to differentiate the restored image from the standard image to obtain a differential image. The detection module is also used for carrying out image preprocessing on the differential image. The detection module is also used for carrying out binarization processing on the difference image after the image preprocessing. The detection module is also used for marking the connected region of the binarized differential image. The detection module is also used for acquiring the characteristic information of each communication area. The detection module is also used for judging whether the lamp beads positioned in any communication area have defects or not according to the characteristic information.
In this embodiment, the functions of each unit and module are similar to those of steps 201 to 231 in the embodiments shown in fig. 2-1, 2-2, 2-3 and 2-4, and will not be described here again.
Referring to fig. 5, an embodiment of a defect detecting device for a micro-display of the present application includes a defect detecting system for a micro-display of the embodiment shown in fig. 3 or fig. 4 and a workbench for placing the micro-display to be detected, and specifically includes a central processing unit 502, a memory 501, an input/output interface 503, a wired or wireless network interface 504 and a power source 505.
Memory 501 is a transient memory or a persistent memory.
The central processor 502 is configured to communicate with the memory 501 and to execute the operations of the instructions in the memory 501 to perform the steps of the embodiments shown in the foregoing figures 1-2-1, 2-2, 2-3 and 2-4.
The defect detection device of the microdisplay also provides a computer readable storage medium comprising instructions that when executed on a computer cause the computer to perform the steps of the embodiments of fig. 1-2-1, 2-2, 2-3 and 2-4 described above.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein.
In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of elements is merely a logical functional division, and there may be additional divisions of actual implementation, e.g., multiple elements or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of the embodiments of the present application. The storage medium includes a usb disk, a removable hard disk, a read-only memory (ROM), a random-access memory (RAM, random access memory), a magnetic disk, an optical disk, or other various media capable of storing program codes.