TW202445514A - Method for monitoring brightness chances in images and device thereof - Google Patents

Abstract

A method for monitoring brightness changes in images and a device thereof are provided. The method includes: capturing and obtaining a plurality of first images from a capture area by an image capture device; generating an ignore mask according to the first images; generating a grayscale average image according to the first images, and masking the grayscale average image by the ignore mask to obtain a reference image; capturing a second image from the capturing area by the image capturing element; masking the second image by the ignore mask and subtracting grayscale values of each pixel position in the reference image from the second image masked by the ignore mask to obtain an image to be tested; and, judging whether brightness of the second image changes according to the image to be tested.

Description

Method and device for monitoring brightness changes in images

The present invention relates to an image recognition technology applied to industrial automation, and in particular to a method and device for monitoring brightness changes in an image.

In industrial automation technology, automated equipment often uses image capture technology to perform operations such as object detection, feature point capture in images, and workpiece alignment. Image capture is affected by the shooting environment and the parameters used when capturing the image. In other words, maintaining a stable shooting environment is an important factor in the application of image capture technology in industrial automation technology.

Currently, image analysis monitoring combined with personnel identification can be used to confirm the stability of the shooting environment. For example, parameters such as brightness or sharpness can be monitored in key areas of the captured image, and personnel can be asked to confirm to ensure that the aforementioned environment has not changed due to external factors (for example, human adjustment/camera aperture setting adjustment, etc.), or some materials or backgrounds that normally have brightness/sharpness variation areas can be avoided through personnel identification to ensure that the image values of the acquired area are not affected by the aforementioned factors and have sufficient reference value. However, errors and even mistakes often occur in the method of personnel identification, which has led to doubts about the accuracy of the aforementioned image analysis monitoring.

The present invention provides a method and device for monitoring brightness changes in an image, which can automatically ignore areas in the image that have no reference value or have a lower reference value, and use areas in the image that have a reference value to determine whether the image brightness has changed, so as to save manpower and reduce the probability of human error in industrial automation technology.

A method for monitoring brightness changes in an image described in an embodiment of the present invention includes the following steps: capturing and obtaining multiple first images from a capture area through an image capture device; generating an ignore mask based on the multiple first images; generating a grayscale average image based on the multiple first images, and masking the grayscale average image by the ignore mask to obtain a reference image; capturing a second image from the capture area through the image capture device; masking the second image by the ignore mask, and subtracting the grayscale value of each pixel position in the reference image from the masked second image to obtain an image to be tested; and determining whether the brightness in the second image has changed based on the image to be tested.

The device for monitoring brightness change in an image described in an embodiment of the present invention comprises an image capturing device and a processor. The image capturing device is used to capture a plurality of first images and second images from a capture area. The processor is coupled to the image capturing device. The processor is configured to: capture and obtain multiple first images from a capture area through the image capture device, generate an ignore mask based on the multiple first images, generate a grayscale average image based on the multiple first images, and mask the grayscale average image by the ignore mask to obtain a reference image, capture a second image from the capture area through the image capture device, mask the second image by the ignore mask, and subtract the grayscale value of each pixel position in the reference image from the masked second image to obtain a test image, and determine whether the brightness in the second image changes based on the test image.

Based on the above, the method and device for monitoring brightness changes in images described in the embodiments of the present invention use the images obtained by the image capture device to automatically calculate and generate an image mask with no reference value or with a lower reference value as an ignore mask to remove the area in the image that may misjudge the image brightness. Then, the ignore mask is used to mask other images and serve as a reference for the brightness change of the image, so as to achieve nearly fully automatic monitoring of image brightness, save manpower and reduce the probability of human error.

FIG. 1 is a schematic diagram of a device 100 for monitoring brightness changes in an image according to an embodiment of the present invention. The device 100 for monitoring brightness changes in an image includes an image capture device 110 and a processor 120. The image capture device 110 is used to capture images from a capture area 109. In detail, as shown in FIG. 1 , the industrial automation equipment has a carrier 105, and an object 107 (e.g., a PCB board) is placed on the carrier 105. The capture area 109 includes a portion of the carrier 105 and a portion of the object 107.

The image capture device 110 captures images from the capture area 109. In this embodiment, these images can be used to monitor brightness changes in images, and can also be used as images used in image processing technologies in industrial automation such as object detection, feature point capture in images, and workpiece alignment.

The processor 120 is coupled to and controls the image capture device 120. The memory 130 is coupled to the processor 120 to temporarily store the image and the corresponding data being processed. The processor 120 is used to execute the method for monitoring brightness changes in the image according to the embodiment of the present invention.

Under normal circumstances, different product carriers, materials or backgrounds may have brightness/sharpness variations or defective areas, and these areas are difficult to use as a reference for judging image brightness changes. In addition, some areas in the capture area 109 are already saturated in brightness, and if the brightness is increased, these areas will still be difficult to change significantly. Similarly, some areas in the capture area 109 are already too dark or extremely low in brightness, and if the brightness is reduced, these areas will still be difficult to change significantly.

Therefore, the device and method for monitoring brightness changes in images described in this embodiment generates a corresponding ignore mask by comparing the pixel positions with larger grayscale differences between the collected images, and then uses this ignore mask to remove areas that may be misjudged as image brightness. The image masked by the ignore mask is then compared with the average brightness of a normal image, thereby achieving the function of automatically monitoring whether the image brightness has changed.

Fig. 2 is a flow chart of a method 200 for monitoring brightness changes in an image according to an embodiment of the present invention. Fig. 3 is a schematic diagram of each image in the method 200. Fig. 2 can be applied to the apparatus 100 for monitoring brightness changes in an image in Fig. 1, and the method 200 is mainly implemented by the processor 120 in Fig. 1.

Please refer to FIG. 2 and FIG. 3 together. In step S210, the processor 120 of FIG. 1 captures and obtains a plurality of first images 310 from the capture area 109 via the image capture device 110 of FIG. 1. The plurality of first images 310 described in this embodiment and the second images 320 described later are all captured by the image capture device 110 of FIG. 1. For the convenience of subsequent description, these images are labeled with different names.

In step S220, the processor 120 of FIG. 1 calculates and generates the ignore mask 315 based on these first images 310. This embodiment uses two first images 310 captured at similar times as the basic images for generating the ignore mask 315 in the embodiment of the present invention. However, as long as the images are captured by the image capture device 110 of FIG. 1, they can be used as the basic images for generating the ignore mask 315 in the embodiment of the present invention, and the present invention is not limited thereto. In the subsequent embodiments (such as FIG. 4 and FIG. 5 and corresponding descriptions), it will be described in detail how to use the two first images 310 captured in sequence to calculate and generate the ignore mask 315.

In step S230, the processor 120 of FIG. 1 calculates and generates a grayscale average image 317 based on the first images 310, and masks the grayscale average image 317 by using the ignore mask 315 to obtain a reference image 319. In this embodiment, the grayscale value of each pixel position in the collected multiple first images 310 is averaged to generate the grayscale average image 317, and the region with a low reference value in the grayscale average image 317 is masked by the ignore mask 315 to generate the reference image 319. The reference image 319 mainly records the pixel positions and grayscale values in the grayscale average image 317 that are not masked by the ignore mask 315. The pixel positions not covered by the ignore mask 315 have a higher reference value for determining whether the brightness in the image has changed. In this embodiment, the first image 310 is used to first organize the pixel positions with higher reference values and their grayscale values into the reference image 319 for use in subsequent determination of whether the brightness in the image has changed.

At this point, in steps S240 to S270, this embodiment uses the ignore mask 315 of step S220 and the reference image 319 of step S230 to process other images (such as the second image 320) to determine whether the brightness in the second image 320 changes.

Specifically, in step S240, the processor 120 of FIG1 captures the second image 320 from the capture area 109 of FIG1 via the image capture device 110 of FIG1. In step S250, the processor 120 of FIG1 masks the second image 320 by using the ignore mask 315 to ignore the area with a low reference value. In step S260, the processor 120 of FIG1 subtracts the grayscale value of each pixel position in the reference image 319 from the masked second image 320 to obtain an absolute value to obtain a difference image between the second image 320 and the reference image 319, i.e., the image to be tested 325. In step S260, the processor 120 of FIG. 1 determines whether the brightness of the second image 320 changes according to the image to be tested 325.

The detailed steps of step S260 are that the processor calculates the average grayscale value of each pixel in the image to be tested 325. It should be noted that some pixel positions of the image to be tested 325 are blocked, so the grayscale values corresponding to these pixel positions are not calculated in the aforementioned average grayscale value. Then, the processor determines whether the aforementioned average grayscale value exceeds a brightness threshold value, and this brightness threshold value can be set by the user of the embodiment of the present invention, for example, to "10". When the aforementioned average grayscale value exceeds the brightness threshold value "10", the processor performs a prompt operation, thereby using this prompt operation to present the information "the brightness of the image to be tested has been changed" to the corresponding personnel. The brightness threshold value can be a single value or a range of values.

The detailed steps of generating the ignore mask 315 in step S220 of FIG2 are described in detail. FIG4 is a detailed flow chart of step S220 of FIG2. FIG5 is a schematic diagram of each image in FIG4.

Please refer to FIG. 4 and FIG. 5 . In step S410 , the processor obtains a difference image 510 between two sequentially captured first images 310 . Specifically, the processor subtracts the grayscale value corresponding to each pixel position in the two first images 310 to generate the difference image 510 .

Step S420 and step S430 of this embodiment are to capture the extremely dark area (e.g., the set of pixel positions with grayscale values less than the first critical grayscale value (e.g., "25")) and the extremely bright area (e.g., the set of pixel positions with grayscale values greater than the second critical grayscale value (e.g., "230")) in the first image 310. These two areas with low reference values are used as brightness masks to exclude pixels with low reference values. The user of this embodiment can adjust the values of the first critical grayscale value and the second critical grayscale value according to their needs. For example, the first critical grayscale value can also be "10" or "245".

In detail, taking the first grayscale value as 25 and the second grayscale value as 230 as an example, in step S420, the processor compares the grayscale value of each pixel position of one of the first images 310 with the first critical grayscale value to capture the area of the first image 310 that is less than the first critical grayscale value, and then obtains the first brightness mask 520. Accordingly, the grayscale value of each pixel in the first image 310 that is masked by the first brightness mask 520 is less than the first critical grayscale value. In step S430, the processor compares the grayscale value of each pixel position of one of the first images 310 with the second critical grayscale value to capture the area of the first image 310 that is greater than the second critical grayscale value, thereby obtaining the second brightness mask 530. Accordingly, the grayscale value of each pixel in the first image 310 that is masked by the second brightness mask 530 is greater than the second critical grayscale value. In this embodiment, the image on the left side of the two first images 310 in FIG. 5 is taken as "one of the first images 310". The user of this embodiment can take any of the first images 310 as "one of the first images 310" described in steps S410 to S440 of this embodiment according to their needs.

In step S440, the processor calculates the ignore mask 315 according to the difference image 510 in step S410, the first brightness mask 520 in step S420, and the second brightness mask 530 in step S430. In the embodiment of the present invention, the simplest way to generate the ignore mask 315 is to add the difference image 510, the first brightness mask 520, and the second brightness mask 530 to generate a rough ignore mask 315. In another embodiment of the present invention, the ignore mask 315 can be generated by steps S441 to S445 in FIG. 4.

In step S441 , the processor adds up the grayscale values of each pixel position in the difference image 510 , the first brightness mask 520 , and the second brightness mask 530 to generate the measurement-ignored image 540 .

In step S442, the processor calculates a grayscale reference value X according to the grayscale value of each pixel in the measured disregarded image 540. For details, please refer to FIG. 6, which is a schematic diagram showing the number of pixels (presented on the Y axis) having each grayscale value (presented on the X axis) in the measured disregarded image 540 in a histogram, that is, the number of pixels corresponding to each grayscale value is presented in a histogram. In FIG. 6, the smaller the grayscale value, the higher the reference value of these pixel positions for the image brightness and the smaller the variation. Therefore, the embodiment of the present invention starts counting and accumulating from the grayscale value 0 to a certain number of pixels, which may be 30% of the total number of pixels of the image and corresponds to the grayscale reference value X calculated in step S442. In other words, the sum of the number of pixels from the grayscale value 0 to the grayscale reference value X is approximately equal to 30% of the total number of pixels of the image, indicating that the corresponding area with a grayscale value lower than the grayscale reference value X is an area with a reference value of 30% of the total image area. 30% of the total number of pixels is an area based on brightness reference. Depending on the shooting situation, if the brightness stable area is known to be extremely small, the percentage may need to be adjusted downward. However, if the general stable area is greater than 30%, 30% of the total number of pixels can be used as the reference area. The above “approximately equal to” may be completely equal to or may not be completely equal to (i.e. within a tolerance range, such as within plus or minus 10%).

Returning to FIG. 4 and FIG. 5 , in step S443, the processor compares the grayscale value of each pixel position in the measured disregarded image 540 based on the grayscale reference value X of step S442. In step S444, when the grayscale value of each pixel position in the measured disregarded image 540 is greater than the grayscale reference value X of step S442, the processor records the corresponding pixel position and generates the ignore mask 315 based on the recorded pixel positions. The grayscale value greater than the grayscale reference value X indicates that the reference value for the image brightness is small and the variation is high, so it can be ignored in the brightness detection. Therefore, the ignore mask 315 is generated in steps S441 to S444. In this embodiment, in order to avoid factors such as image drift or error, in step S445, the processor will appropriately expand each area in the ignore mask 315 by an expanded pixel value (for example, appropriately expand "5" pixels), thereby generating an expanded ignore mask 315.

Although the present embodiment uses two sequentially captured first images 310 in steps S441 to S445 in FIG. 4 and FIG. 5 to generate the measurement-ignored image 540 via the difference image 510, the first brightness mask 520, and the second brightness mask 530, the user of the present embodiment may also generate and collect a large number of measurement-ignored images 540 from any two of the plurality of first images 310 and average these measurement-ignored images 540 to generate an average image 545 of the plurality of measurement-ignored images, and utilize this average image 545 to generate and adjust the ignore mask 315 in the manner of the aforementioned step S444.

The step of judging whether the brightness of the second image has changed includes the following steps: calculating the average grayscale value of each pixel in the image to be tested, and then judging whether the average grayscale value exceeds the brightness threshold value. When the average grayscale value exceeds the brightness threshold value, performing a prompt operation, the prompt operation is used to show that the brightness of the image to be tested has been changed.

In summary, the method and device for monitoring brightness changes in an image described in the embodiment of the present invention utilizes the image obtained by the image capture device to automatically calculate and generate an image mask with no reference value or with a lower reference value as an ignore mask to remove the area in the image that may misjudge the image brightness. Then, the ignore mask is used to mask other images and as a reference for the brightness change of the image, so as to achieve nearly fully automatic monitoring of image brightness, save manpower and reduce the probability of human error.

100: Device for monitoring brightness changes in an image 105: Vehicle 107: Object 109: Capture area 110: Image capture device 120: Processor 130: Memory S210~S270, S410~S445: Steps of a method for monitoring brightness changes in an image 310: First image 315: Ignore mask 317: Grayscale average image 319: Reference image 320: Second image 325: Image to be measured 510: Difference image 520: First brightness mask 530: Second brightness mask 540: Measurement disregarded image 545: Average image of multiple measurement disregarded images X: Grayscale reference value

FIG. 1 is a schematic diagram of a device 100 for monitoring brightness changes in an image according to an embodiment of the present invention. FIG. 2 is a flow chart of a method 200 for monitoring brightness changes in an image according to an embodiment of the present invention. FIG. 3 is a schematic diagram of each image in the method 200. FIG. 4 is a detailed flow chart of step S220 of FIG. 2. FIG. 5 is a schematic diagram of each image in FIG. 4. FIG. 6 is a schematic diagram of measuring the number of pixels (presented on the Y axis) of each grayscale value (presented on the X axis) in an image 540 in a histogram.

S210~S270: Steps of the method for monitoring brightness changes in an image

Claims (10)

A method for monitoring brightness changes in an image, comprising: Capturing and obtaining a plurality of first images from a capture area via an image capture device; Generating a ignore mask based on the plurality of first images; Generating a grayscale average image based on the plurality of first images, and masking the grayscale average image by the ignore mask to obtain a reference image; Capturing a second image from the capture area via the image capture device; Masking the second image by the ignore mask, and subtracting the grayscale value of each pixel position in the reference image from the masked second image to obtain a test image; and Determining whether a brightness in the second image changes based on the test image. As described in claim 1, generating the ignore mask based on the multiple first images includes: Obtaining a difference image between two of the multiple first images; Comparing the grayscale value of each pixel position of one of the multiple first images with a first critical grayscale value to obtain a first brightness mask, wherein the grayscale value of each pixel of the first image masked by the first brightness mask is less than the first critical grayscale value; Comparing the grayscale value of each pixel position of one of the multiple first images with a second critical grayscale value to obtain a second brightness mask, wherein the grayscale value of each pixel of the first image masked by the second brightness mask is greater than the second critical grayscale value, and the first critical grayscale value is less than the second critical grayscale value; and The ignore mask is obtained according to the difference image, the first brightness mask and the second brightness mask. As described in claim 2, obtaining the ignore mask based on the difference image, the first brightness mask and the second brightness mask includes: Adding up the grayscale values of each pixel position in the difference image, the first brightness mask and the second brightness mask to generate a measured ignore image; Calculating a grayscale reference value based on the grayscale value of each pixel in the measured ignore image; Comparing the grayscale value of each pixel position in the measured ignore image based on the grayscale reference value; and When the grayscale value of each pixel position in the measured ignore image is greater than the grayscale reference value, recording the corresponding pixel position, and generating the ignore mask based on the recorded pixel position. A method as described in claim 1, wherein the multiple first images and the second image include a portion of an object. As described in claim 1, judging whether the brightness in the second image has changed based on the image to be tested includes: Calculating an average grayscale value of each pixel in the image to be tested; Judging whether the average grayscale value exceeds a brightness threshold; and When the average grayscale value exceeds the brightness threshold, performing a prompt operation, the prompt operation is used to present that the brightness in the second image has changed. A device for monitoring brightness changes in an image, comprising: an image capture device for capturing multiple first images and a second image from a capture area; and a processor coupled to the image capture device, wherein the processor is configured to: capture and obtain the multiple first images from the capture area via the image capture device, generate an ignore mask based on the multiple first images, generate a grayscale average image based on the multiple first images, and mask the grayscale average image with the ignore mask to obtain a reference image, capture the second image from the capture area via the image capture device, The second image is masked by the ignore mask, and the grayscale value of each pixel position in the reference image is subtracted from the masked second image to obtain a test image, and, Based on the test image, it is determined whether a brightness in the second image changes. The device as described in claim 6, wherein the processor is further configured to: obtain a difference image between two of the plurality of first images, compare the grayscale value of each pixel position of one of the plurality of first images with a first critical grayscale value to obtain a first brightness mask, wherein the grayscale value of each pixel of the first image masked by the first brightness mask is less than the first critical grayscale value, compare the grayscale value of each pixel position of one of the plurality of first images with a second critical grayscale value to obtain a second brightness mask, wherein the grayscale value of each pixel of the first image masked by the second brightness mask is greater than the second critical grayscale value, and the first critical grayscale value is less than the second critical grayscale value, and, The ignore mask is obtained according to the difference image, the first brightness mask and the second brightness mask. The device as described in claim 7, wherein the processor is further configured to: add up the grayscale values of each pixel position in the difference image, the first brightness mask and the second brightness mask to generate a measurement-ignored image, calculate a grayscale reference value based on the grayscale value of each pixel in the measurement-ignored image, compare the grayscale value of each pixel position in the measurement-ignored image based on the grayscale reference value, and, when the grayscale value of each pixel position in the measurement-ignored image is greater than the grayscale reference value, record the corresponding pixel position and generate the ignore mask based on the recorded pixel position. A device as described in claim 6, wherein the multiple first images and the second image include a portion of an object. The device as described in claim 6, wherein the processor is further configured to: calculate an average grayscale value of each pixel in the image to be tested; determine whether the average grayscale value exceeds a brightness threshold; and when the average grayscale value exceeds the brightness threshold, perform a prompt operation, wherein the prompt operation is used to present that the brightness in the second image has been changed.

Images