JP2007318355A - Imaging apparatus and lens dirt detection method - Google Patents

Abstract

It is possible to determine not only remarkable lens contamination but also thin lens contamination, and improve detection accuracy of lens contamination.
A difference image generation unit (103) acquires a plurality of image data captured at different times by a camera (101) from an image storage unit (102), and generates difference image data between these image data. The difference image update unit 104 compares the previous difference image and the current difference image for each pixel, and updates the current difference image with the pixel having the larger difference. When the number of times the difference image has been updated reaches a predetermined number, the latest difference image is output to the dirt determination unit 106. The dirt determination unit 106 determines the presence or absence of lens dirt in the camera 101 based on the latest difference image.
[Selection] Figure 3

Description

  The present invention relates to an imaging apparatus and a lens dirt detection method that can be applied to an in-vehicle camera system mounted on a vehicle such as an automobile.

  As a device that monitors the surroundings of a vehicle using a camera, an in-vehicle camera system that supports safe driving by attaching a camera to the back, side, front, etc. of an automobile and presenting the image of the in-vehicle camera to a driver is known. It has been. As a typical example, as shown in FIG. 9, a vehicle-mounted camera (rear view camera in this case) is installed in a rear trunk portion or the like of an automobile, and an image (rear view) obtained therefrom is shown in FIG. There is a system in which the state behind the vehicle can be known without looking at the mirror by displaying a guide line or the like on the driver.

  Since the camera used in this apparatus is used under all kinds of conditions, it is inevitable that dirt adheres to the lens surface. If the lens becomes very dirty, the image of the outside world taken through the lens will become indistinct or partially hidden, which will hinder driving. Therefore, it is necessary to determine the presence or absence of dirt adhering to the lens of the in-vehicle camera. If there is dirt, it is necessary to urge the driver to clean, or to provide a device for automatically washing the lens.

As a technique for detecting dirt adhering to a lens, for example, there is one described in Patent Document 1. In Patent Document 1, when a vehicle is moving, a difference between density values included in two images captured at different timings is extracted, and a portion below a predetermined threshold is detected as dirt from an image obtained by integrating these. A method is disclosed.
JP 2003-259358 A

  1A to 1C are diagrams showing an outline of lens dirt detection described in Patent Document 1. FIG. FIG. 1A and FIG. 1B are two images captured at different timings while the vehicle is moving. FIG. 1C is an image in which the difference between the density value included in FIG. 1A and the density value included in FIG. 1B is extracted and integrated.

  In FIG. 1C, it can be seen that the image is blackened as the difference is small, and whitened as the difference is large. That is, as the relationship between the image data in FIG. 1A and the image data in FIG. 1B decreases, whitening of the image data proceeds. For example, it can be seen that the whitening of the image data in FIG. 1 (c) is progressing in the portion where the car parked in FIG. 1 (a) disappears in FIG. 1 (b). In addition, the road surface portion should not have a difference between Fig. 1 (a) and Fig. 1 (b), but when considering the characteristics of being taken from a moving object called an automobile and taken for a long time, As a result of taking a difference from an image of a place with other brightness, the difference appears in the integrated image.

  On the other hand, since the dirt region shown in FIG. 1 is noticeable dirt, the input image 1 and 2 are all black from the beginning, and the difference between the input image 1 and the input image 2 is “None”. This gives the same result regardless of the difference between images taken at any timing. Therefore, in FIG. 1C, the dirty area is always black even if the difference is taken. Therefore, the technique described in Patent Literature 1 detects a portion having a density value equal to or less than a threshold in the difference image as a dirty region.

  Japanese Patent Application Laid-Open No. 2004-228561 utilizes the fact that the difference between the density values contained in two images captured at different timings is approximately zero in the pixel where the lens is contaminated. However, the difference is not almost zero in the case of light stains that make the image appear thin. In such a case, even if the integrated image is taken, the difference between the dirty area and the other area is less likely to appear, and it is difficult to detect the dirt by threshold determination. Specific examples in the case where this problem occurs are shown in FIGS. 2C is the same as FIG. 1 in that it is a difference image between FIG. 2A and FIG. 2B.

  In FIG. 2 (c), the background of each of the input images 1 and 2 is seen through the smudged region, so the difference between the input image 1 and the input image 2 is completely 0 (= black). Don't be. Therefore, it is difficult to identify which image is due to dirt adhering to the lens in comparison with the image that was originally taken from black or dark areas. By doing so, it is difficult to detect the dirty area.

  The present invention has been made in view of the above points, and is not only a detection of a noticeable lens stain in which a part of a camera image is completely hidden, but also a thin lens with a light degree such that the background can be seen through. An object of the present invention is to provide an imaging apparatus and a lens dirt detection method capable of detecting dirt.

  The imaging apparatus according to the present invention compares a camera and a plurality of images captured at different timings using the camera with respect to each corresponding pixel, and uses one pixel having a maximum density difference between pixels. A configuration is provided that includes a forming unit that forms a maximum difference image, and a determination unit that determines a pixel having a density equal to or lower than a threshold in the maximum difference image as a dirty pixel.

  According to the present invention, not only significant lens contamination but also thin lens contamination can be determined, and the detection accuracy of lens contamination can be improved.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

(Embodiment 1)
FIG. 3 is a block diagram showing the main configuration of the imaging apparatus according to Embodiment 1 of the present invention.

  The imaging apparatus according to the present embodiment includes a camera 101, an image storage unit 102, a difference image generation unit 103, a difference image update unit 104, a difference image storage unit 105, a stain determination unit 106, a notification unit 107, and a counter 108. .

  Each unit of the imaging apparatus according to the present embodiment performs the following operation.

  The camera 101 is installed in a vehicle such as an automobile, images the surroundings of the vehicle, and outputs image data obtained at a predetermined timing such as every 10 minutes or every hour to the image storage unit 102. Note that shooting itself by the camera 101 is continuously performed while the in-vehicle camera system is activated by an instruction from a driver or the like.

  The image storage unit 102 stores the image data output from the camera 101 for a predetermined period such as several days or months, and outputs the stored image data to the difference image generation unit 103.

  The difference image generation unit 103 acquires a plurality of image data captured at different times by the camera 101 from the image storage unit 102, generates difference image data that is a difference between these image data, and generates a difference image update unit. To 104. Normally, the difference image generation unit 103 generates a difference image between the image 1 obtained at the shooting timing 1 and the image obtained at the shooting timing 2 that is the shooting timing next to the shooting timing 1.

  The difference image update unit 104 compares the previous difference image (stored in the difference image storage unit 105) generated by the difference image generation unit 103 and the current difference image for each pixel, and the larger difference is obtained. The current difference image is updated with the pixels. The updated difference image is output to the difference image storage unit 105 and stored in the difference image storage unit 105.

  The counter 108 counts the number of times the difference image is updated in the difference image update unit 104. When the count value reaches a predetermined number, the counter 108 notifies the difference image update unit 104 to that effect. Therefore, when the difference image is not updated, the count value of the counter 108 is not updated. When the difference image update unit 104 recognizes from the notification of the counter 108 that the number of update of the difference image has reached a predetermined number of times, the difference image update unit 104 uses the latest difference image stored in the difference image storage unit 105 as the stain determination unit 106. Output to.

  The dirt determination unit 106 determines the presence or absence of lens dirt in the camera 101 based on the latest difference image output from the difference image update unit 104, and outputs the determination result to the notification unit 107.

  When the determination result output from the dirt determination unit 106 indicates “dirty”, the notification unit 107 notifies the user of the presence of dirt, for example, by outputting synthesized speech from a speaker (not shown).

  FIG. 4 is a flowchart showing details of the procedure of the lens dirt detection method in the imaging apparatus according to the present embodiment having the above-described configuration.

  First, the imaging apparatus according to the present embodiment sets the count value of the counter 108 to 0 with the start of the lens dirt detection process (ST1010), and the difference image data stored in the difference image storage unit 105 is set. Reset and take an image (ST1020).

  After a predetermined time elapses, an image is captured again to generate a difference image (ST1030), and a difference image between image 2 captured this time and image 1 captured last time (at the previous capturing timing) is generated (ST1040).

  Next, the imaging apparatus according to the present embodiment determines whether or not a previous difference image exists in difference image storage unit 105 (ST1050). That is, it is determined whether or not the difference image generated in ST1020 is the first difference image after the start of the lens dirt detection process. If it is the first difference image (ST1050: NO), no processing is performed. Return to the process of ST1030 without performing. On the other hand, when a saved difference image exists (ST1050: YES), the difference between the saved difference image (previous difference image) and the current difference image is compared for each pixel, and the previous difference is determined. If the pixel of the image is a pixel having a larger difference, the corresponding pixel of the current difference image is updated with this pixel (ST1060).

  In other words, the difference image finally obtained by the difference image update unit 104 is obtained by selecting all the pixels having the larger difference between the previous difference image and the current difference image to form a new image. is there.

  In ST1070, the imaging apparatus according to the present embodiment determines whether or not the count value of counter 108 has reached a predetermined value, and if not, adds 1 to the counter (ST1150), and returns to ST1030 for processing. to continue. On the other hand, if the count value has reached the predetermined value, the process proceeds to ST1080.

  In ST1080, in the difference image finally obtained in ST1060, that is, in the difference image data saved in the difference image saving unit 105, an area composed of pixels having a degree of whitening equal to or less than a predetermined value is extracted, and then in ST1090. Then, the area of this extraction region is calculated.

  Then, it is determined whether or not the calculated area is equal to or greater than a predetermined value (eg, 3% to 70%) (ST1100). If the calculated area is equal to or greater than the predetermined value, this area is determined to be a dirty area, and “dirt” is determined. A signal indicating "no dirt" is output otherwise (ST1130). Thereby, the lens dirt detection process is completed. It should be noted that the predetermined value for determining the contamination is not limited to the above numerical value as long as it is a value for improving the lens detection accuracy.

  Next, the principle of the present invention will be described.

  The inventor found that the density difference between images captured at different times in this region is different from the density difference between the pixels where there is no dirt, even when the dirt is not noticeable and is light. We focused on the phenomenon of being universally small. In other words, a comparison is made with an image captured in the past at the pixel level, and a pixel having a large density difference in the past is retained, thereby generating a final difference image. . That is, it is determined that a pixel having a large density difference in the past is unlikely to be a dirty area. As a result, even if the object to be photographed is blackish or dark, by performing the threshold determination of the density difference for each pixel, a dirty pixel that is highly likely to be stained after a certain period of time has passed. Can be specified. When the area of the area composed of these dirty pixels exceeds a predetermined value, the area can be determined as a dirty area.

  5 and 6 are diagrams for specifically explaining the principle of the present invention.

  In FIGS. 5A to 5C, three examples of input images are shown. Small translucent stains are attached to the upper right of any image. However, it is assumed that the input image 1, the input image 2, and the input image 3 are taken in this order. FIG. 5D shows a difference image between the input image 1 and the input image 2, and FIG. 5E shows a difference image between the input image 2 and the input image 3. In these difference images, the smaller the difference is, the blacker the color is, and the larger the difference is, the whiter the color is. In both of the two difference images of FIG. 5D and FIG. 5E, the area where the dirt is attached is blackened because the density difference is small. However, a portion other than dirt, for example, a dark gray portion on the road surface, also has a blackened portion because the difference is small.

  FIG. 6A is a difference image generated by taking the maximum difference value at each pixel in the two difference images of FIG. 5D and FIG. 5E. Specifically, this is an image stored in the difference image storage unit 105. Looking at this difference image, the smudged area is still black, but the background is still visible because it is still lightly smudged. Therefore, in FIG. 6A, since the entire area is blackened, the smudge area is not clear as compared with the surrounding area.

  However, as shown in FIG. 6 (b), with the passage of time, the dark gray portions of the road surface will eventually become brighter because the brighter road surface image will be input, and the density difference between the pixels in these portions will increase. Is expected to. Therefore, in a differential image that is sequentially updated and is composed of pixels that always have the largest density difference, even if it is a light stain, the difference from the surroundings becomes clear. Therefore, as shown in FIG. 6C, the dirt region can be determined by finally binarizing using threshold determination.

  FIG. 7 is a diagram for explaining how the maximum value of the difference image changes over time. Here, FIGS. 7B to 7E are graphs showing the density of each pixel on the line L1 in FIG. 7A.

  7B to 7E, the upper broken line indicates the maximum theoretical density value of the difference image, that is, the density when the image is completely whitened. In addition, the one-dot chain line in the middle indicates a threshold for determining whether or not it is dirty.

  Thus, it can be seen that the difference in tendency between the dirty region and the region other than the stain clearly appears with the passage of time, and the threshold value determination becomes easy after the predetermined time has elapsed.

  As described above, according to the present embodiment, in an in-vehicle camera system or the like, a plurality of images taken at different timings are compared at a pixel level, and an image constituted by pixels with the largest difference is used. Detect lens dirt. As a result, not only significant lens contamination but also thin lens contamination can be determined, and the detection accuracy of lens contamination can be improved.

  In the present embodiment, the configuration in which lens contamination is detected based on the density difference of each pixel has been described as an example. However, a luminance difference may be used instead of the density difference, and a gradation level may be used instead of density or luminance. The term key may be used.

  In the present embodiment, the counter 108 is used to count the number of times the difference image is updated, and only when the number reaches the predetermined value, it is determined whether or not there is an area greater than or equal to the threshold value in the difference image. Although described as an example, it may be configured to determine whether or not there is an area equal to or greater than the threshold every time the difference image is updated.

  In the present embodiment, the case where the image data stored in the image storage unit 102 is periodic data taken at a predetermined timing such as every 10 minutes or every hour has been described as an example. For example, it may be configured such that image data is always stored in the image storage unit 102 when the vehicle starts moving and when the vehicle ends moving.

  Further, in the present embodiment, the counter 108 has been described with an example in which the count value is not updated when the difference image is not updated. However, the counter 108 is different for each pixel of the previous difference image and the current difference image. It is good also as a structure which counts the frequency | count that the comparison was performed. That is, in such a configuration, the count value is updated even when the difference image is not updated.

(Embodiment 2)
In the first embodiment of the present invention, the configuration in which the difference image between the previously captured image and the current captured image is generated at each capturing timing has been described as an example. However, the difference image having the maximum difference at the pixel level is generated. For the purpose alone, it is not always necessary to generate a difference image at every photographing timing, and it is not essential to generate a difference image itself. For example, a plurality of captured images are stored for a predetermined period, and the plurality of images are compared with each other at a time to directly generate a difference image (hereinafter referred to as a maximum difference image) having a maximum difference at the pixel level. It is good also as such a structure.

  FIG. 8 is a block diagram illustrating a main configuration of the imaging apparatus according to the present embodiment. This imaging apparatus has the same basic configuration as that of the imaging apparatus shown in Embodiment 1, and the same components are denoted by the same reference numerals and description thereof is omitted. In addition, components having the same basic operation but different in detail are distinguished by attaching the same reference numerals with alphabetic lowercase letters to the same numbers, and will be appropriately described.

  The imaging apparatus according to the present embodiment is different from the imaging apparatus shown in Embodiment 1 in that it includes a counter 201 and a maximum difference image generation unit 202.

  The counter 201 counts the number of generated difference images generated by the difference image generation unit 103 and outputs it to the maximum difference image generation unit 202.

  When the count value notified from the counter 201 reaches a predetermined value, the maximum difference image generation unit 202 has a maximum density difference at the pixel level with respect to a plurality of difference image data generated by the difference image generation unit 103. Is selected, a maximum difference image is generated by the selected pixel, and is output to the dirt determination unit 106a. This maximum difference image corresponds to the updated difference image in the first embodiment.

  The dirt determination unit 106 a determines the presence or absence of lens dirt in the camera 101 based on the maximum difference image generated by the maximum difference image generation unit 202, and outputs the determination result to the notification unit 107.

  As described above, according to the present embodiment, not only significant lens contamination but also thin lens contamination can be determined, and the detection accuracy of lens contamination can be improved.

  The embodiments of the present invention have been described above.

  Note that the imaging apparatus and the lens dirt detection method according to the present invention are not limited to the above embodiments, and can be implemented with various modifications.

  In addition, the imaging apparatus and the lens dirt detection method according to the present invention can be used for an in-vehicle camera mounted on a vehicle such as an automobile, thereby providing an in-vehicle camera having the same effects as described above.

  Further, here, the case where the present invention is configured by hardware has been described as an example, but the present invention can also be realized by software. For example, a function similar to that of the imaging apparatus according to the present invention is realized by describing an algorithm of the lens dirt detection method according to the present invention in a programming language, storing the program in a memory, and causing the information processing means to execute the algorithm. can do.

  The imaging device and the lens dirt detection method according to the present invention can be applied to uses such as an in-vehicle camera system mounted on a vehicle such as an automobile.

The figure which shows the outline | summary of the lens dirt detection of patent document 1 The figure which shows the specific example in case a malfunction occurs in patent document 1 1 is a block diagram illustrating a main configuration of an imaging apparatus according to Embodiment 1. FIG. FIG. 5 is a flowchart showing details of the procedure of the lens dirt detection method according to the first embodiment. The figure for demonstrating the principle of this invention concretely The figure for demonstrating the principle of this invention concretely The figure for demonstrating a mode that the maximum value of a difference image changes with progress of time. FIG. 3 is a block diagram illustrating a main configuration of an imaging apparatus according to Embodiment 2. A diagram showing an installation example of a conventional in-vehicle camera The figure which shows the example of a display of the conventional vehicle-mounted camera system

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Camera 102 Image storage part 103 Difference image generation part 104 Difference image update part 105 Difference image preservation | save part 106 Dirt judgment part 107 Notification part 108 Counter

Claims (4)

A camera,
Forming means for comparing a plurality of images captured at different timings using the camera at each corresponding pixel, and forming one maximum difference image using a pixel having a maximum density difference between pixels;
Determination means for determining a pixel having a density equal to or lower than a threshold in the maximum difference image as a dirty pixel;
An imaging apparatus comprising: The forming means includes
Generating means for generating a difference image from two images captured at successive timings using the camera;
An update unit that compares the two difference images generated by the generation unit, and updates the difference image using a pixel having a maximum density difference between the pixels;
Output means for outputting the updated difference image as the maximum difference image when the number of updates or the elapsed time reaches a predetermined value;
The imaging apparatus according to claim 1, further comprising: If the area of the area consisting of pixels determined to be dirty pixels is greater than or equal to a predetermined value, it is determined that the camera is dirty.
The imaging device according to claim 1. Comparing a plurality of images picked up at different timings in corresponding pixels, and forming one maximum difference image using a pixel having a maximum density difference between the pixels;
Determining a pixel having a density equal to or lower than a threshold in the maximum difference image as a dirty pixel;
A lens dirt detection method comprising: