JP2006262242A - Image signal processing device - Google Patents

Abstract

An image signal processing apparatus capable of accurately detecting a water droplet image area in a captured image regardless of an outdoor imaging state (brightness).
An illuminance detection unit that detects imaging illuminance in the monitoring space when the imaging unit captures an image of the monitoring space, and a monitoring space that includes the imaging unit based on the imaging illuminance detected by the illuminance detection unit. The illumination control unit 30 that controls the illumination unit 20 that irradiates the illumination light and the input image captured by the imaging unit 10 determine whether or not an image region due to water droplets exists, and the water droplets to the imaging unit 10 A water droplet detection unit 50 that detects adhesion, and when the illumination unit 20 does not irradiate illumination light, the water droplet detection unit 50 detects a water droplet by the first water droplet determination process, and the illumination unit When 20 irradiates the monitoring space including the imaging unit 10 with illumination light, water droplets are detected by the second water droplet determination process.
[Selection] Figure 1

Description

  The present invention relates to an image signal processing apparatus suitable for detecting that a water droplet has adhered to a lens portion in an imaging apparatus.

  In recent years, security systems that detect intruders or the like that have entered a monitoring space have been widely used. For example, security systems using image sensors have become widespread. A system using an image sensor includes a camera unit that captures a monitoring space to be monitored, an image acquisition unit that receives an image captured by the camera unit, and a storage unit that stores and holds image data used for processing. A processing unit that processes the image data and detects a moving object existing in the monitoring space. The processing unit extracts a variation area in the image based on a difference image composed of a difference between the input image and a background image captured in the past, and detects an attribute value of the area of the input image corresponding to the variation area When the object has a characteristic characteristic, processing such as issuing an alarm as the detection object exists in the monitoring space is performed. For example, when the detection target is “intruder”, the attribute value indicating “humanity” such as the size, aspect ratio, and luminance information of the input image data corresponding to the variable region satisfies a predetermined condition. When the condition is satisfied, processing such as issuing an alarm that an intruder exists is performed.

  Moreover, the system which monitors an intruder etc. using an image sensor is further provided with the illumination part which irradiates light to monitoring space for nighttime monitoring. Then, an image obtained by capturing the monitoring space in a state where the monitoring space is illuminated by the illumination unit is acquired, and the acquired image data is processed to detect a moving object and determine whether the moving object is a detection target. Thus, it is determined whether or not there is a detection object (for example, an intruder) in the monitoring space at night.

  By the way, in order to detect an intruder or the like who has entered an outdoor monitoring space, the above system may be used by installing a camera unit outdoors. In this case, since the camera unit is installed outdoors in a system for monitoring an outdoor monitoring space, a situation may occur in which water drops adhere to the lens portion of the camera unit during rainy weather. Here, when a water droplet adheres to a lens part or the like, an image region in which the water droplet is imaged is reflected in the image data in a blurred state, so that it is difficult to detect a detection target in the blurred water droplet image region. For example, when an intruder exists in the water drop image area in the image, the intruder image may not be clearly captured because it is hidden in the water drop image area, which is a particular problem in the security system.

  Therefore, when water drops adhere to the lens part of the camera unit, in order to prevent the occurrence of the above-mentioned problems, measures such as detecting the water drop adhering and removing it from the lens part are taken immediately. There is a need.

  Conventionally, when detecting the adhesion of water droplets, for example, as a technique for detecting raindrops adhering to the windshield of a vehicle, the illuminance in this region is calculated from image data obtained by imaging the set raindrop detection region Then, based on the obtained illuminance, there is a technique for calculating a threshold of change in illuminance for identifying the outline of the raindrop and detecting the outline of the raindrop using this threshold.

JP 05-284501 A Japanese Patent Laid-Open No. 10-148681

  In a system that uses an image sensor to detect intruders, etc., if the water droplets adhere to the lens part of the camera unit, the image area that captured the water droplets is reflected in the image data in a blurred state. In some cases, it is difficult to detect the detection target in the image area.

  Moreover, in the system using the image sensor, water droplets adhering to the lens portion of the camera unit have been a cause of false alarms.

  In a system using an image sensor, the point of detection is whether or not an attribute value that represents “likeness of a detection target” of a fluctuation region extracted from a difference image between an input image and a background image satisfies a predetermined condition. That is, even when the fluctuation region is not caused by the detection object, a detection error (false report) is issued when the condition of “detection object likelihood” is satisfied. Such false alarms are particularly problematic in security systems that detect intruders.

  Here, in such a system, when a monitoring space is imaged using ambient light in a bright situation such as daytime and an image of the monitoring space is acquired, the water droplet image area in the captured image thus captured is sunlight. There is a feature that it is divided into a bright area caused by the reflection of the image and a dark area caused by the reflection of the ground. A bright area and a dark area in the water drop image area are extracted as a variable area from the difference image between the input image and the background image, respectively, and if the area or shape of any of the variable areas satisfies the condition of “humanity”, As a result, there were cases in which an intruder was reported as false.

  On the other hand, when the monitoring space is imaged in a dark condition such as at night and the monitoring space is illuminated by the illumination unit and an image of the monitoring space is acquired, the water droplet image area in the captured image captured in this way Due to the effect of reflection by water droplets, there is a feature that almost the entire water droplet image region becomes bright. This brightly reflected water drop image area is extracted as a variable area from the difference image between the input image and the background image, and if the area or shape of this variable area satisfies the condition of “humanity”, there is an intruder There was a misinformation.

  As described above, in the case where the monitoring space is imaged using ambient light in a bright situation such as daytime and in the case where the monitoring space is imaged in a state where the monitoring space is illuminated by a lighting unit in a dark situation such as nighttime, The characteristics of the water drop image area are greatly different. Therefore, in order to detect water droplets with high accuracy corresponding to such differences in characteristics, it is preferable to perform a suitable water droplet detection process for each imaging situation in accordance with the imaging situation.

  An object of the present invention is to take an image under any imaging situation when imaging is performed using ambient light without using illumination in a bright situation such as daytime, and when imaging is performed by applying illumination light in a dark situation such as nighttime. An object of the present invention is to provide an image signal processing apparatus capable of increasing the detection accuracy of water droplets attached to an imaging unit even for images.

  The image signal processing apparatus of the present invention is based on the illuminance detection unit that detects the imaging illuminance in the monitoring space when the monitoring space is imaged by the imaging unit, and the imaging illuminance detected by the illuminance detection unit, An illumination control unit that controls an illumination unit that irradiates illumination light to a monitoring space including the imaging unit, and determines whether or not an image region due to water droplets exists in an input image captured by the imaging unit, and the imaging unit A water droplet detection unit that detects adhesion of water droplets to the water droplet detection unit, when the illumination unit does not irradiate illumination light, performs water droplet detection by a first water droplet determination process, When the illumination unit irradiates the monitoring space including the imaging unit with illumination light, water droplets are detected by the second water droplet determination process.

  In the image signal processing device having the above configuration, the first water droplet determination process causes a change in luminance from a difference between a background image captured by the imaging unit and an input image captured after the background image is captured. An input non-illumination difference image that satisfies the condition of satisfying the above condition, and an input high-intensity difference image that satisfies a condition that a corresponding pixel of the input image is greater than or equal to a predetermined luminance value in the non-illumination difference image, and the difference image A non-illuminated differential image generation process for generating an input non-high luminance differential image that satisfies a condition that a corresponding pixel of the input image is less than the predetermined luminance value, and an edge intensity in the input image is a predetermined threshold value Non-illuminated input weak edge image generation processing for generating an unilluminated input weak edge image that satisfies the following condition, edge strength in the input image, and edge in the background image A non-illuminated edge reduced image generation process that generates a non-illuminated edge reduced image that satisfies the condition that edge intensity is reduced in the input image than in the background image, and the non-illuminated differential image, Water droplets may exist due to the logical product of at least one of the input high-intensity difference image and the input non-high-intensity difference image and the image obtained by the logical sum of the unilluminated input weak edge image and the unilluminated edge reduced image. It is preferable to perform a non-illuminated water droplet candidate image generation process for generating a high-quality non-illuminated water droplet candidate image, and determine the presence or absence of water droplets based on the generated non-illuminated water droplet candidate image.

  Furthermore, in the image signal processing device having the above configuration, the second water droplet determination process includes a background image captured by illuminating a monitoring space including the image capturing unit, and the image capturing unit after capturing the background image. The difference between the input image captured by illuminating the monitoring space and generating a positive difference image in which the brightness of the input image is higher than the background image, and the edge intensity in the input image A lighting input weak edge image generation process that generates a lighting input weak edge image that satisfies a condition that the threshold value is equal to or less than a predetermined threshold, and compares the edge strength in the input image with the edge strength in the background image, and the input image Illumination edge reduced image generation processing for generating an illumination edge reduced image that satisfies the condition that the edge intensity is reduced in the background image in the inside, and the illumination input weak edge image And an illumination water drop candidate image generation process for generating an illumination water drop candidate image with a high possibility of the presence of water droplets by a logical product of the image obtained by the logical sum of the illumination edge reduced images and the illumination difference image, It is preferable to determine the presence or absence of water droplets based on the illumination water droplet candidate image.

  According to the present invention, either when the image is captured using ambient light without using illumination light in a bright situation such as daytime, or when the image is captured by applying illumination light from the illumination unit in a dark situation such as nighttime. Even in the captured image under the imaging condition, the accuracy of the water drop detection process is improved. Accordingly, in any of the above-described imaging situations, it is possible to quickly take measures such as detection of water droplet adhesion and removal from the lens portion or the like. In addition, even when imaging is performed using only ambient light without using illumination light, it is possible to prevent unnecessary illumination light irradiation by using a water droplet detection process suitable for such an imaging situation. This is also advantageous in terms of power consumption.

  As shown in FIG. 1, the image sensing device 100 according to the embodiment of the present invention includes an imaging unit 10, an illumination unit 20, an illumination control unit 30, an intruder detection unit 40, a water droplet detection unit 50, and a determination unit 60. The image processing unit 70 and the output unit 80 can be configured by a computer that can execute a program for executing processing described below. In the present embodiment, the detection target is “intruder”, but is not limited to this. Therefore, although the component 40 is named the intruder detection unit, the name can be changed depending on the detection target, and the image sensing device 100 performs a process of detecting a desired detection target. It ’s fine.

  The imaging unit 10 is a camera that includes a photoelectric conversion element such as a CCD element or a C-MOS element that can acquire the monitoring space as two-dimensional image data, and a process that performs processing such as analog / digital conversion on the captured image. And an interface (not shown) for outputting image data. The imaging unit 10 captures an image of the monitoring space at a predetermined time period Ti (for example, 0.1 second) and outputs it as image data.

  The illumination unit 20 has a configuration in which, for example, a plurality of light emitting diode (LED) elements, lamps, and the like are arranged in a matrix, for example. The illumination unit 20 can acquire an image captured by the imaging unit 10 in a state where the monitoring space is irradiated at night or the like. In addition, the imaging part 10 and the illumination part 20 are accommodated in the housing | casing provided with the waterproof performance for outdoors. In the present invention, water droplets attached to the front glass are detected.

  The illumination control unit 30 calculates the average luminance of the entire input image by the imaging unit 10, and irradiates the monitoring space, assuming that it is difficult to monitor only with the brightness due to the ambient light as in the case of nighttime monitoring if it is below a predetermined value. The illumination unit 20 is controlled so that On the other hand, if the average luminance of the entire input image is equal to or higher than a predetermined value, the illumination unit 20 is controlled so that the illumination unit 20 does not perform irradiation, assuming that the brightness by natural light can be sufficiently monitored as in daytime monitoring. . For example, when the monitoring space at night is bright with a streetlight or the like, the illumination unit 20 is controlled according to the average luminance of the entire input image obtained by imaging the monitoring space with the streetlight or the like. Further, the illumination control unit 30 outputs an illumination presence / absence signal indicating which illumination control is performed (whether illumination is present) to the water droplet detection unit 50 (specifically, a program selection unit 58 described later).

  As a condition for determining the presence or absence of illumination, for example, in an AGC (Auto Gain Control) amplifier that amplifies an input image signal built in the imaging unit 10, if the input image signal exceeds the range in which the gain can be adjusted by the AGC amplifier, it is nighttime. It is also possible to set such a condition that it is regarded as being difficult (monitoring is difficult only by the brightness due to natural light), or if the level of the image signal is equal to or less than a predetermined value, it is regarded as nighttime (it is difficult to monitor only due to the brightness due to natural light). In the present embodiment, the illuminance of the monitoring space is determined using the input image data from the imaging unit 10, but an illuminance sensor is provided separately, and whether or not illumination is applied based on the illuminance acquired by the illuminance sensor. You may make it judge.

  The intruder detection unit 40 obtains input image data from the imaging unit 10 at a predetermined time period Ti, as in the conventional case, and includes a difference between the latest input image data and background image data captured in the past. A fluctuation area in the image is extracted based on the difference image data, and the attribute value of the area in the input image data corresponding to the fluctuation area is specific to the detection target (in this embodiment, “intruder”, that is, “person”). The presence of an “intruder” in the monitoring space is detected assuming that the detection target exists in the monitoring space. For example, an intruder is considered to be present when an attribute value representing “humanity” such as the size, aspect ratio, and luminance information of the input image data corresponding to the variable region satisfies a predetermined condition. ”Is detected. In addition, a signal indicating the detection result thus obtained is output to the determination unit 60.

  Similar to the intruder detection unit 40, the water droplet detection unit 50 acquires input image data at a predetermined time period Ti. As shown in FIG. 2, the water drop detection unit 50 includes a background image update unit 51, a background difference calculation unit 52, an edge extraction unit 53, a water drop candidate image generation unit 54, an interframe difference calculation unit 55, and a water drop candidate neighborhood image processing unit. 56, a water drop determination unit 57, a program selection unit 58, and a storage unit 59. The storage unit 59 can be composed of a semiconductor memory, a hard disk device, an optical disk device, or the like. The storage unit 59 stores and holds background image data necessary for calculating a background difference image and past water droplet candidate image data necessary for calculating movement information of a water droplet candidate image described later. The past water droplet candidate image data is image data of a water droplet candidate image extracted as described later in image data captured a predetermined time before the imaging time of the newest input image (for example, one frame before). The Processing in components other than the storage unit 59 will be described later.

  The water droplet detection unit 50 generates difference image data including a difference between the latest input image data and background image data captured in the past. The input image data, the background image data, and the difference image data are subjected to various processes described later, and based on the image data obtained by the various processes, it is determined whether or not the input image is an image caused by water droplets. . If it is determined that the image is caused by a water droplet, a signal indicating that the adhesion of the water droplet has been detected is output to the determination unit 60.

  Based on the results detected by the intruder detection unit 40 and the water droplet detection unit 50, the determination unit 60 determines whether or not a detection target exists in the monitoring space, and a water droplet adheres to the lens portion of the imaging unit 10. Whether or not it is determined. The determination result obtained here is output to the output unit 80.

  As described above, in the present embodiment, the image processing unit 70 in the image sensing device 100 includes the intruder detection unit 40, the water droplet detection unit 50, and the determination unit 60.

  The output unit 80 outputs an alarm signal or the like to the outside of the apparatus when the determination unit 60 determines that a detection target exists in the monitoring space or that water droplets are attached to the lens portion or the like of the imaging unit 10. Used for. For example, an alarm device including an alarm lamp and an alarm buzzer may be used, and a network interface for transmitting an alarm signal to a remote monitoring room is also preferable.

  Next, processing for image data performed by the image sensing device 100 according to the present embodiment will be described with reference to FIG.

  In step S <b> 1, the latest input image data is acquired, and the latest image data of the monitoring space captured by the imaging unit 10 is input to the intruder detection unit 40 and the water droplet detection unit 50. At this time, the illumination unit 20 is controlled by the illumination control unit 30 based on the input image data, and can be sufficiently monitored by the brightness of the ambient light, such as monitoring in bright conditions such as daytime or street lights. In some cases, an image captured using ambient light is acquired as the latest input image data. On the other hand, when it is difficult to monitor only with the brightness of ambient light, such as at night, an image captured in a state where the illumination space 20 illuminates the monitoring space is acquired as the latest input image data.

  In step S <b> 2, the intruder detection unit 40 detects the presence of “intruder” in the monitoring space. The intruder detection unit 40 extracts a fluctuation region in the image based on difference image data that is a difference between the latest input image data and background image data captured in the past. If the attribute value of the area in the input image data corresponding to the variable area has a characteristic characteristic of “intruder”, that is, “person”, for example, the size of the area in the input image data corresponding to the variable area If the attribute value indicating “humanity” such as aspect ratio and luminance information satisfies a predetermined condition, it is assumed that the “intruder” exists in the monitoring space and the presence of the “intruder” in the monitoring space. To detect. And the intruder detection part 40 outputs the signal which shows the detection result obtained in this way to the determination part 60 (step S3).

  In step S <b> 4, the water droplet detection unit 50 detects the adhesion of water droplets to the front surface of the imaging unit 10. The water droplet adhesion detection process performed by the water droplet detection unit 50 will be described with reference to FIG. The following image signal processing can be realized by executing an image signal processing program stored in advance in a storage device (not shown) or the like in the program selection unit 58.

  The image signal processing program includes, for example, a bright monitoring space program for processing a captured image captured using ambient light in a bright condition such as daytime and performing water droplet detection processing, and the illumination unit 20 at night or the like. There is a dark monitoring space program for processing a picked-up image picked up by illuminating light and performing water droplet detection processing. Then, before the following processing is performed, based on the signal from the illumination control unit 30, the program selection unit 58 selects an image signal processing program suitable for the imaging situation, and based on the selected image signal processing program. The following processing is realized by executing each component. In other words, unless otherwise specified, processing is performed on a captured image captured using ambient light without using illumination light even if the process name is the same, and captured image captured using illumination light is processed. There is processing.

  In step S4-1, background difference data is generated based on the latest input image data and background image data. The background difference calculation unit 52 reads the background image data from the storage unit 59 and generates background difference data by calculating the difference between the latest input image data acquired by the imaging unit 10 and the background image data. Specifically, each pixel included in the latest input image data is set as the target pixel in order, and the difference data for the target pixel is obtained by subtracting the luminance value of the pixel of the background image data corresponding to the pixel from the luminance value of the pixel. Is calculated. For example, if the difference data is a value greater than or equal to a predetermined threshold, the pixel value of the background difference data corresponding to the pixel of interest is set to “1”, and if the difference data is smaller than the predetermined threshold, the background difference corresponding to the pixel of interest The binarized background difference data is generated by setting the pixel value of the data pixel to “0”. That is, the pixel value of the image area where the input image data and the background image data fluctuated by more than the threshold value is “1”, and the pixel value of the image area where there was no fluctuation is “0”. It should be noted that an area constituted by a continuous pixel group among pixels having the pixel value “1” included in the background difference data forms one variable area.

  Here, in the processing in step S4-1, a threshold value TH1 of the luminance value (for example, about 200 when the luminance information is set in 256 gradations) is set, and similar binary values are set using this threshold value TH1. The process is also performed. Thereby, with respect to an image region (water droplet image region) generated due to water droplets in the input image data obtained by the above binarization processing, an input high-intensity region that is particularly greater than or equal to the threshold TH1 in the water droplet image region. Difference processing is also carried out separately from other regions (input non-high luminance regions) that are less than the threshold TH1, and input high luminance difference data for the input high luminance regions and input non-high for the input non-high luminance regions. Brightness difference data is generated. This is because when the image is captured using ambient light when it is bright such as in the daytime, the water droplet image region in the image captured using ambient light is a bright region (corresponding to the input high luminance region) and a dark region. This is because it is desirable to process both of them separately in the subsequent processing. On the other hand, when imaging is performed by illuminating the illumination light from the illumination unit 20 at night or the like, the entire area of the water droplet is brightened, and thus the water droplet image region is an input high luminance region. Therefore, in the following description, the processing of the input high-brightness area includes both the input high-brightness area when imaged using ambient light during daytime and the like, and the water droplet image area when imaged by illuminating illumination light during nighttime and the like. The processing for The threshold value TH1 corresponds to an example of a predetermined luminance value in the difference image generation process of the present invention.

  In addition, a threshold value TH2 (for example, about 80 to 160 when the luminance information is set in 256 gradations) is set as the minimum luminance that can be regarded as a water droplet image region, and the luminance value equal to or higher than the threshold value TH2 with respect to the input image data. This pixel may be extracted, and the pixel value of the above binarization process may be set to “1” only for the extracted pixel. According to this, there is a high possibility that an image region with too low luminance is not an image region caused by water droplets, and thus such an image region can be excluded from the processing target of the water droplet adhesion detection process. Each difference data generated in this way is passed to the water droplet candidate image generation unit 54.

  In step S4-2, the edge strength in the image is calculated for each of the latest input image data and background image data, and edge processing is performed. This process utilizes the feature that when a water droplet adheres to a lens portion or the like, an image area in which the water droplet is imaged is reflected in the image data in a blurred state due to the lens effect, and the change of the edge becomes remarkable. The edge extraction unit 53 calculates the edge strength in each image of the latest input image data acquired by the imaging unit 10 and the background image data in the storage unit 59 using an existing calculation method. For example, the edge strength is calculated by calculating the edge strength in the X direction (vertical direction of the image) and the edge strength in the Y direction (horizontal direction of the image), and adding the absolute values of the obtained edge strengths.

  The edge extraction unit 53 performs the above processing on the latest input image data and background image data, and generates the following image data using the input edge image data and background edge image data obtained thereby. First, image data (input weak edge image data) that satisfies the condition that the edge intensity is equal to or less than a predetermined threshold TH3 (for example, about 2 to 3 when luminance information is set in 256 gradations) in the input edge image. Generate. This is for using the feature that the edge intensity is weak inside the waterdrop image area that appears blurred for waterdrop detection. The threshold TH3 corresponds to an example of a predetermined threshold in the input weak edge image generation process of the present invention.

  Also, image data (input strong edge image data) that satisfies the condition that the edge strength is equal to or higher than a predetermined threshold TH4 (for example, about 20 when luminance information is set with 256 gradations) in the input edge image is generated. . This is because the entire area of the input high-brightness area becomes brighter, so the edge strength is not strong inside these input high-brightness areas, but the edge strength is strong near the periphery of these input high-brightness areas. It is for use.

  Further, the edge strength in the input edge image is equal to or higher than the predetermined threshold value TH4, and in the background edge image, the edge strength is equal to or lower than the predetermined threshold value TH5 (for example, about 20 when luminance information is set with 256 gradations). Image data (difference strong edge image data) that satisfies the following condition is generated. As a procedure for generating a differential strong edge image, first, the condition that the edge intensity in the background edge image is greater than or equal to a predetermined threshold TH6 (for example, about 20 when luminance information is set in 256 gradations) is greater than TH5. Image data (background strong edge image data) is generated. Next, difference strong edge image data is generated by taking a difference between the input strong edge image data and the background strong edge image data. This is because a strong edge does not newly appear inside the water drop image area, and the input high-brightness area is almost entirely bright, so a strong edge is likely to be formed in the peripheral part of these areas due to light refraction. In other words, in the area corresponding to the peripheral part of the input high-brightness area, the feature that there are many strong edges that do not exist in the background edge image but appear newly in the input edge image is used for water droplet detection. is there.

  Each edge image data generated by the edge extraction unit 53 as described above is passed to the water droplet candidate image generation unit 54.

  In step S4-3, the input edge image data extracted by the edge extraction unit 53 is compared with the edge intensity of the corresponding pixels in the background edge image data, and an image indicating a change in edge intensity between the two edge images is generated. The The water droplet candidate image generation unit 54 generates difference edge image data between the input edge image data and the background edge image data, and generates the following image data based on the difference edge image data.

  First, image data (edge-reduced image data) that satisfies the condition that the edge strength of the input edge image is lower than a background edge image by a predetermined value TH7 (for example, about 2 to 3 when luminance information is set in 256 gradations) or more. Is generated from the difference edge image. This is for using the feature that the edge intensity is reduced in the water droplet detection in the water droplet image region that appears blurred compared to the image region corresponding to the water droplet image region in the background image.

  Also, image data (edge-enhanced image data) that satisfies the condition that the edge strength of the input edge image is higher than a background edge image by a predetermined value TH8 (for example, about 2 to 3 when luminance information is set in 256 gradations) or more. Is generated from the difference edge image. This is for using the feature that the edge strength increases in the vicinity of the periphery of the water drop image area that appears blurred in comparison with the image area corresponding to the vicinity of the periphery of the water drop image area in the background image for water drop detection. Since the edge strength in the water drop image area in the input image is considered to be weaker than the image area corresponding to the water drop image area in the background image, the edge increase area is used when generating the water drop candidate image. It is effective to exclude.

  Here, for example, when a white wall is projected as a background image and the water droplet appears black due to light, the edge strength is increased inside the water droplet image region compared to the image region corresponding to the water droplet image region in the background image. To do. However, according to the definition of the edge-enhanced image, the edge-enhanced image resulting from such a water-drop image region is excluded when generating the water-drop candidate image, which is not desirable. Therefore, in order to cope with such a case, the threshold TH9 (for example, luminance information is set) in which the edge strength is not increased more than a certain amount in the water drop image area as compared with the image area corresponding to the water drop image area in the background image. When setting with 256 gradations, about 20) is newly set, and image data (edge greatly increased image data) that satisfies the condition that the edge strength increase amount is equal to or greater than the threshold TH9 is generated from the difference edge image. The edge greatly increased image satisfies the condition that the edge strength of the input edge image is higher than the background edge image by a predetermined threshold TH9 or more (here, TH9 ≧ TH8). The edge greatly increased image is excluded when the water droplet candidate image is generated.

  In step S <b> 4-4, a water droplet candidate image for determining whether or not the input image is an image that may have occurred due to the attachment of water droplets to the imaging unit 10 is generated. The water droplet candidate image generation unit 54 generates background difference data acquired from the background difference calculation unit 52, input high luminance difference data and input non-high luminance difference data, each edge image data acquired from the edge extraction unit 53, and water droplet candidate image generation. A water droplet candidate image is generated using each image data generated by the unit 54.

  Here, when an image is captured using ambient light in a bright monitoring space such as daytime, the water drop image area includes the input high-brightness area and the input non-high-brightness area formed in the process of step S4-1. included. Therefore, since the subsequent processing needs to be processed separately for the input high-brightness area and the input non-high-brightness area, in addition to the waterdrop candidate image (all waterdrop candidate images) for the waterdrop image area, Water droplet candidate images (high luminance water droplet candidate images) and water droplet candidate images (non-high luminance water droplet candidate images) for the input non-high luminance region are respectively generated.

  The all water droplet candidate images are images having the following characteristics. A feature 1 is an image (background difference data obtained in step S4-1) having a background luminance difference of a certain level or more and having a luminance of a certain level or more in the input image. Feature 2 is an image with weak edge strength (input weak edge image obtained in step S4-2) or an image with reduced edge strength (edge reduced image obtained in step S4-3). The feature 3 is not an image in which there is no strong edge in the background image but in the input image (difference strong edge image obtained in step S4-2), and an image in which the edge strength has not increased significantly (step S4-). 3). The edge greatly increased image obtained in step 3).

  The high brightness water droplet candidate image is an image having the following characteristics. The feature 1 is an image (the input high luminance difference data obtained in step S4-1) that has a background luminance difference of a certain level or more and satisfies the features of the input high luminance region. Feature 2 is an image with weak edge strength (input weak edge image obtained in step S4-2) or an image with reduced edge strength (edge reduced image obtained in step S4-3). The feature 3 is not an image in which there is no strong edge in the background image but in the input image (difference strong edge image obtained in step S4-2), and an image in which the edge strength has not increased significantly (step S4-). 3). The edge greatly increased image obtained in step 3).

  The non-high brightness water droplet candidate image is an image having the following characteristics. The feature 1 is an image (the input non-high luminance difference data obtained in step S4-1) that has a background luminance difference of a certain level or more and satisfies the features of the input non-high luminance region. Feature 2 and feature 3 are the same as the high brightness water droplet candidate image.

  On the other hand, a water droplet candidate image in the case where an image is taken by illuminating a dark monitoring space such as at night with the illumination unit 20 is an image having the following characteristics. The feature 1 is an image (background difference data obtained in step S4-1) having a positive background luminance difference and having a luminance of a certain level or more in the input image. Feature 2 is an image with weak edge strength (input weak edge image obtained in step S4-2) or an image with reduced edge strength (edge reduced image obtained in step S4-3). The feature 3 is not an image in which there is no strong edge in the background image but in the input image (difference strong edge image obtained in step S4-2), and an image in which the edge strength has not increased significantly (step S4-). 3). The edge greatly increased image obtained in step 3). Therefore, the water droplet candidate image generation unit 54 uses the background difference data, the input weak edge image data, the edge reduced image data, the difference strong edge image data, and the edge greatly increased image data, as the water droplet candidate image. Generate.

  In the following description, if it is simply referred to as a water droplet candidate image, unless otherwise specified, all water droplet candidate images in a bright imaging situation, high-intensity water droplet candidate images, and non-high-intensity water droplet candidate images, It is assumed that all the water drop candidate images in the case of a dark imaging situation are included.

  Here, as the features to be considered when generating the water droplet candidate image, the above-described features 1 to 3 are listed for each water droplet candidate image, but for images satisfying at least the features 1 and 2, the water droplet candidate image Can be generated. By generating an image satisfying not only the feature 1 and the feature 2 but also the feature 3, it is possible to generate a water droplet candidate image with higher accuracy.

  When the water droplet candidate image is generated as described above, the water droplet determination unit 57 determines whether or not a water droplet exists based on the water droplet candidate image. For example, when a region composed of pixels or pixel groups satisfying the above features is present in the water droplet candidate image, it is determined that there is a water droplet, and such a pixel or region is present in the water droplet candidate image. If it does not exist, it is determined that there is no water droplet. The water droplet determination unit 57 outputs a signal indicating that a water droplet has been detected to the determination unit 60 when it is determined that a water droplet is present, and a signal indicating that no water droplet is detected when it is determined that no water droplet is present. . However, the detection accuracy of water droplet detection can be further increased by performing the processing after step S4-5 described below.

  In step S4-5, in the water droplet candidate image generated by the water droplet candidate image generation unit 54, a certain range in the vicinity (peripheral portion) of the region composed of pixels or pixel groups satisfying the characteristics of the water droplet candidate image is It is specified as the peripheral region of the pixel or region. The water drop candidate vicinity image processing unit 56 generates an image indicating the peripheral area as a water drop candidate vicinity image. This is because the edge intensity tends to be weaker than the corresponding part of the background image due to the lens effect inside the water drop image area, whereas the edge intensity is higher than the background image due to refraction in the peripheral part. It is intended to capture the phenomenon of strong edges that tend to increase, and that a strong edge is likely to appear around brightly reflected water droplets. The water droplet candidate vicinity image is a pixel in a region formed by expanding a predetermined number of pixels (for example, 4 pixels) of a region composed of pixels or pixel groups satisfying each feature of the water droplet candidate image. Alternatively, a portion not included in the region is shown as the peripheral region of the pixel or region.

  In step S4-6, the water drop candidate vicinity image processing unit 56 generates an image related to a strong edge included in the water drop candidate vicinity image. First, image data (neighboring strong edge image data) obtained by the logical product of the strong edge image acquired in step S4-2 and the water droplet candidate neighboring image is generated. In this logical product process, a differential strong edge image is used as a strong edge image. This is because it can be assumed that a bright edge is likely to appear newly in the peripheral portion when compared with the background when it appears bright. On the other hand, if the image is not bright, it can be assumed that a strong edge is unlikely to appear compared to the background. In addition, for a non-high brightness water droplet candidate image when imaged using ambient light in the daytime or the like, an input strong edge image may be used as a strong edge image used in the above logical product processing. As described above, it is preferable to use the differential strong edge image. Next, image data (neighboring edge augmented image data) obtained by the logical product of the edge augmented image acquired in step S4-3 and the water droplet candidate nearby image is generated.

  The above two types of image data are used for the following purposes. First, when captured using ambient light without using illumination light, such as in the daytime, it is used to capture the following characteristics of the input high-brightness area and input non-high-brightness area in the water drop image area. It is. First, each of the peripheral areas of the input high-brightness area and the input non-high-brightness area has a feature that edges are increased compared to the background. Further, there is a feature that edge strength is strong in the peripheral region of the input high brightness region, and edge strength is not strong in the peripheral region of the input non-high brightness region. Therefore, by using the neighborhood strong edge image data and the neighborhood edge augmented image data, an image that does not satisfy the edge feature in the peripheral region can be excluded from the water droplet candidate images.

  The above two types of image data are also used to distinguish the water droplet candidate images because they may be due to other causes that are not actually water droplets. Other causes include moving objects that exist in the surveillance space, for example, in light conditions such as daytime, and in addition, for example, in the dark space such as nighttime, there are flashlights and automobiles in the surveillance space. There are areas of bright light created by the insertion of headlights.

  Specifically, in the case of the light region, the area (or the total number of pixels) where the strong edge exists in the neighborhood strong edge image is small and the area of the portion where the edge strength is increased in the neighborhood edge increased image ( Another feature is that the total number of pixels) is large. This is due to the feature of the light region that the edge that becomes stronger than the background increases in the contour portion of the light region, but there are not many strong edges because the luminance changes gently like gradation. In the case of a moving object, the area (or the total number of pixels) where the strong edge exists in the near strong edge image is large, and the area (or the total number of pixels) where the edge strength increases in the near edge increased image. There is a feature that is small. On the contrary, this is due to the feature of the moving object that the edge strength is high because the brightness changes sharply in the contour portion, but the amount of increase of the edge that is stronger than the background is not large. On the other hand, in the case of water droplets, the area (or total number of pixels) where the strong edge exists in the near strong edge image is large, and the area (or total pixel) where the edge strength increases in the near edge increased image. Number) tends to be large. Therefore, by using the near strong edge image data and the near edge augmented image data, it is possible to exclude images caused by other causes from the water droplet candidate images.

  After performing the exclusion process from the water droplet candidate image as described above, the water droplet determination unit 57 determines whether or not a water droplet exists based on the water droplet candidate image after the exclusion process. The method of determination is the same as the determination process in step S4-6. The water droplet determination unit 57 outputs a signal indicating that a water droplet has been detected to the determination unit 60 when it is determined that a water droplet is present, and a signal indicating that no water droplet is detected when it is determined that no water droplet is present. . Thereby, water droplet detection can be performed with high detection accuracy. However, the detection accuracy of further water droplet detection can be further increased by performing the processing after step S4-7 described below.

  In step S4-7, the difference processing with the past water droplet candidate image data in the storage unit 59 is performed on the water droplet candidate image generated by the water droplet candidate image generation unit 54. The inter-frame difference calculation unit 55 is a predetermined time before the latest input image capturing time stored in the storage unit 59 with respect to the water droplet candidate image generated by the water droplet candidate image generation unit 54 (for example, 1 Inter-frame difference data is generated by calculating a difference from the previous water droplet candidate image data before the frame). Then, for example, an inter-frame difference area to be described later is calculated using the generated inter-frame difference data. Thereby, the information regarding the movement of the water droplet candidate image is acquired. This is because higher detection accuracy can be expected by taking in information on the motion of the water droplet candidate image and dividing the determination conditions for water droplets by distinguishing between the case where the water droplets do not move and the case where they flow down.

  The inter-frame difference calculation unit 55 determines the area (or the total number of pixels) of a pixel or a group of pixels determined by exclusive OR of a water droplet candidate image obtained from the current input image and a past water droplet candidate image. Is calculated as an inter-frame difference area. Further, the positions of the centers of gravity of the latest water droplet candidate image data and the past water droplet candidate image data are calculated.

  In step S4-8 and subsequent steps, the water droplet candidate image generated by the water droplet candidate image generation unit 54 is an image caused by water droplets staying without flowing down (staying water droplets) or caused by water droplets flowing down (flowing water droplets). It is determined whether or not the image has been processed. The water drop determination unit 57 is generated by the background difference calculation unit 52 from the water drop candidate image generated by the water drop candidate image generation unit 54 and the movement information of the water drop candidate image calculated by the inter-frame difference calculation unit 55. It is determined whether or not the background difference data thus obtained is an image caused by staying water droplets or an image caused by flowing water droplets.

  In step S4-8, determination information necessary for determining whether the water droplet candidate image is an image caused by a staying water droplet or an image caused by a flowing water droplet is calculated. Here, the following determination information is calculated by each process shown in FIG.

  In step S4-8-1, the area ratio F1, which is the ratio of the area composed of pixels or pixel groups satisfying the characteristics of the water droplet candidate image in one frame of the water droplet candidate image, is expressed as area ratio F1 = (each It is calculated by: (area of pixel or region satisfying feature) / (area of one frame of water drop candidate image). The area ratio F1 is for reflecting the feature that a pixel or a region satisfying each feature of the water drop candidate image should have a size of a certain level or more in the water drop candidate image. The area used here may be the total number of pixels.

  In step S4-8-2, using the inter-frame difference area obtained in step S4-7, the difference area ratio F2 is calculated as follows: difference area ratio F2 = (inter-frame difference area) / (latest and / or past). The area of the waterdrop candidate image that appears). The area of the water droplet candidate image that appears in at least one of the latest and the past is the area of an image generated by the logical sum of the latest water droplet candidate image and the past water droplet candidate image. The difference area ratio F2 is a water droplet candidate image that indicates how many pixels or groups of pixels that satisfy each feature of the water droplet candidate image appear or disappear at a predetermined time interval (for example, one frame). Expressed as a percentage of the total. The area used here may be the total number of pixels.

  In step S4-8-3, using the water droplet candidate vicinity image obtained in step S4-5, the neighborhood image area ratio F3 is calculated as follows: neighborhood image area ratio F3 = (pixel or region satisfying each feature of the water droplet candidate image) The area of the peripheral region of the image) / (the area of one frame of the water droplet candidate vicinity image). The near image area ratio F3 is such that when the water drop image area is projected out of the image frame and the surrounding area of the pixel or area satisfying each feature of the water drop candidate image is largely missing. Since the water droplet determination cannot be performed properly, the information is used for excluding the determination target. The area used here may be the total number of pixels.

  In step S4-8-4, the vicinity strong edge image ratio F4 is set to the vicinity strong edge image using the water droplet candidate vicinity image obtained in step S4-5 and the vicinity strong edge image obtained in step S4-6. Ratio F4 = (area of a region where a strong edge exists in the peripheral region of the pixel or region satisfying each feature of the water droplet candidate image) / (area of one frame of the water droplet candidate vicinity image). The neighborhood strong edge image ratio F4 represents how many strong edges are included in the water droplet candidate neighborhood image. The area used here may be the total number of pixels.

  In step S4-8-5, using the water drop candidate neighborhood image obtained in step S4-5 and the neighborhood edge augmented image obtained in step S4-6, the neighborhood edge augmented image ratio F5 is set to the neighborhood edge augmented image. The ratio F5 = (area of the region where the edge intensity is increased in the peripheral region of the pixel or region satisfying each feature of the water droplet candidate image) / (area of one frame of the water droplet candidate vicinity image). The near edge increased image ratio F5 represents how much the portion where the edge strength is increased in the input image as compared to the background image is included in the water droplet candidate vicinity image. The area used here may be the total number of pixels.

  In step S4-8-6, the position of the center of gravity of the latest water droplet candidate image data and the past water droplet candidate image data is calculated in the inter-frame difference data generated in step S4-7, and the coordinate value in the image is calculated. Is calculated as the amount of change in the center of gravity. The coordinate axes in the image can be arbitrarily set. For example, the upper left corner of the image is set as the origin, the horizontal axis is set as the X axis (right direction is positive), and the vertical axis is set as Y axis (down direction is positive). Then, the coordinates of the center of gravity of the latest water droplet candidate image data and the past water droplet candidate image data are calculated using the set coordinate axes, and the difference between the X coordinates is calculated as (X coordinate of the latest water droplet candidate image data) − (past water droplet candidate image data X coordinate), and a difference in Y coordinate is obtained from (Y coordinate of latest water droplet candidate image data) − (Y coordinate of past water droplet candidate image data). These indicate how much the water droplet image area has moved. If the difference in the X coordinate, that is, the movement amount ΔX in the X direction is positive, it indicates that the water droplet has moved in the right direction on the screen, and the difference in the Y coordinate. That is, if the movement amount ΔY in the Y direction is positive, it indicates that the water droplet has moved downward in the screen. Thus, the movement information of the water droplet is acquired.

  As described above, in step S4-8, each piece of determination information necessary for determining whether the water droplet candidate image is an image caused by a staying water droplet or an image caused by a flowing water droplet is calculated. When each determination information is calculated in this way, the process proceeds to step S4-9 in FIG.

  In step S4-9, the water droplet determination unit 57 determines whether or not the water droplet candidate image is an image caused by a staying water droplet. This processing differs depending on whether the imaging situation is bright or dark. Hereinafter, each imaging situation will be described.

  When the image is picked up using ambient light in the daytime, etc., it is determined whether or not the following conditions are satisfied for the high-brightness water droplet candidate image, the non-high-brightness water droplet candidate image, and the total water droplet candidate image, If at least one of the determination results obtained is determined to be due to staying water droplets, the water droplet candidate image is determined to be an image resulting from the staying water droplets.

  The conditions for the high-intensity water droplet candidate image are: <Residence condition 1> The area ratio F1 is not less than a predetermined threshold THF1-1 (for example, 2%) (since the retained water droplet has a certain area or more), <Residence Condition 2> The difference area ratio F2 is equal to or less than a predetermined threshold value THF2 (for example, 5 to 10%) (because the water droplets in the staying state move less), <Retention condition 3> (2 to 3%) or more (because the water droplet candidate image cannot be properly determined unless the area of the water droplet candidate image has a certain area or more) <Residual condition 4> The near strong edge image ratio F4 is the threshold value THF4. −1 (for example, 10 to 50%) or more (because there are many strong edges around the water droplets), <Residual condition 5> The near edge increased image ratio F5 is not less than the threshold THF5-1 (for example, 10 to 50%). (Because the area of the image in which the edge increases in the peripheral part of the water droplet is large), and if these conditions are satisfied, the high-intensity water droplet candidate image to be judged is an image caused by the staying water droplet judge.

  The conditions for the non-high-intensity water droplet candidate images are the same as those for the high-intensity water droplet candidate images except for <Residence Condition 4> and <Residence Condition 5>. Is less than the threshold THF4-2 (for example, 10%) (because there is no strong edge in the peripheral portion of the water droplet), and for <Residual Condition 5>, the near edge increased image ratio F5 is the threshold THF5-2 (for example, 10% to 50%) (because the area of the image where the edge increases in the surrounding area of the water droplet is large), and if these conditions are satisfied, the non-high-intensity water droplet candidate image to be judged is an image caused by the staying water droplet It is determined that

  The conditions for all the water droplet candidate images satisfy the above <Retention Condition 1> to <Retention Condition 3>, and <Retention Condition 4> and <Retention Condition for each of the high-intensity water droplet candidate image and the non-high-intensity water droplet candidate image. 5> is satisfied. If these conditions are satisfied, it is determined that the all-droplet candidate image to be determined is an image caused by the staying waterdrop.

  On the other hand, when an image is taken with illumination at night or the like, it is determined whether or not the following conditions are satisfied. Each condition is <Residence condition 1> The area ratio F1 is not less than a predetermined threshold THF1-1 (for example, 2%) (because the staying water droplet has a certain area or more), <Residence condition 2> Difference area ratio F2 is equal to or less than a predetermined threshold value THF2 (for example, 5 to 10%) (due to less moving water droplets in a staying state), <Retention Condition 3> The neighborhood image area ratio F3 is equal to or more than a threshold value THF3 (for example, 2 to 3%) (If the area of the water droplet image region is not in a state of a certain area or more, the water droplet determination cannot be performed properly), and <Residual condition 4> The neighborhood strong edge image ratio F4 is a threshold THF4-1 (for example, 10 to 10). 50%) or higher (since there are many strong edges in the surrounding area of the water droplet), <Residual condition 5> The near edge increased image ratio F5 is equal to or higher than the threshold THF5 (for example, 10 to 50%) (the peripheral portion of the water droplet). so Since the area of the image edges is increased greater), it is those such as, determines that satisfy these conditions, water droplets candidate image to be determined is an image due to retention water droplets.

  In step S4-10, the water droplet determination unit 57 determines whether or not the water droplet candidate image is an image caused by a flowing water droplet. Specifically, similarly to the processing in step S4-9, the processing differs depending on whether the imaging condition is bright or dark. Hereinafter, each imaging situation will be described.

  When the image is picked up using ambient light in the daytime, etc., it is determined whether or not the following conditions are satisfied for the high-brightness water droplet candidate image, the non-high-brightness water droplet candidate image, and the total water droplet candidate image, If there is at least one determination result that is based on flowing water droplets among the determination results obtained, it is determined that the water droplet candidate image is an image caused by flowing water droplets.

  The conditions for the high-intensity water droplet candidate image are: <Flow condition 1> The area ratio F1 is equal to or greater than a predetermined threshold THF1-2 (for example, 10%) (because the flow droplet has a certain area or more), <Flow Condition 2> The neighborhood image area ratio F3 is equal to or greater than a threshold THF3 (for example, 2 to 3%) (because the water droplet image area cannot be determined properly unless the area of the water droplet image region has a certain area or more), < Flow condition 3> Near edge strong edge image ratio F4 is greater than or equal to the threshold THF4-1 (for example, 10 to 50%) (because there are many strong edges around the water droplet), <Flow condition 4> Near edge increased image ratio F5 Is greater than or equal to the threshold THF5-1 (for example, 10 to 50%) (because the area of the image in which the edge increases in the peripheral portion of the water droplet is large), <flow condition 5> the movement amount ΔY is a predetermined threshold THΔY (for example, 5%) or more (because the flowing water droplets should have flowed downward for a certain distance or more) <Flow condition 6> The movement amount ΔY is a value greater than or equal to the absolute value of the movement amount ΔX. (For flowing water droplets, the amount of movement in the downward direction should be larger than the amount of movement in the left and right direction.) If these conditions are met, the high-intensity water droplet candidate image to be judged will flow. It is determined that the image is caused by water droplets.

  The conditions for the non-high-intensity water droplet candidate images are the same as the conditions for the high-intensity water droplet candidate images except for <flow condition 3> and <flow condition 4>. For <flow condition 3>, the neighborhood strong edge image ratio F4 Is less than the threshold THF4-2 (for example, 10%) (because there is no strong edge in the surrounding portion of the water droplet), and for <flow condition 4>, the neighboring edge increased image ratio F5 is the threshold THF5-2 (for example, 10% to 50%) or more (because the area of the image where the edge increases in the surrounding area of the water droplet is large), and if these conditions are met, the non-high brightness water droplet candidate image to be judged is an image caused by the flowing water droplet It is determined that

  The conditions for all the water droplet candidate images satisfy the above <flow condition 1>, <flow condition 2>, <flow condition 5>, and <flow condition 6>, and the high brightness water drop candidate image and the non-high brightness water drop candidate image. For each of the above, <flow condition 3> and <flow condition 4> are satisfied. If these conditions are satisfied, it is determined that the determination target all water droplet candidate image is an image caused by the flowing water droplets.

  On the other hand, when an image is taken with illumination at night or the like, it is determined whether or not the following conditions are satisfied. Each condition is <flow condition 1> the area ratio F1 is equal to or greater than a predetermined threshold THF1-2 (for example, 10%) (because the flowing water droplet has a certain area or more), and <flow condition 2> neighboring image area The ratio F3 is equal to or greater than the threshold THF3 (for example, 2 to 3%) (because the water droplet image area cannot be properly determined unless the area of the water droplet image area is a certain area or more), <flow condition 3> strong in the vicinity The edge image ratio F4 is equal to or higher than the threshold THF4-1 (for example, 10 to 50%) (because there are many strong edges around the water droplet), and <flow condition 4> the near edge increased image ratio F5 is the threshold THF5 (for example, 10). ˜50%) (because the area of the image where the edge increases in the peripheral portion of the water droplet is large), <flow condition 5> the movement amount ΔY is greater than or equal to a predetermined threshold THΔY (for example, 5%) (flowing water <Flow condition 6> The amount of movement ΔY is greater than or equal to the absolute value of the amount of movement ΔX. If the movement amount in the downward direction should be larger than the movement amount), and these conditions are satisfied, it is determined that the water droplet candidate image to be determined is an image caused by the flowing water droplets.

  As described above, the water droplet determination unit 57 determines whether the water droplet candidate image is an image caused by a staying water droplet or an image caused by a flowing water droplet in steps S4-9 to S4-11, and the obtained determination result. Based on the above, it is possible to detect a water droplet in consideration of the state of the water droplet. And the water droplet determination part 57 outputs the signal which shows the obtained detection result to the determination part 60 in FIG. 1 (step S5). As the detection result, it is possible to indicate only the presence or absence of water droplets, or it is possible to include information on whether the water droplets are staying or flowing water droplets in the detection results and indicate that there is water droplet adhesion. Thus, the water droplet adhesion detection process in the water droplet detection unit 50 is completed, and then the process proceeds to step S6 in FIG.

  In step S <b> 6, the determination unit 60 determines whether there is a detection target in the monitoring space based on the detection result signals from the intruder detection unit 40 and the water droplet detection unit 50, and drops water on the lens portion of the imaging unit 10. It is determined whether or not the material is attached. When the determination unit 60 detects the presence of an intruder or determines that the attachment of water droplets has been detected, the process proceeds to step S7, and neither the presence of an intruder nor the attachment of water droplets is detected. The process is shifted to step S8.

  In step S7, an alarm signal is output from the output unit 80. When the output unit 80 is an alarm device including an alarm lamp and an alarm buzzer, the lamp is turned on and a buzzer sound is generated. At this time, if only the presence of an intruder is detected, if only the attachment of a water drop is detected, or if both an intruder and a water drop are detected, a different type of alarm signal is output, and a different type of lamp is lit. It is good to make it generate a buzzer sound of another kind. When the output unit 80 is a network interface, an alarm signal can be transmitted to a remote monitoring room via the Internet or a dedicated line. In this case, the alarm signal may be transmitted as another type of alarm signal according to the above three cases. At this time, it is preferable to transmit the input image data to be detected and determined together with the alarm signal. In step S7, when a signal indicating that a water droplet has been detected is output from the output unit 80, based on the output signal, a blower that blows warm air on the front surface of the imaging unit 10 or a wiper that removes attached water droplets. (Not shown) may be activated.

  In step S8, background image data update processing is performed. The background image update unit 51 updates the background image data held in the storage unit 59 with the latest input image data at a predetermined background update cycle Tr (for example, 10 seconds). It is preferable to determine the background update period Tr in consideration of fluctuations in sunlight with respect to the monitoring space. As another method, the background image data may be updated with the input image data determined that “person” does not exist, and the background is obtained by moving average image data of a plurality of input image data acquired in the past. The image data may be updated.

  In the present embodiment, the water droplet detection process is performed for the entire frame of the latest input image acquired by the imaging unit 10, but it is obtained by calculating the difference between the latest input image data and the background image data. The same water droplet detection process may be performed for each fluctuation region in the background difference data. In addition, the water droplet detection process for the entire frame of the latest input image and the water drop detection process for each fluctuation region in the background difference data may be performed in parallel.

  As described above, according to the present embodiment, even when imaging a monitoring space using ambient light without using illumination light in the daytime or the like outdoors, or by applying illumination light at night or the like, the monitoring space Since the water droplet detection process is switched in accordance with each imaging situation even when the image is captured, the water droplet image area in the captured image can be detected with high accuracy in any imaging situation. Thereby, measures such as removing water droplets from the lens portion and the like can be taken promptly. Further, even when imaging is performed using only ambient light without using illumination light, water droplets can be detected with high accuracy, so that unnecessary illumination light irradiation can be prevented, which is advantageous in terms of power consumption.

It is a figure which shows the structure of the sensing apparatus in embodiment of this invention. It is a figure which shows the structure of the water droplet detection part in embodiment of this invention. It is a flowchart of the sensing process in embodiment of this invention. It is a flowchart of the water droplet adhesion detection process in the embodiment of the present invention. It is a flowchart of the water droplet determination information calculation process in embodiment of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 Imaging part, 20 Illumination part, 30 Illumination control part, 40 Intruder detection part, 50 Water drop detection part, 51 Background image update part, 52 Background difference calculation part, 53 Edge extraction part, 54 Water drop candidate image generation part, 55 frames Difference calculation unit, 56 water droplet candidate vicinity image processing unit, 57 water droplet determination unit, 58 program selection unit, 59 storage unit, 60 determination unit, 70 image processing unit, 80 output unit, 100 image sensing device.

Claims (3)

An illuminance detection unit that detects imaging illuminance in the monitoring space when the monitoring space is imaged by the imaging unit;
Based on the imaging illuminance detected by the illuminance detection unit, an illumination control unit that controls an illumination unit that irradiates illumination light to a monitoring space including the imaging unit;
A water droplet detection unit that determines whether or not an image region due to water droplets exists in the input image captured by the imaging unit, and detects adhesion of water droplets to the imaging unit;
Have
The water droplet detector is
When the illumination unit does not irradiate illumination light, water droplets are detected by the first water droplet determination process,
When the illumination unit irradiates the monitoring space including the imaging unit with illumination light, water droplets are detected by the second water droplet determination process.
An image signal processing apparatus. The image signal processing apparatus according to claim 1,
The first water droplet determination process is:
Generating an unilluminated differential image that satisfies a condition that a luminance change has occurred from a difference between a background image captured by the imaging unit and an input image captured after the background image is captured; An input high-intensity difference image that satisfies the condition that the corresponding pixel of the input image is greater than or equal to a predetermined luminance value, and a condition that the corresponding pixel of the input image in the difference image is less than the predetermined luminance value An unilluminated difference image generation process for generating an input non-high luminance difference image satisfying
An unilluminated input weak edge image generation process for generating an unilluminated input weak edge image that satisfies the condition that the edge intensity in the input image is equal to or less than a predetermined threshold; and
Non-illumination that compares the edge strength in the input image with the edge strength in the background image and generates a non-illuminated edge reduced image that satisfies the condition that the edge strength is reduced in the input image as compared with that in the background image Edge reduced image generation processing;
Logical product of at least one of the unilluminated difference image, the input high-intensity difference image, and the input non-high-intensity difference image, and an image obtained by a logical sum of the unilluminated input weak edge image and the unilluminated edge reduced image By the unilluminated water droplet candidate image generation process for generating an unilluminated water droplet candidate image with a high possibility of water droplets,
And determining whether or not there is a water droplet based on the generated unilluminated water droplet candidate image. The image signal processing apparatus according to claim 1 or 2,
The second water drop determination process is:
The difference between the background image captured by illuminating the monitoring space including the imaging unit and the input image captured by illuminating the monitoring space including the imaging unit after capturing the background image is obtained as a background image. Lighting difference image generation processing for generating a positive difference image in which the luminance of the input image is higher,
Illumination input weak edge image generation processing for generating an illumination input weak edge image that satisfies the condition that the edge intensity in the input image is equal to or less than a predetermined threshold;
The edge intensity in the input image is compared with the edge intensity in the background image, and an illumination edge reduction image is generated that satisfies the condition that the edge intensity is reduced in the input image than in the background image. Image generation processing,
Illumination water droplet candidate image generation that generates an illumination water droplet candidate image that is highly likely to have water droplets, based on the logical product of the illumination difference image and the image obtained by the logical sum of the illumination input weak edge image and the illumination edge reduced image Processing,
And determining whether or not there is a water droplet based on the illumination water droplet candidate image.

Images