WO2020179240A1

WO2020179240A1 - Image verification system

Info

Publication number: WO2020179240A1
Application number: PCT/JP2020/001298
Authority: WO
Inventors: 純一富樫
Original assignee: 株式会社日立国際電気
Priority date: 2019-03-01
Filing date: 2020-01-16
Publication date: 2020-09-10
Also published as: JP7214830B2; JPWO2020179240A1

Abstract

Provided is an image verification system with which it is possible to obtain stable verification accuracy without having to change settings on the side of an imaging device. An image quality analysis unit 22 identifies the quality of a shot image captured by an imaging device 10; a face detection unit 23 detects a face image from the shot image; when the face image has been detected from a shot image by the face detection unit 23, a quality evaluation unit 25 evaluates whether or not there is a need to make adjustments to the quality of a registered face image; when the quality evaluation unit 25 has determined that the quality of the registered face image needs adjustments, an image quality adjustment unit 26 makes adjustments to the quality of the registered face image so as to be closer to the quality of the shot image, on the basis of the quality of the registered face image and the quality of the shot image; and a verification unit 27 performs verification on the detected face image using the registered face image that has undergone quality adjustment made by the image quality adjustment unit 26.

Description

Image matching system

The present invention relates to an image matching system that collates an image taken by an imaging device with an image registered in a database.

In recent years, video surveillance systems have been installed for security purposes such as crime deterrence and accident prevention in facilities visited by an unspecified number of people such as airports, train stations, and shopping malls. Examples of such a video monitoring system include an abandoned object detection system that monitors an image and detects an abandoned object, and a person search system that searches a video for a designated person. In addition to the video monitoring system for security purposes, a video analysis system may be constructed for marketing purposes. As such a video analysis system, for example, there is a face image analysis system that identifies a person in a facility and identifies attributes such as age and sex.

International Publication No. 2016/151802

In the case of a video surveillance system that handles face images and aims to identify people and identify attributes, it is necessary to register the face image data of the person you want to identify in advance. For example, Patent Document 1 discloses a face matching system that extracts a feature amount from photographed image data and performs matching by comparing the feature amount with a feature amount registered in a database.

It is difficult to achieve stable matching accuracy in conventional face image analysis systems, especially in systems for matching such as face matching, regardless of environmental changes such as installation conditions of image pickup devices such as cameras and sunshine conditions. Met. For example, in a video surveillance system that pre-registers face images shot in an environment with well-defined indoor lighting conditions, the environment of the camera that shoots in real time changes moment by moment, and the face images are especially changed at night or in the direction of lighting. If a shadow is formed, the image looks completely different from the face image registered in advance, so the matching accuracy tends to decrease.

In order to improve these, it is important to make the video quality of the camera image shot in real time and the pre-registered face image at the same level as much as possible. Therefore, conventionally, the quality of the pre-registered face image is recorded, and the process of bringing the quality of the camera image close to the quality of the pre-registered face image is performed by adjusting the setting of the camera that shoots in real time. It was done.

General cameras are equipped with correction functions such as auto white balance, auto gain, and wide dynamic range (WDR) to reduce the effects of environmental conditions such as lighting and maintain a certain level of image quality. It is designed to have a certain level of image quality. However, when the camera setting is made close to the quality of the face image registered in advance, only the camera image only for improving the accuracy of face matching can be captured. Therefore, in some cases, the video quality may be insufficient for monitoring purposes. That is, in the case of the video surveillance system, it is difficult to satisfy both the surveillance purpose and the matching purpose.

The present invention has been made in view of the above-mentioned conventional circumstances, and an object of the present invention is to provide an image matching system capable of obtaining stable matching accuracy without changing the settings on the imaging device side.

In order to achieve the above object, in the present invention, the image matching system is configured as follows.
That is, in an image collation system that collates a captured image captured by an imaging device with a registered image registered in advance, a storage unit that stores the registered image and its quality, and a specifying unit that identifies the quality of the captured image. Adjusting means for adjusting the quality of the registered image so as to approach the quality of the taken image based on the quality of the taken image and the quality of the registered image, and the registered image after adjustment by the adjusting means. It is characterized by comprising a collating means for collating the photographed images.

As described above, in the image collation system according to the present invention, the registered image is temporarily adjusted so as to approach the quality of the photographed image, and is used for collation of the photographed image. Thereby, the captured image can be collated in a state where the quality of the captured image and the quality of the registered image are adjusted to a level suitable for image collation (for example, a similar level). Therefore, since the quality of the captured image does not need to be adjusted, stable matching accuracy can be obtained without changing the setting on the imaging device side. Furthermore, since it is not necessary to change the setting on the imaging device side, it is possible to use the captured image not only for collation purposes but also for monitoring purposes.

In addition, the adjustment means, when the difference between the quality of the captured image and the quality of the registered image is a predetermined value or more, when the quality of the captured image deviates from a predetermined condition, or when a timing scheduled in advance has arrived. In some cases, it is preferable to configure the registered image so as to adjust the quality. As a result, it is possible to avoid making adjustments even when adjustment of the quality of the registered image is unnecessary, and it is possible to reduce the processing load.

Here, in the image matching system according to the present invention, it is possible to use various qualities that may affect matching accuracy as the quality, such as contrast, white balance, compression level, and brightness level. 1 or more can be used.

Further, the image matching system according to the present invention can be used in various systems. For example, when a person's face image is used as the registered image and a person's face image is detected from the captured image, the person's face image is used. A face collation system that collates the detected face image by using the registered image whose quality is adjusted so as to be closer to the detected face image can be provided.

According to the present invention, it is possible to provide an image matching system that can obtain stable matching accuracy without changing the setting on the imaging device side.

It is a figure which shows the schematic structure of the face matching system which concerns on one Embodiment of this invention. It is a figure which shows the structural example of the image processing apparatus in the face collation system of FIG. It is a figure which shows the example of the flowchart of the face matching by the face matching system of FIG.

An embodiment of the present invention will be described with reference to the drawings. In the following, a face matching system, which is an example of the image matching system, will be described as an example.
FIG. 1 shows a schematic configuration of a face matching system according to an embodiment of the present invention. The face matching system of this example includes an imaging device 10, an image processing device 20, a recording device 30, and an operation terminal 40.

The imaging device 10 is basically a setting suitable for the purpose of monitoring, and the setting is not changed one by one for the purpose of matching. Further, the image pickup apparatus 10 has a function of automatically adjusting the quality of the photographed image in response to a change in environmental conditions at the time of photography (lighting / extinguishing, change in weather, etc.). In FIG. 1, as the imaging device 10, three network cameras 10-1, an analog camera 10-2, and a network camera 10-3 are illustrated, but the number of these devices is arbitrary. The network cameras 10-1 and 10-3 have a built-in encoder that encodes the captured image for communication, and are directly connected to the network. The analog camera 10-2 is connected to the network via a separate encoder 12 that encodes the captured image for communication. The image captured by the image pickup device 10 is transmitted to the image processing device 20 via the network.

The recording device 30 stores a face image of a person registered in advance and data of the quality thereof on a hard disk or the like. It should be noted that the recording device 30 may also store data such as attributes such as a person's name and sex, and face feature amounts obtained by analyzing a face image. As the face image of the person to be pre-registered, for example, an image taken in an environment in which indoor lighting conditions are prepared is used. Contrast, white balance, compression level, brightness level and the like are used as the quality of the face image, but various other qualities that may affect the accuracy of face matching may be used.

The image processing apparatus 20 detects a face image of a person from the image captured by the image capturing apparatus 10 and performs face matching to match the detected face image with a face image registered in advance in the recording device 30. In the following, when distinguishing a face image detected from a captured image and a face image registered in advance, the former is referred to as a “detected face image” and the latter is referred to as a “registered face image”. Details of the image processing device 20 will be described later.
The operation terminal 40 is a terminal used by the user of the face matching system, and displays the result of face matching by the image processing device 20 and the captured image by the imaging device 10.

FIG. 2 shows a configuration example of the image processing device 20 in the face matching system of this example.
The image processing apparatus 20 includes a captured image reception unit 21, an image quality analysis unit 22, a face detection unit 23, a registration data acquisition unit 24, a quality evaluation unit 25, an image quality adjustment unit 26, and a collation unit 27. Equipped with.

The captured image receiving unit 21 communicates with the image capturing apparatus 10 and receives in real time the captured image obtained by capturing with the image capturing apparatus 10.
The image quality analysis unit 22 analyzes the captured image received by the captured image reception unit 21, and specifies a plurality of types of quality such as contrast, white balance, compression level, and brightness level of the captured image. For example, for white balance, the RGB levels (R (Red), G (Green), and B (Blue) color levels) of each pixel included in the captured image are analyzed. The registered face image is corrected so as to approach the RGB level. As for the brightness level, a part (for example, nose) from the position of the face detected from the captured image is analyzed. The registered face image is corrected so as to approach the brightness level. In addition, when the imaging device 10 transmits the quality data to the captured image in association with each other, the quality of the captured image can be obtained without analyzing the captured image. The quality of the captured image specified by the image quality analysis unit 22 is temporarily stored in the memory together with the captured image for subsequent processing.

The face detection unit 23 performs a face detection process on the captured image received by the captured image reception unit 21 to detect a face image included in the captured image. As the face detection processing, various known methods can be used.
The registration data acquisition unit 24 communicates with the recording device 30 to acquire the registered face image and its quality data recorded in the recording device 30.

When the face image is detected from the captured image by the face detection unit 23, the quality evaluation unit 25 determines the quality of the captured image specified by the image quality analysis unit 22 and the registered face image acquired by the registration data acquisition unit 23. The quality of the registered face image is evaluated by comparing the quality of the registered face image with the quality of the registered face image. As an example, the difference between the quality value of the captured image and the quality value of the registered face image is calculated for each quality type such as contrast, white balance, compression level, and brightness level, and the threshold value set in advance for each quality type is calculated. In comparison, if the difference is found to be greater than or equal to the threshold, it is determined that the quality of the registered face image needs to be adjusted. As another example, the difference between the quality value of the captured image and the quality value of the registered face image is calculated for each quality type, multiplied by a preset weighting coefficient for each quality type, and totaled, and this total value is calculated in advance. Compared with the set threshold value, if the total value is equal to or greater than the threshold value, it is determined that the quality of the registered face image needs to be adjusted. It should be noted that these are merely examples, and the quality of the photographed image and the quality of the registered face image may be compared by other methods.

Here, when the quality of the registered face image recorded in the recording device 30 is uniform (for example, when all the registered face images are taken under the same shooting conditions), the quality evaluation unit 25 is any one. The registered face image may be selected as a representative and compared with the quality of the captured image. In addition, when a plurality of registered face images can be grouped with similar quality, the quality evaluation unit 25 selects a representative registered face image for each group and compares it with the quality of the captured image. May be good. On the other hand, if the quality of the registered face images is not uniform or cannot be grouped, the quality evaluation unit 25 compares each of the plurality of registered face images recorded in the recording device 30 with the quality of the captured image. There is a need to do.

The image quality adjustment unit 26 adjusts the quality of the registered face image acquired from the recording device 30 when the quality evaluation unit 24 determines that the quality of the registered face image needs to be adjusted. The quality of the registered face image is adjusted so that the quality of the registered face image approaches the quality of the captured image. It is determined that the quality of the registered face image needs to be adjusted when the quality of the captured image is deteriorated due to the deterioration of the shooting environment. Therefore, it is necessary to adjust the quality of the registered face image. Will be applied. The quality of the registered face image may be adjusted for all types of quality, but the quality of the type in which the difference from the quality value of the captured image is greater than or equal to the threshold value (or some It is possible to reduce the processing load by limiting the processing load to (quality of type).

When the face detection unit 23 detects a person's face image from the captured image, the collation unit 27 performs face collation to collate the detected face image with the registered face image. The collation unit 27 normally collates with the registered face image itself acquired from the recording device 30, but the registered face image determined to require quality adjustment is the one after adjustment by the image quality adjustment unit 26. Match with. The collation result by the collation unit 27 is transmitted to the operation terminal 40 and displayed on the screen by the operation terminal 40.

FIG. 3 is a diagram showing an example of a flowchart of face matching by the face matching system of this example.
As pre-processing, the face image to be registered (registered face image) and its quality data are recorded in the recording device 30 (steps S11 and S12), and it is determined whether registration is completed (step S13). When the unprocessed registered face image remains, it is determined that the registration is not completed, and the processes of steps S11 and S12 are repeated until there are no unprocessed registered face images. When there is no unprocessed registered face image, it is determined that the registration has ended, and the preprocessing is ended.

After completing the pre-processing, the captured image receiving unit 21 acquires a captured image from the image capturing device 10 (step S14). Next, the image quality analysis unit 22 analyzes the photographed image acquired by the photographed image receiving unit 21 and specifies its quality (step S15). Next, the face detection unit 23 performs face detection processing on the captured image acquired by the captured image reception unit 21 (step S16). Next, it is determined whether or not a face image is detected from the captured image (step S17). When the face image is not detected from the captured image, the process returns to step S14, and the next captured image is acquired from the imaging device 10. When the face image is detected from the captured image, the quality evaluation unit 25 evaluates the necessity of adjusting the quality of the registered face image (step S18). In this example, the registration data acquisition unit 24 sequentially reads the quality of the registered face image from the recording device 30 and compares the quality with the quality of the captured image, and adjusts the quality of the registered face image having a difference in quality of a predetermined value or more. Determined to be necessary. When it is determined that the quality of the registered face image needs to be adjusted, the image quality adjustment unit 26 causes the quality of the registered face image to approach the quality of the captured image (that is, the quality of the detected face image). Is adjusted (step S19). Next, the collation unit 27 collates the detected face image with the registered face image acquired from the recording device 30 or the registered face image adjusted by the image quality adjustment unit 26 (step S20).

Describing with a specific example, the image processing apparatus 20 of the present example analyzes a captured image obtained in real time from the imaging apparatus 10 and records qualities such as contrast, white balance, compression level, and brightness level. Then, when there is a scene in which a face image is obtained from the captured image, the quality data of the registered face image recorded in the recording device 30 is matched with the quality data specified by analyzing the captured image. , By comparing the image quality, the difference in quality between the registered face image and the captured image is clarified. Based on the result of this comparison, a registered face image whose quality difference is equal to or greater than a predetermined value is searched, and the registered face image is corrected for brightness level and contrast so as to be closer to a real-time captured image. Give. Further, since the compression level differs depending on the image pickup apparatus 10, and particularly the compression level tends to be strongly applied to a real-time captured image, the compression level is analyzed to adjust the compression level of the registered face image to compress the captured image. Bring it closer to the level. In this way, when face matching is performed, the registered face images that have been pre-registered are temporarily corrected, and the detected face images detected from the captured images are compared, so that the image quality of both images is at the same level. Perform face matching processing with.

As described above, in the face matching system of this example, the recording device 30 stores the registered face image registered in advance and its quality data. Then, in the image processing device 20, the image quality analysis unit 22 identifies the quality of the captured image captured by the imaging device 10, the face detection unit 23 detects the face image from the captured image, and the quality evaluation unit 25 When the face detection unit 23 detects a face image from the captured image, the necessity of adjusting the quality of the registered face image is evaluated, and the image quality adjustment unit 26 causes the quality evaluation unit 25 to determine the quality of the registered face image. When it is determined that adjustment is necessary, the registered face image is adjusted in quality so as to approach the quality of the captured image based on the quality of the captured image and the quality of the registered face image, and the matching unit 27 determines the image quality. The detected face image is collated using the registered face image whose quality has been adjusted by the adjustment unit 26.

As described above, in the face matching system of this example, the registered face image is temporarily adjusted so as to approach the quality of the captured image (that is, the quality of the detected face image), and is used for matching the detected face image. I have it. As a result, the detected face image can be collated in a state where the quality of the detected face image and the quality of the registered face image are adjusted to a level (for example, about the same level) suitable for image collation. Therefore, since the quality of the captured image does not need to be adjusted, stable matching accuracy can be obtained without changing the setting on the imaging device side. Furthermore, since it is not necessary to change the setting on the imaging device side, it is possible to use the captured image not only for collation purposes but also for monitoring purposes.

Here, in the face matching system of this example, the storage means according to the present invention is configured by the recording device 30, the specific means according to the present invention is configured by the image quality analysis unit 22, and the adjusting means according to the present invention is image quality. The adjusting unit 26 and the matching unit according to the present invention are configured by the matching unit 27, but it goes without saying that the matching unit may be realized by another structure.

In the above description, the quality evaluation unit 25 evaluates the necessity of adjusting the quality of the registered face image by comparing the quality of the photographed image with the quality of the registered face image, but other methods Therefore, the necessity of adjusting the quality of the registered face image may be evaluated. For example, the necessity of adjusting the quality of the registered face image may be evaluated by comparing the quality of the captured image with a predetermined condition (condition that does not depend on the quality of the registered face image). That is, as an example, a reference range is set in advance for each type of quality, and if the quality value of the captured image deviates from the reference range, it may be determined that the quality of the registered face image needs to be adjusted. Good.

Also, for example, when the timing scheduled in advance arrives, it may be determined that the quality of the registered face image needs to be adjusted. As an example, when the timing of turning on/off the lighting device around the imaging device 10 is scheduled in advance, whether or not to adjust the quality of the registered face image may be switched according to the timing. This method is effective when the timing of the environmental change that affects the quality of the captured image is known in advance.

The case where the image matching system according to the present invention is applied to the face matching system has been described above as an example. However, the present invention is not limited to such a face matching system, and a vehicle, its number, an abandoned object, a grounded object, etc. It goes without saying that it can be widely applied to various image matching systems that match other images with registered images.
The present invention also provides, for example, a method and method for executing the process according to the present invention, a program for realizing such method and method by using hardware resources such as a memory and a processor, and a storage for storing the program. It is also possible to provide it as a medium or the like.

The present invention can be used in various image collation systems that collate a captured image captured by an imaging device with a registered image registered in advance.

10: imaging device, 20: image processing device, 30: recording device, 40: operating terminal,
21: photographed image reception unit, 22: image quality analysis unit, 23: face detection unit, 24: registered data acquisition unit, 25: quality evaluation unit, 26: image quality adjustment unit, 27: collation unit

Claims

In an image collation system that collates a photographed image photographed by an imaging device with a registered image registered in advance,
Storage means for storing the registered image and its quality,
Specifying means for specifying the quality of the captured image,
Adjusting means for adjusting the quality of the registered image so as to approach the quality of the captured image based on the quality of the captured image and the quality of the registered image;
An image collation system comprising: collation means for collating the photographed image using the registered image adjusted by the adjusting means.
The image matching system according to claim 1,
When the difference between the quality of the captured image and the quality of the registered image is a predetermined value or more, when the quality of the captured image deviates from a predetermined condition, or when a timing scheduled in advance arrives. An image collation system, characterized in that the registered image is adjusted in quality.
The image matching system according to claim 1,
The image matching system is characterized in that the quality is one or more of contrast, white balance, compression level, and brightness level.
The image matching system according to claim 2,
The image matching system is characterized in that the quality is one or more of contrast, white balance, compression level, and brightness level.
The image matching system according to claim 1,
The registered image is a face image of a person,
When a face image of a person is detected from the photographed image, the detected face image is collated using the registered image whose quality is adjusted so that the quality is close to the detected face image. Image matching system.
The image matching system according to claim 2,
The registered image is a face image of a person,
When a face image of a person is detected from the photographed image, the detected face image is collated using the registered image whose quality is adjusted so that the quality is close to the detected face image. Image matching system.
The image matching system according to claim 3,
The registered image is a face image of a person,
When a face image of a person is detected from the photographed image, the detected face image is collated using the registered image whose quality is adjusted so that the quality is close to the detected face image. Image matching system.
The image matching system according to claim 4,
The registered image is a face image of a person,
When a face image of a person is detected from the photographed image, the detected face image is collated using the registered image whose quality is adjusted so that the quality is close to the detected face image. Image matching system.