KR102588696B1 - Chip for face recognition based on heterogeneous sensors and face recognition device using the same - Google Patents

Abstract

A heterogeneous sensor-based facial recognition chip and facial recognition device are disclosed.
A facial recognition chip based on heterogeneous sensors according to an embodiment of the present invention includes a plurality of buffer memories that store target area images generated by a plurality of image sensors separately for each image sensor that generated the target area images; The operation mode of the face recognition chip is set to single image mode or multiple image mode according to the distance of the object that appears in the target area and sensed by the distance sensor, and each of the face recognition chips is supplied to the plurality of buffer memories according to the set operation mode. a first processor that controls power; And when the single image mode is set, face recognition is performed using target area images stored in a main buffer memory among the plurality of buffer memories, and when the multiple image mode is set, a plurality of target area images stored in the plurality of buffer memories are performed. It includes a second processor that performs face recognition using.

Description

Chip for face recognition based on heterogeneous sensors and face recognition device using the same {Chip for face recognition based on heterogeneous sensors and face recognition device using the same}

The present invention relates to a facial recognition chip based on heterogeneous sensors and a facial recognition device using the same. More specifically, facial recognition is performed based on a distance sensor that senses the distance of an object and an image sensor that generates an image of the object. It relates to a facial recognition chip and a facial recognition device using the same.

In general, face recognition technology detects face images that exist in still images or videos captured by a camera, and performs preprocessing such as face alignment, luminance correction, and image normalization on the detected face images. , refers to a technology that extracts features from preprocessed face images and identifies the face.

Recently, this facial recognition technology has been used in various local devices such as smart door-locks, wall-pads, closed circuit televisions (CCTVs), mobile terminals, Internet of Things (IoT) devices, and autonomous vehicles. As it is applied, interest in and requests for facial recognition devices that enable low-power operation while ensuring accuracy of facial recognition are increasing.

However, as disclosed in Korean Patent Publication No. 10-1988279, the existing technology has a problem in that it uses different types of image sensors for object detection but does not provide a way to operate these image sensors in a power-efficient manner.

In addition, as disclosed in Korean Patent Publication No. 10-2014-0089697, the existing technology recognizes the user's face through an image captured by a single camera installed in the terminal, so face recognition depends on the performance of the camera. There is a problem in that the possible distance is limited, and the farther away the user is, the lower the resolution of the user's face appears in the captured image, making it impossible to guarantee the accuracy of face recognition. In addition, when using a high-resolution camera to increase the distance at which a face can be recognized, the manufacturing cost of the face recognition device increases and there is a problem in that it is less cost-effective when recognizing the face of a user located at a close distance.

The technical problem to be solved by the present invention is a heterogeneous sensor-based facial recognition chip that improves the power efficiency of facial recognition devices and reduces manufacturing costs, while alleviating distance limitations for facial recognition and ensuring the accuracy of facial recognition, and a chip using the same. The goal is to provide a facial recognition device.

A facial recognition chip based on heterogeneous sensors according to an embodiment of the present invention includes a distance sensor that senses the distance of an object that appears in a target area, and a plurality of target area images that are photographed from different perspectives. Performing face recognition in conjunction with an image sensor, comprising: a plurality of buffer memories that store target area images generated by the plurality of image sensors separately for each image sensor that generated the target area images; A first device that sets the operation mode of the face recognition chip to single image mode or multiple image mode according to the distance sensed by the distance sensor, and controls power supplied to each of the plurality of buffer memories according to the set operation mode. processor; And when the single image mode is set, face recognition is performed using target area images stored in a main buffer memory among the plurality of buffer memories, and when the multiple image mode is set, a plurality of target area images stored in the plurality of buffer memories are performed. and a second processor that performs face recognition using, wherein when the single image mode is set, the first processor controls power provided to buffer memories other than the main buffer memory among the plurality of buffer memories to the first processor. While controlling the power level, the power provided to the main buffer memory is controlled to a second level that is greater than the first level, and when the multi-image mode is set, all power provided to each of the plurality of buffer memories is adjusted to the second level. It is configured to control.

In one embodiment, the first processor sets the operation mode to a sleep mode when an object does not appear in the target area and controls all power provided to each of the plurality of buffer memories to the first level. It is configured to block or block.

In one embodiment, the face recognition chip further includes a clock provider that provides an operation clock to the second process, and the first processor controls the clock provider when the single image mode is set to An operation clock of 1 frequency is provided to the second processor, and when the multiple image mode is set, the clock provider is controlled to provide an operation clock of a second frequency higher than the first frequency to the second processor.

In one embodiment, the first processor sets the operation mode to sleep mode when an object appears in the target area and controls the clock provider to block the operation clock provided to the second processor. It is composed.

In one embodiment, the first processor sets the operation mode to the single image mode when the distance sensed by the distance sensor does not exceed a predetermined threshold distance, and the sensed distance exceeds the threshold distance. When exceeded, the operation mode is configured to be set to the multi-image mode.

In one embodiment, the first processor is configured to designate a main image sensor among the plurality of image sensors before setting the operation mode, and the main buffer memory is configured to specify the main image sensor among the plurality of buffer memories. It corresponds to a buffer memory that stores the target area image created by .

In one embodiment, when the single image mode is set, the second processor detects a face image from the target area image stored in the main buffer memory and applies the detected face image to a previously learned deep learning model for face recognition. It is configured to perform face recognition by inputting.

In one embodiment, when the multi-image mode is set, the second processor detects a face image from a plurality of target area images stored in the plurality of buffer memories, and selects the corresponding face images through the detected face images. It is configured to generate a high-resolution face image and perform face recognition by inputting the high-resolution face image into a pre-trained deep learning model for face recognition.

A face recognition device according to an embodiment of the present invention is a device that performs face recognition using the face recognition chip, and is configured to include the distance sensor and the plurality of image sensors.

According to the present invention, the power efficiency of the face recognition device is improved by adaptively changing the number of image sensors and buffer memories used for face recognition and the power provided to the buffer memory and processor according to the distance of the face recognition target object. And while reducing manufacturing costs, it is possible to relax the distance limitation of face recognition and ensure the accuracy of face recognition.

In addition, target area images are generated by photographing the same target area from different perspectives through a plurality of image sensors, the face image of the face recognition target object is detected from each of these target area images, and the face image of the object subject to face recognition is detected based on the detected face images. By generating high-resolution facial images to be used for face recognition, the accuracy and reliability of face recognition even when low-cost, low-resolution image sensors are used or when the object to be recognized is located at a long distance where face recognition cannot be performed with a single image sensor. can be guaranteed to a high level.

In addition, by having the image sensors provided in the face recognition device alternately perform the role of the main image sensor used for both near-distance and far-distance face recognition, unbalanced deterioration of the image sensors is prevented and the face recognition device Maintenance and repair can be facilitated.

Furthermore, those skilled in the art to which the present invention pertains will be able to clearly understand from the following description that various embodiments according to the present invention can solve various technical problems not mentioned above.

Figure 1 is a block diagram showing a face recognition device according to an embodiment of the present invention.
Figure 2 is a flowchart showing an operation mode setting process performed by the first processor of the facial recognition chip according to an embodiment of the present invention.
Figure 3 is a flowchart showing a face recognition process performed by a second processor of a face recognition chip according to an embodiment of the present invention.
FIG. 4 is a diagram showing capturing areas of image sensors included in the face recognition device of FIG. 1.
FIG. 5 is a diagram illustrating an example of target area images generated by the face recognition device of FIG. 1.
Figure 6 is a diagram showing a high-resolution facial image generation method of a facial recognition chip according to an embodiment of the present invention.
Figure 7 is a flowchart showing a main image sensor change process of a face recognition chip according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings to clarify solutions to the technical problems of the present invention. However, in describing the present invention, if the description of related known technology rather obscures the gist of the present invention, the description thereof will be omitted. Additionally, the terms used in this specification are terms defined in consideration of the functions in the present invention, and may vary depending on the intention or custom of the designer, manufacturer, etc. Therefore, definitions of terms described below should be made based on the content throughout this specification.

Figure 1 shows a block diagram of a face recognition device 10 according to an embodiment of the present invention.

As shown in FIG. 1, the face recognition device 10 according to an embodiment of the present invention includes an object detection sensor 12, a distance sensor 14, a plurality of image sensors 16a to 16d, and a power unit 18. It includes, and further includes a facial recognition chip 100 according to an embodiment of the present invention.

The object detection sensor 12 is configured to detect an object appearing in a predetermined target area. This object detection sensor 12 may be configured in various ways. For example, the object detection sensor 12 may be configured to detect an object by sensing the motion of the object using an ultrasonic sensor or an infrared sensor, or may be configured to detect the object by contacting the object.

The distance sensor 14 is configured to sense the distance to an object appearing in the target area. This distance sensor 14 transmits a wave signal such as an infrared signal or an ultrasonic signal to the target area, receives a reflected signal reflected by an object located in the target area, and calculates the Time of Flight (ToF) of the wave signal. , It can be configured to sense the distance to the object using the moving speed of the wave signal and the calculated ToF.

Depending on the embodiment, the distance sensor 14 may be integrated with the object detection sensor 12. That is, the distance sensor 14 may be configured to transmit a wave signal at any time or periodically and detect an object that appears in the target area through whether a reflected signal is received or a ToF change in the wave signal.

The plurality of image sensors 16a to 16d are independently controlled image sensors and are configured to generate target area images obtained by photographing the target area from different perspectives at adjacent positions. To this end, each image sensor may include a Complementary Metal-Oxide Semiconductor (CMOS) or a Charge Coupled Device (CCD). Although the face recognition device 10 is shown in FIG. 1 as including four image sensors 16a to 16d, the number of image sensors included in the face recognition device 10 may vary depending on the embodiment.

The power supply unit 18 is configured to supply power required for operation of the face recognition device 10. To this end, the power supply unit 18 may optionally include a battery, a battery management system (BMS), a transformer, a rectifier, etc.

The face recognition chip 100 according to the present invention is applied to the face recognition device 10 and is configured to perform face recognition based on the distance sensor 14 and the image sensors 16a to 16d.

To this end, the facial recognition chip 100 according to an embodiment of the present invention includes a plurality of buffer memories 110a to 110d, a first processor 120, a second processor 130, a clock provider 140, and It may include RAM 150. This facial recognition chip 100 may be implemented in the form of a SoC (System on Chip).

The plurality of buffer memories (110a to 110d) are connected one-to-one with the plurality of image sensors (16a to 16d), and use the target area images generated by each image sensor (16a to 16d) to generate the corresponding target area image. It is configured to be stored separately for each image sensor. Additionally, the plurality of buffer memories 110a to 110d are each configured to receive power through independent power lines.

The first processor 120 sets the operation mode of the face recognition chip 100 to single image mode or multiple image mode according to the distance of the face recognition target object sensed by the distance sensor 14, and performs the set operation. It is configured to control the power supplied to each of the plurality of buffer memories 110a to 110d according to the mode.

As will be explained again below, in the present invention, the 'single image mode' is a facial recognition chip 100 that selects a face image from a target area image generated by a pre-designated main image sensor among the plurality of image sensors 16a to 16d. It is an operation mode that detects and performs face recognition, and in the 'multiple image mode', the face recognition chip 100 detects each face image from a plurality of target area images generated through the plurality of image sensors 16a to 16d. This is an operation mode that performs face recognition.

In this case, the first processor 120 sets the operation mode to a single image mode if the distance sensed by the distance sensor 14 does not exceed a predetermined threshold distance, and if the sensed distance does not exceed a predetermined threshold distance, the first processor 120 sets the operation mode to a single image mode. If exceeded, the operation mode may be configured to set to multi-image mode.

The threshold distance, which is the standard for setting the operating mode, may be determined differently depending on the performance of the image sensors 16a to 16d. That is, the higher the resolution of the image generated through each image sensor, the longer the critical distance can be determined. On the other hand, the lower the resolution of the image captured through each image sensor, the closer the critical distance may be determined. This is because as the performance of the image sensor increases, the object distance at which accurate face recognition is possible in single image mode increases.

In addition, the first processor 120 may be configured to set the operation mode of the face recognition chip 100 to 'sleep mode (or standby mode)' before the face recognition target object appears in the target area. You can.

The second processor 130 is a processor configured independently from the first processor 120. When the sleep mode is set, the second processor 130 maintains a sleep state that does not perform face recognition, and when the single image mode is set, the plurality of buffer memories ( Face recognition is performed using target area images stored in predetermined main buffer memories (110a to 110d), and when the multi-image mode is set, a plurality of target area images distributedly stored in the plurality of buffer memories (110a to 110d) are used. It is configured to perform face recognition using.

The clock provider 140 provides an operation clock to the second processor 130 and is configured to provide operation clocks of different frequencies according to the operation mode set by the first processor 120.

The RAM (Random Access Memory) 150 is configured to store computer programs, data, etc. necessary for the operation of the face recognition chip 100. To this end, the RAM 150 may optionally include Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), etc.

FIG. 2 shows a flowchart of an operation mode setting process performed by the first processor 120 of the facial recognition chip 100 according to an embodiment of the present invention. With reference to FIG. 2, detailed operations of the face recognition chip 100 will be described in chronological order.

As shown in FIG. 2, the first processor 120 of the facial recognition chip 100 may monitor the target area at any time or periodically through the object detection sensor 12 (S200).

If the face recognition target object does not appear in the target area, the first processor 120 sets the operation mode of the face recognition chip 100 to ‘sleep mode’ (S210, S240).

On the other hand, when a face recognition target object appears in the target area and the object is detected through the object detection sensor 12, the first processor 120 controls the distance sensor 14 to sense the distance to the object ( S210, S220).

Next, the first processor 120 sets the operation mode of the face recognition chip 100 to 'single image mode' or 'multiple image mode' according to the distance of the object sensed by the distance sensor 14. (S230, S250, S260).

That is, if the distance sensed by the distance sensor 14 does not exceed a predetermined threshold distance, the first processor 120 sets the operation mode to the single image mode (S250). On the other hand, if the distance sensed by the distance sensor 14 exceeds a predetermined threshold distance, the first processor 120 sets the operation mode to the multi-image mode (S260).

In this way, when the operation mode of the face recognition chip 100 is set, the first processor 120 provides power to each of the plurality of buffer memories 110a to 110d according to the type of the set operation mode, and the second processor Controls the operation clock provided to 130.

That is, when the sleep mode is set (S240), the first processor 120 controls or blocks all power provided to each of the plurality of buffer memories 110a to 110d to a first level, which is a low power level (S242), The clock provider 140 is controlled to block the operation clock provided to the second processor 130 (S244).

Meanwhile, when the single image mode is set (S250), the first processor 120 reduces the power provided to the remaining buffer memories except the pre-designated main buffer memory among the plurality of buffer memories 110a to 110d to the first level, which is a low power level. While controlling, the power provided to the main buffer memory is controlled to a second level greater than the first level (S252), and the clock provider 140 is controlled to provide an operation clock of the first frequency to the second processor ( 130) (S254).

In addition, when the multi-image mode is set (S260), the first processor 120 controls the power provided to each of the plurality of buffer memories 110a to 110d to a second level, which is a high power level (S262), and the clock The provision unit 140 is controlled to provide an operation clock of a second frequency higher than the first frequency to the second processor 130 (S264).

As a result, the second processor 130 performs different face recognition processes depending on the type of operation mode set by the first processor 120 (S270).

The first processor 120 may repeat the above-described operation mode setting process while the facial recognition device 10 continues to operate (S280).

FIG. 3 shows a flowchart of a face recognition process performed by the second processor 130 of the face recognition chip 100 according to an embodiment of the present invention. With reference to FIG. 3, detailed operations of the face recognition chip 100 will be described in chronological order.

As shown in FIG. 3, when the sleep mode is set as the operation mode of the face recognition chip 100, the second processor 130 does not receive an operation clock from the clock provider 140 and maintains the sleep state. (S300).

On the other hand, when the single image mode is set as the operation mode of the face recognition chip 100 (S310), the second processor 130 receives the target area image stored in the main buffer memory among the plurality of buffer memories 110a to 110d. Then (S320), a face image is detected from the received target area image (S322). As will be explained again below, the main buffer memory is a buffer memory (e.g., FIG. 110a).

In this case, the second processor 130 may detect a face image from the received target area image using a deep learning model for face image detection. The deep learning model for face image detection may include a CNN (Convolutional Neural Network) trained to detect face images using a large number of images containing face images. The face image detected in the entire image can be defined by relative coordinates (X, Y) and confidence values determined based on the entire image. The reliability is a kind of probability value and ranges from 0 to 1, but only the part of the image with reliability higher than an experimentally or theoretically determined threshold can be detected as a face image.

Next, the second processor 130 performs face recognition by inputting the detected face image into a pre-trained deep learning model for face recognition (S340).

Meanwhile, when the multi-image mode is set as the operation mode of the face recognition chip 100 (S310), the second processor 130 receives a plurality of target area images stored in the plurality of buffer memories 110a to 110d, and (S330), face images are detected from each of the received target area images (S332). The received target area images are images of the same target area, and are images of the target area taken at the same time and from different perspectives.

Next, the second processor 130 selects a plurality of face images corresponding to the same face among the detected face images (S334), and provides location information and pixel value information of pixels obtained from each of the selected face images. By performing interpolation on the pixels that will form the high-resolution face image, a high-resolution face image with higher resolution than the corresponding face images is generated (S336).

In general, the amount of computation and processing speed when processing an image increases in proportion to the size of the image. Therefore, the present invention does not generate a high-resolution target area image by directly using each target area image generated through the plurality of image sensors 16a to 16d, but uses only the face image detected from each target area image. The goal is to create high-resolution facial images.

Next, the second processor 130 performs face recognition by inputting the generated high-resolution face image into the deep learning model for face recognition (S340). In this case, the deep learning model for face recognition can extract a feature vector from an input face image and calculate similarity by comparing the extracted feature vector with the feature vector of a pre-registered face image. The similarity between the feature vector of the pre-registered face image and the feature vector of the input face image can be calculated using a comparison method such as cosine-similarity. If the calculated similarity is greater than or equal to the reference threshold, recognition succeeds, otherwise recognition fails. It can be judged as

Next, the second processor 130 outputs the face recognition result (S350). That is, if the similarity calculated through the deep learning model for face recognition is greater than or equal to the reference threshold, the second processor 130 determines that face recognition is successful and outputs the face recognition result through a predetermined output device. On the other hand, if the similarity calculated through the deep learning model for face recognition is less than the reference threshold, the second processor 130 may determine that face recognition has failed and output the corresponding face recognition result through the output device.

FIG. 4 shows the capturing areas of the image sensors 16a to 16d included in the face recognition device 10 of FIG. 1 .

As shown in FIG. 4, the image sensors 16a to 16d of the face recognition device 10 are each arranged to capture a certain range of capturing areas A1 to A4, but include the same target area At. They are placed in adjacent locations for filming.

As will be explained again below, the target area At corresponds to a disparity support area that can generate a high-resolution image by fusing or combining images generated through a plurality of image sensors 16a to 16d. do.

FIG. 5 shows an example of target area images generated by the face recognition device 10 of FIG. 1 .

As shown in FIG. 5, images (I1 to I4) generated through a plurality of image sensors (16a to 16d) arranged to photograph the same target area (At) each capture the target area (At). It contains the image part (It) that it represents. Accordingly, when a face recognition target object appears in the target area At, the images I1 to I4 taken at the same time through the plurality of image sensors 16a to 16d commonly include an image of the object. Included.

The image of the object included in each of the images I1 to I4 may be defined by location information and pixel value information of pixels constituting the image. In this case, the location information may include coordinate values (Xn, Yn) of each pixel determined based on the entire image, and the pixel value information may include the RGB value of each pixel determined between 0 and 255. You can.

Figure 6 shows a high-resolution facial image generation method of the facial recognition chip 100 according to an embodiment of the present invention.

As shown in FIG. 6, the second processor 130 of the face recognition chip 100 processes low-resolution facial images (F1 to F1) taken at adjacent viewpoints through a super resolution technique. By combining F4), a high-resolution face image (Fo) can be generated. That is, the second processor 130 uses the position information and pixel value information of the pixels obtained from the low-resolution face images F1 to F4 to interpolate the pixel values of the pixels to form the high-resolution face image to create a high-resolution face image. A face image (Fo) can be created.

FIG. 7 shows a flowchart of a main image sensor change process performed by the facial recognition chip 100 according to an embodiment of the present invention. With reference to FIG. 7, additional detailed operations of the face recognition chip 100 will be described in chronological order.

As shown in FIG. 7, the first processor 120 of the face recognition chip 100 designates one image sensor to be used as the main image sensor among the plurality of image sensors 16a to 16d, and It is configured to re-designate the image sensor to be used as the subsequent main image sensor according to the usage of the image sensor designated as the image sensor. In this case, the amount of image sensor usage may be determined based on the data processing amount or usage time of the image sensor.

Specifically, before setting the operation mode of the face recognition chip 100, the first processor 120 designates image sensor 1 (16a) as the main image sensor among the plurality of image sensors (16a to 16d) and then , Monitors the usage of the image sensor used while the face recognition process is performed (700), calculates the cumulative usage for each image sensor, and generates sensor usage information including the cumulative usage for each image sensor or updates the generated sensor usage information. Do it (S710).

Then, when the cumulative usage of image sensor 1 (16a), which is designated as the main image sensor, reaches a predetermined reference usage amount, the first processor 120 uses image sensor 2 (16b) instead of image sensor 1 (16a). Designated as the subsequent main image sensor.

That is, the first processor 120 refers to the generated or updated sensor usage information and checks the usage of image sensor 1 (16a), which is designated as the main image sensor, and the difference in usage from other image sensors (S720).

As a result of the confirmation, if it is determined that the usage of the image sensor 1 (16a) or the difference in usage with other image sensors has reached a predetermined standard usage (S730), the first processor 120 replaces the image sensor 1 (16a). The main camera is changed by designating image sensor 2 (16b) as the subsequent main camera (S740). In this case, image sensor 2 (16b), which is designated as the next main camera, is the image sensor with the lowest cumulative usage according to the sensor usage information, or is an image sensor that is predetermined to be designated as the main image sensor after image sensor 1 (16a). It can be.

In this way, when the operation mode of the face recognition chip 100 is set to a single image mode after the image sensor 2 (16b) is designated as the subsequent main image sensor, the second processor ( 130) receives the target area image only from the buffer memory 110b that stores the image of the image sensor 2 (16b), detects the face image, and inputs the detected face image into the deep learning model to perform face recognition. do. These processes can be repeated until the operation of the face recognition device 10 is stopped (S750).

Meanwhile, embodiments according to the present invention may be implemented with a computer system and a computer program that runs the computer system. When embodiments of the present invention are implemented as a computer program, the components of the present invention are program segments that execute the corresponding operation or task through the computer system. These computer programs or program segments may be stored in various computer-readable recording media. Computer-readable recording media include all types of media that record data that can be read by a computer system. For example, computer-readable recording media may include ROM, RAM, EEPROM, registers, flash memory, CD-ROM, magnetic tape, hard disk, floppy disk, or optical data recording device. Additionally, these recording media can be distributed across computer systems connected to various networks to store or execute program codes in a distributed manner.

As described above, according to the present invention, the number of image sensors and buffer memories used for face recognition, the power provided to the buffer memory and the processor, etc. are adaptively changed according to the distance of the face recognition object, thereby providing a face recognition device. It can improve the power efficiency of and reduce manufacturing costs, while relaxing the distance limitations of facial recognition and ensuring the accuracy of facial recognition.

In addition, target area images are generated by photographing the same target area from different perspectives through a plurality of image sensors, the face image of the face recognition target object is detected from each of these target area images, and the face image of the object subject to face recognition is detected based on the detected face images. By generating high-resolution facial images to be used for face recognition, the accuracy and reliability of face recognition even when low-cost, low-resolution image sensors are used or when the object to be recognized is located at a long distance where face recognition cannot be performed with a single image sensor. can be guaranteed to a high level.

In addition, by having the image sensors provided in the face recognition device alternately perform the role of the main image sensor used for both near-distance and far-distance face recognition, unbalanced deterioration of the image sensors is prevented and the face recognition device Maintenance and repair can be facilitated.

Furthermore, of course, the embodiments according to the present invention can solve various technical problems other than those mentioned in this specification in the relevant technical field as well as in the related technical field.

So far, the present invention has been described with reference to specific embodiments. However, those skilled in the art will clearly understand that various modified embodiments can be implemented within the technical scope of the present invention. Therefore, the previously disclosed embodiments should be considered from an explanatory perspective rather than a limiting perspective. In other words, the scope of the true technical idea of the present invention is shown in the claims, and all differences within the scope of equivalents should be construed as being included in the present invention.

10: Face recognition device
12: object detection sensor
14: Distance sensor
16a-16d: Image sensor
18: power unit
100: Chip for face recognition
110a-110d: buffer memory
120: first processor
130: second processor
140: clock provider
150: RAM

Claims (9)

For face recognition based on heterogeneous sensors that perform face recognition in conjunction with a distance sensor that senses the distance of an object appearing in the target area and a plurality of image sensors that generate target area images taken from different perspectives. As a chip,
a plurality of buffer memories that store target area images generated by the plurality of image sensors separately for each image sensor that generated the target area images;
The operation mode of the face recognition chip is set to single image mode or multi-image mode according to the distance sensed by the distance sensor, and the power supplied to each of the plurality of buffer memories according to the set operation mode is connected to an independent power line. a first processor controlled through; and
When the single image mode is set, face recognition is performed using target area images stored in a main buffer memory among the plurality of buffer memories, and when the multiple image mode is set, a plurality of target area images stored in the plurality of buffer memories are performed. It includes a second processor that performs facial recognition using
When the single image mode is set, the first processor controls the power provided to the buffer memories other than the main buffer memory among the plurality of buffer memories to a first level and adjusts the power provided to the main buffer memory to the first level. Controlling to a second level greater than 1 level, and controlling all power provided to each of the plurality of buffer memories to the second level when the multi-image mode is set,
The first processor sets the operation mode to the single image mode when the distance sensed by the distance sensor does not exceed the predetermined threshold distance, and sets the operation mode to the single image mode when the sensed distance exceeds the threshold distance. Configured to set to the multi-image mode,
The second processor selects a plurality of face images corresponding to the same face among the recognized face images, and creates a high-resolution face image using the position information and pixel value information of pixels obtained from each of the selected face images. A high-resolution face image is generated by performing interpolation on the pixels to be configured, and the high-resolution face image is generated using only the face image detected in each target area image,
The face recognition chip further includes a clock provider that provides an operation clock to the second processor,
When the single image mode is set, the first processor controls the clock provider to provide an operation clock of a first frequency to the second processor. When the multiple image mode is set, the first processor controls the clock provider to provide the first processor. A facial recognition chip based on heterogeneous sensors configured to provide an operation clock of a second frequency higher than the frequency to the second processor. According to paragraph 1,
The first processor is configured to set the operation mode to a sleep mode before an object appears in the target area, and to control or block all power provided to each of the plurality of buffer memories to the first level. A facial recognition chip based on heterogeneous sensors. delete According to paragraph 1,
The first processor is configured to set the operation mode to a sleep mode before an object appears in the target area and to control the clock provider to block the operation clock provided to the second processor. Sensor-based facial recognition chip. delete According to paragraph 1,
The first processor is configured to designate a main image sensor among the plurality of image sensors before setting the operation mode,
The main buffer memory is a heterogeneous sensor-based facial recognition chip, characterized in that the main buffer memory is a buffer memory that stores a target area image generated by the main image sensor among the plurality of buffer memories. According to paragraph 1,
When the single image mode is set, the second processor detects a face image from the target area image stored in the main buffer memory and performs face recognition by inputting the detected face image into a pre-trained deep learning model for face recognition. A chip for facial recognition based on heterogeneous sensors, characterized in that it is configured to do so. According to paragraph 1,
When the multi-image mode is set, the second processor detects face images from a plurality of target area images stored in the plurality of buffer memories, and creates a high-resolution face image with a higher resolution than the corresponding face images through the detected face images. A heterogeneous sensor-based face recognition chip, characterized in that it is configured to perform face recognition by generating and inputting the high-resolution face image into a pre-trained deep learning model for face recognition. A facial recognition device using the facial recognition chip according to any one of claims 1, 2, 4, or 6 to 8, wherein the face recognition device includes the distance sensor and the plurality of image sensors. Recognition device.