KR102174208B1 - FACE RECOGNITION METHOD AND DEVICE USING INFRARED AND DEPTH IMAGE, and Non-Transitory COMPUTER READABLE RECORDING MEDIUM - Google Patents

Abstract

An embodiment includes the steps of generating a first depth image by detecting a face area in the depth image; Generating a first infrared image corresponding to the face area; Encoding the first depth image based on a comparison result of a depth value between a first target pixel of the first depth image and a first adjacent pixel; Encoding the first infrared image based on a result of comparing gray scale values between a second target pixel and a second adjacent pixel in the first infrared image; Generating a first histogram of the encoded first depth image and a second histogram of the encoded first infrared image; Comparing a histogram of a second infrared image of each of the plurality of people with the second histogram to select at least one of the plurality of people; And comparing the histogram of the second depth image of the selected at least one person with the first histogram to recognize the identity of the person; including a depth image in which a face is photographed and an infrared image. Can provide.

Description

The method and device for identification recognition through the depth and infrared images of the face, and a non-transitory computer-readable recording medium {FACE RECOGNITION METHOD AND DEVICE USING INFRARED AND DEPTH IMAGE, and Non-Transitory COMPUTER READABLE RECORDING MEDIUM}

The present invention relates to an identity recognition method and apparatus, and to a non-transitory computer-readable recording medium. In detail, it relates to a technology capable of recognizing an identity using a depth image and an infrared image in which a face is captured.

While research and application of user authentication and control systems using biometrics are gradually increasing, face recognition is drawing attention among them. Face recognition is a non-aggressive and non-coercive authentication method, and has the advantage of being able to perform authentication without requiring a user to perform a special operation for authentication. In addition, the identity recognition technology through image-based face recognition has the advantage that it can be applied to various fields such as security and marketing, so research is being actively conducted. However, the existing face recognition method has a disadvantage in that it requires an additional authentication method due to the disadvantage of being vulnerable to the external environment and having a security risk that may result in false recognition through printed photos.

As an algorithm for face recognition, PCA (Principal Component Analysis), SIFT (Scale Invariant Feature Transform), Haar-like feature, LBP (Local Binary Pattern), etc. are used. The face recognition method using PCA is a method of extracting features of an object and analyzing its main components in face recognition. Eigenfaces are extracted and used for face detection. PCA is a representative algorithm that reduces the dimension of a high-dimensional feature vector to a low-dimensional feature vector so that loss of information is minimized.After finding similarities in the training face images, reducing the dimension of the face image to obtain an average face, then training The weight of the image to be compared with the image is obtained, and a face is identified using the Nearest Neighbor (NN) method. PCA has the advantage of having a simple algorithm and a high speed, but there is a problem that the accuracy is lowered due to the influence of the pose and illumination of the face. The face recognition method using SIFT is an algorithm that extracts vector components after selecting feature points that are easily identifiable in an image, such as corners or vertices. It is an algorithm for the gradient direction and size of pixels (pixels) belonging to blocks around the feature points. After the histogram is obtained, a vector in which the histogram values are connected in a row is used for face comparison. SIFT has the advantage of maintaining accuracy even when the size of the comparison image is changed or deformation due to rotation, but has a disadvantage in that it is difficult to recognize a face in real time because a large amount of data is generated and there are many processes of repetitive calculation. The face detection method using the Haar-like feature uses the Haar-like feature using the difference in brightness between the area in the image and the area. After defining various types of elementary features consisting of squares in the bright and dark areas of the image, This is a method of finding meaningful feature points through the difference between pixel values included in each area. Here, the selection of meaningful features is done through automatic learning algorithms such as Boosting. The method using the Haar-like feature has the advantage of maintaining accuracy even with respect to changes in shape and position within a region, but has a disadvantage in that accuracy is deteriorated according to changes or rotation of the light source.

Color images are mainly used as images for face recognition. The color image-based face recognition method has the advantage of being able to use the existing image as it is. However, the color image-based face recognition method has the disadvantage of being vulnerable to changes in lighting or changes in facial expressions. In addition, the color image-based method has a flaw in that it cannot be used alone in the security field because it is weak in distinguishing a real face from a printed color picture. In order to overcome the shortcomings of such a color image-based method, a method of using an infrared image or a depth image is being studied. Compared to color images, infrared images have the advantage that pixels change less even when lighting changes, but there is a disadvantage in that contrast contrast is low compared to color images. In addition, there is a problem that infrared images become face recognition for printed color photos like color images. The depth image is less affected by lighting, similar to the infrared image, and has the advantage of obtaining distance information that cannot be obtained from a color image or an infrared image. However, the depth image has a disadvantage in that the noise in the image is included more than the color image or the infrared image.

Korean Registered Patent Publication No. 10-1074953

It is an object of the present invention to provide an identity recognition method in consideration of both characteristics of an infrared image and a depth image in order to solve the above-described problem.

It is an object of the present invention to provide a method and apparatus capable of accurately recognizing an identity by maximizing the advantages of both images and minimizing the disadvantages by recognizing faces using both a depth image and an infrared image.

An object of the present invention is to provide a method and apparatus capable of reducing the risk of false recognition in existing identity recognition such as color photographs.

An object of the present invention is to provide a method and apparatus for recognizing an identity using an infrared image in order to solve a problem in which a color image is heavily influenced by lighting.

An object of the present invention is to improve the accuracy of identity recognition by utilizing a depth image that does not recognize false images such as color photos in order to compensate for the limitation of false recognition even for false images such as color photos. It is to provide a method and apparatus that can be improved.

An embodiment includes the steps of generating a first depth image by detecting a face area in the depth image; Generating a first infrared image corresponding to the face area; Encoding the first depth image based on a comparison result of a depth value between a first target pixel of the first depth image and a first adjacent pixel; Encoding the first infrared image based on a result of comparing gray scale values between a second target pixel and a second adjacent pixel in the first infrared image; Generating a first histogram of the encoded first depth image and a second histogram of the encoded first infrared image; Comparing a histogram of a second infrared image of each of the plurality of people with the second histogram to select at least one of the plurality of people; And comparing the histogram of the second depth image of the selected at least one person with the first histogram to recognize the identity of the person; including a depth image in which a face is photographed and an infrared image. Can provide.

In another aspect, the encoding of the first depth image includes comparing a depth value of the first target pixel with a depth value between the first adjacent pixels and assigning 0 or 1 to the first adjacent pixel. ; Generating level 1 allocation information representing a value allocated to the first adjacent pixel as a first binary number, converting the first binary number to a first decimal value based on the level 1 allocation information; And generating a first-level coded image by encoding the depth value of the first target pixel as the first decimal value; and an identity recognition method using a depth image and an infrared image in which a face is photographed. .

In yet another aspect, the encoding of the first depth image includes a value allocated to the first adjacent pixel based on information on a difference between a depth value of the first target pixel and a depth value between the first adjacent pixels. Generating level 2 to 4 allocation information represented by one second binary number; Converting the second binary number to a second decimal value based on each of the level 2 to 4 allocation information; And generating a second to fourth level coded image by encoding a depth value of the first target pixel as the second decimal value; and an identity recognition method using a depth image and an infrared image in which a face is photographed. Can provide.

In another aspect, the encoding of the first infrared image includes comparing a gray scale value of the second target pixel with a gray scale value between the second adjacent pixels and assigning 0 or 1 to the second adjacent pixel. step; Generating level 1 allocation information representing a value allocated to the second adjacent pixel in one first binary number; Converting the first binary number to a first decimal value based on the level 1 allocation information; And generating a first level coded image by encoding the grayscale value of the second target pixel as the first decimal value; and an identity recognition method using a depth image and an infrared image in which a face is photographed. .

In another aspect, the encoding of the first infrared image includes a value allocated to the second adjacent pixel based on information on a difference between a gray scale value of the second target pixel and a gray scale value between the second adjacent pixels Generating level 2 to 4 allocation information represented by one second binary number; Converting the second binary number to a second decimal value based on each of the level 2 to 4 allocation information; And generating a second to fourth level coded image by encoding the grayscale value of the second target pixel as the second decimal value; and an identity recognition method using a depth image and an infrared image in which a face is photographed. Can provide.

In yet another aspect, measuring a similarity between a histogram of an encoded image of any one of the first to fourth level encoded images and a histogram of a depth image of a plurality of people stored in advance; And selecting at least one person from among the plurality of people based on the similarity. The method may further include a depth image in which a face is photographed and an infrared image to recognize an identity.

In yet another aspect, measuring a similarity between a histogram of an encoded image of any one of the first to fourth level encoded images and a histogram of an infrared image of a plurality of people stored in advance; And selecting at least one person from among the plurality of people based on the similarity. The method may further include a depth image in which a face is photographed and an infrared image to recognize an identity.

In another aspect, the generating of the level 2 to 4 allocation information includes: calculating an absolute value of a difference between a depth value of the first target pixel and a depth value between the first adjacent pixels; Generating binary variables that satisfy the absolute value of the difference; It is possible to provide a method for recognizing an identity through a depth image and an infrared image including generating the level 2 to 4 allocation information from the set of binary variables.

In another aspect, at least one memory for storing instructions; And at least one processor; wherein the instructions are executable by the processor to cause the processor to perform operations, and the operations include: detecting a face region in a depth image to generate a first depth image, and , Generating a first infrared image corresponding to the face region, encoding the first depth image based on a comparison result of a depth value between a first target pixel of the first depth image and a first adjacent pixel, and In the first infrared image, the first infrared image is encoded based on the result of comparing the gray scale values between the second target pixel and the second adjacent pixel, and the first histogram of the encoded first depth image and the first histogram of the encoded first infrared image are 2 A histogram is generated, and at least one of the plurality of people is selected by comparing the histogram of the second infrared image of each of the plurality of people with the second histogram, and a second depth image of the selected at least one person It is also possible to provide an apparatus for identifying an identity through an infrared image and a depth image in which the face is photographed, including recognizing the identity of the person by comparing the histogram of and the first histogram.

According to the exemplary embodiment, the accuracy of face recognition may be improved by encoding an image in consideration of the size of the difference as well as comparing the size between pixels and comparing it with an existing image.

In addition, according to the embodiment, the accuracy of face recognition may be increased and the complexity of operation may be reduced by using an encoded image having a level that best represents the characteristics of a person in the depth image and the infrared image.

In addition, the embodiment may provide a method for face improvement that can reduce sensitivity to changes in lighting and prevent misrecognition of photos.

The effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood from the following description.

1 is a block diagram illustrating an identity recognition apparatus according to an embodiment of the present invention.
2 is an exemplary flowchart of an identity recognition method according to an embodiment of the present invention.
3 is a flowchart of a method for detecting a face area.
4 is an exemplary diagram for detecting a face area in a depth image.
5 is a flowchart for encoding a first depth image.
6 is a flowchart for encoding a first infrared image.
7 shows images of different levels according to the change of R.

Since the present invention can apply various transformations and have various embodiments, specific embodiments are illustrated in the drawings and will be described in detail in the detailed description. Effects and features of the present invention, and a method of achieving them will be apparent with reference to the embodiments described later in detail together with the drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various forms. In the following embodiments, terms such as first and second are not used in a limiting meaning, but are used for the purpose of distinguishing one component from another component. In addition, expressions in the singular include plural expressions unless the context clearly indicates otherwise. In addition, terms such as include or have means that the features or elements described in the specification are present, and do not preclude the possibility of adding one or more other features or elements in advance. In addition, in the drawings, the size of components may be exaggerated or reduced for convenience of description. For example, the size and thickness of each component shown in the drawings are arbitrarily shown for convenience of description, and the present invention is not necessarily limited to what is shown.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, and when describing with reference to the drawings, the same or corresponding components are assigned the same reference numerals, and redundant descriptions thereof will be omitted. .

1 is a block diagram illustrating an identity recognition apparatus according to an embodiment of the present invention.

The identity recognition apparatus 100 may receive image data from the image detection apparatus 200.

The image detection apparatus 200 may be a depth image capturing device that generates a depth image. In addition, the image detection apparatus 200 may be an infrared photographing device that generates an infrared image. Further, the image detection apparatus 200 may be a photographing device capable of generating a depth image and an infrared image.

In some implementations, the identity recognition apparatus 100 and the image detection apparatus 200 may be equipped with a configuration 10 for wireless communication.

In some implementations, the image detection apparatus 200 may process the detected image data and transmit it to the identity recognition apparatus 100.

In some implementations, the identity recognition apparatus 100 may receive image-processed image data from a computer-readable medium.

The computer-readable medium may include any type of medium or device capable of moving image data processed from the image detection apparatus 200 to the identity recognition apparatus 100. For example, the computer-readable medium may include a communication medium, such as a transmission channel, which enables the image detection device 200 to directly transmit image data to the identity recognition device 100 in real time.

The image-processed image data may be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to the identity recognition apparatus 100. The communication medium may comprise any wireless or wired communication medium, such as a radio frequency spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include routers, switches, base stations, or any other equipment that may be useful to facilitate communication between the identity recognition device 100 and the image detection device 200. In some examples, the image-processed image data may be output from the output interface of the image detection apparatus 200 to a computer-readable storage medium, such as a non-transitory computer-readable storage medium, that is, a data storage device. . Likewise, image data may be accessed from the storage device through an input interface of the identity recognition apparatus 100. The storage device may be a variety of distributed or distributed storage media such as hard drives, Blu-ray disks, DVDs, CD-ROMs, flash memory, volatile or nonvolatile memory, or any other suitable digital storage media for storing image data. It may include any of the non-transitory data storage media that are accessed locally. In a further example, the storage device may correspond to a file server or other intermediate storage device that may store image data generated by the image detection apparatus 200.

The identity recognition apparatus 100 may access stored image data from a storage device through streaming or download.

In some implementations, the identity recognition apparatus 100 may include an image source 110 and an image processing unit 120. The identity recognition apparatus 100 may receive an image captured by the image detection apparatus 200 and store it in the image source 110. In addition, the image processing unit 120 may analyze the captured image stored in the image source 110.

In another example, the identity recognition apparatus 100 and the image detection apparatus 200 may include different components.

The image source 110 of the identity recognition device 100 includes an image detection device 200, such as a camera, an archive containing previously captured depth images, and/or depth images and depth from an infrared image content provider. And an interface for receiving an infrared image.

In some implementations, the image detection apparatus 200 may provide a depth image in which depth information of a scene is expressed in an integer data type of 16 bits per pixel. The number of bits for representing one pixel of the depth image may be changed. The image detection apparatus 200 may provide a depth image having a value proportional or inversely proportional to the distance by measuring a distance from the image detection apparatus 200 to an object and a background using infrared rays.

The pixel value of the depth image may be, for example, not color information of RGB, but depth information of an integer of mm unit (but not limited thereto).

Each of the identity recognition apparatus 100 and the image detection apparatus 200 includes at least one memory and at least one microprocessor, digital signal processors (DSPs), custom integrated circuits (ASICs), field programmable gate arrays ( FPGAs), individual logic circuits, software, hardware, firmware, or any combination thereof.

The memory contains instructions (eg, executable instructions) such as computer readable instructions or processor readable instructions. Instructions may include one or more instructions executable by a computer, such as by each of one or more processors.

For example, the one or more instructions may be executable by one or more processors for performing operations including processing an image to allow the one or more processors to recognize an identity.

In detail, the image processing unit 120 may include one or more memories 121 for storing instructions and at least one processor 122 for executing the instructions.

The processor 122 of the image processing unit 120 may be configured to apply techniques for recognizing an identity from a depth and infrared image.

In some implementations, the identity recognition apparatus 100 may be configured to transmit, display, analyze, or the like image data from the image processing unit 120 to an external device.

Although not shown in FIG. 1, in some embodiments, the identity recognition apparatus 100 and the image detection apparatus 200 may be integrated devices.

2 is an exemplary flowchart of an identity recognition method according to an embodiment of the present invention.

Referring to FIG. 2, the method (S100) for identifying an object through a depth image and an infrared image in which a face is photographed according to an embodiment of the present invention includes detecting an object area in the depth image (S110), Detecting and generating a first depth image (S120), generating a first infrared image corresponding to a face area (S130), comparing a depth value between a first target pixel of a first depth image and a first adjacent pixel Encoding a first depth image based on the result (S140), encoding a first infrared image based on a result of comparing gray scale values between a second target pixel and a second adjacent pixel in the first infrared image (S150) , Generating a first histogram of the encoded first depth image and a second histogram of the encoded first infrared image (S160), comparing the histogram of the second infrared image and the second histogram of each of the plurality of people Selecting at least one of the selected at least one person (S170) and comparing the histogram of the second depth image of the selected at least one person with the first histogram to recognize the identity of the person (S180).

Detecting an object area in the depth image (S110) .

The processor 122 may detect an object area in the captured depth image and/or the infrared image. For example, the processor 122 may detect the object area based on the location and depth value of the pixel of the depth image and/or the location and gray scale value of the pixel of the infrared image. In some implementations, when there is a background image in a state in which the object does not exist, the processor 122 generates a difference image between the captured depth image and/or infrared image and a pre-stored background depth image and/or infrared image. Then, after generating a binarized image from the difference image, the binarized image may be labeled to generate a labeling image. Further, the processor 122 may block the labeling image and divide it into a background, a boundary between a background and an object, and an area inside the object. However, the present invention is not limited thereto, and various techniques for detecting an object may be applied.

Generating a first depth image by detecting a face area within the object area (S120) .

3 is a flowchart of a method for detecting a face area, and FIG. 4 is an exemplary diagram for detecting a face area from a depth image.

With further reference to FIGS. 3 and 4, the processor 122 may detect a face area within the captured depth image. In some implementations, the processor 122 may detect the face area within the detected object area.

In some implementations, the processor 122 may detect a face area of a person based on the tip of the nose, which may be the most feature point on the face. Since this is a portion where the tip of the nose protrudes most from the human face, a pixel having the smallest depth value in the current depth image, that is, a portion closest to the image detection apparatus 200 can be searched. In detail, the processor 122 may detect the minimum depth value pixel location, which is the location of the pixel having the smallest depth value in the current depth image (S121). In some embodiments, the processor 122 may detect a minimum depth value pixel position among pixels within an area of a current depth image corresponding to a block corresponding to an inner area of an object. In addition, the processor 122 may set a face area, which is an area having a size of w*(WxH), which is set in advance based on the pixel position of the minimum depth value (S122). Here, W and H may each have a fixed size of 56, but are not limited thereto. In addition, w is a weight for increasing and decreasing the area according to the depth value, and can be calculated through Equation 1.

[Equation 1]

In Equation 1, dnose is a minimum depth value, and a depth value of a pixel, and α and β may be experimentally determined as constants according to the image detection apparatus 200. Exemplarily, α and β may be 2.83 and 456, but are not limited thereto.

In addition, the processor 122 may normalize the above-described face area to maintain face size information.

In some embodiments, the processor 122 may detect a minimum depth value pixel location and detect depth values of a plurality of pixel values in an adjacent region based on the minimum depth value pixel location. Further, the processor 122 may analyze first distribution information of depth values of adjacent pixels based on the minimum depth value pixel based on the detected depth value. In addition, the processor 122 may compare the first distribution information with second distribution information, which is depth value distribution information for each area of the face previously stored. In addition, the processor 122 may extract the first distribution information from among the second distribution information and the third distribution information showing a degree of similarity equal to or greater than a preset value or the highest degree of similarity. When the third distribution information corresponds to distribution information of the nose region, the processor 122 may determine that the minimum depth value pixel corresponds to the tip of the nose. In contrast, it is determined whether the third distribution information matches the depth value distribution information of the nose region, and if not, the processor 122 performs an additional minimum depth in a region spaced apart from the pixel position by a predetermined distance in a specific direction. The value pixel can be detected again. Here, the predetermined distance and specific direction may vary according to the third distribution information. That is, the predetermined distance and a specific direction may be determined according to the distribution information of the depth value in which area among the face areas as the third distribution information. For example, when the subject of identity identification wears a hat, the minimum depth value pixel may be a pixel corresponding to the tip of the hat. Further, the processor 122 may determine that the minimum depth value pixel is a pixel corresponding to the cap end from the third distribution information most similar to the first distribution information. In addition, the processor 122 may detect the minimum depth value pixel again within an area spaced from the minimum depth value pixel by a predetermined distance in the downward direction. Here, since the nose region of the person to be identified will be included in the region separated by a predetermined distance in the downward direction, a pixel corresponding to the tip of the nose can be detected by re-detecting a pixel with a minimum depth value within the region. Accordingly, even in a situation in which the pixel of the minimum depth value does not become the tip of the nose by wearing a hat on the head or other accessories, the pixel corresponding to the tip of the nose can be detected.

The processor 122 may extract an area corresponding to the set face area from the current depth image. In addition, by extracting a region matching the corresponding face region from the current infrared image, both a face region depth image and a face region infrared image may be generated (S123). In addition, the processor 122 may generate a face region depth image and a face region infrared image by extracting a region corresponding to a preset face region from a depth image and an infrared image continuously received from the image detection apparatus 200.

Generating a first infrared image corresponding to the face area (S130) .

As described above, when detecting the face area, the processor 122 may generate the first infrared image by cropping an area at a location corresponding to the face area from the captured infrared image.

Encoding a first depth image based on a comparison result of a depth value between a first target pixel and a first adjacent pixel of the first depth image (S140) .

Encoding a first infrared image based on a result of comparing gray scale values between a second target pixel and a second adjacent pixel in the first infrared image (S150) .

A step of encoding each of the first depth image and the first infrared image will be described in detail.

If a circle region having a radius R around a position of a target pixel in the image is defined, P adjacent pixels of sampling points adjacent to the target pixel may exist in the circle region. Here, the radius R may be defined as a pixel distance. For example, when R is 1, the number of adjacent pixels adjacent to the target pixel may be 8, and when R is 2, the number of adjacent pixels adjacent to the target pixel may be 16.

For example, when R=1, the processor 122 may compare with the target pixel while searching for the remaining adjacent pixels in one direction from any one of the eight adjacent pixels. One direction here may be clockwise or counterclockwise.

The processor 122 may compare the target pixel and the adjacent pixel while searching for adjacent pixels according to Equations 2 and 3, and perform an operation process of encoding the target pixel according to the comparison result.

[Equation 2]

[Equation 3]

In Equation 2, EC is the encoded value of the target pixel, P is the number of adjacent pixels adjacent to the target pixel in the original region with radius R, and pi is the depth of each of the adjacent pixels in the original region with radius R Alternatively, it is a grayscale value, and pc is a depth or grayscale value of the target pixel.

In detail, referring to FIG. 5, in encoding a first depth image, the processor 122 may compare a depth value of a target pixel with a depth value of an adjacent pixel to be compared (S141). When the depth value of the adjacent pixel is greater than or equal to the depth value of the target pixel, 1 may be allocated to the adjacent pixel (S142), and when the depth value of the adjacent pixel is less than the depth value of the target pixel, 0 may be allocated to the adjacent pixel. (S143). In this way, after allocating 1 or 0 to adjacent pixels, the processor 122 may generate one binary value by collecting values allocated to all adjacent pixels (S144). In addition, the processor 122 may convert one binary value to a decimal value and encode the depth value of the target pixel as a decimal value (S145).

As described above, the processor 122 may encode the first depth image by encoding depth values of all pixels in the first depth image (S147).

Further, referring to FIG. 6, in encoding a first infrared image, the processor 122 may compare a grayscale value of a target pixel with a grayscale value of an adjacent pixel to be compared (S151). When the grayscale value of the adjacent pixel is greater than or equal to the grayscale value of the target pixel, 1 may be allocated to the adjacent pixel (S152), and when the grayscale value of the adjacent pixel is smaller than the grayscale value of the target pixel, 0 may be allocated to the adjacent pixel. (S153). In this way, after allocating 1 or 0 to adjacent pixels, the processor 122 may generate one binary value by collecting values allocated to all adjacent pixels (S154). In addition, the processor 122 may convert one binary value to a decimal value and encode the depth value of the target pixel as a decimal value (S155).

As described above, the processor 122 may encode the first infrared image by encoding depth values of all pixels in the first infrared image (S157).

On the other hand, it is possible to encode an image by setting the P and R values differently, and in this case, the position of the pixel may appear as a decimal point (for example, (115.4, 244.8)) rather than an integer (for example, (155,245)). . In order to infer the value of the pixel at the position of the decimal point, an interpolation method such as a bilinear interpolation method may be applied.

In addition, the processor 122 may perform encoding by converting the depth value or grayscale value of the target pixels into a decimal value according to Equations 2 and 3, and removing some characteristics to be described later.

In detail, the processor 122 may perform an encoding process by removing a characteristic that has no discrimination power or insignificant from a value encoded before generating the first and second histograms. In detail, when P is 8, 256 characteristics of an encoded image can be obtained, and when P is 16, 65536 characteristics of an encoded image can be obtained. However, as the number of features increases, computational complexity increases, and some of the features lack discrimination or have insignificant features, so that the number of features for an encoded image may be reduced.

For example, when searching for each number in order in one binary number, a pattern in which a change from 0 to 1 or a change from 1 to 0 is less than 2 times is defined as a uniform pattern, and one label is assigned to each. And, a pattern with a change of 3 or more times can be defined as a non-uniform pattern. Then, after grouping them into a group, only one label is assigned to the entire group. For example, patterns such as 01110000 (change of 2), 00000001 (change of 1), and 11111111 (change of 0) are uniform patterns, so each one recognizes individuality, while 00011101 (change of 3), 11100101 (change of 4) Changes) and 10101000 (5 changes) can be collected and treated as one. Through this, a value having no discrimination power or an insignificant characteristic may be removed from the encoded values of the pixels of the image. In addition, when P is 8, only 59 bins (58 bins for uniform patterns and 1 bin for non-uniform patterns) can be used in 256 bins (from 0 to 255 in decimal).

Meanwhile, the depth value of the nose tip region of the face may be minimized compared to other regions of the face. Accordingly, even if the faces of different people are different, if the pixel at the tip of the nose becomes the target pixel, the depth value of the target pixel is lower than that of the adjacent pixels. Accordingly, since all values allocated to adjacent pixels are the same, the depth values of the target blocks, which are pixels corresponding to the nose ends of different faces, may finally be encoded to be the same value. In other words, there is a limit to discerning the faces of different people.

In addition, due to the nature of the depth image, there is a high possibility that noise is contained in the depth image. Accordingly, if the depth image is encoded through a process of comparing the depth value of the target pixel with the depth value of the adjacent pixel, there is a high risk that the characteristic of the pixel will be erroneously detected.

In consideration of this limitation, the processor 122 may encode not only size comparison between the target pixel and the neighboring pixels, but also information on the size of the difference, that is, the size of the difference between the target pixel and the adjacent pixel. This makes it possible to distinguish between different surfaces having different depth differences in a 3D object such as a face surface.

In detail, the processor 122 may generate level 1 allocation information in which 1 or 0 is allocated to the adjacent pixel based on the comparison of the sizes of the target pixel and the adjacent pixel. For example, if the depth value or gray scale value of the target pixel is greater than that of the adjacent pixel, the processor 122 may allocate 0 to the adjacent pixel, otherwise it may allocate 1. In addition, by converting the level 1 allocation information into a decimal value EC according to Equation 2, the target pixel may be primary encoded.

In addition, the processor 122 may generate level 2 to 4 allocation information based on information on a size of a difference between a target pixel and an adjacent pixel.

In detail, according to an experiment, when R is 2, more than 93% of the depth difference between pixels appears to be smaller than 7. Accordingly, binary units i2, i3, and i4 for encoding the absolute value DD, which is the magnitude of each depth difference between the three pixels and their neighbors, can be defined. In some embodiments, if the absolute value DD is greater than 7, the absolute value DD may be assigned as 7.

The absolute value DD and the binary unit satisfy Equation 4.

[Equation 4]

In addition, i1 is defined as a value of 1 or 0 allocated based on the comparison of the sizes of the target pixel and the adjacent pixel, which satisfies Equation 5. That is, a value of 1 or 0 is assigned to i1 according to the DD value, which is the difference between the depth value or grayscale value of the target pixel and the depth value or grayscale value of the adjacent pixel.

[Equation 5]

Accordingly, the processor 122 may generate level 2 to 4 allocation information that satisfies the absolute value DD representing the size of the difference between the target pixel and the adjacent pixel according to Equation (5).

Hereinafter, a method of generating level 1 to level 4 allocation information will be described in detail by way of example.

It is assumed that grayscale values of pixels in a block in an image photographing a human face are the same as the map below.

The processor 122 generates a difference value between the gray scale value 255 of the target pixel and the gray scale values 254, 253, 252, 250 251, 254, 250, and 249 of the eight pixels adjacent to the target pixel. These difference values are shown in the map below.

And, the difference values DD are listed in a line as follows.

When DD, which is a difference in grayscale values between the target pixel and the adjacent pixel, is applied to Equation 5, the processor 122 may assign the value of i1 as 1 or 0 as follows. That is, the processor 122 may generate level 1 allocation information.

<Level 1 allocation information>

Further, according to Equation 4, when the difference value DD=-6, the processor 122 may allocate 1 or 0 to i2 to i4 to satisfy |DD|=6.

According to Equation 4, when i2=1, i3=1, and i4=0, |DD|=6 is satisfied.

Accordingly, the rightmost values in the map below for explaining the level 2 to 4 allocation information are 1, 1, and 0 in order.

Further, according to Equation 4, when the difference value DD=-5, 1 or 0 may be assigned to i2 to i4 to satisfy |DD|=5.

Therefore, the values of the second blank from the rightmost in the map below are 1, 0, and 1 in order.

In this way, the processor 122 determines a binary variable value that satisfies the absolute value of the difference between the depth value or gradation value of the target pixel and the depth value or gradation value of the adjacent pixel, and from the set of binary variables to level 2 to 4 Assignment information can be created.

<Level 2 allocation information>

Assign 1 or 0 to i2

<Level 3 allocation information>

Assign 1 or 0 to i3

<Level 4 allocation information>

Assign 1 or 0 to i4

Also, the processor 122 may convert the level 1 to 4 allocation information into one binary number and generate a decimal value from the binary number.

Taking the above maps as an example, since one binary number is 00000000 in the level 1 allocation information, the decimal value is 0.

In the level 2 allocation information, since one binary number is 00011011, the value of the decimal number is 27.

In the level 3 allocation information, since one binary number is 01100001, the decimal value is 97.

Finally, in the level 4 allocation information, since one binary number is 10110110, the decimal value is 182.

Accordingly, as indicated in the map below, four encoding values can be obtained for the target pixel, and accordingly, first to fourth level encoded images can be obtained for one image. Here, the first level encoded image is generated based on the level 1 allocation information, the second level encoded image is generated based on the level 2 allocation information, and the third level encoded image is generated based on the level 3 allocation information. And the fourth level encoded image is generated based on the level 4 allocation information.

Generating a first histogram of the encoded first depth image and a second histogram of the encoded first infrared image (S160) .

The processor 122 may encode all pixels of the first depth image and may encode all pixels of the first infrared image. In addition, the processor 122 may generate a first histogram for the encoded first depth image and a second histogram for the encoded first infrared image.

For example, when P=8, a total of 256 bins (one section of the histogram) of the histogram. That is, 256 characteristics of the encoded image can be obtained.

In some embodiments, a histogram with 59 bins (58 bins for uniform patterns and 1 bin for non-uniform patterns) may be generated. That is, 59 characteristics of the encoded image can be obtained.

In some embodiments, the processor 122 may generate four coded blocks for one block based on the level 1 to level 4 allocation information, and may obtain a histogram for each of them.

Comparing the histogram of the second infrared image of each of the plurality of people and the second histogram to select at least one of the plurality of people (S170) .

The histogram of the second infrared image of each of the plurality of people may be stored in advance in the memory 121. In some embodiments, the histogram of the second infrared image of each of the plurality of people may be a histogram having 256 characteristics or 59 characteristics described above. In some embodiments, the histogram of the second infrared image of each of the plurality of people may be a histogram from four encoded infrared images of one infrared image encoded according to level 1 to 4 allocation information.

The processor 122 may compare the second histogram, which is a histogram of the captured infrared image, and the histogram of the second infrared image of each of the plurality of persons previously stored.

Meanwhile, in some embodiments, the processor 122 may generate a histogram only for one of the encoded images of the first to fourth levels. In detail, see FIG. 7.

7 shows images of different levels according to the change of R.

The images of FIG. 7 are maintained at P=8 and are images of different levels according to different radius values (R=1, 2, 6).

In the case of a depth image, when R=2, encoded images of the first and second levels can best express facial features.

In addition, in an infrared image, a first-level encoded image at R=2 and R=6 can best express facial features. Accordingly, the processor 122 may compare the histogram of at least one coded depth image among the first and second level coded depth images when P=8 and R=2 with a previously stored histogram. Also, the processor 122 may compare a histogram of at least one encoded infrared image among the encoded infrared images of the first level when P=8, R=2, or R=6 with a previously stored histogram. As described above, according to the embodiment, the number of feature dimensions can be reduced by using the encoded image having the most characteristic level, and accordingly, the complexity of the operation can be reduced.

Comparing the histogram of the depth image of the selected at least one person with the first histogram to recognize the identity of the person (S180) .

The histogram of the second depth image of each of the plurality of people may be stored in advance in the memory 121. In some embodiments, the histogram of the second depth image of each of the plurality of people may be a histogram having 256 characteristics or 59 characteristics described above. In some embodiments, the histogram of the second depth image of each of the plurality of people may be a histogram from four encoded depth images obtained by encoding one depth image according to level 1 to 4 allocation information.

The processor 122 may compare the first histogram, which is a histogram of the photographed depth image, and the histogram of the second depth image of each of the plurality of persons stored in advance.

In some embodiments, the processor 122 first filters information on a plurality of people in step S170, compares the histogram of the depth image of the filtered people and the first histogram, You can also verify your identity. That is, the person with the highest accuracy may be determined by comparing the facial features stored in advance through the photographed infrared image, and whether the person is actually the same person using the photographed depth image.

Meanwhile, the processor 122 may measure a similarity between the histogram of the captured image and the previously stored histogram according to an operation that satisfies Equation (6).

[Equation 6]

In more detail, the processor 122 may measure the similarity of the two histograms by calculating a value of each bin of the histogram to be compared according to Equation 6.

In Equation 6, S may be a pre-stored histogram, and M may be a histogram related to the captured image. In addition, n denotes the number of bins in the histogram, and i denotes the order of bins in the histogram.

In addition, the processor 122 may compare histograms related to the infrared image to select candidates having high similarity with the captured object. Further, the processor 122 may determine whether there is a candidate matching the photographed object among the selected candidates by comparing the histogram related to the depth image. In this case, if the similarity is greater than or equal to a preset threshold, the processor 122 may determine the identity of the photographed object. If not, the processor 122 may display an identity verification failure and terminate the identity verification process. In some embodiments, if there is no candidate matching the photographed object among the selected candidates, the processor 122 may additionally determine whether there is a candidate matching the photographed object among unselected candidates.

The embodiment according to the present invention can be utilized in the fields of portable device lock control, security system, access control, and close-out management.

The embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded in the computer-readable recording medium may be specially designed and configured for the present invention or may be known and usable to those skilled in the computer software field. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magnetic-optical media such as floptical disks. medium), and a hardware device specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of the program instructions include not only machine language codes such as those produced by a compiler but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device can be changed to one or more software modules to perform the processing according to the present invention, and vice versa.

The specific implementations described in the present invention are examples and do not limit the scope of the present invention in any way. For brevity of the specification, descriptions of conventional electronic configurations, control systems, software, and other functional aspects of the systems may be omitted. In addition, the connection or connection members of the lines between the components shown in the drawings exemplarily represent functional connections and/or physical or circuit connections. Connections, or circuit connections, may be represented. In addition, if there is no specific mention such as “essential” or “importantly”, it may not be an essential component for the application of the present invention.

In addition, although the detailed description of the present invention has been described with reference to a preferred embodiment of the present invention, the spirit of the present invention described in the claims to be described later if a person skilled in the art or those of ordinary skill in the art And it will be understood that various modifications and changes can be made to the present invention within a range not departing from the technical field. Accordingly, the technical scope of the present invention should not be limited to the contents described in the detailed description of the specification, but should be determined by the claims.

Claims (9)

Generating a first depth image by detecting a face area in the depth image;
Generating a first infrared image corresponding to the face area;
Encoding the first depth image based on a comparison result of a depth value between a first target pixel of the first depth image and a first adjacent pixel;
Encoding the first infrared image based on a result of comparing gray scale values between a second target pixel and a second adjacent pixel in the first infrared image;
Generating a first histogram of the encoded first depth image and a second histogram of the encoded first infrared image;
Comparing a histogram of a second infrared image of each of the plurality of people with the second histogram to select at least one of the plurality of people; And
Comprising: comparing the histogram of the second depth image of the selected at least one person with the first histogram to recognize the identity of the person; including
An identity recognition method through depth images and infrared images of a face. The method of claim 1,
Encoding the first depth image,
Comparing a depth value of the first target pixel with a depth value between the first adjacent pixels and allocating 0 or 1 to the first adjacent pixel;
Generating level 1 allocation information representing a value allocated to the first adjacent pixel in one first binary number,
Converting the first binary number to a first decimal value based on the level 1 allocation information; And
Generating a first-level encoded image by encoding the depth value of the first target pixel into the first decimal value; including
An identity recognition method through depth images and infrared images of a face. The method of claim 2,
Encoding the first depth image,
Generates level 2 to 4 allocation information representing a value allocated to the first adjacent pixel in one second binary number based on information on the difference between the depth value of the first target pixel and the depth value between the first adjacent pixels Step to do;
Converting the second binary number to a second decimal value based on each of the level 2 to 4 allocation information; And
Encoding the depth value of the first target pixel as the second decimal value to generate second to fourth level encoded images; further comprising
An identity recognition method through depth images and infrared images of a face. The method of claim 1,
The step of encoding the first infrared image,
Comparing a gray scale value of the second target pixel with a gray scale value between the second adjacent pixels and allocating 0 or 1 to the second adjacent pixel;
Generating level 1 allocation information representing a value allocated to the second adjacent pixel in one first binary number,
Converting the first binary number to a first decimal value based on the level 1 allocation information; And
Generating a first-level encoded image by encoding the grayscale value of the second target pixel as the first decimal value;
An identity recognition method through depth images and infrared images of a face. The method of claim 4,
The step of encoding the first infrared image,
Generates level 2 to 4 allocation information representing a value assigned to the second adjacent pixel in one second binary number based on information on the difference in the size of the gray level value between the second target pixel and the second adjacent pixel Step to do;
Converting the second binary number to a second decimal value based on each of the level 2 to 4 allocation information; And
Generating a second to fourth level encoded image by encoding the grayscale value of the second target pixel as the second decimal value;
An identity recognition method through depth images and infrared images of a face. The method of claim 3,
Measuring a similarity between a histogram of an encoded image of any one of the first to fourth level encoded images and a histogram of a depth image of a plurality of people stored in advance; And
Further comprising: selecting at least one person from among the plurality of people based on the similarity
An identity recognition method through depth images and infrared images of a face. The method of claim 5,
Measuring a similarity between the histogram of the encoded image of any one of the first to fourth level encoded images and the histogram of the infrared image of a plurality of people stored in advance; And
Further comprising: selecting at least one person from among the plurality of people based on the similarity
An identity recognition method through depth images and infrared images of a face. The method of claim 3,
Generating the level 2 to 4 allocation information,
Calculating an absolute value of a difference between a depth value of the first target pixel and a depth value between the first adjacent pixels;
Generating binary variables that satisfy the absolute value of the difference;
Generating the level 2 to 4 assignment information from the set of binary variables; including
An identity recognition method through depth images and infrared images of a face. At least one memory for storing instructions; And
Including; at least one processor,
The instructions are executable by the processor to cause the processor to perform operations,
The above actions are:
A first depth image is generated by detecting a face area in the depth image,
Generating a first infrared image corresponding to the face region,
Encode the first depth image based on a comparison result of a depth value between a first target pixel of the first depth image and a first adjacent pixel,
In the first infrared image, the first infrared image is encoded based on a result of comparing gray scale values between a second target pixel and a second adjacent pixel,
Generate a first histogram of the encoded first depth image and a second histogram of the encoded first infrared image,
At least one of the plurality of people is selected by comparing the histogram of the second infrared image of each of the plurality of people with the second histogram,
Comparing the histogram of the second depth image of the selected at least one person with the first histogram to recognize the identity of the person
An identity recognition device through the depth image and infrared image of the face.

Images