CN110866469A - A method, device, equipment and medium for facial feature recognition - Google Patents

Abstract

The invention discloses a face facial feature recognition method, a face facial feature recognition device, face facial feature recognition equipment and a face facial feature recognition medium. The method comprises the following steps: acquiring a facial five sense organs image to be recognized; extracting facial features from facial features to be recognized through a feature extraction model; determining standard features of the five sense organs matched with the features of the five sense organs based on the similarity between the features of the five sense organs and the features of a plurality of standard five sense organs, wherein each standard feature of the five sense organs is marked with a corresponding attribute type of the five sense organs; acquiring the attribute type of the five sense organs corresponding to the matched standard five sense organ characteristics; taking the attribute type of the facial features corresponding to the matched standard facial features as the attribute type of the facial features to be recognized; the feature extraction model corresponds to a network structure used for extracting feature information of facial features in the facial features classification model, the facial features classification model is obtained by performing machine learning training on a plurality of facial features sample images, and the facial features sample images carry classification labeling information of the belonging users.

Description

A method, device, equipment and medium for facial feature recognition

technical field

The present invention relates to the technical field of Internet communication, and in particular, to a method, device, device and medium for identifying facial features.

Background technique

With the development of Internet communication technology, various biometric identification technologies have emerged, and face recognition is one of them. Face recognition, which belongs to the field of computer vision, is a biometric technology for identification based on human facial feature information. As an important biological feature of the human body, the face has the good characteristics of uniqueness and is not easy to be copied. At the same time, the collection of face images is relatively convenient, so face recognition plays an important role in many fields.

The facial features (eyebrows, eyes, ears, nose, and mouth) in a human face often have corresponding facial features. For example, eyebrows include willow leaf eyebrows, sword eyebrows, etc., and eyes include peach blossom eyes, Danfeng eyes, etc. It is also of great significance to perform more fine-grained facial feature recognition. However, in the prior art, the obtained facial feature recognition solution needs to label a large number of facial feature sample images with facial feature attribute types, so there is a problem of a large amount of labeling workload, such as labeling each eyebrow sample image corresponding to Types of facial features (willow leaf eyebrow, sword eyebrow, etc.). In addition, the types of facial features can be marked based on different classification dimensions (for example, eyebrow sample images can be marked with the eyebrow shape classification dimension, or the eyebrow thickness classification dimension). Differences between different classification dimensions may cause Identifying difficult problems. The problems existing in the prior art affect the efficiency of establishing a relevant model for facial feature recognition. Therefore, it is necessary to provide a more effective recognition solution for facial features.

SUMMARY OF THE INVENTION

In order to solve the problems of low efficiency and high cost of establishing related models when the prior art is applied to recognize facial features, the present invention provides a facial feature recognition method, device, equipment and medium:

In one aspect, the present invention provides a method for identifying facial features, the method comprising:

Obtain the facial features image to be recognized;

Extracting facial features from the facial facial features image to be recognized through a feature extraction model;

Based on the similarity between the facial features and a plurality of standard facial features, determine standard facial features that match the facial features, and each of the standard facial features is marked with a corresponding facial feature type;

Obtain the facial feature attribute type corresponding to the matched standard facial feature;

Taking the facial feature attribute type corresponding to the matched standard facial features as the facial feature attribute type of the facial facial features image to be recognized;

Wherein, the feature extraction model corresponds to a network structure used for extracting feature information of facial features in a facial feature classification model, and the facial feature classification model is obtained by performing machine learning training on multiple facial feature sample images, The facial features sample image carries the classification and labeling information of the user to which it belongs.

Another aspect provides a facial feature recognition device, the device comprising:

Image acquisition module: used to acquire facial features images to be recognized;

Feature extraction module: used to extract facial features from the facial features image to be recognized through a feature extraction model;

Feature matching module: used to determine the standard facial features matching the facial features based on the similarity between the facial features and a plurality of standard facial features, and each of the standard facial features is marked with a corresponding facial feature type;

The facial features attribute type acquisition module: used to obtain the facial feature attribute types corresponding to the matched standard facial features;

Facial feature attribute type labeling module: used to use the facial feature attribute type corresponding to the matched standard facial feature feature as the facial feature attribute type of the facial facial feature image to be recognized;

Wherein, the feature extraction model corresponds to a network structure used for extracting feature information of facial features in a facial feature classification model, and the facial feature classification model is obtained by performing machine learning training on multiple facial feature sample images, The facial features sample image carries the classification and labeling information of the user to which it belongs.

Another aspect provides an electronic device, the electronic device includes a processor and a memory, the memory stores at least one instruction, at least a piece of program, a code set or an instruction set, the at least one instruction, the at least one piece of The program, the code set or the instruction set is loaded and executed by the processor to implement the above-mentioned method for facial feature recognition.

Another aspect provides a computer-readable storage medium, the storage medium stores at least one instruction, at least one piece of program, code set or instruction set, the at least one instruction, the at least one piece of program, the code set Or the instruction set is loaded and executed by the processor to implement the above-mentioned method for recognizing facial features.

A method, device, equipment and medium for facial feature recognition provided by the present invention have the following technical effects:

The invention obtains a facial feature classification model by training, and constructs a feature extraction model based on the network structure in the facial feature classification model for extracting facial feature image feature information. The facial features of the facial facial features image to be recognized are obtained through the feature extraction model. Based on the similarity between the facial features and the standard facial features, label the facial facial features image corresponding to the facial facial features corresponding to the standard facial features to be recognized. In the training of related models, there is no need to label specific facial feature attribute types for facial facial feature sample images, which reduces the difficulty and workload of manual labeling. The user's facial features are identified in the form of feature comparison, and the information of the fine-grained facial features of the face can be accurately obtained.

Description of drawings

In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or in the prior art, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description The drawings are only some embodiments of the present invention, and for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

1 is a schematic diagram of an application environment provided by an embodiment of the present invention;

2 is a schematic flowchart of a method for identifying facial features according to an embodiment of the present invention;

FIG. 3 is a schematic flow chart of obtaining an image of facial features to be recognized according to an embodiment of the present invention;

FIG. 4 is a schematic flowchart of a facial feature classification model obtained by training according to an embodiment of the present invention;

5 is a schematic diagram of an application scenario of a feature extraction model provided by an embodiment of the present invention;

6 is also a schematic diagram of an application scenario of a feature extraction model provided by an embodiment of the present invention;

7 is also a schematic diagram of an application scenario of a feature extraction model provided by an embodiment of the present invention;

8 is a block diagram of a configuration of a device for identifying facial features provided by an embodiment of the present invention;

FIG. 9 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present invention.

It should be noted that the terms "comprising" and "having" in the description and claims of the present invention and the above-mentioned drawings, as well as any variations thereof, are intended to cover non-exclusive inclusion, for example, including a series of steps or units A process, method, system, product or server is not necessarily limited to those steps or units expressly listed, but may include other steps or units not expressly listed or inherent to such process, method, product or device.

Please refer to FIG. 1. FIG. 1 is a schematic diagram of an application environment provided by an embodiment of the present invention, which may include a client 01 and a server 02, and the client and the server are connected through a network. The facial features images to be recognized can be sent by the client to the server. The server performs image processing on the received facial facial features image to be recognized to mark the corresponding facial feature attribute type for it. It should be noted that FIG. 1 is only an example.

Specifically, the client 01 may include physical devices such as smart phones, desktop computers, tablet computers, notebook computers, digital assistants, smart wearable devices, etc., and may also include software running in the physical devices. For example, some service providers provide The user's web page may also be an application provided to the user by these service providers.

Specifically, the server 02 may include an independently running server, or a distributed server, or a server cluster composed of multiple servers. The server 02 may include a network communication unit, a processor, and memory, among others. Specifically, the server 02 may provide background services for the above-mentioned clients.

In practical applications, the solutions provided by the embodiments of the present invention may involve related technologies of artificial intelligence, which will be described in the following specific embodiments. Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, including both hardware-level technology and software-level technology. The basic technologies of artificial intelligence generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation/interaction systems, and mechatronics. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning/deep learning. With the research and progress of artificial intelligence technology, artificial intelligence technology has been researched and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, autonomous driving, drones It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important value.

Among them, Machine Learning (ML) is a multi-field interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in how computers simulate or realize human learning behaviors to acquire new knowledge or skills, and to reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent, and its applications are in all fields of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, teaching learning and other technologies.

A specific embodiment of a method for recognizing facial features of the present invention is described below. FIG. 2 is a schematic flowchart of a method for recognizing facial features according to an embodiment of the present invention. This specification provides the method described in the embodiment or the flowchart. operational steps, but may include more or fewer operational steps based on routine or non-creative work. The sequence of steps enumerated in the embodiments is only one of the execution sequences of many steps, and does not represent the only execution sequence. When an actual system or server product is executed, it can be executed sequentially or in parallel (for example, in a parallel processor or multi-threaded processing environment) according to the embodiments or the methods shown in the accompanying drawings. Specifically, as shown in Figures 2 and 6, the method may include:

S201: Obtain an image of facial features of the face to be recognized;

In this embodiment of the present invention, the facial features image to be recognized may be an image collected by a device that uses optical principles to image and record images. The device that uses optical principles to image and record images may be a digital camera, a camera of a terminal device (such as a camera of smart glasses, a camera of a mobile phone). The collected facial feature images to be recognized may have different sizes, and accordingly, the size range of the facial feature images to be recognized may be relatively wide. The facial features image to be recognized may be an image directly imaged by a device that uses optical principles to image and record images, or an image that has been preprocessed on the above directly imaged image. The preprocessing method may include denoising processing, grayscale processing, etc. on the image.

In a specific embodiment, as shown in FIG. 3 , the step of acquiring an image of facial features to be recognized includes:

S301: collect the image to be recognized;

The image to be recognized can be an image captured by a digital camera or a camera, or the image to be recognized can also be an image captured from a video captured by a digital camera or a camera, or the image to be recognized can also be uploaded by the user to the server through the network. image. The image to be recognized may be a static image or a dynamic image. Certainly, the acquisition manner of the image to be recognized may also include other possible manners, which are not limited in this embodiment of the present invention.

S302: performing face detection on the to-be-recognized image to obtain a face image;

Face detection (face detection) can search for any given image using intelligent strategies to determine whether it contains a human face. Using face detection technology, images to be recognized that do not contain faces can be filtered out. For an image to be recognized that contains a face, the face region in the image can also be cropped out to obtain a human image.

S303: Perform face key point detection on the face image to obtain at least one facial feature image to be identified;

The face key point detection may be to locate the position of the facial feature key points on the human face, and the facial feature key points may include at least one selected from the group consisting of eyebrows, eyes, ears, nose, and mouth. Using the face key point detection technology, the area that does not contain the face key points can be cut out from the face image. Correspondingly, the areas containing the face key points in the face image can be cut out respectively to obtain at least one to be Identify facial features. The facial features images to be recognized include eyebrow objects, eye objects, ear objects, nose objects or mouth objects. In this way, the interference of redundant features on the recognition of facial features can be reduced, and the amount of calculation in the recognition process can be reduced.

In practical applications, after acquiring the collected video data, frames with facial features are extracted from the video data. Then face detection is performed on the frame to obtain a rectangular frame of face coordinates. Then, according to the facial coordinate rectangle frame, the facial features key points are positioned to obtain the facial features key points. Then, the coordinate positions of the facial features are determined according to the key points of the facial features. Then determine the specific eyebrow position, eye position, ear position, nose position or mouth position according to the coordinate positions of the facial features, and correspondingly, obtain the eyebrow image, eye image, ear image, nose image Oral image or oral image.

S202: Extract facial features from the to-be-recognized facial features image by using a feature extraction model;

In the embodiment of the present invention, the feature extraction model corresponds to the network structure used to extract the feature information of facial features in the facial feature classification model, and the facial feature classification model is a machine-generated facial feature classification model based on a plurality of facial feature sample images. Obtained through learning and training, the facial feature sample image carries the classification and labeling information of the user to which it belongs.

In a specific embodiment, the extracting the facial features from the to-be-recognized facial feature image by using a feature extraction model includes: determining a relationship with the facial feature object according to the facial feature object in the to-be-recognized facial feature image The corresponding target feature extraction model; the facial features are extracted from the facial features image to be recognized through the target feature extraction model.

The facial features objects may be eyebrow objects, eye objects, ear objects, nose objects, or mouth objects. The target feature extraction model corresponds to the network structure used to extract the feature information of facial features in the facial features classification model for the same feature object. For example, if the facial features in the facial features image to be recognized are eyebrow objects, the target feature extraction model is used to extract eyebrow features, and the facial features classification model pointed to by the network structure used to construct the target feature extraction model is also Used to implement eyebrow classification.

In practical applications, for the received face image, at least one facial feature image to be recognized can be obtained by cropping based on facial feature objects, and then each facial feature image to be recognized is respectively input into the corresponding target feature extraction model. For example, the eyebrow image is input into the feature extraction model for extracting eyebrow features, and the eye image is input into the feature extraction model for extracting eye features.

In another specific embodiment, as shown in Figures 4 and 5, the training process of the facial features classification model includes the following steps:

S401: Obtain a sample image of facial features;

The facial features sample image includes eyebrow object, eye object, ear object, nose object or mouth object. The facial feature sample image may be obtained by cropping the face sample image based on the detection of the key points of the face. Here, referring to the aforementioned step S303, the face image is cropped based on the detection of the key points of the face to obtain the correlation of the facial feature image to be recognized. The process will not be repeated. The face sample image can correspond to the training of the face recognition model, so that the classification and labeling information of the user carried by the facial feature sample image can reuse the labeling information (identity label) of the face sample image.

Specifically, a first number threshold and a second number threshold may be determined, where the first number threshold represents the number of candidate users, and the second number threshold represents the number of images corresponding to the same candidate user. Then, the facial feature sample image is acquired according to the first quantity threshold and the second quantity threshold. When acquiring sample images for training, the dimension of the number of candidate users and the dimension of the number of images of the same candidate user are set. Further, the face sample images (corresponding to the training of the face recognition model) can be obtained according to the first quantity threshold and the second quantity threshold, such as obtaining images of N people, each of which is 100-300 indefinite. number of images. Then, based on the detection of the key points of the face, the face sample image is cropped to obtain the facial feature sample image, and the face sample image and the annotation information carried by it are reused.

S402: Input the facial features sample image into a neural network model for image classification training;

A size threshold can be determined, and then the size value of the facial features sample image is adjusted according to the size threshold, and then the face feature sample image with the size value adjusted is input into the neural network model for image classification training. Set the size of the facial feature sample input to the neural network model to M*N pixels (for example, 112*56 pixels), and adjust the initial size of the facial feature sample image accordingly. Of course, in the aforementioned step S303, the face image can also be cropped according to the size threshold to obtain the facial features image to be recognized.

The network structure of the neural network model includes a residual network ResNet18 (which can be composed of a convolutional layer, a pooling layer, and a fully connected layer) for extracting facial features image feature information and a feature based on the facial features image features. A classifier (or classification layer) for classifying facial features. Of course, the network structure of the neural network model can be adjusted by adding or subtracting convolutional layers, and the network structure of the neural network model is not limited to this.

In practical applications, the eyebrow sample images that meet the size threshold requirements are input into the neural network model. Then, the feature of the eyebrow sample image is obtained by ResNet18. The feature can be 256-dimensional, and the feature can be specifically output by the conv5-3 layer (the third convolution layer in the fifth convolution block) in ResNet18. A classifier (or classification layer) is then used to classify based on this feature.

S403: During the training process, adjust the model parameters of the neural network model so that the classification result output by the neural network model matches the classification and annotation information of the input facial features sample image;

The adjustment process of the model parameters includes the following steps: obtaining a loss value according to an intermediate value and a labeled value, the intermediate value corresponds to the classification result output by the neural network model, and the labeled value corresponds to the input facial features sample image. the classification and labeling information; obtain the gradient of the model parameter according to the loss value; update the model parameter based on the gradient of the model parameter. The loss value corresponds to the forward computation phase of model training, in which the model samples a certain number of sample images for the current iteration of training. The forward calculation of each sample image will generate a loss value, and the size of the loss value indicates the learning quality of this sample image. Then, in the back-propagation stage of model training, the gradient of the model parameters is calculated through the loss value, and then the model parameters are adjusted.

S404: Use the neural network model corresponding to the adjusted model parameters as the facial feature classification model;

The trained facial features classification model can output the classification result of the user to which the image to be predicted belongs, such as which user the eyebrows represented by the image to be predicted belong to. Use multiple labeled facial features sample images to perform machine learning training to obtain a facial feature classification model. The obtained facial feature classification model has high generalization ability, which can improve the adaptability of image classification, and can improve image classification. reliability and validity.

In practical applications, the feature extraction model corresponds to the network structure used for extracting feature information of facial features in the facial features classification model. Extracting the facial features from the to-be-recognized facial features image through the feature extraction model can be achieved according to the following steps: input the facial features image to be recognized into the facial facial features classification model, and take out the output of the middle layer of the facial facial features classification model. As a result, the middle layer corresponds to the residual network ResNet18 used to extract the feature information of facial features in the facial features classification model, specifically the output results of the conv5-3 layers. The output here is the facial features.

S203: Based on the similarity between the facial features and a plurality of standard facial features, determine standard facial features that match the facial features, and each of the standard facial features is marked with a corresponding facial feature type;

In the embodiment of the present invention, the same facial feature object may correspond to a plurality of standard facial features, and each of the standard facial features is marked with a corresponding facial feature attribute type. There is a certain difference between the facial features pointed to by the standard facial features, and the facial features pointed to by each standard facial feature have clear features in the relevant images. For example, multiple standard facial features of an eyebrow object may include standard facial features pointing to Liu Yemei, standard facial features pointing to sword eyebrows, standard facial features pointing to arched eyebrows, and standard facial features pointing to tall eyebrows, etc. Correspondingly, " Willow eyebrows, sword eyebrows, arched eyebrows, and tall eyebrows can be used as the types of facial features marked by the corresponding standard facial features, respectively. Further, standard facial features can be defined, a standard facial feature image carrying a standard facial feature object can be obtained, and the standard facial feature image can be input into the feature extraction model to obtain standard facial features. The standard facial feature image carries a type annotation describing the corresponding facial feature attribute type. Of course, defining standard facial features can be based on different dimensions, such as race, gender, shape, color, and more.

The similarity can be calculated from Euclidean distance (Euclidean metric, euclidean metric: refers to the true distance between two points in m-dimensional space, or the natural length of a vector), cosine similarity (cosine similarity: It is to evaluate their similarity by calculating the cosine value of the angle between the two vectors; the vector is drawn into a vector space according to the coordinate value, such as the most common two-dimensional space), relative entropy (Kullback-Leibler divergence, is two At least one selected from the group consisting of an asymmetry measure of the difference between probability distributions) is determined.

In a specific embodiment, similarity calculation may be performed on the facial features and the plurality of standard facial features, respectively, to obtain a similarity set. For example, the facial features corresponds to the eyebrow object, and correspondingly, the multiple standard facial features are multiple standard eyebrow features: standard eyebrow feature 1 , standard eyebrow feature 2 , and standard eyebrow feature 3 . The facial features are calculated by similarity with the multiple standard eyebrow features, respectively, to obtain the similarity of the facial features corresponding to each standard eyebrow feature: similarity 1 (corresponding to standard eyebrow feature 1), similarity 2 (corresponding to standard eyebrow feature 2) and similarity 3 (corresponding to standard eyebrow feature 3). Similarity 1, similarity 2 and similarity 3 form a similarity set. Then, the maximum similarity is determined in the similarity set. For example, similarity 1 is 70%, similarity 2 is 65%, and similarity 3 is 90%. Then the maximum similarity is similarity 3. Then, the standard facial feature corresponding to the maximum similarity is determined as the matched standard facial feature. In this way, the matched standard facial features are standard eyebrow feature 3 (corresponding to the maximum similarity: similarity 3).

S204: Obtain the facial feature attribute type corresponding to the matched standard facial feature;

In the embodiment of the present invention, standard facial features can be defined, a standard facial feature image carrying a standard facial feature object is obtained, and the standard facial feature image is input into the feature extraction model to obtain standard facial features. The standard facial feature image carries a type annotation describing the corresponding facial feature attribute type. Standard facial features and corresponding facial feature types may be pre-stored, and after the matched standard facial features are determined, the facial features corresponding to the matched standard facial features are acquired based on the correspondence in the stored data.

S205: Use the facial feature attribute type corresponding to the matched standard facial features as the facial feature attribute type of the facial facial feature image to be recognized;

In the embodiment of the present invention, the facial feature image of the face to be recognized is annotated by using the facial feature attribute type corresponding to the matched standard facial feature feature.

In practical applications, for the received face image, at least one facial feature image to be recognized can be obtained by cropping based on facial feature objects, and then each facial feature image to be recognized is input into the corresponding feature extraction model. Referring to steps S202-S205, each to-be-recognized facial feature image is sequentially subjected to feature extraction, similarity calculation, facial feature type acquisition and labeling, and then more accurate facial facial features labeling on the face image is realized, as shown in Figure 7. display, which can be displayed to the user.

It can be seen from the technical solutions provided by the above embodiments of this specification that a facial feature classification model is obtained by training in the embodiments of this specification, and a feature extraction model is constructed based on the network structure in the facial feature classification model for extracting facial feature image feature information. The facial features of the facial facial features image to be recognized are obtained through the feature extraction model. Based on the similarity between the facial features and the standard facial features, label the facial facial features image corresponding to the facial facial features corresponding to the standard facial features to be recognized. In the training of related models, there is no need to label specific facial feature attribute types for facial facial feature sample images, which reduces the difficulty and workload of manual labeling. The user's facial features are identified in the form of feature comparison, and the information of the fine-grained facial features of the face can be accurately obtained.

The embodiment of the present invention also provides a facial feature recognition device, as shown in FIG. 8 , the device includes:

Image acquisition module 810: used to acquire an image of facial features to be recognized;

Feature extraction module 820: used to extract facial features from the to-be-recognized facial features image through a feature extraction model;

Feature matching module 830: for determining the standard facial features matching the facial features based on the similarity between the facial features and a plurality of standard facial features, and each of the standard facial features is marked with a corresponding facial feature type ;

A facial feature type acquisition module 840: used to acquire the facial feature attribute type corresponding to the matched standard facial features;

Facial feature attribute type labeling module 850: used to use the facial feature attribute type corresponding to the matched standard facial feature feature as the facial feature attribute type of the facial facial feature image to be recognized;

Wherein, the feature extraction model corresponds to a network structure used for extracting feature information of facial features in a facial feature classification model, and the facial feature classification model is obtained by performing machine learning training on multiple facial feature sample images, The facial features sample image carries the classification and labeling information of the user to which it belongs.

It should be noted that the apparatus and method embodiments in the apparatus embodiments are based on the same inventive concept.

An embodiment of the present invention provides an electronic device, the electronic device includes a processor and a memory, the memory stores at least one instruction, at least one piece of program, code set or instruction set, the at least one instruction, the at least one piece of program, the at least one piece of program, the The code set or the instruction set is loaded and executed by the processor to implement the facial feature recognition method provided by the above method embodiments.

Further, FIG. 9 shows a schematic diagram of a hardware structure of an electronic device for implementing the method for recognizing human face and facial features provided by an embodiment of the present invention. Face recognition device. As shown in FIG. 9, the electronic device 90 may include one or more (shown as 902a, 902b, . processing means of a logic device FPGA or the like), a memory 904 for storing data, and a transmission means 906 for communication functions. In addition, may also include: display, input/output interface (I/O interface), universal serial bus (USB) port (may be included as one of the ports of the I/O interface), network interface, power supply and/or camera. Those of ordinary skill in the art can understand that the structure shown in FIG. 9 is only a schematic diagram, which does not limit the structure of the above-mentioned electronic device. For example, the electronic device 90 may also include more or fewer components than shown in FIG. 9 , or have a different configuration than that shown in FIG. 9 .

It should be noted that the one or more processors 902 and/or other data processing circuits described above may generally be referred to herein as "data processing circuits." The data processing circuit may be embodied in whole or in part as software, hardware, firmware or any other combination. Furthermore, the data processing circuitry may be a single, stand-alone processing module, or incorporated in whole or in part into any of the other elements in the electronic device 90 (or mobile device). As referred to in the embodiments of the present application, the data processing circuit acts as a kind of processor control (eg, selection of a variable resistance termination path connected to an interface).

The memory 904 can be used to store software programs and modules of the application software, such as a program instruction/data storage device corresponding to the method described in the embodiment of the present invention, the processor 902 executes the software programs and modules stored in the memory 94 by running the software programs and modules. Various functional applications and data processing realize the above-mentioned method for identifying facial features. Memory 904 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, memory 904 may further include memory located remotely from processor 902, which may be connected to electronic device 90 through a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

Transmission means 906 is used to receive or transmit data via a network. The specific example of the above-mentioned network may include a wireless network provided by the communication provider of the electronic device 90 . In one example, the transmission device 906 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through the base station so as to communicate with the Internet. In one embodiment, the transmission device 906 may be a radio frequency (Radio Frequency, RF) module, which is used to communicate with the Internet in a wireless manner.

The display may be, for example, a touch screen-style liquid crystal display (LCD) that enables a user to interact with the user interface of the electronic device 90 (or mobile device).

An embodiment of the present invention further provides a storage medium, which can be set in an electronic device to store at least one instruction, at least one section of program, at least one instruction, at least one section of program, A code set or instruction set, the at least one instruction, the at least one piece of program, the code set or the instruction set is loaded and executed by the processor to implement the facial feature recognition method provided by the above method embodiments.

Optionally, in this embodiment, the above-mentioned storage medium may be located in at least one network server among multiple network servers of a computer network. Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk Or various media such as optical discs that can store program codes.

It should be noted that: the above-mentioned order of the embodiments of the present invention is only for description, and does not represent the advantages and disadvantages of the embodiments. And the foregoing describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that in the embodiments and still achieve desirable results. Additionally, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the various embodiments may be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the apparatus and electronic device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and for related parts, refer to the partial descriptions of the method embodiments.

Those of ordinary skill in the art can understand that all or part of the steps of implementing the above embodiments can be completed by hardware, or can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. The storage medium mentioned may be a read-only memory, a magnetic disk or an optical disk, etc.

The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection of the present invention. within the range.

Claims (10)

1. A facial feature recognition method, the method comprising:

acquiring a facial five sense organs image to be recognized;

extracting facial features from the facial features image to be recognized through a feature extraction model;

determining standard features of the five sense organs matched with the features of the five sense organs based on the similarity between the features of the five sense organs and a plurality of standard features of the five sense organs, wherein each standard feature of the five sense organs is marked with a corresponding attribute type of the five sense organs;

acquiring the attribute type of the facial features corresponding to the matched standard facial features;

taking the attribute type of the facial features corresponding to the matched standard facial features as the attribute type of the facial features to be recognized;

the feature extraction model corresponds to a network structure used for extracting feature information of facial features in the facial features classification model, the facial features classification model is obtained by performing machine learning training on a plurality of facial features sample images, and the facial features sample images carry classification labeling information of the users.

2. The method according to claim 1, wherein the step of obtaining the facial features image to be recognized comprises:

collecting an image to be identified;

carrying out face detection on the image to be recognized to obtain a face image;

carrying out face key point detection on the face image to obtain at least one facial feature image to be recognized;

the facial feature image to be recognized comprises an eyebrow object, an eye object, an ear object, a nose object or an oral object.

3. The method according to claim 1, wherein the extracting facial features from the facial features image to be recognized through a feature extraction model comprises:

determining a target feature extraction model corresponding to a facial feature object according to the feature object in the facial feature image to be recognized;

and extracting the facial features from the facial features image to be recognized through the target feature extraction model.

4. The method of claim 1, wherein determining a standard facial feature that matches the facial feature based on the similarity between the facial feature and a plurality of standard facial features comprises:

respectively carrying out similarity calculation on the features of the five sense organs and the standard features of the five sense organs to obtain a similarity set;

determining the maximum similarity in the similarity set;

and determining the standard facial features corresponding to the maximum similarity as the matched standard facial features.

5. The method according to claim 1, wherein the training process of the facial feature classification model comprises the following steps:

acquiring a facial five sense organ sample image;

inputting the facial five sense organ sample image into a neural network model for image classification training;

in the training process, adjusting the model parameters of the neural network model until the classification result output by the neural network model is matched with the input classification label information of the facial five sense organ sample image;

taking the neural network model corresponding to the adjusted model parameters as the facial five sense organs classification model;

wherein the facial five sense organs sample image comprises an eyebrow object, an eye object, an ear object, a nose object or an oral object.

6. The method of claim 5, wherein the adjusting process of the model parameters comprises the steps of:

obtaining a loss value according to an intermediate value and an annotation value, wherein the intermediate value corresponds to a classification result output by the neural network model, and the annotation value corresponds to the input classification annotation information of the facial feature sample image;

obtaining the gradient of the model parameter according to the loss value;

updating the model parameters based on the gradient of the model parameters.

7. The method of claim 5, wherein said obtaining the facial five sense organ sample image comprises:

determining a first quantity threshold value and a second quantity threshold value, wherein the first quantity threshold value represents the quantity of candidate users, and the second quantity threshold value represents the quantity of images corresponding to the same candidate user;

acquiring the facial five sense organ sample image according to the first quantity threshold and the second quantity threshold.

8. A facial feature recognition apparatus, the apparatus comprising:

an image acquisition module: the facial feature recognition system is used for acquiring facial feature images to be recognized;

a feature extraction module: extracting facial features from the facial features image to be recognized through a feature extraction model;

a feature matching module: the facial features are used for determining standard facial features matched with the facial features based on the similarity between the facial features and a plurality of standard facial features, and each standard facial feature is labeled with a corresponding facial attribute type;

the five sense organs attribute type acquisition module: the method is used for acquiring the attribute type of the facial features corresponding to the matched standard facial features;

and a five sense organs attribute type marking module: the facial feature recognition system is used for taking the facial feature attribute type corresponding to the matched standard facial feature as the facial feature attribute type of the facial feature image to be recognized;

the feature extraction model corresponds to a network structure used for extracting feature information of facial features in the facial features classification model, the facial features classification model is obtained by performing machine learning training on a plurality of facial features sample images, and the facial features sample images carry classification labeling information of the users.

9. An electronic device, comprising a processor and a memory, wherein at least one instruction, at least one program, set of codes, or set of instructions is stored in the memory, and wherein the at least one instruction, the at least one program, the set of codes, or the set of instructions is loaded and executed by the processor to implement the facial feature recognition method according to any one of claims 1-7.

10. A computer readable storage medium having stored therein at least one instruction, at least one program, a set of codes, or a set of instructions, which is loaded and executed by a processor to implement the method of facial feature recognition according to any one of claims 1-7.

Images