CN110728255A - Image processing method, device, electronic device and storage medium - Google Patents

Abstract

The application discloses an image processing method, an image processing device, electronic equipment and a storage medium, and relates to the technical field of image processing. The method comprises the following steps: acquiring image data to be processed; inputting image data to be processed into a plurality of pre-trained specific networks to obtain attribute labels corresponding to the image data, wherein each specific network is used for determining the attribute labels corresponding to the image data, and the attribute labels determined by each specific network are different from each other; inputting the attribute labels determined by each specific network into a pre-trained shared network to obtain an image recognition result; and outputting an image recognition result. Therefore, the plurality of specific networks jointly analyze the image data and obtain the plurality of attribute tags, the obtaining speed of the attribute tags can be improved, the shared network can obtain the image recognition result by combining the correlation of the attribute tags, and the accuracy and the overall performance of the recognition result are improved.

Description

Image processing method, device, electronic device and storage medium

technical field

The present application relates to the technical field of image processing, and more particularly, to an image processing method, apparatus, electronic device and storage medium.

Background technique

The existing image attribute recognition technical solutions are mainly based on traditional machine learning attribute recognition solutions and attribute recognition solutions based on convolutional neural network models. However, the most commonly used image attribute recognition technology is based on a single model and realizes a single attribute judgment, and when multi-attribute recognition is used, the efficiency is not high.

SUMMARY OF THE INVENTION

The present application proposes an image processing method, apparatus, electronic device and storage medium to improve the above-mentioned defects.

In a first aspect, an embodiment of the present application provides an image processing method, including: acquiring image data to be processed; inputting the image data to be processed into a plurality of pre-trained specific networks to obtain corresponding image data corresponding to the image data. attribute labels, wherein each of the specificity networks is used to determine the attribute labels corresponding to the image data, and the attribute labels determined by each of the specificity networks are different from each other; The attribute labels are input into a pre-trained shared network to obtain the image recognition result, wherein the shared network is used to determine the image recognition result according to each attribute label and the correlation of each attribute label; and output the image recognition result.

In a second aspect, the embodiments of the present application further provide an image processing method, including: acquiring multiple sample image data, each of which corresponds to multiple attribute labels; setting a shared network and multiple specific networks, each Each of the specific networks can identify at least one attribute label, and the attribute labels that can be identified by each of the specific networks are different from each other; input the plurality of sample image data into the shared network and the plurality of specific networks for training to obtain a shared network and multiple specific networks after training; obtain image data to be processed, and process the image data to be processed according to the shared network after training and multiple specific networks to obtain image recognition result.

In a third aspect, the embodiments of the present application further provide an image processing apparatus, including: a data acquisition unit, an attribute determination unit, a result acquisition unit, and an output unit. The data acquisition unit is used to acquire the image data to be processed. an attribute determination unit, configured to input the image data to be processed into a plurality of pre-trained specific networks to obtain attribute labels corresponding to the image data, wherein each of the specific networks is used to determine the image attribute labels corresponding to the data, and the attribute labels determined by each of the specific networks are different from each other. The result acquisition unit is used to input the attribute labels determined by each specific network into a pre-trained shared network to obtain image recognition results, wherein the shared network is used to Correlation determines image recognition results. An output unit, configured to output the image recognition result.

In a fourth aspect, the embodiments of the present application further provide an image processing apparatus, including: a sample acquisition unit, a setting unit, a network training unit, and an identification unit. A sample acquisition unit, configured to acquire a plurality of sample image data, each of the sample image data corresponds to a plurality of attribute tags. The setting unit is used for setting a shared network and a plurality of specific networks, each of the specific networks can identify at least one attribute label, and the attribute labels that can be identified by each of the specific networks are different from each other. A training unit, configured to input the plurality of sample image data into the shared network and the plurality of specific networks for training, so as to obtain the trained shared network and the plurality of specific networks. The recognition unit is used for acquiring the image data to be processed, and processing the image data to be processed according to the shared network after training and a plurality of specific networks to obtain an image recognition result.

In a fifth aspect, embodiments of the present application further provide an electronic device, comprising: one or more processors; a memory; and one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to perform the above method.

In a sixth aspect, an embodiment of the present application further provides a computer-readable medium, where the readable storage medium stores program code executable by a processor, and when multiple instructions in the program code are executed by the processor The processor is caused to perform the above method.

The image processing method, device, electronic device and storage medium provided by this application are pre-trained with a shared network and a plurality of specific networks, each specific network is used to determine the attribute label corresponding to the image data, and each specific network is used to determine the attribute label corresponding to the image data. The attribute labels determined by the specific network are different from each other, then when the image data to be processed is obtained, the image data to be processed is input into multiple specific networks, and each specific network can identify the specific network. Attributes that can be identified, so that multiple attribute tags corresponding to the image data to be processed can be identified by multiple specific networks respectively, which improves the identification of multiple attribute tags in the overall image data, so that the corresponding attribute tags of the image data can be obtained. attribute label, and then input the attribute label corresponding to the image data to the sharing network, the sharing network determines the image recognition result according to each attribute label and the correlation of each attribute label, and outputs the image recognition result. Therefore, multiple specific networks jointly analyze the image data and obtain multiple attribute labels, which can improve the speed of obtaining attribute labels, while the shared network can combine the correlation of each attribute label to obtain image recognition results, which improves the accuracy and accuracy of the recognition results. overall performance.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the drawings that are used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can also be obtained from these drawings without creative effort.

FIG. 1 shows a method flowchart of an image processing method provided by an embodiment of the present application;

FIG. 2 shows a method flowchart of an image processing method provided by another embodiment of the present application;

FIG. 3 shows a method flowchart of an image processing method provided by another embodiment of the present application;

FIG. 4 shows a method flowchart of S310 in the image processing method shown in FIG. 3 provided by an embodiment of the present application;

FIG. 5 shows a method flowchart of S310 in the image processing method shown in FIG. 3 provided by another embodiment of the present application;

FIG. 6 shows a schematic diagram of a measurement area provided by an embodiment of the present application;

FIG. 7 shows a schematic diagram of the connection of a specific network and a shared network provided by an embodiment of the present application;

FIG. 8 shows a schematic diagram of sub-image data provided by an embodiment of the present application;

FIG. 9 shows a schematic diagram of a face orientation provided by an embodiment of the present application;

FIG. 10 shows a method flowchart of an image processing method provided by still another embodiment of the present application;

FIG. 11 shows a module block diagram of an image processing apparatus provided by an embodiment of the present application;

FIG. 12 shows a module block diagram of an image processing apparatus provided by another embodiment of the present application;

FIG. 13 shows a module block diagram of an image processing apparatus provided by another embodiment of the present application;

FIG. 14 shows a module block diagram of an electronic device provided by an embodiment of the present application;

FIG. 15 shows a storage unit provided by an embodiment of the present application for storing or carrying a program code for implementing the graphics processing method according to the embodiment of the present application.

Detailed ways

In order to make those skilled in the art better understand the solutions of the present application, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application.

Face recognition is a technology that identifies different people based on human facial features. It has a wide range of applications and has been researched and applied for decades. With the development of related technologies such as big data and deep learning in recent years, the effect of face recognition has been improved by leaps and bounds, and it has been widely used in scenarios such as identity authentication, video surveillance, and beauty entertainment. Among them, the problem of witness comparison, that is, the problem of face recognition between standard ID photos and life photos, because the identification of the target person only needs to deploy their ID photos in the database, the target person is not required to collect life photos in the system for registration. The troubles are getting more and more attention.

Existing face attribute recognition technology solutions are mainly based on traditional machine learning attribute recognition solutions and attribute recognition solutions based on convolutional neural network CNN model and so on.

In some face recognition technologies, the concept of multi-task learning is used for reference. After the face detection algorithm is applied to extract the face region from the image or image, the convolutional neural network is used to learn the volume of the preset analysis task in the face database. Layer up, get the face analysis model, and complete the prediction of face emotion.

In other face recognition technologies, based on the idea of multi-task learning, multi-task cascade learning is realized by adding auxiliary information such as gender, whether to smile, whether to wear glasses, and posture in the training process. These attributes are used as labels, and face alignment is achieved by cascading.

The multi-task learning method not only performs single face attribute recognition, but also can achieve multi-attribute prediction. For example, multi-task learning methods can be introduced into face image race and gender recognition, with different semantics as different tasks, and semantic-based multi-task feature selection can be proposed, which can be applied to race and gender recognition, but when constructing the network structure, Race and gender are still solved separately as two tasks, the model has a lot of redundancy and cannot be predicted in real time.

Therefore, the most commonly used face attribute recognition technology is based on a single model and realizes a single attribute judgment, that is, only one task is learned at a time under a unified model, and complex problems are first decomposed into theoretically independent sub-problems , in each sub-problem, the samples in the training set only reflect the information of a single task. However, face images contain various attribute information such as race, gender, age, etc., and there are correlations between recognition tasks corresponding to different information, and certain relevant information is shared between tasks in the learning process. The multi-task learning method is introduced into the recognition of race, gender, and age in face images, and different semantics are used as different tasks, and a semantic-based multi-task feature selection is proposed. Applied to multi-attribute recognition, it can significantly improve the generalization ability and recognition of the learning system. Effect.

Although, in response to the above problems, multi-task learning-based face image race and gender recognition methods have emerged. It introduces the multi-task learning method into face image race and gender recognition, takes different semantics as different tasks, and proposes multi-task feature selection based on semantics. Although it significantly improves the generalization ability and recognition effect of the learning system, it adopts It is the traditional machine learning method, which greatly reduces the efficiency.

Existing patents also propose to combine deep learning technology with attribute recognition tasks, but only use multi-task learning, and propose a three-stage training process to learn face positions and faces on convolutional networks. The three features of action unit and emotional space value are used to complete the task of facial emotion analysis, and there is no multi-attribute output result.

Because, in order to solve the above-mentioned defects, the embodiment of the present application provides an image processing method. As shown in FIG. 1 , the method includes: S101 to S104.

S101: Acquire image data to be processed.

The image data may be an offline image file that has been downloaded in advance in the electronic device, or may be an online image file. In this embodiment of the present application, the image data may be online image data, for example, may be acquired in real time image.

Wherein, the online image data corresponds to a certain frame of images or multiple frames of images in a video file, and the online image data is the data that the video file has sent to the electronic device, for example, the video file is a certain movie, and the electronic device The received data is the playing time of the XX movie from 0 to 10 minutes, and the online image data corresponding to the XX movie is the data of the playing time of the XX movie from 0 to 10 minutes. Then, after acquiring each online image data, the client can decode each online image data to obtain its corresponding layer to be rendered, and then combine and display, so that multiple video images can be displayed on the screen.

As an embodiment, the electronic device includes multiple clients capable of playing video files. When the client of the electronic device plays a video, the electronic device can obtain the video file to be played, and then decode the video file. Specifically , the above-mentioned soft decoding or hard decoding can be used to decode the video file. After decoding, the multi-frame image data to be rendered corresponding to the video file can be obtained, and then the multi-frame image data needs to be rendered before being displayed on the display screen.

As another implementation manner, the image data may also be an image captured by a specified application in the electronic device through a camera of the electronic device. Specifically, when the specified application performs a certain function, the camera is called to capture an image, and the electronic device is requested to The device determines the image recognition result through the method of the present application, and sends the image recognition result to a designated application program, and the designated application program performs corresponding operations according to the image recognition result.

S102: Input the image data to be processed into a plurality of pre-trained specific networks to obtain attribute labels corresponding to the image data.

Wherein, each of the specific networks is used to determine an attribute label corresponding to the image data, and the attribute labels determined by each of the specific networks are different from each other.

Specifically, when learning the specificity network in advance, the sample image data input into the specificity network includes multiple attribute labels, for example, the color of the hair in the face image is black, the color of the white car in the vehicle image, etc., and The value of each attribute tag is 0 or 1, 0 means not having this attribute, 1 means not having this attribute, and these attribute tags are preset image eigenvalues for obtaining image recognition results. The role of the sex network determines whether a predefined attribute tag is included within the image.

Specifically, each specific network can determine an attribute label corresponding to the image data. In some embodiments, each specific network can determine at least one attribute label corresponding to the image data. For example, the multiple specific networks include The first specific network and the second specific network, and the attribute labels include label 1, label 2 and label 3, then the first specific network is used to identify the recognition result of label 1 corresponding to the image data, that is, if the image data includes labels 1, the image data recognized by the first specific network corresponds to label 1, or, a recognition result about label 1 is given, that is, label 1 is 1, and if there is no label 1, the given recognition result is label 1 is 0. The second specificity network is used to determine label 2 and label 3. Therefore, label 1, label 2 and label 3 are separately identified by different specific networks, which can improve the recognition efficiency and avoid label 1, label 2 and label 3 from being identified by the same One specific network recognizes and results in an excessive amount of computation, and the first specific network is only used to recognize label 1, so there is no need to learn and train the recognition for labels 2 and 3, and the training cost is also reduced.

In addition, it should be noted that multiple specific networks can be executed at the same time, that is, they can be operated simultaneously under multiple threads, rather than a cascade relationship, that is, the output result of a specific network does not require other specific input to the sexual network. Specifically, the structure of the specificity network is introduced in the following embodiments.

In the embodiment of the present application, the specific network is mainly used to segment the target object from the image and identify the target object, that is, the specific network can also be a target detection network. Obviously, the specific network is to divide the target object The segmentation and recognition are combined into one. Commonly used target detection networks include GOTURN network, MobileNet-SSD deep convolutional neural network, FasterRCNN neural network, Yolo neural network and SPP-Net (Spatial Pyramid Pooling) neural network. GOTURN neural network is a target detection algorithm that uses convolutional neural network for offline training. It uses the existing large-scale classification dataset pre-trained CNN classification network to extract features and identify the features.

S103: Input the attribute label determined by each specific network into a pre-trained shared network to obtain an image recognition result.

Wherein, the shared network is used to determine the image recognition result according to each attribute tag and the correlation of each attribute tag. Specifically, the shared network focuses on learning the shared information of all attribute tags. For example, when the attribute tag with the raised corner of the mouth and the attribute tag with rolling eyes appear at the same time, the emotion expressed is thinking, while the attribute tag with the raised corner of the mouth and the attribute tag with rolling eyes appear at the same time. The correlation of is the identification result identified by the shared network and obtained according to the correlation. That is to say, after the shared network is pre-trained, it can identify the correlation between each attribute label, and obtain the image recognition result based on it.

S104: Output the image recognition result.

The way of outputting the image recognition result may be to display the image recognition result on the screen, or send the image recognition result to the requesting end requesting to obtain the image recognition result, and the requesting end may be the server communicating with the electronic device, or it may be other The electronic device can also be an application program installed in the electronic device, then the execution subject of the above method can be the application program capable of image recognition in the electronic device, or the operating system of the electronic device, then after the image recognition result is obtained, the image The recognition result is sent to the requesting end, and the requesting end performs an operation according to the image recognition result, for example, transaction payment or screen unlocking.

Referring to FIG. 2, an image processing method provided by an embodiment of the present application is shown, and the method includes: S201 to S207.

S201: Acquire original image data.

The original image data may be a grayscale value corresponding to the image, that is, the value of each pixel in the image is a value in the [0, 255] interval, that is, a grayscale value. Then, as an implementation manner, when the electronic device obtains an image, the image may be a color image, and then the color image is binarized to obtain a grayscale image, and then each pixel in the grayscale image is obtained. The grayscale values of , constitute the data of the original image.

In addition, it should be noted that the original image data may be data collected by a camera of an electronic device. For example, the image processing method is applied to real-time analysis of the image data collected by the camera, and the analysis is based on the attributes of face recognition. analysis.

Specifically, the image data can also be an image collected by a designated application in the electronic device through the camera of the electronic device. Specifically, when the designated application performs a certain function, the camera is called to capture an image, and the electronic device is requested to pass the application. The method determines an image recognition result, and sends the image recognition result to a designated application program, and the designated application program performs corresponding operations according to the image recognition result. Wherein, the designated application may be a screen unlocking APP in the electronic device or a payment APP. For example, the screen unlocking APP performs face recognition through the face image collected by the camera to determine the identity information, and judges whether the face image matches the preset face image. If it matches, it is determined that the unlocking is successful. , it is determined that the unlocking is unsuccessful.

The preset face image may be a face image preset by the user, which may be stored in the mobile terminal, or may be stored in a certain server or memory. Set up a face image. Specifically, it may be preset feature information of a preset face image, then if the face image is a two-dimensional image, the preset feature information is the facial feature point information of the face image pre-entered by the user, if the face image is If a three-dimensional image is used, the preset feature information is the face three-dimensional information of the face image pre-entered by the user. Then the way of judging whether the face image satisfies the preset condition is to obtain the feature point information of the face image, and compare the collected feature information of the face image with the preset feature information pre-entered by the user, and if it matches, If it is determined that the face image satisfies the preset conditions, it is determined that the face image has the authority to unlock the screen of the mobile terminal; if it does not match, it is determined that the face image does not meet the preset conditions and has no authority to unlock the screen.

In the embodiment of the present application, the image data includes a face image, and the recognition for the image data is face recognition, then when performing the acquisition of the image data to be processed, it can be first judged whether the image data includes a human face, if included, Then perform subsequent operations. Specifically, the image collected by the camera is a two-dimensional image. By finding out whether there are facial features in the image, it can be determined whether a face image is collected. If collected, the collected face image is sent to the mobile terminal. The processor enables the processor to analyze the face image and perform the screen unlocking operation. As another embodiment, if the camera includes structured light, it is determined whether there is three-dimensional face information according to the three-dimensional information collected by the structured light, and if so, the collected image is sent to the processor of the mobile terminal.

In addition, if the image collected by the camera does not include a face image, return to and continue to perform the operation of judging whether the image collected by the camera includes a face image, or send a face collection reminder message to remind the user A face image is collected using the camera. Specifically, the face collection reminder information may be displayed on the current interface of the electronic device.

S202: Normalize the original image data to obtain image data to be processed.

Normalize each pixel value in the original image, that is, the original value of 0-255 becomes the value in the range of 0-1, which can improve the calculation speed of the subsequent specific network and shared network, and improve the overall The speed of image processing, specifically, normalization of the original image data by means of mean-variance normalization or grayscale transformation normalization.

In addition, after normalizing the original image data, redundant information may be removed, where the redundant information refers to the gap between the compressed distributions.

The original image data after normalization processing is regarded as the image data to be processed.

S203: Determine the attribute label corresponding to each specific network.

Specifically, the attribute label that can be identified by the specific network is the attribute label corresponding to the specific network. The attribute labels that can be identified by the specific network are already set when the specific network is trained. For details, reference may be made to subsequent embodiments.

S204: Divide the image data into multiple sub-image data according to the attribute label corresponding to each specific network.

Since each attribute tag in the image data corresponds to a position in the image, taking a face image as an example, the position in the image corresponding to the attribute tag of hair color is the position of the hair, and the image corresponding to the attribute tag of eye color The position in the data is the eye position, so the position corresponding to each attribute label can be predetermined. For example, when training a specific network, the area corresponding to each attribute label is set. Each pixel value corresponds to a pixel coordinate, so the position of the region corresponding to each attribute label in the pixel coordinate can be determined, and then the position of the attribute label recognized by each specific network in the image can be determined. Furthermore, the image data can be divided into multiple sub-image data, and each sub-image data corresponds to an area in the image, and the attribute labels in this area all correspond to the same specific network, that is, each specific network can identify The attribute label of is located in the area corresponding to the sub-image data.

Thus, the image sub-region corresponding to each specific network can be obtained, that is, the sub-image data corresponding to each specific network. For example, the image is divided into a first area, a second area and a third area, the attribute labels that can be recognized by the first specific network are distributed in the first area, and the attribute labels that can be recognized by the second specific network are distributed in the first area. In the second area, the attribute labels that can be recognized by the third specific network are distributed in the third area, and the image data is divided into three sub-image data, namely the first sub-image data, the second sub-image data and the third sub-image data. image data, the first sub-image data corresponds to the first area, the second sub-image data corresponds to the second area, and the third sub-image data corresponds to the third area.

In addition, in order to better divide multiple areas according to the pixel coordinates, the image can be uniformly adjusted in one direction, so that the designated area is located in the same position. For the purpose of face image, when acquiring the original image, you can take a screenshot of the face area in the original image, and adjust the image rotation according to a certain direction, so that the face is fixed to a certain position, for example, always keep the forehead of the face at the position of the face. The upper part of the image, the chin is in the lower part of the image.

S205: Input the sub-image data into a specific network corresponding to the sub-image data.

As an implementation manner, after determining the region where the attribute label corresponding to each specific network is located in the image, the image can be divided into multiple regions, and each region corresponds to a specific network, according to the determined The multiple regions divide the image into multiple sub-images, and the pixel data corresponding to each sub-image is used as the sub-image data, that is, the pixel data processed in S202.

For example, the above-mentioned first area, second area and third area, and then, the image is divided into three sub-images, namely the first sub-image, the second sub-image and the third sub-image, and then the first sub-image is divided into three sub-images. The pixel data corresponding to each pixel value in the second sub-image is input into the first specific network, the pixel data corresponding to each pixel value in the second sub-image is input into the second specific network, and the pixel corresponding to each pixel value in the third sub-image is input. Data entered into the third specificity network.

Therefore, it is not necessary to input the entire image data into each specific network, but only the sub-image data corresponding to the attribute label that can be recognized by the specific network is input into the specific network, which reduces the calculation amount of the specific network. Improved overall recognition speed.

S206: Input the attribute label determined by each specific network into the pre-trained shared network to obtain an image recognition result.

S207: Output the image recognition result.

It should be noted that, for parts not described in detail in the above steps, reference may be made to the foregoing embodiments, and details are not described herein again.

In addition, before executing S102 or S204, it is also necessary to train the specific network and the shared network. Specifically, the training process can be after S101 and before S102 or after S201 and before S204, or before S101 and S201, In this embodiment of the present application, the specific network and the shared network may be trained before executing the image recognition method.

Specifically, please refer to FIG. 3 , which shows the training process of the specific network and the shared network training in the image processing method provided by the embodiment of the present application. Specifically, as shown in FIG. 3 , the method includes: S310 to S370 .

S310: Acquire a plurality of sample image data, each of which corresponds to a plurality of attribute tags.

Specifically, the sample image data is already marked image data, which may be a pre-acquired image, and the image is manually marked, and each marked point corresponds to an attribute label. For example, the sample image data may be the CelebA face attribute data set as the experimental data set. The dataset contains about 200,000 face images, and each image provides 40 face attribute annotations and 5 face key points location information. According to the official standards of CelebA, this paper takes about 100,000 face images for the training of the network model, about 10,000 images for verification, and 10,000 images for testing the network model.

For the public face attribute data set, each face image corresponds to 40 attribute labels, and the value of each label is 0 or 1, 0 means not having this attribute, 1 means not having this attribute.

It should be noted that the sample image data and the image data to be processed both contain face images, that is to say, the image processing method provided by the embodiment of the present application is applied to face attribute recognition, and the training of the specific network and the shared network It is also trained for face attribute recognition.

Further, in order to improve the accuracy of image recognition, the faces may be aligned. Specifically, please refer to FIG. 4 , and S310 may include: S311 to S314.

S311: Acquire multiple sample image data.

Specifically, for this step, reference may be made to the above description, which will not be repeated here.

S312: Identify the position information of the face key points in the sample image in each of the sample image data.

Specifically, the face key points may be facial features in the face image, for example, the face key points may be eyes, nose, mouth, and the like. The specific recognition method can be, through the facial features recognition method of the face image, for example, the PCA (principal component analysis) analysis method to determine the facial features in the face image, so as to determine the position information of the key points of the face in the face image, i.e. pixel coordinates.

Specifically, after obtaining the sample image, obtain the face area image, crop the face area, perform face correction on the sample data to be trained, and determine the position information of the key points of the face, such as eyes, nose, mouth, etc. .

S313: Adjust the orientation of the face in each of the sample image data to conform to the preset orientation according to the position information of the key points of the face in each of the sample image data.

Specifically, the preset orientation may be that the human face faces straight ahead. Specifically, the face facing straight ahead means that the forehead part of the human face is in the upper part of the image, and the chin part of the human face is in the lower part of the image. Specifically, the orientation of the face in the image can be determined by the position information of the key points of the face. Specifically, the same pixel coordinate system is set for each sample image, that is, the pixel coordinate system can be established with the vertex on the top left of the sample image as the origin, so that the pixel coordinates of the key points of the face in the face image are It can be obtained, and the positional relationship between the forehead and the chin of the person can be determined through the position information of the key points of the face. Specifically, if it is determined that the eyes are on the left side of the image, and the mouth is on the right side of the image, and the two If the difference between the ordinates is less than the specified value, it can be determined that the eyes and the mouth are on the same horizontal line, and the face orientation in the sample image data can be made to conform to the preset orientation by rotating 90° clockwise. As an implementation manner, the preset orientation may be within 15 degrees of the face facing straight ahead.

In addition, after adjusting the face orientation in each of the sample image data to conform to the preset orientation according to the position information of the face key points of each of the sample image data, in order to reduce the amount of calculation, the orientation of the sample image can also be adjusted. size, so as to adjust the size. Specifically, through the positioning of key points of the face, such as eyes, nose, mouth, etc., the direction of the object to be predicted is adjusted according to the preset direction standard to ensure that the face of each object to be predicted faces. Within 15 degrees in front, the face is aligned, and a predetermined proportion of margin is added to the face area. At the same time, in order to reduce the amount of calculation, the image size is set to a specified size, for example, the specified size can be 112*112. Specifically, the size of the entire image can be compressed to a specified size, or the sample image can be cropped with a window of the specified size. Specifically, the center point of the sample image can be used as the center point of the window, and the size of the window can be detected. The image in the corresponding image area is used as the resized image. As an implementation manner, the size of the window may be 112*112.

S314: Use the adjusted sample image data as the sample image data used for training the shared network and multiple initial specificity networks this time.

Specifically, if the above adjustment is to adjust the face orientation in each of the sample image data to conform to the preset orientation according to the position information of the face key points of each of the sample image data, the face orientation adjustment will be performed. The subsequent sample image data is used as the sample image data for training the shared network and multiple initial specificity networks this time. If the above adjustment includes the position information of the face key points of each of the sample image data, the adjustment of each The face orientation in each of the sample image data conforms to the preset orientation and the size of the sample image is adjusted to the specified size, then the sample image data after face orientation adjustment and size adjustment will be used for training the sharing. Sample image data for the network and multiple initial specificity networks.

Further, in order to increase the data samples and improve the image processing model after training, that is, the face recognition model, that is, the generalization type of the shared network and multiple specific networks, specifically, please refer to FIG. 5, S310 may include: S311 , S315, S316 and S317.

S311: Acquire multiple sample image data.

Specifically, for this step, reference may be made to the above description, which will not be repeated here.

S315: Perform data enhancement processing on the plurality of sample image data, so that the illumination intensity and contrast of each of the sample image data are randomly distributed within a preset interval.

Specifically, the illumination intensity of the object to be trained is transformed according to a preset illumination intensity interval, and data in which the illumination intensity of each object to be trained is randomly distributed in the preset illumination intensity interval is obtained; the object to be trained is transformed according to a preset contrast interval The contrast of each object to be trained is randomly distributed in the preset contrast interval.

Specifically, the preset illumination intensity interval may be a preset illumination intensity area, and after acquiring the illumination intensity of each pixel in the sample image, the illumination intensity of the pixel may be adjusted to be within the preset illumination intensity interval . As an embodiment, the distribution of the illumination intensity of each pixel in the sample image can be counted, so that the pixel with higher illumination intensity is also located at the value of higher illumination intensity within the preset illumination intensity interval, and the illumination intensity The lower pixel points are also located at the lower value of the illumination intensity in the preset illumination intensity interval. In addition, the continuity of the distribution of the illumination intensity of each pixel in the preset illumination intensity interval can also be increased, that is, multiple pixel values. In the distribution sub-region of the illumination intensity, the difference between the intensity values of two adjacent sub-regions is not greater than the specified value, so that the illumination intensity is randomly distributed within the preset illumination intensity interval, thereby increasing the diversity of data. Specifically, The illumination intensity of the pixels in each sample image data can be randomly distributed within its corresponding preset illumination intensity interval, and the preset illumination intensity intervals corresponding to each sample image data may not be all the same, thereby further increasing the data density. Diversity, thereby improving the generalization of later model training.

Similarly, to transform the contrast of the object to be trained according to the preset contrast interval, to obtain the data of random distribution of the contrast of each object to be trained in the preset contrast interval, reference may be made to the above-mentioned adjustment process of the illumination intensity. Therefore, the contrast of the pixels in each sample image data can be randomly distributed in its corresponding preset contrast interval, and the preset contrast interval corresponding to each sample image data can be different, thereby further increasing the variety of data. This improves the generalization of later model training.

In addition, before performing the above-mentioned enhancement processing, the sample data obtained by the processing can be normalized, and the pixel values of the sample data can be normalized from [0, 255] to [0, 1] to remove the redundancy contained in the sample data. information.

S316: Crop each of the sample image data after the enhancement processing according to a preset random cropping ratio, and the size of each cropped sample image data is a preset size.

Crop the object to be trained according to a preset random cropping ratio, and adjust it to a preset size, where the preset size may be 112*112; flip the object to be trained in a horizontal direction.

It should be noted that, for the above-mentioned cropping of the object to be trained according to the preset random cropping ratio, and adjusting it to a preset size, reference may be made to the aforementioned cropping method of adjusting to a specified size, and the preset size is the same as the specified size.

S317: Use the cropped sample image data as the sample image data used for training the shared network and multiple initial specificity networks this time.

It should be noted that the above steps S311 to S314 can replace S310, that is, S320 is executed after S311, S312, S313 and S314, or S310 can be replaced by steps S311, S315, S316 and S317, that is, S311, S315, S316 and S320 is executed after S317, or S310 can be replaced together with S311 to S317, that is, S320 is executed after S311, S312, S313, S314, S315, S316 and S317.

S320: Setting a shared network and multiple specific networks.

Each of the specific networks can identify at least one attribute label, and the attribute labels that can be identified by each of the specific networks are different from each other. If a specific network is configured for each attribute label, there will be a huge computational cost. For example, taking face image recognition as an example, assuming that there are 40 attribute labels in total, if 40 face attributes are regarded as 40 independent tasks, there is a huge computational cost to directly regard 40 face attributes as 40 independent tasks. , and ignores the displayed positional correlation between face attributes. Therefore, a face can be divided into multiple areas, and each area corresponds to a specific network. Specifically, the specific implementation of setting a shared network and multiple specific networks can be divided into multiple measurement areas, each of which is measured. Corresponding to different areas of the face; multiple specific networks are set according to the multiple measurement areas, each specific network corresponds to a measurement area, and each of the specific networks is used to confirm the attribute label in the corresponding measurement area .

Specifically, four specificity networks can be set, which are the upper specific network, the middle specific network, the lower specific network and the whole face specific network. Correspondingly, the attribute labels are divided into four groups, namely the upper group, the middle group, the lower group and the full face group, each group corresponds to the attribute label, and the attribute labels of each group are different, that is, each specific network The attribute label that can identify the corresponding group. According to their corresponding positions, the attribute classification of each group can be treated as a separate attribute learning task. After dividing the attributes into 4 groups, the attribute classification problem of each attribute group is regarded as a subtask, specifically, the attribute labels are shown in the following table:

As shown in FIG. 6 , the sample image is divided into four areas, namely the upper area m1, the middle area m2, the lower area m3 and the full face area m4. As an embodiment, the upper area m1 is the top side of the image to the The area between the abscissas of the position of the lowermost eye in both eyes, specifically, as shown in Figure 6, the abscissa of the position of the lowermost eye in both eyes is provided with a straight line parallel to the abscissa axis , denoted as the first straight line, then the area between the first straight line and the top side of the image is taken as the upper area. Select a position point in the area between the nose and the upper lip, which can be a middle position point, and set a line parallel to the abscissa axis, denoted as the second line, and the interval before the first line and the second line is regarded as the middle The second execution and the area from the bottom side of the image to the present are taken as the lower area, and the area between the chin end of the face to the top of the hair is taken as the full face area, wherein the full face area can enclose the face and hair. Among them, the above-mentioned areas are measurement areas, that is, the upper area m1 , the middle area m2 , the lower area m3 and the full face area m4 are four measurement areas.

In the embodiment of the present application, a total of four specific networks and one shared network are included, and each task has an independent specific network. Unlike the branching structure, in order to better preserve the specificity of each task, the parameters of the specificity network between tasks are not shared. As an independent grid, the shared network does not correspond to a specific learning task, but extracts complementary information between tasks in order to learn the correlation between tasks. Through a simple connection unit, the specific network and the shared network are connected, so as to maximize the information flow between the two. Specifically, as shown in Figure 7, the connection relationship between the specific network and the shared network is shown in Figure 7. The input of each layer of the shared network, in addition to the output features of the previous layer, also includes all the The output features of the specificity network. These features are concatenated together to form the input to each layer of the specificity network. At the same time, the input of each layer of the specific network, in addition to the output features of the previous layer, also includes the output features of the shared network in the previous layer. The two are concatenated together to form the final input. In addition, FIG. 7 only shows the connection relationship between the two specific networks and the shared network, and the connection relationship for the four specific networks can also be reasonably obtained with reference to what is shown in FIG. 7 .

The multi-task attribute recognition model constructed in the embodiment of the present application includes 4 specific networks and 1 shared network. The specificity network focuses on learning the specific features of each feature group, while the shared network focuses on learning the shared information of all feature groups. The specific network and the shared network are connected and exchanged through the local shared unit, thus forming the whole local shared multi-task face multi-attribute classification network. Each specific network and shared network have the same network structure. Both contain 5 convolutional layers and 2 fully connected layers. At the same time, each convolutional layer and fully connected layer is followed by a normalization layer and a ReLU (Rectified Linear Unit) activation layer. The number of output channels of each layer between the specific networks is also the same, while the number of output channels of the shared network is different from that of the specific network.

S330: Input the multiple sample image data into the shared network and multiple specific networks for training, so as to obtain the trained shared network and multiple specific networks.

As an embodiment, the entire sample image data can be input into each specific network for training, but each specific network can only identify the corresponding attribute label, for example, the upper specific network can recognize the upper The attribute tag of the group, while other attribute tags are not recognized.

As another embodiment, in order to reduce the amount of computation and training speed, different parts of different sample image data may be input for each specific network. Specifically, according to the attribute label corresponding to each specific network, the The sample image data is divided into a plurality of sub-sample image data, and the sub-sample image data is input into a specific network corresponding to the sub-image data.

As shown in Figure 8, the same sample image is divided into four sub-sample images, the upper left image is the sub-sample image data corresponding to the upper region m1 in the sample image shown in 6, and the upper right image is the sample shown in 6. The subsample image data corresponding to the middle area m2 in the image, the lower left image is the subsample image data corresponding to the lower area m3 in the sample image shown in 6, and the lower right image is the full face area in the sample image shown in 6 For the sub-sample image data corresponding to m4, the sub-sample image data corresponding to the upper region m1 is input into the upper specific network for training the upper specific network, and the sub-sample image data corresponding to the middle region m2 is input into the middle specific network, using For training the middle specific network, input the sub-sample image data corresponding to the lower area m3 into the lower specific network for training the lower specific network, input the sub-sample image data corresponding to the full-face area m4 into the full-face specific network, use for training full face-specific networks.

In addition, it should be noted that a part of the sample image data obtained in step S311 is used for training the above-mentioned network model, and the other part is used for testing the above-mentioned network model. It is randomly divided into training set and test set with a ratio of 8:2. The training set is used for the training of the face multi-attribute recognition model, and the test set is used for the test of the face multi-attribute recognition model, to ensure that the data of the same person only appears in a set.

Send the test data set to the trained specific network and shared network for testing to verify the accuracy of the network model, and send the wrongly judged samples in the above test data set to the network model again for fine-tuning to improve the generalization of the model sex.

In the embodiment of the present application, the Adam gradient descent algorithm is used, and Adam is an efficient calculation method, which can improve the gradient descent convergence speed. During the training process, the training set is input into the convolutional neural network model and the preset number of epochs is iterated. In this method, the epochs are set to 90 times. In each iteration calculation process, the Adam gradient descent algorithm is used to optimize the objective function. In this method, the batch_size is set to 64, that is, 64 input images are sent in each round of training. Among them, the specific network and the shared network are both trained based on the convolutional neural network model.

For multi-attribute problems, this method uses cross-entropy as a loss function for training, which is used as a standard to measure the cross-entropy between the target and the output. The formula is as follows:

代表第i个属性第j个样本的标签值，而

指第i个属性第j个样本的预测值。In the above formula, m represents the total number of attributes, n _i represents the total number of samples of the ith attribute,

represents the label value of the jth sample of the ith attribute, and

Refers to the predicted value of the jth sample of the ith attribute.

S340: Acquire image data to be processed.

It should be noted that, in addition to referring to the above steps to obtain the image data to be processed, the detection of the face orientation can also be added. Whether the face orientation in the image satisfies the preset orientation. Specifically, taking the camera in the electronic device to collect the face image as an example, the electronic device calls the camera to collect the face image in response to the face recognition request, and recognizes the position information of the key points of the user's face in the face image, so as to determine the Whether the orientation of the face is the default orientation.

As shown in Figure 9, the face of the left image is oriented to the right of the user whose face image is being collected, and several key points on the image can be determined, namely, left eye a1, right eye a2, nose a3 and lips a4, and from the vertical symmetry line of the image, it can be seen that the right eye a2, the nose a3 and the lips a4 are all located on the left side of the symmetry line, so it can be determined that the user's face is biased to the right, specifically, it can be A collection frame is displayed on the screen, and the user needs to move his face in the collection frame. If the user is not facing the screen, or the orientation of the face does not conform to the preset orientation, a reminder message can be sent Prompt the user to adjust the face orientation for the camera. As shown in Figure 9, in the face picture on the right, the left eye b1, right eye b2, nose b3 and lips b4 are located near the line of symmetry, and the left eye b1 and the right eye b2 are separated on both sides of the line of symmetry, it can be determined that the person If the face is facing the screen, if it conforms to the preset orientation, the face image can be collected normally and recognized later.

S350: Input the image data to be processed into a plurality of pre-trained specific networks to acquire attribute labels corresponding to the image data.

S360: Input the attribute label determined by each specific network into the pre-trained shared network to obtain an image recognition result.

S370: Output the image recognition result.

It should be noted that the face image not only includes face attribute information such as facial features, race, gender, age, expression, etc., but also can express the identity information of the person. Therefore, face attribute recognition has broad application prospects in the fields of age-related access control, face retrieval of face attributes, security and human-computer interaction.

Referring to FIG. 10 , an image processing method provided by an embodiment of the present application is shown, and the method includes: S1001 to S1004.

S1001: Acquire multiple sample image data, each of which corresponds to multiple attribute tags.

S1002: Set a shared network and a plurality of specific networks, each of the specific networks can identify at least one attribute label, and the attribute labels that can be identified by each of the specific networks are different from each other.

S1003: Input the multiple sample image data into the shared network and multiple specific networks for training, so as to obtain a trained shared network and multiple specific networks.

Among them, S1001 to S1003 are the training process of the shared network and multiple specific networks, and the specific implementation thereof may refer to the aforementioned S310 to S330, which will not be repeated here.

S1004: Acquire image data to be processed, and process the image data to be processed according to the trained shared network and multiple specific networks to obtain an image recognition result.

To obtain an image recognition result by processing the image data to be processed according to the trained shared network and multiple specific networks, reference may be made to the foregoing embodiments, and details are not repeated here.

Please refer to FIG. 11 , which shows a structural block diagram of an image processing apparatus 1100 provided by an embodiment of the present application. The apparatus may include: a data acquisition unit 1110 , an attribute determination unit 1120 , a result acquisition unit 1130 , and an output unit 1140 .

The data acquisition unit 1110 is used to acquire image data to be processed.

The attribute determination unit 1120 is configured to input the image data to be processed into a plurality of pre-trained specific networks to obtain attribute labels corresponding to the image data, wherein each specific network is used to determine the The attribute labels corresponding to the image data, and the attribute labels determined by each of the specific networks are different from each other.

The result obtaining unit 1130 is configured to input the attribute labels determined by each of the specific networks into a pre-trained shared network to obtain image recognition results, wherein the shared network is used to The correlation determines the image recognition results.

The output unit 1140 is configured to output the image recognition result.

Please refer to FIG. 12, which shows a structural block diagram of an image processing apparatus 1200 provided by an embodiment of the present application. The apparatus may include: a training unit 1210, a data acquisition unit 1220, an attribute determination unit 1230, a result acquisition unit 1240, and an output unit 1250.

The training unit 1210 is used for training the shared network and the plurality of specific networks.

Specifically, the training unit 1210 includes a sample acquisition subunit 1211 , a setting subunit 1212 and a training subunit 1213 .

The obtaining subunit 1211 is configured to obtain a plurality of sample image data, each of which corresponds to a plurality of attribute tags.

The setting subunit 1212 is configured to set a shared network and a plurality of specific networks, each of the specific networks can identify at least one attribute label, and the attribute labels that can be identified by each of the specific networks are different from each other.

The training subunit 1213 is configured to input the plurality of sample image data into the shared network and the plurality of specific networks for training, so as to obtain the trained shared network and the plurality of specific networks.

Further, the sample image data and the image data to be processed both contain face images, and the acquisition subunit 1211 is also used to acquire multiple sample image data; identify the key points of the face in each of the sample image data in the the position information in the sample image; according to the position information of the face key points of each of the sample image data, adjust the face orientation in each of the sample image data to conform to the preset orientation; the adjusted sample image data As the sample image data for training the shared network and multiple initial specificity networks this time.

Further, the acquisition subunit 1211 is also used to acquire multiple sample image data; perform data enhancement processing on the multiple sample image data, so that the illumination intensity and contrast of each of the sample image data are randomly within a preset interval. distribution; each of the sample image data after the enhancement processing is cropped according to a preset random cropping ratio, and the size of each cropped sample image data is a preset size; the cropped sample image data is used as This time the sample image data used to train the shared network and multiple initial specificity networks.

Further, the sample image data and the image data to be processed both contain face images, and the plurality of attribute labels corresponding to each of the sample image data correspond to different positions of the face; the setting subunit 1212 is also used to divide multiple measurement areas, each of which corresponds to a different area of the face; multiple specific networks are set up according to the multiple measurement areas, each specific network corresponds to a measurement area, and each specific network is used for Check the attribute label in the corresponding measurement area.

The data acquisition unit 1220 is used for acquiring image data to be processed.

Further, the data acquisition unit 1220 is further configured to acquire original image data; and normalize the original image data to obtain image data to be processed.

The attribute determination unit 1230 is configured to input the image data to be processed into a plurality of pre-trained specific networks to obtain attribute labels corresponding to the image data, wherein each specific network is used to determine the The attribute labels corresponding to the image data, and the attribute labels determined by each of the specific networks are different from each other.

Further, the attribute determining unit 1230 is further configured to determine the attribute label corresponding to each specific network, wherein the attribute label that can be identified by the specific network is the attribute label corresponding to the specific network; The attribute label corresponding to the specific network divides the image data into a plurality of sub-image data; the sub-image data is input into the specific network corresponding to the sub-image data.

The result obtaining unit 1240 is configured to input the attribute labels determined by each specific network into a pre-trained shared network to obtain image recognition results, wherein the shared network is used to The correlation determines the image recognition results.

The output unit 1250 is configured to output the image recognition result.

Please refer to FIG. 13 , which shows a structural block diagram of an image processing apparatus 1300 provided by an embodiment of the present application. The apparatus may include: a sample acquisition unit 1310 , a setting unit 1320 , a network training unit 1330 , and an identification unit 1340 .

The sample obtaining unit 1310 is configured to obtain a plurality of sample image data, each of which corresponds to a plurality of attribute tags.

The setting unit 1320 is configured to set a shared network and a plurality of specific networks, each of the specific networks can identify at least one attribute label, and the attribute labels that can be identified by each of the specific networks are different from each other.

The network training unit 1330 is configured to input the plurality of sample image data into the shared network and the plurality of specific networks for training, so as to obtain the trained shared network and the plurality of specific networks.

The sample acquisition unit 1310 , the setting unit 1320 , and the network training unit 1330 correspond to the above-mentioned training unit 1210 . As an embodiment, the sample acquisition unit 1310 corresponds to the acquisition subunit, the specific implementation of the sample acquisition unit 1310 may refer to the acquisition subunit, the setting unit 1320 corresponds to the setting subunit, and the specific implementation of the setting unit 1320 may refer to the setting subunit unit, the network training unit 1330 corresponds to the training subunit, and the specific implementation of the network training unit 1330 can refer to the training subunit.

The identification unit 1334 is configured to acquire image data to be processed, and process the image data to be processed according to the trained shared network and multiple specific networks to obtain an image recognition result.

Specifically, the identification unit 1334 is configured to acquire image data to be processed; input the image data to be processed into multiple pre-trained specific networks to acquire attribute labels corresponding to the image data, wherein each of the The specific network is used to determine the attribute label corresponding to the image data, and the attribute labels determined by each specific network are different from each other; the attribute label determined by each specific network is input into the pre-trained shared network , to obtain the image recognition result, wherein the shared network is used to determine the image recognition result according to each attribute label and the correlation of each attribute label; and output the image recognition result.

As an embodiment, the identification unit 1334 corresponds to a data acquisition unit, an attribute determination unit, a result acquisition unit, and an output unit, and the specific implementation can refer to the data acquisition unit, the attribute determination unit, the result acquisition unit, and the output unit.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, for the specific working process of the above-described devices and modules, reference may be made to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

In several embodiments provided in this application, the coupling between the modules may be electrical, mechanical or other forms of coupling.

In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist physically alone, or two or more modules may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware, and can also be implemented in the form of software function modules.

Please refer to FIG. 14 , which shows a structural block diagram of an electronic device provided by an embodiment of the present application. The electronic device 100 may be an electronic device capable of running an application program, such as a smart phone, a tablet computer, an electronic book, or the like. The electronic device 100 in the present application may include one or more of the following components: a processor 110, a memory 120, and one or more application programs, wherein the one or more application programs may be stored in the memory 120 and configured to be executed by One or more processors 110 execute, and one or more programs are configured to execute the methods described in the foregoing method embodiments.

The processor 110 may include one or more processing cores. The processor 110 uses various interfaces and lines to connect various parts of the entire electronic device 100, and executes by running or executing the instructions, programs, code sets or instruction sets stored in the memory 120, and calling the data stored in the memory 120. Various functions of the electronic device 100 and processing data. Optionally, the processor 110 may employ at least one of a digital signal processing (Digital Signal Processing, DSP), a Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA), and a Programmable Logic Array (Programmable Logic Array, PLA). implemented in hardware. The processor 110 may integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU), a modem, and the like. Among them, the CPU mainly handles the operating system, user interface and application programs, etc.; the GPU is used for rendering and drawing of the display content; the modem is used to handle wireless communication. It can be understood that, the above-mentioned modem may also not be integrated into the processor 110, and is implemented by a communication chip alone.

The memory 120 may include random access memory (Random Access Memory, RAM), or may include read-only memory (Read-Only Memory). Memory 120 may be used to store instructions, programs, codes, sets of codes, or sets of instructions. The memory 120 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.) , instructions for implementing the following method embodiments, and the like. The storage data area may also store data (such as phone book, audio and video data, chat record data) created by the electronic device 100 during use.

Please refer to FIG. 15 , which shows a structural block diagram of a computer-readable storage medium provided by an embodiment of the present application. The computer-readable medium 1500 stores program codes, and the program codes can be invoked by the processor to execute the methods described in the above method embodiments.

The computer-readable storage medium 1500 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 1500 includes a non-transitory computer-readable storage medium. Computer readable storage medium 1500 has storage space for program code 1510 to perform any of the method steps in the above-described methods. These program codes can be read from or written to one or more computer program products. Program code 1510 may be compressed, for example, in a suitable form.

In summary, the image processing method, device, electronic device and storage medium provided by this application are pre-trained with a shared network and multiple specific networks, each of which is used to determine the attributes corresponding to the image data. label, and the attribute labels determined by each of the specific networks are different from each other, then when the image data to be processed is acquired, the image data to be processed is input into multiple specific networks, and each specific network can identify the The attributes that can be identified by the specific network, so that multiple attribute labels corresponding to the image data to be processed can be identified by multiple specific networks respectively, which improves the identification of multiple attribute labels of the overall image data, so that it can be The attribute label corresponding to the image data is obtained, and then the attribute label corresponding to the image data is input to the sharing network, and the sharing network determines the image recognition result according to each attribute label and the correlation of each attribute label, and outputs the image recognition result. Therefore, multiple specific networks jointly analyze the image data and obtain multiple attribute labels, which can improve the speed of obtaining attribute labels, while the shared network can combine the correlation of each attribute label to obtain image recognition results, which improves the accuracy and accuracy of the recognition results. overall performance.

This application proposes a face multi-attribute recognition algorithm and system based on deep learning. The method divides 40 face attributes into 4 face attribute groups according to the picture positions corresponding to the attributes, and takes into account the display between the face attributes. Location correlation, which treats the problem of attribute classification for each attribute group as a subtask, builds a model with 4 specific networks and 1 shared network.

The specificity network aims to learn the specificity between tasks, so each attribute group is configured with a separate specificity network, while the shared network aims to learn the complementary information between tasks and facilitate the interaction between tasks. The rich connections between the specific network and the shared network promote the exchange of information between each other, which is conducive to mining the correlation between tasks and improves the overall performance.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand: it can still be Modifications are made to the technical solutions described in the foregoing embodiments, or some technical features thereof are equivalently replaced; and these modifications or replacements do not drive the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims (12)

1. An image processing method, comprising:

acquiring image data to be processed;

inputting the image data to be processed into a plurality of pre-trained specific networks to obtain attribute labels corresponding to the image data, wherein each specific network is used for determining the attribute labels corresponding to the image data, and the attribute labels determined by each specific network are different from each other;

inputting the attribute labels determined by each specific network into a pre-trained shared network to obtain an image recognition result, wherein the shared network is used for determining the image recognition result according to the attribute labels and the correlation of the attribute labels;

and outputting the image recognition result.

2. The method according to claim 1, wherein the inputting the image data to be processed into a plurality of pre-trained specific networks to obtain attribute labels corresponding to the image data comprises:

determining an attribute label corresponding to each specific network, wherein the attribute label which can be identified by the specific network is the attribute label corresponding to the specific network;

dividing the image data into a plurality of sub-image data according to the attribute label corresponding to each specific network;

and inputting the sub-image data into a specific network corresponding to the sub-image data.

3. The method of claim 1, wherein the acquiring image data to be processed comprises:

acquiring original image data;

and normalizing the original image data to obtain image data to be processed.

4. The method of claim 1, wherein before inputting the image data to be processed into a plurality of pre-trained specific networks to obtain the attribute labels corresponding to the image data, the method further comprises:

obtaining a plurality of sample image data, wherein each sample image data corresponds to a plurality of attribute labels;

setting a sharing network and a plurality of specific networks, wherein each specific network can identify at least one attribute label, and the attribute labels which can be identified by each specific network are different from each other;

and inputting the plurality of sample image data into the shared network and the plurality of specific networks for training so as to obtain the trained shared network and the plurality of specific networks.

5. The method according to claim 4, wherein the sample image data and the image data to be processed each contain a face image; the acquiring a plurality of sample image data includes:

acquiring a plurality of sample image data;

identifying position information of a face key point in each sample image data in the sample image;

adjusting the face orientation in each sample image data to accord with a preset orientation according to the position information of the face key point of each sample image data;

and taking the adjusted sample image data as the sample image data used for training the shared network and the plurality of initial specific networks.

6. The method of claim 4, wherein said acquiring a plurality of sample image data comprises:

acquiring a plurality of sample image data;

performing data enhancement processing on the plurality of sample image data to ensure that the illumination intensity and the contrast of each sample image data are randomly distributed in a preset interval;

clipping each sample image data after enhancement processing according to a preset random clipping proportion, wherein the size of each clipped sample image data is a preset size;

and taking the clipped sample image data as the sample image data used for training the shared network and the plurality of initial specific networks.

7. The method according to claim 4, wherein the sample image data and the image data to be processed each contain a face image, and the plurality of attribute labels corresponding to each sample image data correspond to different positions of a face; the setting shared network and a plurality of specific networks comprise:

dividing a plurality of measurement areas, wherein each measurement area corresponds to a different area of the human face;

and setting a plurality of specific networks according to the plurality of measurement areas, wherein each specific network corresponds to one measurement area and is used for confirming the attribute labels in the corresponding measurement area.

8. An image processing method, comprising:

obtaining a plurality of sample image data, wherein each sample image data corresponds to a plurality of attribute labels;

setting a sharing network and a plurality of specific networks, wherein each specific network can identify at least one attribute label, and the attribute labels which can be identified by each specific network are different from each other;

inputting the sample image data into the shared network and the specific networks for training to obtain a trained shared network and specific networks;

and acquiring image data to be processed, and processing the image data to be processed according to the trained shared network and the plurality of specific networks to obtain an image recognition result.

9. An image processing apparatus, characterized in that the apparatus comprises:

the data acquisition unit is used for acquiring image data to be processed;

the attribute determining unit is used for inputting the image data to be processed into a plurality of pre-trained specific networks to obtain attribute labels corresponding to the image data, wherein each specific network is used for determining the attribute labels corresponding to the image data, and the attribute labels determined by the specific networks are different from each other;

the result acquisition unit is used for inputting the attribute labels determined by each specific network into a pre-trained shared network to acquire an image recognition result, wherein the shared network is used for determining the image recognition result according to the attribute labels and the correlation of the attribute labels;

and the output unit is used for outputting the image recognition result.

10. An image processing apparatus, characterized in that the apparatus comprises:

the system comprises a sample acquisition unit, a data processing unit and a data processing unit, wherein the sample acquisition unit is used for acquiring a plurality of sample image data, and each sample image data corresponds to a plurality of attribute labels;

a setting unit, configured to set a shared network and a plurality of specific networks, where each specific network is capable of identifying at least one attribute tag, and the attribute tags that each specific network is capable of identifying are different from each other;

the network training unit is used for inputting the sample image data into the shared network and the specific networks for training so as to obtain the trained shared network and the specific networks;

and the identification unit is used for acquiring image data to be processed, and processing the image data to be processed according to the trained shared network and the plurality of specific networks to obtain an image identification result.

11. An electronic device, comprising:

one or more processors;

a memory;

one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to perform the method of any of claims 1-7.

12. A computer-readable storage medium storing program code executable by a processor, wherein a plurality of instructions in the program code, when executed by the processor, cause the processor to perform the method of any one of claims 1-7.

Images