CN111401196A - Method, computer device and computer readable storage medium for self-adaptive face clustering in limited space - Google Patents
Method, computer device and computer readable storage medium for self-adaptive face clustering in limited space Download PDFInfo
- Publication number
- CN111401196A CN111401196A CN202010162711.4A CN202010162711A CN111401196A CN 111401196 A CN111401196 A CN 111401196A CN 202010162711 A CN202010162711 A CN 202010162711A CN 111401196 A CN111401196 A CN 111401196A
- Authority
- CN
- China
- Prior art keywords
- node
- clustering
- face
- category
- current node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 62
- 238000004364 calculation method Methods 0.000 claims abstract description 109
- 238000001514 detection method Methods 0.000 claims abstract description 30
- 239000013598 vector Substances 0.000 claims abstract description 27
- 238000004590 computer program Methods 0.000 claims description 24
- 230000003044 adaptive effect Effects 0.000 claims description 21
- 238000004422 calculation algorithm Methods 0.000 description 18
- 210000000887 face Anatomy 0.000 description 11
- 238000000605 extraction Methods 0.000 description 7
- 238000012549 training Methods 0.000 description 6
- 238000013135 deep learning Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 238000012544 monitoring process Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000001815 facial effect Effects 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000001186 cumulative effect Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Biology (AREA)
- Oral & Maxillofacial Surgery (AREA)
- General Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a method for self-adaptive face clustering in a limited space, a computer device and a computer readable storage medium, wherein the method comprises the steps of carrying out face detection on an original image, acquiring face images and calculating a feature vector of each face image; calculating the similarity of the feature vectors of any two face images, constructing an undirected graph according to the similarity of the feature vectors of any two face images and a similarity threshold value, wherein each face image is used as a node in the undirected graph, and each node is assigned with a category; confirming the category of the current node according to the category of the neighbor node of each node, obtaining the result of primary clustering calculation, judging whether the result of the current clustering calculation meets the preset requirement, if not, adjusting the similarity threshold value to construct an undirected graph again, and performing clustering calculation again. The invention also provides a computer device and a computer readable storage medium for realizing the method. The invention can improve the efficiency and accuracy of clustering calculation.
Description
Technical Field
The invention relates to the technical field of image recognition, in particular to a method for self-adaptive face clustering in a limited space, a computer device for realizing the method and a computer readable storage medium.
Background
With the development of intelligent technology, the identification of a person by means of image identification has been widely applied in a plurality of fields, for example, the detection of a face in a digital image and the clustering of a plurality of detected faces is a wide demand, and besides being applied to general digital media management and mobile phone album management, the system can also be applied to a plurality of real scenes, such as an on-board video monitoring system, face capture, clustering, identification and analysis of drivers and passengers in a vehicle, and the like.
In the prior art, a method for performing face detection and clustering face images generally includes acquiring a plurality of original images, performing face detection on each original image, determining a face region in each original image, performing key point positioning and feature extraction on the positioned face region, and implementing clustering by using extracted face feature vectors.
Specific methods generally include the following two: the first method generally uses classical clustering algorithms such as K-means, mean-shift, etc. to cluster face images together, thereby clustering images belonging to the same person. However, such clustering algorithms often require a predefined number of classes and are sensitive to initial cluster centers and outliers, resulting in a less accurate result. In addition, the clustering algorithm usually assumes that the picture of the same person is closer to the central point, and due to the diversity of the human face gestures, such as being affected by factors such as illumination, skin color, occlusion, and human face gestures, the assumption often does not meet the requirements of practical application.
The second method is that after extracting the feature vector of the face image, a similarity matrix is constructed according to the extracted feature vector, connected graphs are searched in the similarity matrix, and the face corresponding to each connected graph is classified into one class. However, this method needs to traverse the whole graph formed by face nodes, has high computational complexity and large time consumption, and is generally difficult to output the result of face image clustering in a short time.
Disclosure of Invention
The invention mainly aims to provide a method for self-adaptive face clustering in a limited space, which is small in calculation amount and high in calculation accuracy.
Another object of the present invention is to provide a computer device for implementing the above method for adaptive face clustering in a restricted space.
It is still another object of the present invention to provide a computer readable storage medium for implementing the above method for adaptive face clustering in a restricted space.
In order to realize the main purpose of the invention, the method for self-adaptive face clustering in the limited space comprises the steps of obtaining at least one original image, carrying out face detection on the original image, obtaining face images in the original image, and calculating a feature vector of each face image; calculating the similarity of the feature vectors of any two face images, constructing an undirected graph according to the similarity of the feature vectors of any two face images and a similarity threshold value, wherein each face image is used as a node in the undirected graph, and each node is assigned with a category; confirming the category of the current node according to the category of the neighbor node of each node, obtaining a clustering calculation result after iterative calculation for preset times, judging whether the current clustering calculation result meets preset requirements, if not, adjusting a similarity threshold, building an undirected graph again, and performing clustering calculation again.
According to the scheme, the category of each node is determined by constructing the undirected graph and performing multiple iterative computations, and after the multiple iterative computations, the number of the categories of the nodes in the undirected graph is reduced, so that the multiple face images are divided into a limited number of multiple categories. Because the method carries out clustering through repeated iterative computation, the computation amount of each iterative computation is not complex, the computation amount of the clustering computation is less, and the efficiency of the clustering computation can be improved.
On the other hand, the final clustering calculation result is determined by multiple clustering calculations, and the similarity threshold is adaptively adjusted by using the result of the last clustering calculation during the multiple clustering calculations, for example, if the number of categories is too large as the result output by the last clustering calculation, the number of categories after the clustering calculation is reduced by improving the similarity threshold, and the clustering calculation requirement in the limited space can be well met.
Preferably, the confirming the class of the current node according to the class of the neighbor node of each node includes: and during initial iterative calculation, the category of any neighbor node of the current node is given to the current node.
Therefore, during initial iterative computation, because the classes of all the nodes are different, the iterative computation can be started most quickly by giving the class of any one neighbor node to the current node, so that the computation amount of the iterative computation is minimized, and the clustering computation is operable.
Further, the step of confirming the category of the current node according to the categories of the neighbor nodes of each node comprises: and during subsequent iterative calculation, calculating the category of the current node according to the categories of all the neighbor nodes of the current node.
Therefore, in the subsequent iterative computation process, all neighbor nodes are required to be used as references for computing the class of the current node, so that the conditions of all neighbor nodes of the current node can be fully considered, and the class identification accuracy of the current node is improved.
Further, the calculating the category of the current node according to the categories of all the neighbor nodes of the current node comprises: determining the weight value of a connecting line between the current node and each neighbor node, and calculating the category of the current node according to the weight value of the connecting line between the current node and the neighbor node and the category of the neighbor node.
Therefore, the class of the current node is calculated by using the weight value of the connecting line between the current node and the neighbor node and the class of the neighbor node, so that the class of the neighbor node with higher weight value can be ensured to have larger influence on the class determination of the current node, and the accuracy of the class calculation of the current node is improved.
Preferably, the determining the weight value of the connection line between the current node and the neighbor node includes: judging whether the similarity of the feature vector of the current node and the neighbor node is greater than or equal to a similarity threshold value, if so, determining the weight value of a connecting line between the current node and the neighbor node as a first weight value, otherwise, determining the weight value of the connecting line between the current node and the neighbor node as a second weight value; wherein the first weight value is greater than the second weight value.
Therefore, the category of the neighbor node has larger weight for the neighbor node with higher similarity, and the category of the neighbor node has smaller weight for the neighbor node with lower similarity, so that the category of the current node is kept consistent with the most similar neighbor node, and the accuracy of node category calculation is improved.
Further, the determining the weight value of the connection line between the current node and the neighboring node comprises: and taking the similarity of the feature vectors of the current node and the neighbor node as a weight value.
Therefore, the similarity of the feature vectors of the two nodes is directly used as the weight value, the calculation amount of the weight value calculation is simplified, the node type calculation time is favorably shortened, and the face image clustering calculation efficiency is improved.
Further, the step of determining whether the current clustering result meets a preset requirement includes: and judging whether the category quantity of all the nodes in the current clustering calculation result is in a preset range, if so, confirming that the preset requirement is met.
Because the category number range of the nodes can be preset in the limited space, whether the current clustering result meets the requirement or not can be determined only by judging whether the category number of the nodes in the current clustering calculation result is in the preset range or not, and the subsequent calculation can be rapidly performed.
The method comprises the following steps of acquiring a section of video, carrying out face detection on each frame of image of the video, and calculating the maximum number of faces appearing in each frame of image in the section of video; determining whether the result of the current clustering calculation meets a preset requirement includes: the number of the types of all the nodes is not less than the maximum number of the faces appearing in each frame of image in the video.
A further scheme is that when the method is applied to a limited space, the number of people that can be accommodated in the limited space has an upper limit, for example, when the method is applied to a vehicle-mounted video monitoring system, because the upper limit of the number of people in a vehicle compartment is determined, the upper limit of the number of categories obtained by clustering is also determined, if the number of categories obtained by clustering is greater than the upper limit value, it indicates that the clustering calculation clusters the faces of the same person into at least two categories, and at this time, the similarity threshold value needs to be further adjusted, and the clustering calculation is performed again, so that the accuracy of the clustering calculation result can be ensured.
In order to achieve the above another object, the present invention provides a computer device, which includes a processor and a memory, wherein the memory stores a computer program, and the computer program, when executed by the processor, implements the steps of the method for adaptive face clustering in a restricted space.
To achieve the above-mentioned further object, the present invention provides a computer-readable storage medium, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the steps of the above-mentioned method for adaptive face clustering in a restricted space.
Drawings
FIG. 1 is a flowchart of an embodiment of a method for adaptive face clustering in a restricted space according to the present invention.
FIG. 2 is a flowchart of acquiring a face image and calculating a feature vector of the face image in the embodiment of the method for adaptively clustering faces in a limited space of the present invention.
FIG. 3 is a flow chart of constructing an undirected graph in an embodiment of the method for adaptive face clustering in a restricted space of the present invention.
FIG. 4 is a flow chart of primary cluster calculation in the embodiment of the method for adaptive face clustering in a restricted space of the present invention.
FIG. 5 is a flowchart of adaptive face cluster calculation in an embodiment of the method for adaptive face clustering in a restricted space of the present invention.
The invention is further explained with reference to the drawings and the embodiments.
Detailed Description
The method for self-adaptive face clustering in the limited space is applied to intelligent equipment, preferably, the intelligent equipment is provided with a camera device such as a camera, and the intelligent equipment performs image analysis by using video data acquired by the camera device, for example, each frame of image in a video is acquired, the face in each frame of image is detected, and then the cluster calculation is performed on the face region image obtained by detection. The method mainly aims at face clustering calculation in a limited space environment, such as a carriage and other limited spaces. Preferably, the intelligent device is provided with a processor and a memory, the memory is stored with a computer program, and the processor implements the method for self-adaptive face clustering in the limited space by executing the computer program.
The embodiment of the method for self-adaptive face clustering in the limited space comprises the following steps:
in this embodiment, a computer program for face detection is used for implementation, that is, an algorithm for face detection needs to be used in the clustering process.
The embodiment includes that a section of video data is obtained, then face region detection is carried out on multiple frames of images in the video data, a face region in each frame of image is obtained, the number of the face regions in each frame of image is calculated, the most detected face type number in the section of video can be determined according to the frame of image with the most face region number, and the most detected face type number is the lower limit of the number of people in a carriage considering that the possibility of missing detection exists in the actual application of a face detection algorithm. Because the face images need to be clustered in the embodiment, and the purpose of clustering is to identify the face images of the same person, the number of the face image categories in the clustering identification result needs to be determined according to the number range of the identified persons in the compartment, and the number is used as a judgment standard for judging whether the result of one-time clustering calculation meets the preset requirement.
The operation of the present embodiment will be described in detail with reference to fig. 1 to 5. Referring to fig. 1, step S1 is first executed to acquire a plurality of original images. The acquired original image can be a multi-frame image in a section of video, and the multi-frame image can be a continuous multi-frame image or a discontinuous multi-frame image. Because the motion of the person is continuous, even if the acquired images are not continuous, the result of the clustering calculation is not substantially affected, and on the contrary, if continuous multi-frame images are acquired from a section of video, the number of the images acquired by the clustering calculation is too large, the calculation amount is too large, and the efficiency of the clustering calculation is affected, so that discontinuous multi-frame images can be acquired. For example, one frame of image is acquired at preset time intervals, or a video segment is divided into a plurality of time intervals, and one or more frames of images are acquired randomly from each time interval. For example, for a video with a time length of 20 seconds, the video may be divided into 20 segments, each segment has a time length of 1 second, and one or two frames of images are randomly acquired from each segment of video as original images.
Then, step S2 is executed to perform face detection on each original image, and acquire a plurality of face images. Specifically, face detection is performed on each original image, and each detected face is stored as a face image. In this embodiment, the method for detecting a face is to search the original image by using a certain policy to determine whether the current original image contains a face, and if so, return the position, size, and the like of the face. The specific algorithms for face detection may include conventional detection algorithms and detection algorithms based on deep learning, wherein the conventional algorithms include algorithms based on statistics, algorithms based on structural features, and the like. In recent years, with the great success of deep learning neural networks in pattern recognition, for example, in the fields of image classification, target detection, image segmentation, speech recognition, machine translation, and the like, the detection algorithm based on deep learning has far superior performance to the traditional shallow model, and has attracted much attention and application. For each original image, face detection can be performed by only adopting one face detection algorithm, in order to improve the detection effect, face detection can also be performed by adopting a mode of combining multiple algorithms, and the specific face detection algorithm can be determined according to actual needs.
Specifically, referring to fig. 2, firstly, step S11 is executed to obtain position information of key points of each portion included in one face image, and the face image is corrected according to the position information of at least one portion of the key points, namely, step S12 is executed, wherein one method is that firstly, an included angle between a central connecting line and a horizontal line of two eyes is calculated according to the key point information of left and right eyes in the face image, the face image is rotated to a horizontal position according to the included angle, namely, in the rotated face image, the central connecting line of the two eyes is parallel to the horizontal line.
Finally, step S14 is executed to extract facial feature vectors from the corrected and scaled facial images respectively by using the feature extraction model obtained by pre-training. In order to train to obtain the feature extraction model, in this embodiment, a plurality of training samples need to be obtained in advance, and the feature extraction model is obtained by training the plurality of training samples. The training to obtain the feature extraction model can be realized based on the existing algorithm, and is not described herein again. Preferably, the feature extraction model is obtained by applying the deep learning method-based training in the embodiment, and because the feature extraction model trained by the deep learning method is compared with the conventional model, the face features with higher recognition degree can be extracted, and the performance of face clustering calculation can be further improved, for example, the accuracy of a clustering result is improved.
Next, step S4 is executed to calculate the feature vector similarity of any two face images. The most common method is an angle cosine algorithm, the difference between two individuals is measured according to the cosine value of an angle between two vectors in a vector space, the closer the cosine value is to 1, the closer the angle is to 0 degree, and the higher the similarity between the two individuals is. For example, feature vectors of two face images are used as two vectors in a vector space, so that an included angle cosine value between the two vectors can be calculated. In step S4, the similarity between two face images is calculated.
Then, step S5 is executed to construct an undirected graph using a preset similarity threshold and the similarity between any two face images. An undirected graph is a graph in which connecting lines have no direction. The following describes a specific flow of constructing an undirected graph with reference to fig. 3.
First, step S21 is executed to correspond each face image participating in the clustering calculation to a node in the undirected graph. A connecting line can be arranged between the two nodes or not, and the connecting line between the nodes represents the similarity relation between the two nodes, namely the similarity relation between the two face images. Of course, the connection line between the two nodes does not need to be set in step S21.
Then, step S22 is executed to acquire an initial value or an adjustment value of the similarity threshold. If the current clustering calculation is the undirected graph constructed for the first time, namely the clustering calculation is performed for the first time, the used similarity threshold value is an initial value, and the initial value is a preset value. If the current clustering calculation is not the undirected graph constructed for the first time, namely the clustering calculation is not carried out for the first time, the used similarity threshold is an adjustment value, and the adjustment value is a value adjusted according to the result of the last clustering calculation.
Then, step S23 is executed to calculate the relationship between the similarity between any two nodes and the similarity threshold obtained in step S22, i.e., to determine whether the similarity between any two nodes is greater than or equal to the similarity threshold, i.e., step S24 is executed. If the similarity between the two nodes is greater than or equal to the similarity threshold, step S25 is performed to set a connection line between the two nodes, and if the similarity between the two nodes is less than the similarity threshold, step S26 is performed to set no connection line between the two nodes.
Then, step S27 is executed to determine whether all nodes in the undirected graph are traversed, if so, step S28 is executed to output the undirected graph, where a plurality of nodes in the undirected graph are connected to other nodes through connecting lines, and the similarity of any two nodes with the connecting lines is greater than the similarity threshold. If the judgment result in the step S27 is no, which indicates that there are nodes in the undirected graph that have not been subjected to the similarity judgment, the process returns to step S23, and a node that has not been subjected to the processing is obtained, and whether the node has a similarity larger than the similarity threshold is judged, and if yes, a connecting line is set between the two nodes.
It should be noted that the purpose of setting a connection line or no connection line between two nodes is to determine the weight set when each node is classified, in one embodiment, the weight value of the connection line between two nodes may be determined depending on whether there is a connection line between two nodes, in another embodiment, a connection line is set between any two nodes, but if the similarity between two nodes is less than the similarity threshold, the weight value of the connection line is set to 0. The method for setting the weight value of the connecting line between two nodes and the specific method applied to the node type calculation will be described in detail later.
And after constructing the undirected graph, performing cluster calculation on the constructed undirected graph, and obtaining a cluster calculation result. In practical applications, an expected result may not be obtained only by one clustering calculation, multiple clustering calculations are usually required, and the similarity threshold needs to be adjusted continuously when performing subsequent clustering calculations. Moreover, each clustering calculation is an iterative calculation process, that is, each clustering calculation needs to be performed through multiple iterative calculations, and the number of iterative calculations can be preset.
Specifically, step S6 is executed first, each node in the undirected graph is assigned with a category, and iterative computation is performed for a preset number of times, then step S7 is executed, whether the result of the current clustering computation meets the requirement is judged, if the result meets the requirement, step S8 is executed, the result of the current clustering computation is output, otherwise step S9 is executed, the similarity threshold is adjusted, step S5 is executed again, an undirected graph is reconstructed, clustering computation is performed again by using the newly constructed undirected graph, and whether the result of the clustering computation meets the preset requirement is judged again. Thus, after a plurality of clustering calculations, the expected result can be obtained.
The process of primary cluster calculation is described below with reference to fig. 4. First, step S301 is executed to determine whether the current iterative computation is the first iterative computation, and if so, step S302 is executed to assign each node a category. Therefore, in the initial state, each node in the undirected graph has one unique class, that is, in the initial state, each face image in the undirected graph belongs to one class, and the number of the classes is equal to that of the nodes in the undirected graph.
Then, step S303 is executed to acquire a node as the current node. In one case, the nodes are sequentially acquired as the current node according to a preset arrangement order. Preferably, one node is randomly acquired from all the nodes of which the category is not calculated yet as the current node, and the efficiency of clustering calculation can be improved by properly applying randomness.
Then, step S304 is executed to randomly acquire a neighbor node of the current node, and assign the category of the neighbor node to the current node. The neighbor node is a node having a connection line with the current node, and in one case, the connection line is set only when the similarity between the two nodes is greater than the similarity threshold, and then the neighbor node is a node having a higher similarity with the current node. In another case, a connecting line is provided between any two nodes, and if the similarity between two nodes is lower than the similarity threshold, the weight value of the connecting line between two nodes is lower, in which case, the neighbor node of the current node is any node other than the current node.
After a neighbor node of the current node is randomly acquired, the category of the neighbor node is given to the current node. For example, the category of the current node is X1, the category of the randomly selected neighbor node is X2, and the category of the current node is changed to X2 after the category of the neighbor node is given to the current node.
Then, step S305 is executed to determine whether all the nodes have been traversed, if not, it indicates that there are nodes that have not been subjected to the category calculation, then the method returns to step S303, and continues to acquire other nodes that have not been subjected to the category calculation as the current nodes to perform the category calculation.
If the determination result in step S305 is yes, which indicates that all nodes in the undirected graph have completed one category calculation, i.e., have completed one iteration calculation, step S310 is executed to determine whether the preset number of iterations is reached. Because the classes of all the nodes cannot be reduced to the preset number only by one iterative computation, the face clustering is performed by setting multiple iterative computations, and therefore, the number of iterative computations required to be performed by each clustering computation is preset, for example, 5 or 8 iterative computations are required to be performed by each clustering computation. Step S310 is to determine whether the current iterative computation reaches a preset number, and if not, the process returns to step S301, and then the subsequent iterative computation is performed.
After returning to step S301, it is determined whether the current iterative computation is the first iterative computation, if not, step S306 is executed to obtain a node as the current node, and the manner of obtaining the node is the same as that of step S303.
Then, step S307 is executed to obtain the categories of all the neighboring nodes of the current node, and step S308 is executed to determine the category of the current node according to the categories of all the neighboring nodes, specifically, the category of the current node is calculated according to the categories of all the neighboring nodes of the current node and the weight values of the connection lines between the current node and the neighboring nodes.
In one case, the connection line is set only when the similarity between two nodes is greater than or equal to the similarity threshold, and at this time, the category of the current node may be determined by calculating the cumulative sum of the similarity weights between the current node and its neighboring nodes, for example, directly using the similarity between the current node and a neighboring node as the weight value of the connection line between the two nodes. Assume that the current node a has 3 neighboring nodes, which are neighboring nodes B, C, D respectively, wherein the similarity between the current node a and the neighboring node B is 0.9, the category of the neighboring node B is X3, the similarity between the current node a and the neighboring node C is 0.8, the category of the neighboring node C is X4, the similarity between the current node a and the neighboring node D is 0.7, and the category of the neighboring node D is X3. After the accumulation calculation, the sum of the weight values of the current node a with the category X3 is 0.9+0.7 to 1.6, and the sum of the weight values of the current node a with the category X4 is 0.8, obviously, the sum of the weight values of the current node a with the category X3 is greater than the sum of the weight values of the category X4, and therefore, the category of the current node a is determined to be X3.
In another case, a connecting line is arranged between any two nodes, and if the similarity between two nodes is lower than the similarity threshold, the weight value of the connecting line between two nodes is lower, and at this time, the category calculation of the current node may be performed in the following manner: if the similarity between the current node and a certain neighbor node is greater than or equal to the similarity threshold, the weight value of the connecting line between the current node and the neighbor node is set to be a first weight value, for example, the first weight value is 1, and if the similarity between the current node and a certain neighbor node is less than the similarity threshold, the weight value of the connecting line between the current node and the neighbor node is set to be a second weight value, for example, the second weight value is 0. Therefore, the first weight value is larger than the second weight value, namely the similarity between the two nodes is higher, the weight value is higher, the similarity between the two nodes is lower, and the weight value is also lower, so that the higher weight value occupation ratio of the current node and the neighbor node with the higher similarity can be ensured when the category of the current node is calculated, and the accuracy of the category calculation of the current node is improved.
Assuming that the current node a has 5 neighbor nodes, wherein the weight values of the connection lines between 3 neighbor nodes and the current node a are 1, the categories of the 3 neighbor nodes are X5, and the weight values of the connection lines between the other 2 nodes and the current node a are 0, the category of the current node a is determined by the categories of the 3 neighbor nodes whose weight values of the connection lines with the current node a are 1, that is, the category of the current node a is set to be X5.
If the connecting lines between the neighbor nodes of two or more categories and the current node have the same weight value, one of the two categories is randomly selected to be endowed to the current node. For example, the current node a has 4 neighbor nodes, and the weight values of the connection lines between the neighbor nodes and the current node a are all 1, where the category of two neighbor nodes is X6, and the category of the other two neighbor nodes is X7, then one of the categories X6 and X7 is randomly selected to be assigned to the current node, that is, the category of the current node may be X6 or X7, and the probabilities are equal.
After step S308 is executed, step S309 is executed to determine whether all nodes are traversed, if not, it indicates that there are nodes that do not perform the category calculation, then the method returns to step S306, and continues to acquire other nodes that do not perform the category calculation as the current nodes to perform the category calculation.
If the judgment result in the step S309 is yes, it indicates that all the nodes in the undirected graph have completed one category calculation, that is, one iteration calculation, the step S310 is continuously executed to judge whether the preset iteration number is reached, if the preset iteration number is not reached, the step S301 is executed again to continuously execute the next iteration calculation, and if the preset iteration number is reached, the step S311 is executed to output the result of the iteration calculation, that is, the result of one clustering calculation.
It should be noted that, in some limited spaces, for example, the number of people inside a small vehicle is often limited, and therefore, the number of faces in a video picture captured by a camera is also limited, and therefore, the cluster calculation may use the number of people in a recorded video as a reference to determine the number of categories of the cluster, for example, in a section of video captured by the camera, after face detection is performed on multiple frames of images, the number of faces of one frame of image with the largest number of faces in the multiple frames of images is the smallest number of categories that can be used for reference in the face cluster calculation, which is a case of missing detection that may occur when face detection is performed.
Meanwhile, the maximum number of people capable of being accommodated in the limited space is limited, so that the maximum number of people capable of being accommodated can be used as the maximum category number in face clustering calculation. For example, for five cars, it may be set that the number of categories does not exceed 5 as a result of the clustering calculation. Therefore, the maximum number of faces appearing in the image and the maximum number of people capable of being accommodated in the limited space are recorded and updated, whether the number of categories output by face clustering calculation meets the preset requirement or not can be checked, if the number of categories does not meet the preset requirement, the similarity threshold value is adjusted in a self-adaptive mode, and clustering calculation is executed again until the clustering calculation result meets the preset requirement.
The process of performing multiple clustering calculations is described below in conjunction with fig. 5. First, step S41 is executed to obtain an initially set similarity threshold, and then step S42 is executed to apply the initially set similarity threshold and the detected face image to construct an undirected graph, where the specific construction method of the undirected graph is described in detail above. Then, step S43 is executed to perform a clustering calculation, i.e., the calculations of steps S31 to S37 are executed, and then step S44 is executed to determine whether the current clustering calculation result satisfies the preset requirement. In this embodiment, the condition that the result of the cluster calculation meets the preset requirement is that in the result of the cluster calculation, the number of the types of the nodes is within a preset range, that is, the number of faces of one frame of image with the largest number of faces detected in each frame of image of the video is not less than the number of persons accommodated in the limited space.
If the clustering result does not meet the preset requirement, step S46 is required to be executed to adjust the similarity threshold, specifically, if the clustering result is that the number of the categories obtained by clustering is large, the similarity threshold may be adjusted upward, for example, the similarity threshold is adjusted from 0.7 to 0.8, so as to reduce the number of neighbor nodes of each node, or the number of neighbor nodes whose similarity with the current node is greater than the similarity threshold is reduced, and the category number of the node is also reduced after the finite iteration calculation. If the clustering result is that the number of the categories obtained by clustering is less, the similarity threshold value can be adjusted downwards. And after the similarity threshold is adjusted, returning to the step of executing S42, constructing an undirected graph by using the adjusted similarity threshold, and performing next clustering calculation until the result of the clustering calculation meets the preset requirement. If the determination result in the step S44 is yes, indicating that the current clustering result has met the preset requirement, then the step S45 is executed to output the clustering result.
In this embodiment, an undirected graph-based adaptive clustering algorithm is adopted, and the categories of all neighboring nodes corresponding to each node are determined by constructing an undirected graph and iteratively searching the categories of all neighboring nodes in the undirected graph, so as to implement clustering. Meanwhile, based on the application scene information, the similarity threshold value can be adjusted in a self-adaptive mode, so that the clustering result is continuously optimized. Compared with the traditional clustering calculation, the clustering result of the embodiment has higher accuracy and more robust performance. In addition, the calculation amount of the embodiment is small, the calculation amount of the cluster calculation is small, and the face cluster calculation efficiency can be improved.
It should be noted that, in the above embodiment, a plurality of original images are acquired, but in practical application, only one original image may be acquired, and the face detection is performed on the acquired original image, and then the cluster calculation is performed.
The embodiment of the computer device comprises:
the computer device of this embodiment may be an intelligent device, such as a vehicle-mounted monitoring apparatus with image processing capability, and the computer device includes a processor, a memory, and a computer program stored in the memory and running on the processor, and when the processor executes the computer program, the processor implements the steps of the method for adaptive face clustering in a restricted space.
For example, a computer program may be partitioned into one or more modules that are stored in a memory and executed by a processor to implement the modules of the present invention. One or more of the modules may be a series of computer program instruction segments capable of performing certain functions, which are used to describe the execution of the computer program in the terminal device.
The Processor may be a Central Processing Unit (CPU), or may be other general-purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, a discrete hardware component, or the like. The general-purpose processor may be a microprocessor or the processor may be any conventional processor or the like, the processor being the control center of the terminal device and connecting the various parts of the entire terminal device using various interfaces and lines.
The memory may be used to store computer programs and/or modules, and the processor may implement various functions of the terminal device by running or executing the computer programs and/or modules stored in the memory and invoking data stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone, and the like. In addition, the memory may include high speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), at least one magnetic disk storage device, a Flash memory device, or other volatile solid state storage device.
A computer-readable storage medium:
the computer program stored in the computer device may be stored in a computer-readable storage medium if it is implemented in the form of a software functional unit and sold or used as a separate product. Based on such understanding, all or part of the processes in the method according to the above embodiments may also be implemented by a computer program, which may be stored in a computer readable storage medium, and when the computer program is executed by a processor, the computer program may implement the steps of the method for adaptive face clustering in a restricted space.
Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying computer program code, recording medium, U.S. disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution media, and the like. It should be noted that the computer readable medium may contain other components which may be suitably increased or decreased as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, in accordance with legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunications signals.
Finally, it should be emphasized that the present invention is not limited to the above-mentioned embodiments, such as the change of the initial value of the similarity threshold, or the change of the preset number of iterations, and such changes should also be included in the protection scope of the claims of the present invention.
Claims (10)
1. The method for self-adaptive face clustering in the limited space comprises the following steps:
acquiring at least one original image, carrying out face detection on the original image, acquiring face images in the original image, and calculating a feature vector of each face image;
the method is characterized in that:
calculating the similarity of the feature vectors of any two human face images, constructing an undirected graph according to the similarity of the feature vectors of any two human face images and a similarity threshold value, wherein each human face image is used as a node in the undirected graph, and each node is assigned with a category;
confirming the category of the current node according to the category of the neighbor node of each node, obtaining a clustering calculation result after iterative calculation for preset times, judging whether the current clustering calculation result meets preset requirements, if not, adjusting the similarity threshold value to construct an undirected graph again, and performing clustering calculation again.
2. The method for adaptive face clustering in a restricted space according to claim 1, wherein:
confirming the class of the current node according to the classes of the neighbor nodes of each of the nodes comprises: and during initial iterative calculation, the category of any neighbor node of the current node is given to the current node.
3. The method for adaptive face clustering in a restricted space according to claim 2, wherein:
confirming the class of the current node according to the classes of the neighbor nodes of each of the nodes comprises: and during subsequent iterative calculation, calculating the category of the current node according to the categories of all the neighbor nodes of the current node.
4. The method for adaptive face clustering in a restricted space according to claim 3, wherein:
calculating the class of the current node according to the classes of all neighbor nodes of the current node comprises: determining the weight value of a connecting line between the current node and each neighbor node, and calculating the category of the current node according to the weight value of the connecting line between the current node and the neighbor node and the category of the neighbor node.
5. The method for adaptive face clustering in a restricted space according to claim 4, wherein:
determining a weight value of a connecting line between a current node and a neighbor node comprises: judging whether the similarity of the feature vector of the current node and the feature vector of the neighbor node is greater than or equal to the similarity threshold value, if so, determining the weight value of the connecting line as a first weight value, otherwise, determining the weight value of the connecting line as a second weight value;
wherein the first weight value is greater than the second weight value.
6. The method of adaptive face clustering in a restricted space according to claim 4, wherein:
determining a weight value of a connecting line between a current node and a neighbor node comprises: and taking the similarity of the feature vectors of the current node and the neighbor node as the weight value.
7. The method for adaptive face clustering in the restricted space according to any one of claims 1 to 6, wherein:
judging whether the current clustering calculation result meets the preset requirements comprises the following steps: and judging whether the category quantity of all the nodes in the current clustering calculation result is in a preset range, if so, confirming that the preset requirement is met.
8. The method for adaptive face clustering in a restricted space according to claim 7, further comprising:
acquiring a video, carrying out face detection on each frame of image of the video, and calculating the number of faces of one frame of image with the largest number of faces contained in a plurality of frames of images in the video; determining whether the result of the current clustering calculation meets a preset requirement includes: the category number of all the nodes is not less than the face number of one frame of image with the largest face number in the multi-frame images in the video.
9. Computer arrangement, characterized in that it comprises a processor and a memory, said memory storing a computer program that, when being executed by the processor, carries out the steps of the method of adaptive face clustering in a restricted space according to any one of claims 1 to 8.
10. A computer-readable storage medium having stored thereon a computer program, characterized in that: the computer program, when being executed by a processor, carries out the steps of the method for adaptive face clustering in a restricted space according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010162711.4A CN111401196A (en) | 2020-03-10 | 2020-03-10 | Method, computer device and computer readable storage medium for self-adaptive face clustering in limited space |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010162711.4A CN111401196A (en) | 2020-03-10 | 2020-03-10 | Method, computer device and computer readable storage medium for self-adaptive face clustering in limited space |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111401196A true CN111401196A (en) | 2020-07-10 |
Family
ID=71428707
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010162711.4A Pending CN111401196A (en) | 2020-03-10 | 2020-03-10 | Method, computer device and computer readable storage medium for self-adaptive face clustering in limited space |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111401196A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112101238A (en) * | 2020-09-17 | 2020-12-18 | 浙江商汤科技开发有限公司 | Clustering method and device, electronic equipment and storage medium |
| CN112270361A (en) * | 2020-10-30 | 2021-01-26 | 重庆紫光华山智安科技有限公司 | Face data processing method, system, storage medium and equipment |
| CN112347842A (en) * | 2020-09-11 | 2021-02-09 | 博云视觉（北京）科技有限公司 | Off-line face clustering method based on association graph |
| CN112507315A (en) * | 2021-02-05 | 2021-03-16 | 红石阳光（北京）科技股份有限公司 | Personnel passing detection system based on intelligent brain |
| CN113849653A (en) * | 2021-10-14 | 2021-12-28 | 鼎富智能科技有限公司 | Text classification method and device |
| CN114385845A (en) * | 2021-12-14 | 2022-04-22 | 浙江飞图影像科技有限公司 | Image classification management method and system based on graph clustering |
| CN114495946A (en) * | 2021-12-31 | 2022-05-13 | 思必驰科技股份有限公司 | Voiceprint clustering method, electronic device and storage medium |
| WO2023240991A1 (en) * | 2022-06-14 | 2023-12-21 | 青岛云天励飞科技有限公司 | Clustering connection graph construction method and apparatus, device and readable storage medium |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105161093A (en) * | 2015-10-14 | 2015-12-16 | 科大讯飞股份有限公司 | Method and system for determining the number of speakers |
| CN107241277A (en) * | 2017-05-17 | 2017-10-10 | 天津大学 | Node method for annealing in SDN |
| CN107609466A (en) * | 2017-07-26 | 2018-01-19 | 百度在线网络技术（北京）有限公司 | Face cluster method, apparatus, equipment and storage medium |
| CN107633074A (en) * | 2017-09-22 | 2018-01-26 | 咪咕文化科技有限公司 | Information extraction method and device and storage medium |
| CN109063737A (en) * | 2018-07-03 | 2018-12-21 | Oppo广东移动通信有限公司 | Image processing method, device, storage medium and mobile terminal |
| CN109829337A (en) * | 2019-03-07 | 2019-05-31 | 广东工业大学 | A kind of method, system and the equipment of community network secret protection |
| CN109969891A (en) * | 2019-03-25 | 2019-07-05 | 浙江新再灵科技股份有限公司 | A kind of elevator passenger weight discriminance analysis system based on deep learning |
| CN110110593A (en) * | 2019-03-27 | 2019-08-09 | 广州杰赛科技股份有限公司 | Face Work attendance method, device, equipment and storage medium based on self study |
| CN110263633A (en) * | 2019-05-13 | 2019-09-20 | 广州烽火众智数字技术有限公司 | The personnel that are involved in drug traffic based on space time correlation detect method for early warning, system and storage medium |
| CN110458078A (en) * | 2019-08-05 | 2019-11-15 | 高新兴科技集团股份有限公司 | A kind of face image data clustering method, system and equipment |
| CN110738577A (en) * | 2019-09-06 | 2020-01-31 | 平安科技（深圳）有限公司 | Community discovery method, device, computer equipment and storage medium |
-
2020
- 2020-03-10 CN CN202010162711.4A patent/CN111401196A/en active Pending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105161093A (en) * | 2015-10-14 | 2015-12-16 | 科大讯飞股份有限公司 | Method and system for determining the number of speakers |
| CN107241277A (en) * | 2017-05-17 | 2017-10-10 | 天津大学 | Node method for annealing in SDN |
| CN107609466A (en) * | 2017-07-26 | 2018-01-19 | 百度在线网络技术（北京）有限公司 | Face cluster method, apparatus, equipment and storage medium |
| CN107633074A (en) * | 2017-09-22 | 2018-01-26 | 咪咕文化科技有限公司 | Information extraction method and device and storage medium |
| CN109063737A (en) * | 2018-07-03 | 2018-12-21 | Oppo广东移动通信有限公司 | Image processing method, device, storage medium and mobile terminal |
| CN109829337A (en) * | 2019-03-07 | 2019-05-31 | 广东工业大学 | A kind of method, system and the equipment of community network secret protection |
| CN109969891A (en) * | 2019-03-25 | 2019-07-05 | 浙江新再灵科技股份有限公司 | A kind of elevator passenger weight discriminance analysis system based on deep learning |
| CN110110593A (en) * | 2019-03-27 | 2019-08-09 | 广州杰赛科技股份有限公司 | Face Work attendance method, device, equipment and storage medium based on self study |
| CN110263633A (en) * | 2019-05-13 | 2019-09-20 | 广州烽火众智数字技术有限公司 | The personnel that are involved in drug traffic based on space time correlation detect method for early warning, system and storage medium |
| CN110458078A (en) * | 2019-08-05 | 2019-11-15 | 高新兴科技集团股份有限公司 | A kind of face image data clustering method, system and equipment |
| CN110738577A (en) * | 2019-09-06 | 2020-01-31 | 平安科技（深圳）有限公司 | Community discovery method, device, computer equipment and storage medium |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112347842A (en) * | 2020-09-11 | 2021-02-09 | 博云视觉（北京）科技有限公司 | Off-line face clustering method based on association graph |
| CN112347842B (en) * | 2020-09-11 | 2024-05-24 | 博云视觉（北京）科技有限公司 | Offline face clustering method based on association graph |
| JP2022552034A (en) * | 2020-09-17 | 2022-12-15 | チョーチアン センスタイム テクノロジー デベロップメント カンパニー，リミテッド | CLUSTERING METHOD AND DEVICE, ELECTRONIC DEVICE AND STORAGE MEDIUM |
| CN112101238A (en) * | 2020-09-17 | 2020-12-18 | 浙江商汤科技开发有限公司 | Clustering method and device, electronic equipment and storage medium |
| WO2022057302A1 (en) * | 2020-09-17 | 2022-03-24 | 浙江商汤科技开发有限公司 | Clustering method and apparatus, electronic device, and storage medium |
| CN112270361A (en) * | 2020-10-30 | 2021-01-26 | 重庆紫光华山智安科技有限公司 | Face data processing method, system, storage medium and equipment |
| CN112270361B (en) * | 2020-10-30 | 2021-10-22 | 重庆紫光华山智安科技有限公司 | Face data processing method, system, storage medium and equipment |
| CN112507315A (en) * | 2021-02-05 | 2021-03-16 | 红石阳光（北京）科技股份有限公司 | Personnel passing detection system based on intelligent brain |
| CN112507315B (en) * | 2021-02-05 | 2021-06-18 | 红石阳光（北京）科技股份有限公司 | Personnel passing detection system based on intelligent brain |
| CN113849653A (en) * | 2021-10-14 | 2021-12-28 | 鼎富智能科技有限公司 | Text classification method and device |
| CN113849653B (en) * | 2021-10-14 | 2023-04-07 | 鼎富智能科技有限公司 | Text classification method and device |
| CN114385845A (en) * | 2021-12-14 | 2022-04-22 | 浙江飞图影像科技有限公司 | Image classification management method and system based on graph clustering |
| CN114495946A (en) * | 2021-12-31 | 2022-05-13 | 思必驰科技股份有限公司 | Voiceprint clustering method, electronic device and storage medium |
| WO2023240991A1 (en) * | 2022-06-14 | 2023-12-21 | 青岛云天励飞科技有限公司 | Clustering connection graph construction method and apparatus, device and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111401196A (en) | Method, computer device and computer readable storage medium for self-adaptive face clustering in limited space | |
| CN110807385B (en) | Target detection method, target detection device, electronic equipment and storage medium | |
| CN112446270B (en) | Person re-identification network training method, person re-identification method and device | |
| CN107529650B (en) | Closed loop detection method and device and computer equipment | |
| US10402627B2 (en) | Method and apparatus for determining identity identifier of face in face image, and terminal | |
| EP3910507B1 (en) | Method and apparatus for waking up screen | |
| CN103971386B (en) | A kind of foreground detection method under dynamic background scene | |
| US20170213080A1 (en) | Methods and systems for automatically and accurately detecting human bodies in videos and/or images | |
| CN110765860A (en) | Tumble determination method, tumble determination device, computer apparatus, and storage medium | |
| CN107944381B (en) | Face tracking method, face tracking device, terminal and storage medium | |
| CN111860147A (en) | Pedestrian re-identification model optimization processing method and device and computer equipment | |
| CN112633159B (en) | Human-object interaction relation identification method, model training method and corresponding device | |
| KR20220076398A (en) | Object recognition processing apparatus and method for ar device | |
| CN109033955A (en) | A kind of face tracking method and system | |
| CN111709377B (en) | Feature extraction method, target re-identification method and device and electronic equipment | |
| EP4332910A1 (en) | Behavior detection method, electronic device, and computer readable storage medium | |
| CN111914668A (en) | Pedestrian re-identification method, device and system based on image enhancement technology | |
| CN111444817A (en) | A person image recognition method, device, electronic device and storage medium | |
| CN112215840B (en) | Image detection and driving control method and device, electronic equipment and storage medium | |
| CN113221922A (en) | Image processing method and related device | |
| CN112257628A (en) | A kind of identification method, device and equipment for outdoor competition athletes | |
| WO2024244416A1 (en) | Video coding motion estimation method and apparatus based on multi-target tracking | |
| CN112819859B (en) | Multi-target tracking method and device applied to intelligent security | |
| CN116612521A (en) | A face recognition method, device, chip and terminal | |
| CN114140822A (en) | Pedestrian re-identification method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200710 |