CN117079063A - Feature extraction model processing, sample retrieval method and device and computer equipment - Google Patents

Abstract

The present application relates to a feature extraction model processing, sample retrieval method, device and computer equipment. The method includes: performing feature extraction on the training sample through a feature extraction model to be trained to obtain a training feature vector; classifying and predicting the training sample based on the training feature vector to obtain a predicted semantic vector; and obtaining a label semantic vector of a training category label corresponding to the training sample. , the label semantic vector includes at least two activation label vector components; based on the label semantic vector, determine the expected semantic vector of the training sample, and the expected semantic vector contains the expected activation vector component corresponding to the position distribution of at least two activation label vector components; Based on the difference between each expected activation vector component in the expected semantic vector and the predicted vector component at the corresponding position in the predicted semantic vector, the feature extraction model is trained to obtain the target feature extraction model. Using this method can improve the accuracy of feature extraction.

Description

Feature extraction model processing, sample retrieval method, device and computer equipment

Technical field

The present application relates to the field of computer technology, and in particular to a feature extraction model processing method, device, computer equipment, storage medium and computer program product, and a sample retrieval method, device, computer equipment, storage medium and computer program product.

Background technique

With the development of computer technology, machine learning technology has emerged. Machine learning can be used to train various machine learning models. For example, a feature extraction model for feature extraction can be trained. The feature extraction model can extract the features of the input sample. , the input sample can be identified based on this feature, and the identification result can be obtained. To give a practical example, feature extraction can be performed on a sentence to obtain a feature vector representing the sentence, and the sentence can be translated based on the feature vector.

In traditional technology, the feature extraction model can be trained through training samples. However, there are often situations where the accuracy of features extracted by the trained feature extraction model is relatively low.

Contents of the invention

Based on this, it is necessary to address the above technical problems and provide a feature extraction model processing method, device, computer equipment, computer readable storage medium and computer program product that can improve the accuracy of feature extraction.

On the one hand, this application provides a feature extraction model processing method. The method includes: performing feature extraction on the training sample through a feature extraction model to be trained to obtain a training feature vector; classifying and predicting the training sample based on the training feature vector to obtain a predicted semantic vector; and obtaining the corresponding training sample The label semantic vector of the training category label, the label semantic vector includes at least two activation label vector components; based on the label semantic vector, determine the expected semantic vector of the training sample, the expected semantic vector includes the The expected activation vector component corresponding to the position distribution of the at least two activation label vector components; based on the relationship between each of the expected activation vector components in the expected semantic vector and the predicted vector component of the corresponding position in the predicted semantic vector The difference is determined to determine the classification loss; the feature extraction model is trained based on the classification loss, and when the training stop condition is met, the target feature extraction model is obtained; the target feature extraction model is used to extract the sample feature vector of the input sample.

On the other hand, this application also provides a feature extraction model processing device. The device includes: a feature extraction module, used to perform feature extraction on training samples through a feature extraction model to be trained, to obtain a training feature vector; a classification prediction module, used to perform classification prediction on the training sample based on the training feature vector , obtain the predicted semantic vector; the label vector acquisition module is used to obtain the label semantic vector of the training category label corresponding to the training sample, and the label semantic vector includes at least two activation label vector components; the expectation vector determination module is used to determine based on The label semantic vector determines the expected semantic vector of the training sample, and the expected semantic vector includes the expected activation vector component corresponding to the position distribution of the at least two activation label vector components; the classification loss determination module uses Determine the classification loss based on the difference between each of the expected activation vector components in the expected semantic vector and the predicted vector components at respective corresponding positions in the predicted semantic vector; a model training module for determining based on the classification loss The feature extraction model is trained, and when the training stop condition is met, a target feature extraction model is obtained; the target feature extraction model is used to extract the sample feature vector of the input sample.

On the other hand, this application also provides a computer device. The computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps of the feature extraction model processing method are implemented.

On the other hand, this application also provides a computer-readable storage medium. The computer-readable storage medium has a computer program stored thereon, and when the computer program is executed by a processor, the steps of the feature extraction model processing method are implemented.

On the other hand, this application also provides a computer program product. The computer program product includes a computer program that implements the steps of the feature extraction model processing method when executed by a processor.

The above feature extraction model processing method, device, computer equipment, storage medium and computer program product extracts features from training samples through the feature extraction model to be trained to obtain training feature vectors, and classifies and predicts training samples based on the training feature vectors to obtain Predict the semantic vector, obtain the label semantic vector of the training category label corresponding to the training sample, determine the expected semantic vector of the training sample based on the label semantic vector, and then determine the corresponding position in the predicted semantic vector based on each expected activation vector component in the expected semantic vector. The difference between the predicted vector components determines the classification loss. The feature extraction model is trained through this classification loss. The final target feature extraction model can learn features that tend to the label semantic vector, so that it can be used in the vector space where the training feature vector is located. The semantic space where the label semantic vector is located is simulated, so that the extracted feature vector can store feature information representing the semantics, which improves the accuracy of feature extraction, and because the label semantic vector includes at least two activation label vector components, it can The training category labels are more accurately characterized, and the expected semantic vector is determined based on the label semantic vector and contains the expected activation vector components corresponding to the position distribution of at least two activation label vector components, which can make the calculated classification loss more accurate. , further improving the accuracy of feature extraction.

On the other hand, this application provides a sample retrieval method. The method includes: obtaining a query sample and a candidate recall sample set; respectively inputting the query sample and the candidate recall sample in the candidate recall sample set into a target feature extraction model to obtain the query feature vector and candidate recall sample corresponding to the query sample. The corresponding candidate recall feature vector; wherein, the target feature extraction model is obtained by training the feature extraction model to be trained through classification loss, and the classification loss is based on each of the expected activation vector components in the expected semantic vector, The predicted semantic vector is determined by the difference between the predicted vector components corresponding to the corresponding positions in the predicted semantic vector. The predicted semantic vector is obtained by classifying and predicting the training sample based on the training feature vector. The expected semantic vector is based on the training The label semantic vector of the training category label corresponding to the sample is determined. The label semantic vector includes at least two activation label vector components. The expected semantic vector includes the position distribution corresponding to the at least two activation label vector components. Desired activation vector component, the training feature vector is obtained by feature extraction of the training sample through the feature extraction model to be trained; based on the query feature vector and the candidate recall feature vector, from the candidate recall The target retrieval sample corresponding to the query sample is determined in the sample set.

On the other hand, this application also provides a sample retrieval device. The device includes: a sample acquisition module, used to obtain a query sample and a candidate recall sample set; a feature extraction module, used to input the query sample and the candidate recall sample in the candidate recall sample set into a target feature extraction model to obtain the desired feature extraction model. The query feature vector corresponding to the query sample and the candidate recall feature vector corresponding to the candidate recall sample; wherein, the target feature extraction model is obtained by training the feature extraction model to be trained through classification loss, and the classification loss is based on the The difference between each expected activation vector component in the expected semantic vector and the predicted vector component at the corresponding position in the predicted semantic vector is determined, and the predicted semantic vector is used to classify the training sample based on the training feature vector. Predicted, the expected semantic vector is determined based on the label semantic vector of the training category label corresponding to the training sample, the label semantic vector includes at least two activation label vector components, and the expected semantic vector includes the The expected activation vector component corresponding to the position distribution of at least two activation label vector components, the training feature vector is obtained by feature extraction of the training sample through the feature extraction model to be trained; a retrieval module, used to perform feature extraction based on the The query feature vector and the candidate recall feature vector are used to determine the target retrieval sample corresponding to the query sample from the candidate recall sample set.

On the other hand, this application also provides a computer device. The computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps of the above sample retrieval method are implemented.

On the other hand, this application also provides a computer-readable storage medium. The computer-readable storage medium has a computer program stored thereon, and when the computer program is executed by a processor, the steps of the above sample retrieval method are implemented.

On the other hand, this application also provides a computer program product. The computer program product includes a computer program that implements the steps of the sample retrieval method when executed by a processor.

In the above sample retrieval methods, devices, computer equipment, storage media and computer program products, since the target feature extraction model is trained through the classification loss, the final target feature extraction model can learn features that tend to the label semantic vector, so that it can The semantic space where the label semantic vector is located is simulated in the vector space where the training feature vector is located, so that feature information representing semantics can be stored in the extracted feature vector, which improves the accuracy of feature extraction, and because the label semantic vector includes at least two The activation label vector component can more accurately characterize the training category label, and the expected semantic vector is determined based on the label semantic vector, and contains the expected activation vector component corresponding to the position distribution of at least two activation label vector components, so that The calculated classification loss is more accurate, further improving the accuracy of feature extraction. Using the feature vector extracted by the target feature extraction model for sample retrieval can improve the accuracy of sample retrieval.

Description of the drawings

Figure 1 is an application environment diagram of the feature extraction model processing method and sample retrieval method in one embodiment;

Figure 2 is a schematic flowchart of a feature extraction model processing method in one embodiment;

Figure 3 is a mapping relationship diagram between training feature vectors and expected semantic vectors in one embodiment;

Figure 4 is a comparison diagram of feature learning effects in one embodiment;

Figure 5 is a schematic diagram of the training process of the feature extraction model in one embodiment;

Figure 6 is a schematic flow chart of a sample retrieval method in one embodiment;

Figure 7 is a schematic diagram of the specific process of retrieval through indexing in one embodiment;

Figure 8 is a structural block diagram of a feature extraction model processing device in one embodiment;

Figure 9 is a structural block diagram of a sample retrieval device in one embodiment;

Figure 10 is an internal structure diagram of a computer device in one embodiment;

Figure 11 is an internal structure diagram of a computer device in another embodiment.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present application more clear, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application and are not used to limit the present application.

Artificial Intelligence (AI) is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is the study of the design principles and implementation methods of various intelligent machines, so that the machines have the functions of perception, reasoning and decision-making.

Artificial intelligence technology is a comprehensive subject that covers a wide range of fields, including both hardware-level technology and software-level technology. Basic artificial intelligence technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation/interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, machine learning/deep learning, autonomous driving, smart transportation and other major directions.

Computer vision technology (Computer Vision, CV) is a science that studies how to make machines "see". Furthermore, it refers to using cameras and computers instead of human eyes to perform machine vision such as target identification and measurement, and further to do graphics. Processing, so that computer processing becomes an image more suitable for human eye observation or transmitted to instrument detection. As a scientific discipline, computer vision studies related theories and technologies, trying to build artificial intelligence systems that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and map construction, Technologies such as autonomous driving and smart transportation also include common biometric recognition technologies such as facial recognition and fingerprint recognition.

Machine Learning (ML) is a multi-field interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. Specializes in studying how computer equipment can simulate or implement human learning behavior to acquire new knowledge or skills, and reorganize existing knowledge structures to continuously improve its performance. Machine learning is the core of artificial intelligence and the fundamental way to make computer equipment intelligent. Its applications cover all fields of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, teaching learning and other technologies.

The solutions provided by the embodiments of this application involve artificial intelligence, computer vision, machine learning and other technologies, which are specifically explained through the following examples:

The feature extraction model processing method and sample retrieval method provided by this application can be applied in the application environment as shown in Figure 1. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store data that the server 104 needs to process, such as training sample data. The data storage system can be integrated on the server 104, or placed on the cloud or other network servers. The terminal 102 may be, but is not limited to, a laptop computer, a smartphone, a tablet computer, a desktop computer, a smart TV, a vehicle-mounted terminal, and a portable wearable device. The terminal can be provided with an application program, through which the input sample can be retrieved and the target retrieval sample can be obtained. The application program can refer to a client installed in the terminal, and the client (also known as an application client, APP Client) refers to a program installed and run in the terminal; an application can also refer to an installation-free application, that is, an application that can be used without downloading and installing. This type of application is also commonly known as a small program, which is usually used as a sub-program. The program runs in the client; the application can also refer to a web application opened through a browser; etc. The server 104 can be implemented as an independent server or a server cluster or cloud server composed of multiple servers. It can be understood that the embodiments of the present application do not limit the number of terminals and servers. There can be one or more terminals and one or more servers. The specific settings can be set as needed. Where multiple refers to at least two.

It can be understood that the feature extraction model processing method and sample retrieval method provided by the embodiment of the present application can be executed by the terminal 102, the server 104, or the terminal 102 and the server 104. For example, the server 104 can obtain the target feature extraction model through the feature extraction model processing method provided in the embodiment of the present application, and send the target feature extraction model to the terminal 102. The terminal 102 can implement sample retrieval based on the target feature extraction model.

In one embodiment, as shown in Figure 2, a feature extraction model processing method is provided, which is executed by a computer device. The computer device can be the terminal 102 in Figure 1, or the server 104 in Figure 1, or It may be a system composed of the terminal 102 and the server 104. Specifically, the feature extraction model processing method includes the following steps:

Step 202: Extract features from the training samples through the feature extraction model to be trained to obtain training feature vectors.

Among them, training samples refer to content samples used to train the feature extraction model, and the content can be any one of text, audio, or images. There is a corresponding training category label for the training sample, and the training category label corresponding to the training sample is used to identify the category of the training sample. For example, if the training samples are images, the training category labels corresponding to the training samples can be animal species such as dogs, cats, and fish, plant species such as corals, pine trees, osmanthus, or object types such as magnifying glasses, cabinets, and water bottles. The category labels corresponding to the training samples can be one or more, and multiple refers to at least two. When a training sample has multiple category labels, it means that the training sample can be classified into multiple different categories. Taking the training sample as a picture as an example, if the training sample contains both cats and dogs, then the training sample can be classified into The cat category can also be classified into the dog category.

The feature extraction model to be trained refers to the feature extraction model that requires parameter adjustment. The feature extraction model is a machine learning model used to extract features and output feature vectors. The feature extraction model at least includes an embedding model. The embedding model is used to output feature vectors. The feature vector output by the embedding model can be called an embedding vector. That is, the embedding represents the vector. The embedding model can be a model composed of one or more fully connected layers (fullconnection).

In one embodiment, the embedding model can normalize the output feature vector so that each feature component of the feature vector ranges from -1 to 1. In other embodiments, the embedding model can also perform binary quantization on the feature vector to obtain binary quantized features. Among them, binary quantization refers to the process of binary encoding of features. For example, the features can be encoded into binary codes with values 0 and 1. Bit compression can also be performed during the encoding process. For example, the features can be Vectors are compressed to 48 bits. The features obtained by binary quantization of the feature vector can be called hash features. At this time, the embedding model can be called a hash quantization model.

In one embodiment, the feature extraction model may only include an embedding model. The input end of the embedding model may be connected to the output end of the trained basic neural network model and receive the output of the basic neural network model as input. The basic neural network model may be a model used to extract feature information contained in the content. The basic neural network model may be a neural network based on artificial intelligence, for example, it may be a convolutional neural network (Convolutional Neural Networks, CNN), or it may be Networks such as ResNet101 (deep residual network 101) or ResNet18 (deep residual network 18). The convolutional neural network is a type of feedforward neural network (Feedforward Neural Networks) that includes convolutional calculations and has a deep structure. Specifically, after acquiring the training sample, the computer device inputs the training sample into the trained basic neural network model, extracts feature information through the trained basic neural network model, and inputs the extracted feature information into the embedding model to be trained, Get the training feature vector. For example, taking the training sample as an image, the image can be input into the trained CNN model to extract the image features, and the extracted image features can be input into the embedding model to obtain the training feature vector of the image.

In other embodiments, the feature extraction model may include a basic neural network model and an embedding model, in which case the basic neural network model and the embedding model are trained together as a feature extraction model. Specifically, after acquiring the training sample, the computer device inputs the training sample into the basic neural network model to be trained, extracts feature information through the basic neural network model to be trained, and inputs the extracted feature information into the embedding model to be trained, Get the training feature vector.

It is understandable that different feature extraction models can be trained for different types of samples. For example, if the sample is an image type sample, then a feature extraction model for extracting image features can be trained. If the sample is a speech type sample, then Feature extraction models for extracting speech features can be trained.

In one embodiment, the computer device locally stores a set of annotated training samples. The computer device can obtain training samples from the locally stored training sample set, input the training samples into a feature extraction model to be trained, and output training through the feature extraction model. The training feature vector corresponding to the sample. It is understandable that for the same training sample, different training feature vectors can be obtained under different feature extraction models. For example, the training feature vector can be a D-dimensional feature vector directly output by the embedding model, or a feature obtained by normalizing the feature vector, or it can also be a hash feature.

In one embodiment, the training samples in the training sample set can be labeled by searching from an Internet search engine. Taking the training sample as an image as an example, specifically, for a certain label, you can enter it in the search engine For this label, take the first N images returned as images with this label. For example, if you search for golden retriever on a certain search engine, save the first 500 images returned as image samples with the "golden retriever" label in the training sample set.

Step 204: Classify and predict the training samples based on the training feature vectors to obtain predicted semantic vectors.

Among them, classification prediction refers to predicting the category to which the training sample belongs based on the training feature vector to determine the specific category to which the training sample belongs. The predicted semantic vector refers to the probability vector of the classification prediction output. This probability vector can represent the category information of the training sample and is a probability vector containing semantic information.

Specifically, the computer device can perform classification prediction on the training sample based on the training feature vector to obtain the predicted semantic vector.

In one embodiment, the computer device can use a classification model to classify the training feature vector, that is, input the training feature vector into the classification model, and output the predicted semantic vector through the classification model. Among them, the classification model refers to a machine learning model that can perform category recognition. In some embodiments, the classification model may include one or more fully connected layers. When the classification model includes multiple fully connected layers, the first few fully connected layers can extract semantic features from the training feature vector, so it is called semantic Layer Semantic_layer, the last fully connected layer is used for classification, called the classification layer. The number of semantic layers can be determined as needed, and the semantic layers can be deepened if more feature intersections are required. By setting up a semantic layer, high-order features can be abstracted to more fully mine classification information.

Step 206: Obtain the label semantic vector of the training category label corresponding to the training sample. The label semantic vector includes at least two activation label vector components.

Among them, the label semantic vector is used to characterize the semantics of the training category label. The label vector component refers to the vector component contained in the label semantic vector. For example, assuming that the label semantic vector is (1, 1, 0, 0), then 1, 1, 0, 0 are the label vector components. The activated label vector component refers to the activated label vector component in the label semantic vector, that is, the label vector component is the activation value. For example, 1 can be used to represent activation and 0 can be used to represent inactivity, then the label semantic vector is (1, 1, 0, 0), which includes two activation label vector components. In the embodiment of the present application, for each training category label, a label semantic vector including at least two activation label vector components is used to express its semantic information, which can be more specific and accurate than one-hot encoding (one-bot). express semantic information.

Specifically, the training samples belong to the training sample set, and the training category labels of each training sample in the training sample set form a label set. A label semantic vector is generated for each training category label in the label set in advance, and a relationship between the label semantic vector and the training category label is established. Therefore, when training, the computer device can obtain the label semantic vector of the training category label from the pre-established correlation between the training category label and the label semantic vector.

In one embodiment, each training category label in the label set can be element-split, each training category label can be split into multiple elements, and co-occurring elements between each training category label can be extracted, such as forest and park. There are grass elements. The vector dimension of the label semantic vector is determined based on the number of all extracted elements. Each element corresponds to a feature bit. The feature bit refers to the sorting position of the feature component, and then the label of each training category can be generated. Label semantic vector. For example, assume that the label set includes three training category labels, where training category label 1 includes three elements (a, b, c), training category label 2 includes three elements (a, d, e), and training category label 3 includes three elements (c, e), then it can be determined that the label semantic vector of training category label 1 is (1, 1, 1, 0, 0), and the label semantic vector of training category label 2 is (1, 0, 0) , 1, 1), the label semantic vector of training category label 3 is (0, 0, 1, 0, 1).

In another embodiment, the computer device can also generate an initial semantic vector for each training list label in the label set through the trained word2vec, and compare each feature component in the initial semantic vector with the semantic activation threshold. If a feature component is greater than If the activation threshold is set, the feature component is determined to be the activated feature component, and the value of the feature bit is set to the activation value. The activation value can be, for example, 1. If a feature component is less than the activation threshold, the feature component is determined to be inactive. feature component, and set the value of the feature bit to an inactive value. The inactive value can be, for example, 0, and finally the label semantic vector is obtained.

Step 208: Determine the expected semantic vector of the training sample based on the label semantic vector. The expected semantic vector includes the expected activation vector component corresponding to the position distribution of at least two activation label vector components.

Among them, the expected semantic vector refers to the semantic vector expected to be output when classifying and predicting training samples. This semantic vector can be used as supervision information in the training process, so that the feature extraction model can learn semantic information during the training process. The expected activation vector component refers to the activated expected vector component. The expected vector component here refers to the vector component contained in the expected semantic vector. For example, assuming that the expected semantic vector is (1, 0, 0, 1), then Among them, 1, 1, 0, 0 are the expected vector components. The expected semantic vector contains the expected activation vector component corresponding to the position distribution of at least two activation label vector components, that is, if the label semantic vector has an activation label vector component on a certain feature bit, then the corresponding position in the expected semantic vector There is a desired activation vector component at . For example, assuming that the label semantic vector is (1, 1, 0, 0), the label semantic vector has activation vector component 1 in both the first feature bit and the second feature bit, then in the expected semantic vector, the One flag and the second flag must be 1.

Specifically, the training category label corresponding to the training sample is used to characterize the category to which the training sample belongs. The label semantic vector of the training category label can express the semantic information of the training sample. Therefore, based on the label semantic vector, the computer device can determine the training sample. The expected semantic vector, because the expected semantic vector contains expected activation vector components corresponding to the position distribution of at least two activation label vector components, can therefore contain semantic information expressed by the label semantic vector.

In one embodiment, when a training sample corresponds to a training category label, the semantic information represented by the training category label is the semantic information contained in the training sample, so the computer device can use the label semantic vector as the expected semantics of the training sample. vector.

In another embodiment, when the training sample corresponds to multiple training category labels, it is expected that the semantic vector needs to be able to express the semantic information of each training category label. At this time, the computer device can fuse the semantic vectors of each label to obtain the semantic vector of each training category. The semantic information of the training category labels is jointly expressed as an expected semantic vector.

In one embodiment, when fusing each tag semantic vector, the computer device ensures that the dimension of the expected semantic vector is consistent with the dimension of the tag semantic vector, so as to avoid the dimension of the expected semantic vector being too large due to too many tags. .

Step 210: Determine the classification loss based on the difference between each expected activation vector component in the expected semantic vector and the predicted vector component at the corresponding position in the predicted semantic vector.

The position correspondence between the expected activation vector component and the predicted vector component refers to the sorting position of the expected activation vector component in the expected semantic vector, which is the same as the sorted position of the predicted vector component in the predicted semantic vector. For example, assuming that the expected semantic vector is (a1, a2, a3, a4) and the predicted semantic vector is (b1, b2, b3, b4), then a1 corresponds to the b1 position, a2 corresponds to the b2 position, and a3 corresponds to the b3 position. , a4 corresponds to the position of b4.

Specifically, since the expected semantic vector contains expected activation vector components corresponding to the position distribution of at least two activation label vector components, therefore, the expected semantic vector contains at least two expected activation vector components, for each expected activation vector component , the computer device can calculate the difference between the expected activation vector component and the predicted vector component at the corresponding position in the predicted semantic vector, and perform statistics on the corresponding differences of each expected activation vector component to determine the classification loss.

It can be understood that the expected semantic vector may also include undesired activation vector components. For each undesired activation vector component, the computer device can calculate the relationship between the undesired activation vector component and the predicted vector component at the corresponding position in the predicted semantic vector. Finally, the corresponding differences of each expected activation vector component and the corresponding differences of each undesired activation vector component are counted to obtain the classification loss.

Step 212: Train the feature extraction model based on the classification loss. When the training stop condition is met, the target feature extraction model is obtained; the target feature extraction model is used to extract the sample feature vector of the input sample.

Specifically, the computer device can adjust the model parameters of the feature extraction model based on the classification loss and continue training until the training stop condition is met, and the target feature extraction model is obtained. The obtained target feature extraction model is used to extract the sample feature vector of the input sample. Similar samples can be retrieved from the sample database through the sample feature vector. The input sample can also be classified and identified through the sample feature vector to determine the category to which the input sample belongs. .

In one embodiment, during the training process, the computer device may use a stochastic gradient descent algorithm, Adagrad (Adaptive Gradient, adaptive gradient) algorithm, Adadelta (an improvement of the AdaGrad algorithm), RMSprop (an improvement of the AdaGrad algorithm), Adam ( Adaptive Moment Estimation, adaptive moment estimation) algorithm, etc. to adjust the model parameters of the feature extraction model.

In one embodiment, the training stop condition may be that the model parameters no longer change, the loss may reach a minimum value, the training number may reach a maximum iteration number, and so on. In other embodiments, the computer device can perform multiple rounds (epoch) of iterative training on the feature extraction model. After each round of training is completed, the target loss of the round will be obtained. The training stop condition may be that the average target loss no longer decreases in a certain epoch. .

In the above feature extraction model processing method, the above feature extraction model processing method, device, computer equipment, storage medium and computer program product perform feature extraction on the training sample through the feature extraction model to be trained to obtain a training feature vector, based on the training feature vector Classify and predict the training samples to obtain the predicted semantic vector, obtain the label semantic vector of the training category label corresponding to the training sample, determine the expected semantic vector of the training sample based on the label semantic vector, and then based on each expected activation vector component in the expected semantic vector, The difference between the predicted vector components at the corresponding positions in the predicted semantic vector determines the classification loss, and the feature extraction model is trained through this classification loss. The final target feature extraction model can learn features that tend to the label semantic vector, thereby The semantic space where the label semantic vector is located can be simulated in the vector space where the training feature vector is located, so that the extracted feature vector can store feature information representing the semantics, which improves the accuracy of feature extraction, and because the label semantic vector includes at least two The activation label vector components can more accurately represent the training category labels, and the expected semantic vector is determined based on the label semantic vector and contains the expected activation vector components corresponding to the position distribution of at least two activation label vector components, which can This makes the calculated classification loss more accurate and further improves the accuracy of feature extraction.

In one embodiment, the training sample corresponds to multiple training category labels. Based on the label semantic vector, determining the expected semantic vector of the training sample includes: forming a feature component set by label vector components with the same sorting position in each label semantic vector; for Activate the feature component set of the label vector component, and set the expected vector component at the sorting position to which the feature component set belongs as the activation value; for the feature component set that does not contain the activation label vector component, set the expected vector component at the sorting position to which the feature component set belongs. Set to an inactive value; each expected vector component is combined according to its corresponding sorting position to form the expected semantic vector of the training sample.

Among them, the sorting position refers to the position of the label vector component in the label semantic vector. For example, for the label semantic vector (a, b, c), the sorting position of a is the first position, and the sorting position of b is the second position. The sorting position of c is third.

Specifically, the computer device can combine the label vector components with the same sorting position in each label semantic vector to form a feature component set, so that corresponding to each sorting position, a feature component set containing multiple label vector components can be obtained. For each feature Component set, the computer device can determine whether the feature component set contains an activation label vector component, and if the feature component set contains an activation label vector component, the expected vector component at the sorting position to which the feature component set belongs can be set as the activation value , if the feature component set does not contain an activation label vector component, the expected vector component at the sorting position to which the feature component set belongs can be set to an inactive value, and finally the expected vector components are combined according to their corresponding sorting positions to form training The expected semantic vector of the sample.

For example, assume that the training sample corresponds to two training category labels A and B, where the label semantic vector of A is (1, 0, 1, 0), and the label semantic vector of B is (0, 0, 1, 1), where If the feature component set at the first sorting position contains the activation label vector component 1, then the expected vector component at the sorting position is 1, and the feature component set at the second sorting position does not contain the activation label vector component, then the expected vector component at the sorting position is 1. The expected vector component at the sorting position is 0, and the feature component set at the third sorting position contains the activation label vector component 1, then the expected vector component at the sorting position is 1, and the feature component set at the fourth sorting position contains activation label vector component 1, then the expected vector component at this sorting position is 1, and the final expected semantic vector is (1, 0, 1, 1).

In the above embodiment, by setting the expected vector component at the sorting position of the feature component set containing the activation label vector component as the activation value, the expected vector component at the sorting position of the feature component set not containing the activation label vector component is set as The non-activation value can make the expected semantic vector represent unlimited multiple labels through limited feature bits, thereby saving the amount of calculation in the loss calculation process and improving training efficiency.

In one embodiment, determining the classification loss based on the difference between each expected activation vector component in the expected semantic vector and the predicted vector component at the respective corresponding position in the predicted semantic vector includes: for each expected vector in the expected semantic vector Component, based on the difference between the expected vector component and the predicted vector component corresponding to the sorted position in the predicted semantic vector, determine the initial value of the classification loss component at the sorted position where the expected vector component is located; for the feature component set at the sorted position where the expected vector component is located The activation label vector components in are counted, and the activation degree at the sorted position of the expected vector component is determined based on the statistical results; the initial value of the classification loss component is weighted based on the activation degree to obtain the classification loss component target value; each classification loss component target is counted value to determine the classification loss.

The statistics may include statistics on the number of activation label vector components. The degree of activation based on statistical results is positively correlated with the statistical results, that is, the larger the numerical value obtained by statistics, the higher the degree of activation. On the contrary, the smaller the numerical value obtained by statistics, the lower the degree of activation.

Specifically, after the computer device determines the initial value of the classification loss component at the sorting position where each desired vector component is located, for each sorting position, the number of activation label vector components in the feature component set at the sorting position can be counted, Obtain the quantitative value, and then determine the activation degree at the sorting position based on the quantitative value. Use the activation degree as the weight of the initial value of the classification loss component at the sorting position. Multiply this weight with the initial value of the classification loss component to obtain the classification loss component. target value, and finally the computer device can add the individual classification loss component target values to determine the classification loss.

In one embodiment, the computer device determines the activation degree at the sorting position of the sample feature component based on the statistical results by: for a feature component set containing the activation label vector component, the statistically obtained quantitative value is used as the sorting position of the feature component set. The activation degree at the location, and for the feature component set that does not contain the activation label vector component, the quantity value is 0, then it can be determined that the activation degree at the sorted position of the feature component set is 1.

In other embodiments, after the computer device obtains the quantitative value statistically, it can further normalize the quantitative value to obtain the final activation degree, wherein, in order to avoid ignoring the loss component at the sorting position where the quantitative value is 0, The target quantity value can be obtained by adding the statistical quantity value at each sorting position to the preset offset value. Then the activation degree can be (target quantity value/sum of the target quantity values corresponding to each sorting position), where the preset The offset value may be 1, for example. For example, in the above example, for the training category labels A and B, the quantity values corresponding to each sorting position can be statistically obtained as 1, 0, 2, 1. After adding the preset offset value of 1, the target quantity is obtained. The values are 2, 1, 3, and 2 respectively, and the final calculated activation levels are 1/4, 1/8, 3/8, and 1/4 respectively.

In one embodiment, the computer device can perform cross-entropy loss calculation based on the difference between the expected vector component and the predicted vector component corresponding to the sorted position in the predicted semantic vector, and obtain an initial value of the classification loss component at the sorted position where the expected vector component is located. .

In the above embodiment, by counting the activation label vector components in the feature component set at the sorted position where the expected vector component is located, the activation degree at the sorted position where the expected vector component is located is determined based on the statistical results, and the classification loss component is initialized based on the activation degree. The value is weighted to obtain the classification loss component target value. The obtained loss classification target value can accurately reflect the sharing of the classification loss by the initial value of each classification loss component, and then count the target values of each classification loss component to determine the classification loss. It can be obtained More accurate classification loss.

In one embodiment, before training the feature extraction model based on the classification loss and obtaining the target feature extraction model when the training stop condition is met, the above method further includes: based on the training sample between the training feature vector and the expected semantic vector. Difference, determine the semantic constraint loss, the semantic constraint loss is used to semantically constrain the training feature vector extracted by the feature extraction model; the feature extraction model is trained based on the classification loss, and when the training stop condition is met, the target feature extraction model is obtained, including : Determine the target loss based on semantic constraint loss and classification loss; train the feature extraction model based on the target loss. When the training stop condition is met, the target feature extraction model is obtained.

Among them, the semantic constraint loss is positively related to the difference between the training feature vector and the expected semantic vector of the training sample, and is used to semantically constrain the training feature vector extracted by the feature extraction model. By applying semantic constraints to the training feature vector, the training feature vector can learn the distribution of semantic information in the expected semantic vector, making the semantic information expressed by the training feature vector more accurate.

Specifically, the computer device can perform regression loss calculation based on the difference between the training feature vector of the training sample and the expected semantic vector to obtain the semantic constraint loss, and then determine the target loss based on the semantic constraint loss and the classification loss, and finally determine the feature based on the target loss. The extraction model performs parameter adjustment and continues loss until the training stop condition is reached, and the target feature extraction model is obtained.

In the above embodiment, by calculating the semantic constraint loss and combining the semantic constraint loss and the classification loss for model training, the feature extraction model can learn more accurate semantic information and further improve the accuracy of feature extraction.

It can be understood that in order to ensure that the extracted training feature vector can accurately learn the semantic information distribution in the expected semantic vector, it must be ensured that the dimension of the training feature vector is greater than or equal to the dimension of the expected semantic vector. Based on this, this application The following examples are provided.

In one embodiment, determining the semantic constraint loss based on the difference between the training feature vector of the training sample and the expected semantic vector includes: if the dimension of the training feature vector is consistent with the dimension of the expected semantic vector, then calculating the training feature The vector distance between the vector and the expected semantic vector, using the vector distance as the semantic constraint loss corresponding to the training sample.

In this embodiment, if the dimension of the training feature vector is consistent with the dimension of the expected semantic vector, then for each training vector component in the training feature vector, there is a corresponding expected vector component in the expected semantic vector, that is, training There is a one-to-one mapping relationship between the training vector component in the feature vector and the expectation vector component in the expected semantic vector. For example, refer to (a) in Figure 3. Assume that the dimensions of the training feature vector and the expected semantic vector are both 3. The black circle in (a) represents the expected vector component in the expected semantic vector, and the white circle represents the training As for the training vector component in the feature vector, it can be seen from Figure (a) that there is a one-to-one mapping relationship between the training vector component and the expected vector component.

In this embodiment, considering that the training feature vector and the expected semantic vector are vectors with the same dimension, the computer device can directly calculate the vector distance between the training feature vector and the expected semantic vector to obtain the regression loss, thereby obtaining the semantics corresponding to the training sample. Constraint loss. In specific implementation, the vector distance may be, for example, L2 distance.

In the above embodiment, the semantic constraint loss is obtained by using the vector distance between the training feature vector and the expected semantic vector, which can improve the calculation efficiency of the regression loss, thereby improving the training efficiency.

Consider that when characterizing a sample, it is necessary to characterize the sample as a whole, and the classification information is only a high degree of abstraction of the sample. For different samples under the same classification, it is necessary to introduce sample-specific information after the semantic information that characterizes the classification to distinguish the same Different samples under classification, therefore the dimension of the training feature vector can be larger than the dimension of the expected semantic vector, so as to leave certain feature bits as a supplement to the classification information. Based on this, this application further provides the following embodiments.

In one embodiment, determining the semantic constraint loss based on the difference between the training feature vector of the training sample and the expected semantic vector includes: if the dimension of the training feature vector is greater than the dimension of the expected semantic vector, then for the expected semantic vector For each desired vector component, select a training vector component from the training feature vector, and use the selected training vector component as the mapping vector component of the desired vector component; calculate the difference between the desired vector component and the mapping vector component, and obtain Expect the semantic constraint loss component corresponding to the vector component; count each semantic constraint loss component to obtain the semantic constraint loss.

In this embodiment, the dimension of the training feature vector is greater than the dimension of the expected semantic vector. Then when calculating the semantic constraint loss, for each expected vector component in the expected semantic vector, the corresponding expected vector component can first be determined from the training feature vector. For the corresponding training vector component, the determined training vector component is used as the mapping vector component of the expected vector component. That is, in the learning process, the characteristic bits of each mapping vector component are used to learn the corresponding expected vector components. Then it can be based on the expected The difference between the vector component and the mapping vector component is used for regression loss calculation to obtain the semantic constraint loss component corresponding to the expected vector component. Finally, each semantic constraint loss component is counted to obtain the semantic constraint loss component. The statistics here can be summation or averaging.

It can be understood that the feature bits where the unselected training vector components in the training feature vector are located are representation bits that can be used to supplement sample information other than category information.

In one embodiment, for each expected vector component in the expected semantic vector, the computer device can determine the training vector component corresponding to the expected vector component from the training feature vector according to certain rules, for example, for each expected vector component, The training vector component corresponding to the position can be selected from the training feature vector, and the selected training vector component can be used as the mapping vector component of the expected vector component. For example, assuming that the training feature vector is a 5-dimensional vector and the expected semantic vector is a 3-dimensional vector, then for the expected vector component on the first feature bit in the expected semantic vector, the first feature bit in the training feature vector will be The training vector component is used as the mapping vector component. For the expected vector component on the second feature bit in the expected semantic vector, the training vector component on the second feature bit in the training feature vector is used as the mapping vector component. For the expected semantic vector in The expected vector component on the third feature bit of the training feature vector is the training vector component on the third feature bit as the mapping vector component.

It can be understood that in some other embodiments, the computer device can also select training vector components according to other rules, as long as it is ensured that there is a corresponding mapping vector component for each desired vector component.

In the above embodiment, if the dimension of the training feature vector is greater than the dimension of the expected semantic vector, the difference between the expected vector component and the mapping vector component is calculated, the semantic constraint loss component corresponding to the expected vector component is obtained, and each semantic constraint loss component is counted. , the semantic constraint loss is obtained, the semantic constraint loss can be accurately calculated, and the accuracy of feature extraction can be improved.

In one embodiment, selecting a training vector component from the training feature vector, and using the selected training vector component as a mapping vector component of the expected vector component includes: mapping the training feature vector according to the dimension of the expected semantic vector. The training vector components in are grouped to obtain multiple vector component groups whose number is consistent with the dimension of the expected semantic vector; for each expected vector component in the expected semantic vector, a vector component group is selected from multiple feature component groups as Target vector component group; select a training vector component from the target vector component group as the mapping vector component corresponding to the desired vector component.

Specifically, the computer device can group the training vector components in the training feature vector according to the dimension of the desired semantic vector to obtain multiple vector component groups whose number is consistent with the dimension of the desired semantic vector. The vector component groups can be grouped according to the dimension of the desired semantic vector. The sorted positions of the included vectors are sorted, so that for each desired vector component in the desired semantic vector, the computer device can select a vector component group corresponding to the sorted position from the plurality of feature component groups as a target vector component group, from the target vector Select a training vector component from the component group as the mapping vector component corresponding to the desired vector component.

In one embodiment, if the dimension Nh of the training feature vector is an integer multiple of the dimension n of the desired semantic vector, that is, n*P=Nh (P is greater than 1), then the training vector components in the training feature vector can be Group evenly. For example, refer to (b) in Figure 3, in which the expected semantic vector is a 3-dimensional vector and the training feature vector is 6-dimensional. The black circle in (b) represents the expected vector component in the expected semantic vector, and the white circle represents The training vector components in the training feature vector. In the figure (b), the training vector components in the training feature vector are divided into three groups in order. Each two training vector components are divided into one group, and the first one in each group is divided into three groups. The training vector component serves as the mapping vector component corresponding to the expected vector component.

In the above embodiment, by grouping the training vector components in the training feature vector and selecting one training vector component from each group as the mapping vector component corresponding to the expected vector component, each feature bit in the training feature vector can be better The learning expects the semantic information in the semantic vector.

It can be seen from the above embodiment that the vector dimension of the training feature vector is greater than or equal to the dimension of the expected semantic vector, and the expected semantic vector is obtained based on the training category label. When there are multiple training category labels in the training sample, the expected semantic vector The semantic vector needs to be obtained by fusing multiple tag semantic vectors. Therefore, it is expected that the dimension of the semantic vector is greater than or equal to the tag semantic vector. Based on this, this application also provides the following embodiments to generate the tag semantic vector.

In one embodiment, the training samples belong to a training sample set, and the training category labels corresponding to each training sample in the training sample set form a label set. The above method also includes: constructing a target semantic space; the target vector dimension of the target semantic space does not exceed the training The vector dimension of the feature vector; map each training category label in the label set to the target semantic space to obtain the label semantic vector of each training category label.

Specifically, the computer device can construct the target semantic space and ensure that the target vector dimension of the target semantic space does not exceed the vector dimension of the training feature vector, and then map each training category label in the label set to the target semantic space to obtain each The computer device can further establish a mapping relationship between the training category labels and the label semantic vectors of the training category labels, so that during the training process, the computer device can obtain the label semantic vectors of the training category labels based on the mapping relationship.

Further, in a specific embodiment, constructing the target semantic space includes: generating a target matrix whose matrix order matches the dimension of the target vector. The target matrix includes multiple candidate representation vectors, and the multiple candidate representation vectors form the target semantic space. ; The candidate representation vector includes an equal number of first and second values; each training category label in the label set is mapped to the target semantic space, and the respective label semantic vector of each training category label is obtained, including: for semantics For each semantic category label in the category label set, one candidate representation vector is selected from multiple candidate representation vectors as the label semantic vector corresponding to each semantic category label.

Wherein, the target matrix includes a plurality of candidate representation vectors, and the candidate representation vectors include an equal number of first values and second values. The first values and the second values are different values, and the first value can represent the value used as Activation vector component, the second value can be used as a non-activation vector component. Since the candidate representation vector includes an equal number of first values and second values, when the candidate representation vector is used as a label semantic vector, the semantic space constructed is a uniformly distributed semantic space.

Specifically, the computer device may determine the target vector dimension of the target semantic space, and after determining the target vector dimension, generate a target matrix whose matrix order matches the target vector dimension. The target matrix may be a Hadamard matrix. It is known that for the n-order Hadamard matrix H, its n+1-order Hadamard matrix is:

Therefore, the Hadamard matrix of each order can be obtained. For example, the 4th order Hadamard matrix is as follows:

Among them, each value in the matrix can be 1 or -1, 1 means activation, and -1 means inactivation. At the same time, the first row is all 1, and the matrix is a symmetric matrix, and the sum of the elements on the diagonal of the matrix (the trace of the matrix) is 0.

Except for the first row, the vectors in the other rows of the Hadamard matrix can be used as candidate representation vectors. These candidate representation vectors constitute the target semantic space. The computer device can select a candidate representation vector from the target semantic space as the label corresponding to the semantic category label. Semantic vectors, wherein for different semantic category labels, the computer device selects different candidate label vectors as label semantic vectors.

Since the numbers of -1 and 1 in each candidate representation vector of the Hadamard matrix are the same, when these candidate representation vectors are used as vectors in the target semantic space, the effect of uniform activation can be achieved, and the resulting target semantic vector is a uniformly distributed semantic space. In the embodiment of the present application, by mapping the training category labels to the uniformly activated semantic space, the obtained semantic label vectors can make the feature vectors extracted by the trained feature extraction model simulate the uniformly distributed semantic space, improving feature extraction. The semantic representation effect of the features extracted by the model prevents certain semantic information from being redundant or compressed in the feature bits.

Refer to Figure 4, which is a comparison diagram of feature learning effects in one embodiment. The diagram on the left represents a uniformly distributed semantic space. Different circles represent the distribution of different labels in the semantic space. The largest circle is the overall semantic space. Different circles The labels are distributed on the circle. Since the semantic space uses vectors in the Hadamard matrix to represent each label, the distance between two labels is the same. The picture on the right represents the hash space. In this embodiment, the hash space is used for metric learning. Therefore, similar samples are close to each other in the hash space. For example, the two underlined Xs in Figure 4 represent pairs of similar samples. Referring to (a) in Figure 4, the uniformly distributed semantic space is not directly related to the hash similarity space, so samples of different categories are mixed and distributed in the hash space, as shown in (a), the solid line X represents the samples belonging to the label represented by small circle 1, and the dotted line Cross each other. Referring to (b) in Figure 4, in the hash space based on semantic space fine-tuning, in addition to the proximity of sample pairs, each sample pair is distributed in the direction of the label semantic vector where its label is located, so a close semantic can be simulated in the hash space Evenly distributed space.

In the above embodiment, by simulating a uniformly distributed semantic space, the semantic representation effect of the extracted features can be further improved.

In one embodiment, the feature extraction model is trained based on the classification loss. When the training stop condition is met, the target feature extraction model is obtained, which includes: obtaining the comparison feature vector corresponding to the training sample, and comparing the training feature vector and the training sample based on the comparison The difference between feature vectors is used to obtain the feature extraction loss; based on the feature extraction loss and classification loss, the target loss is obtained; based on the target loss, the parameters of the feature extraction model are adjusted and training continues. When the training stop condition is met, the target feature extraction model is obtained.

Among them, the comparison training sample refers to the training sample used for comparison with the training sample to determine the feature extraction loss. The comparison training samples may include at least one of positive comparison training samples or negative comparison training samples. Positive comparison training samples refer to training samples that are similar to the training samples, and negative comparison training samples refer to training samples that are similar to the current training samples. Dissimilar training samples.

Specifically, the computer device can perform feature extraction on the comparison training sample based on the feature extraction model to be trained, obtain the training feature vector corresponding to the comparison training sample as the comparison feature vector, and obtain the feature extraction based on the difference between the training feature vector and the comparison feature vector. Loss, perform a weighted sum of feature extraction loss and classification loss to obtain the target loss. Based on the target loss, adjust the parameters of the feature extraction model to be trained and continue training. When the training stop condition is met, the target feature extraction model is obtained.

In one embodiment, the computer device obtains the feature extraction loss based on the difference between the training feature vector and the comparison feature vector. Specifically, the method may be: calculating the cosine similarity between the comparison feature vector and the training feature vector, and using the obtained cosine similarity to characterize the two. The difference between the cosine similarity and the training label can be calculated to obtain the feature extraction loss. Among them, when the comparison training sample is a positive comparison training sample, the training label is 1, and when the comparison training sample is a negative comparison training sample, the training label is 0.

In some embodiments, the computer device obtains the feature extraction loss based on the difference between the training feature vector and the comparison feature vector. Specifically, the method may be: performing similarity classification on the training feature vector and the comparison feature vector, and dividing them into two categories, similar and dissimilar, to obtain Similar probability and dissimilar probability, take the larger value of the probability as the classification result, use the probability value of the classification result to represent the difference between the training feature vector and the comparison feature vector, and then calculate the difference between the probability of the classification result and the training label Get the feature extraction loss. Among them, when the comparison training sample is a positive comparison training sample, the training label is 1, and when the comparison training sample is a negative comparison training sample, the training label is 0.

In one embodiment, the computer device obtains the feature extraction loss based on the difference between the training feature vector and the comparison feature vector. Specifically, the feature extraction loss can be calculated by: calculating the feature distance between the training feature vector and the comparison feature vector, and using the feature distance to characterize the training feature vector and the comparison feature vector. The difference in feature vectors is used as the feature extraction loss, where the feature distance can be, for example, Euclidean distance or L2 distance.

In the above embodiment, when the computer device trains the feature extraction model to be trained, the feature extraction loss and the classification loss are simultaneously adjusted to adjust parameters, so that the features extracted by the trained feature extraction model are more accurate.

In one embodiment, the comparison feature vector includes a positive comparison feature vector corresponding to a positive comparison training sample and a negative comparison feature vector corresponding to a negative comparison training sample; the feature is obtained based on the difference between the training feature vector and the comparison feature vector. The extraction loss includes: obtaining the positive feature difference value, which is the feature difference value between the training feature vector and the positive comparison feature vector; obtaining the negative feature difference value, which is the feature difference value between the training feature vector and the positive comparison feature vector. Negatively compare the feature difference values between feature vectors; determine the feature extraction loss based on the positive feature difference value and the negative feature difference value.

Specifically, the computer device extracts features from the positive contrast training sample based on the feature extraction model to be trained, obtains the positive contrast feature vector corresponding to the positive contrast training sample, and extracts features from the negative contrast training sample based on the feature extraction model to be trained. , obtain the negative contrast feature vector corresponding to the negative contrast training sample, obtain the feature difference value between the training feature vector and the positive comparison feature vector, obtain the positive feature difference value, and obtain the difference between the training feature vector and the negative comparison feature vector The feature difference value is obtained to obtain the negative feature difference value, and finally the feature extraction loss is determined based on the positive feature difference value and the negative feature difference value.

In one embodiment, the computer device can determine the feature extraction loss with reference to the following formula (1), where x _a is the training feature vector, x _p is the positive contrast feature vector, x _n is the negative contrast feature vector, ||x _a -x _p || represents the L2 distance between x _a and x _p , that is, the positive feature difference value, ||x _a -x _n || represents the L2 distance between x _a and x _n , that is, the negative feature difference value value, the purpose of formula (1) is to make the distance between the training sample and the negative comparison training sample larger than the distance between the training sample and the positive comparison training sample α, α is the margin (spacing term), and the value of α can be as needed Make settings.

L _tri =max(||x _a -x _p ||-||x _a -x _n ||+α,0) Formula (1)

It can be seen from formula (1) that the feature extraction loss value is 0 only when the distance between the training sample and the negative comparison training sample is greater than the distance α between the training sample and the positive comparison training sample. Otherwise, the feature extraction loss value is 0. The value is greater than 0, so in the process of reducing the loss value, the distance between the training sample and the negative contrast training sample develops in the direction of α greater than the distance between the training sample and the positive contrast training sample, thus making the feature extraction model extract Features can express semantic information more accurately.

In the above embodiment, since the feature extraction loss value is determined based on the positive feature difference value and the negative feature difference value, when the feature extraction model performs similarity metric learning, it takes into account that the feature distance between classes is too small for classification. The impact further improves the semantic accuracy of features extracted by the feature extraction model.

It can be understood that when the feature extraction model is trained in combination with the feature extraction loss calculated by the above formula (1), the feature extraction model can perform metric learning, so that the extracted features have similarity measurement capabilities.

In one embodiment, extracting the training feature vector of the training sample through the feature extraction model to be trained includes: extracting the initial sample features of the training sample through the feature extraction model to be trained, and performing quantification processing on the initial sample features to obtain the training sample. training feature vector; the feature extraction model is trained based on the classification loss. When the training stop condition is met, the target feature extraction model is obtained, including: determining the quantification target corresponding to each quantization value in the training feature vector based on the preset symbolic function, based on The difference between each quantified value and the corresponding quantified target determines the quantified loss; based on the quantified loss and the classification loss, the target loss is obtained; based on the target loss, adjust the parameters of the feature extraction model and continue training. When the training stop condition is met, we get Target feature extraction model.

Among them, the quantization loss refers to the loss of calculating the quantization effect (whether it is close enough to -1 or 1). During the training process, it is expected that each quantized value in the training feature vector is close enough to 1 or -1. The quantization loss is positively related to the difference between the quantization value and the quantization target corresponding to the quantization value.

In one embodiment, for the training feature vector obtained by quantization processing of each training sample, Q _i is the value of the training feature vector Q obtained by quantization processing of the training sample at the i-th position, and B _i is the i-th position value. Quantification target, B _i is generated by Q _i through the preset sign function - sign function -. By using the sign function in the following formula (3), the target code Bi can be calculated separately for each bit Q _i of the training feature vector Q. Finally, The target encoding of Q is B, and then the quantization loss is calculated by referring to formula (2).

In one embodiment, after obtaining the quantified loss, the computer device can perform a weighted sum of the two losses through the preset weights of the quantified loss and the classification loss to obtain the target loss, and then adjust the model of the feature extraction model based on the target loss. parameters to obtain the adjusted feature extraction model.

In the above embodiment, the target loss is obtained by combining the quantification loss and the classification loss. The model is trained based on the target loss. The trained feature extraction model can extract accurate quantitative features.

In a specific embodiment, as shown in Figure 5, it is a schematic diagram of the training process of the feature extraction model. In this embodiment, the content of the training sample is an image, and the training sample is an image sample. The feature extraction model to be trained includes CNN module, embedding module, and hash quantization module. Referring to Figure 5, the computer device first uses the Hadamard matrix to establish a uniformly expressed semantic multi-label space corresponding to the training sample set. During the training process, the training samples are input into the CNN module to obtain basic feature information, and the basic feature information is input into the embedding module to obtain the embedding. The vector is then quantized through the hash quantization module to obtain the training feature vector. The training feature vector is further input into the classification model. The predicted semantic vector output by the classification model and the expected semantic vector of the training sample calculate the classification loss, and based on the expected semantics The difference between the vector and the training feature vector calculates the semantic constraint loss, which is weighted with the metric loss and the classification loss to obtain the target loss. The metric loss can be obtained by referring to the weighted sum of the quantization loss and the feature extraction loss in the above embodiment.

In this embodiment, in addition to learning metric capabilities in the hash feature learning space, each sample is used to simulate a semantic space with uniform semantic distribution, so that the hash features of each image are uniformly distributed while satisfying the metric learning similarity. The label semantic vector adjustment, compared with the hash and semantic dual space models, more effectively improves the multi-bit activation effect of the hash feature, thereby reducing its excessive concentration on certain feature information or excessive redundancy of the hash bits. question. The hash code metric constraint loss in Figure 5 is an important condition for fine-tuning the output of the hash layer in the non-semantic space with the help of a uniform distribution model of the semantic space, thereby supporting the simulation of the semantic space in the non-semantic space and fine-tuning the hash representation of the non-semantic space. of learning.

In one embodiment, as shown in Figure 6, a sample retrieval method is provided. For illustration, the method is executed by a computer device. It can be understood that the computer device can be the terminal 102 shown in Figure 1, or It may be the server 104, or it may be a system composed of the terminal 102 and the server 104. In this embodiment, the sample retrieval method includes the following steps:

Step 602: Obtain query samples and candidate recall sample sets.

Among them, the candidate recall sample set refers to a set of recallable content samples in the database. Query samples refer to content samples that need to recall similar samples from the database. For example, given an image A, similar images of image A are recalled from the database, then image A is a query sample.

Step 604: Input the query sample and the candidate recall sample in the candidate recall sample set into the target feature extraction model to obtain the query feature vector corresponding to the query sample and the candidate recall feature vector corresponding to the candidate recall sample.

Among them, the target feature extraction model is obtained by training the feature extraction model to be trained through classification loss. The classification loss is based on each expected activation vector component in the expected semantic vector and the predicted vector component at the corresponding position in the predicted semantic vector. The difference is determined. The predicted semantic vector is obtained by classifying and predicting the training sample based on the training feature vector. The expected semantic vector is determined based on the label semantic vector of the training category label corresponding to the training sample. The label semantic vector includes at least two activation label vectors. component, the expected semantic vector contains the expected activation vector component corresponding to the position distribution of at least two activation label vector components, and the training feature vector is obtained by feature extraction of the training sample through the feature extraction model to be trained.

Specifically, the computer device can input the query sample and each candidate recall sample in the candidate recall sample set into the target feature extraction model, respectively perform feature extraction on the query sample and each candidate recall sample through the target feature extraction model, and obtain the query sample corresponding to Query feature vectors and candidate recall feature vectors corresponding to each candidate recall sample.

Step 606: Based on the query feature vector and the candidate recall feature vector, determine the target retrieval sample corresponding to the query sample from the candidate recall sample set.

In one embodiment, the computer device can calculate the similarity between the query feature vector and each candidate recall feature vector, and determine the candidate recall sample corresponding to the candidate recall feature vector whose similarity satisfies the similarity condition as the target retrieval sample. The similarity may be cosine similarity, and the similarity condition may be, for example, that the similarity is greater than a preset similarity threshold or that the similarity is sorted before the preset sorting threshold. For example, the calculated similarities are sorted from large to small, The top N similarities are selected, and the candidate recall samples corresponding to the candidate recall feature vectors corresponding to these similarities are determined as target retrieval samples.

In another embodiment, the computer device can calculate the degree of difference between the query feature vector and each candidate recall feature vector, and determine the candidate recall sample corresponding to the candidate recall feature vector whose difference satisfies the difference condition as the target retrieval sample. The difference degree may be a feature distance, and the difference degree condition may be, for example, that the difference degree is less than a preset difference degree threshold or the difference degree is sorted before the preset sorting threshold. For example, the calculated difference degrees are sorted from small to large to select For the first N differences, the candidate recall samples corresponding to the candidate recall feature vectors corresponding to these differences are determined as target retrieval samples.

In the above sample retrieval method, since the target feature extraction model is trained through the classification loss, the final target feature extraction model can learn features that tend to the label semantic vector, so that the label semantic vector can be compared in the vector space where the training feature vector is located. The semantic space is simulated, so that the extracted feature vector can store feature information representing semantics, which improves the accuracy of feature extraction. And because the label semantic vector includes at least two activation label vector components, the training category labels can be more refined. Accurately characterized, and the expected semantic vector is determined based on the label semantic vector and contains the expected activation vector component corresponding to the position distribution of at least two activation label vector components, which can make the calculated classification loss more accurate and further improve the feature The accuracy of extraction, and then using the feature vector extracted by the target feature extraction model for sample retrieval, can improve the accuracy of sample retrieval.

In one embodiment, the above sample retrieval method also includes: performing feature clustering on the candidate recall feature vectors of each candidate recall sample in the candidate recall sample set to obtain multiple clusters; each cluster cluster has a corresponding cluster. center; for each clustering center, establish the association between the clustering center and each candidate recall feature vector in the same cluster; based on the query feature vector and the candidate recall feature vector, determine the query sample correspondence from the candidate recall sample set Target retrieval samples include: determining the target cluster center from each cluster center based on the feature distance between the query feature vector and each cluster center; obtaining each candidate recall feature vector that is associated with the target cluster center, Based on the feature distance between the query feature vector and each obtained candidate recall feature vector, the target retrieval sample is determined from each obtained candidate recall feature vector.

Specifically, after the computer device performs clustering, it establishes an association between each clustering center and each candidate recall feature vector in the same clustering cluster. Then the clustering center can be used as the index of the modified clustering cluster, thereby performing the clustering process. During retrieval, you can first calculate the feature distance between the query feature vector and these indexes, thereby filtering out the clusters where similar samples are located, and perform retrieval from the filtered clusters to obtain the target retrieval samples. Index query greatly reduces the amount of calculation in the retrieval process and improves retrieval efficiency.

For example, assuming that the candidate recall sample set includes 1,000 candidate recall samples, and after clustering these candidate recall samples, 10 clusters are obtained. Then, during retrieval, the computer device can separate the query feature vector and The clustering centers of these 10 clusters are used to calculate feature clusters, and the first three clusters with the smallest feature distance are selected, and the query feature vector and each candidate recall feature vector in these three clusters are used to calculate feature clusters, so that The target retrieval sample is obtained by retrieval, but for the remaining 7 clusters, no calculation is required.

In one embodiment, refer to FIG. 7 , which is a schematic diagram of an index retrieval system in one embodiment. (a) in Figure 7 shows that for all inventory images, the target feature extraction model trained in the embodiment of this application extracts image feature vectors and performs clustering to obtain multiple clusters. Clustering clusters and clustering Center correlation to build an index system, (b) in Figure 7, in the specific retrieval process, for the query sample, first extract the query feature vector through the target feature extraction model, and compare it with the cluster center of each cluster cluster. To determine the target cluster, and then recall the target retrieval samples from the target cluster through the index system and sort them back.

In one embodiment, the above sample retrieval method also includes a feature extraction model processing step. This step specifically includes: performing feature extraction on the training sample through the feature extraction model to be trained to obtain a training feature vector; and extracting the training sample based on the training feature vector. Perform classification prediction to obtain the predicted semantic vector; obtain the label semantic vector of the training category label corresponding to the training sample, the label semantic vector includes at least two activation label vector components; based on the label semantic vector, determine the expected semantic vector of the training sample, the expected semantic vector , contains the expected activation vector components corresponding to the position distribution of at least two activation label vector components; based on the difference between each expected activation vector component in the expected semantic vector and the predicted vector component at the corresponding position in the predicted semantic vector, determine Classification loss; the feature extraction model is trained based on the classification loss. When the training stop condition is met, the target feature extraction model is obtained; the target feature extraction model is used to extract the sample feature vector of the input sample.

This application also provides an application scenario, which is applicable to the above-mentioned feature extraction model processing method and sample retrieval method. In this application scenario, the content is an image and the training sample is an image sample. The feature extraction model processing method and sample retrieval method can be applied to image deduplication retrieval.

In related technologies, image retrieval uses metric learning plus relaxed constraints of symbolic quantization to generate target hash features, and similar samples are recalled through the target hash features. However: 1) This type of hash feature is constrained by training samples and cannot be perceived due to data Missing samples cause problems such as insufficient hash coding and difficulty in characterizing out-of-domain samples other than training samples. 2) It is not conducive to subsequent parsing and analysis: Since hash features are learned randomly, each hash feature will have varying degrees of redundant bits (for example, an important feature may end up using different numbers of hashes such as 0, 1, 2, etc. Bit representation), which brings great inconvenience to the subsequent analysis of difficult samples of the model. If the hash feature can effectively represent all possible sample spaces, based on the perception of the overall space, using the overall space to constrain the hash distribution, and then performing sufficient hash learning on the target sample space, it will greatly alleviate the above-mentioned hashing problems. The main problem with features.

Based on this, this application provides a feature extraction model processing method, which first uses rich multi-labels to establish the overall data space, and maximizes the classification representation distance between each multi-label category so that the classification targets are evenly distributed in the semantic space; at the same time, it makes The sample hash feature space is directly related to the semantic space of its classification, so that the hash feature can further learn the feature measurement capabilities required for duplicate retrieval in the uniform semantic space, thereby making the hash representation dispersed enough to support maximizing multi-label category representation. requirements, thus avoiding the original initialized random distribution hash representation being overly concentrated on certain hash bits, or the extracted features being overly concentrated on certain image textures; it also avoids the isolation of the hash feature space and the semantic space. Through iterative updates of the model, the hash feature is finally affected by the uniform spatial representation of the classification, improving the representation ability of multi-bit hashing.

Specifically, the following description takes the method being executed by a computer device as an example. It can be understood that the computer device may be the terminal 102 shown in FIG. 1 , the server 104 , or a combination of the terminal 102 and the server 104 . system. The application of the above feature extraction model processing method and sample retrieval method in this application scenario is as follows:

1. Data preparation

1. Collect similar sample pairs. Two frames of images can be randomly extracted from the image database to form an image pair, and similar images can be combined into similar sample pairs by manually annotating whether they are similar images.

2. Triplet mining: Taking similar sample pairs as input, in each batch (in training, the full amount of data is used for epoch rounds of iterations, and in each round of iterations, the full amount of N similarity sample pairs is used as a batch. A total of N/bs batches) contains bs sample pairs, and the triplet is obtained by mining as follows: for any picture x in a certain sample pair, randomly select one from the remaining bs-1 sample pairs (each pair Calculate the embedding of the sample pair from the samples of each image), calculate the distance between the embedding and For bs = 64, after the first 4), the first 10 samples are taken as negative samples, and they are combined with the positive sample pairs in x to form triples, so each sample generates 10 triples, and the entire batch gets 10*bs Triplets, in which the image x in each triplet is recorded as anchor(a), the image that is a similar sample to x is positive(p), the negative sample mined is negative(n), and the triplet is Marked as (apn).

In this embodiment, it is necessary to ensure that the extracted feature vectors can be predicted to be the same (the metric distance is as small as possible) for the same image (exactly the same, extremely similar images), and for different images (including only a small amount of similarity but not similar enough, dissimilar ) needs to be as large as possible (it also needs to satisfy the order-preserving effect: the more dissimilar it is, the farther away it is). Except for similar images such as the front and back frames of the same video shot, adding tone transformation and other attack methods to the image, the probability that most images are similar to each other is very low, so in this embodiment, each batch ( Sampling from all samples, it can be understood that most of the two sample pairs in the batch are not the same, but there are still a small number of identical positive sample pairs called noise. For each shot of 3 similar sample pairs, these 3 themselves are similar to each other. If a certain batch draws two of the sample pairs, there must be noise. Such similar samples perform the following processing on weakly supervised data in this embodiment: remove the most similar small number of samples (top K%) from the remaining batch samples. It is considered that the same samples may exist in the samples here and should not be used as negative samples. learning), the rest are all valid negative samples, and the 10 negative samples with the smallest distance are selected as difficult negative samples. Among them, K is a controllable value. The greater the noise in the training sample set, the larger k.

3. Multi-label annotation: For the images of the above similar sample pairs, label multiple labels for each image (i.e., the training category labels above). You can refer to the multi-labels corresponding to the training sample set (i.e., which categories of labels are involved). Large-scale open source imagenet or open-image tag system. Images can be manually labeled with multiple labels, or a multi-label model can be used to predict image labels: For example, the open source resnet101 network structure is used to assist the imagenet pre-trained multi-label model to produce multi-label prediction results for the images in the above training set.

4. For each image, generate a joint multi-label Hadamard vector representation under its multi-label annotation (i.e., the expected semantic vector above). According to the hash feature bit Nh to be learned, set the order n of the Hadamard matrix to ensure that n*P=Nh and n>Nclass, that is, the order of the Hadamard matrix is less than or equal to the hash bit length and greater than or equal to half the hash bits. The length is greater than or equal to the number of categories with multiple labels at the same time. Considering that the hash feature needs to represent the entire image, and the classification information is only a high degree of abstraction of the image, the length of n should be less than or equal to the hash length; also considering that the full image representation needs to introduce image-specific features in addition to the classification representation of the image. Information is used to distinguish different images of the same classification, so the Hadamard order (n) of P>=1 times is reserved as a supplement to the classification information. Where Nclass is the number of all labels involved in the training sample set.

For each label involved in the training sample set, select one of the vectors in the other rows of the Hadamard matrix except the first row as its Hadamard vector representation (i.e., the label semantic vector above). Since a Hadamard vector only It can represent a label. If there are multiple labels in the image, the Hadamard vectors of each label can be fused to obtain the joint multi-label Hadamard vector representation of the image. Specifically, it can be judged that multiple Hadamard vectors belong to the same order. Whether there is an activation value in the vector component of the position. If there is an activation value, the sorting position is treated as an activation value. If there is no activation value, the sorting position is treated as an inactivation value. Finally, each activation value and inactivation value are processed according to Their sorted positions are arranged to form a joint multi-label Hadamard vector representation of the image.

It can be understood that if the image has only one label, the Hadamard vector of the label is the expected semantic vector of the image.

2. Training process

Among them, the model to be trained includes three parts: basic feature extraction module, hash quantization module, and classification model. Among them, the basic feature extraction model and the hash quantization module together constitute the feature extraction model. The basic feature extraction model uses resnet101. The parameters are as shown in Table 1, including convolution layer 1, convolution layer 2-convolution layer 5 and pooling layer. 6 parts. Convolutional layer 1 is a 7×7×64 convolution with a stride of 2. Convolutional layer 2 includes a 3×3 maximum pool layer (max pool) and 3 ResNet modules (block). , Convolutional layer 3-convolutional layer 5 include 3 ResNet modules, 4 ResNet modules, 23 ResNet modules and 3 ResNet modules respectively.

Table 1

The parameters of the hash quantization module are shown in Table 2, including a layer of fully connected layer, which takes the output of the maximum pooling layer as input and outputs a floating point of 1x256. This floating point vector can be mapped to a binary vector through the sign function. The value vector (0 or 1) is the hash feature in the final application. The parameters of the classification model are shown in Table 3, including the classification layer (Fc_class). The input of this classification layer is the output of the hash feature generation layer, and the output of this classification layer is the n-dimensional predicted probability. Among them, n is the same as the order of Hadamard matrix.

Table 2

layer name Output size layer Hash quantization layer 1x256 Fully connected layer

table 3

layer name Output size layer classification layer 1xn Fully connected layer

It should be noted that the basic feature extraction model, hash quantization model and classification model can also use other model structures. For example, the basic feature extraction model uses resnet18CNN, and the hash quantization model uses multi-layer fully connected layer connections. The specific training process is as follows:

1. Parameter initialization:

In the pre-training process, convolutional layers 1 to 5 use the parameters of ResNet101 pretrained on the ImageNet data set. Newly added layers such as the feature layer are initialized with a Gaussian distribution with a variance of 0.01 and a mean of 0 - ha The quantification layer and the classification layer are initialized with a random normal distribution with a variance of 0.01 and a mean of 0.

2. Set learning parameters: All parameters in Table 1, Table 2 and Table 3 need to be learned.

3. Set the learning rate: lr=0.0005 learning rate is used for the basic feature extraction model and hash quantization model, and 0.005 learning rate is used for the classification model. After every 10 iterations, lr becomes 0.1 times its original value. During gradient backpropagation, the semantic loss (i.e., classification loss) first updates the parameters of the classification model (i.e., Table 3), and then is transmitted back to the hash quantization module (Table 2) and the basic feature extraction model (Table 1). The learning rate of the hash feature is set to be smaller than the semantic label here, which can prevent the multi-label semantic loss from being completely transferred to the quantized hash layer, causing semantic learning to excessively affect the measurement effect of the hash quantization module, and causing the hash quantization module to be redundant.

4. Learning process: Perform epoch rounds of iterations on the full amount of data; each iteration processes a full amount of samples until the average epoch loss no longer decreases under a certain epoch, and the image feature extraction model and target multi-label classification model are obtained. Among them, the specific operations in each iteration of each epoch are as follows: divide the entire image N similar sample pairs, each bs sample pair into a batch (batch), divide it into Nb=N/bs batches, for each A batch, obtain the triples through the above triple mining, proceed:

4.1. Forward calculation: Set all parameters to the state that needs to be learned. During training, perform forward calculation on an input picture to obtain the hash feature and the n-dimensional probability vector output by the classification layer, represented by Q and P, where Q is a 1x256 vector representing the hash feature, P is a 1*n-dimensional probability vector, and each bit represents the predicted probability of the feature bit.

4.2. Loss calculation: 1) For each triplet, calculate the quantification loss and metric loss; 2) For the bs sample pair, for each image, obtain the expected semantic vector of the image, and calculate the n-dimensional probability of the classification layer output The multi-label cross-entropy loss function of the vector and the desired semantic vector; 3) For the bs pair of sample pairs, calculate the L2 loss of the Adama multi-label valid bits and the Adama multi-label target in the hash output result, so that the sample hash bits The medium Hadamard multi-label bits are consistent with the Hadamard label of the sample to simulate Hadamard space in the hash space. Calculate the sum of the three losses as the total loss. details as follows:

For each sample pair in the batch (bs sample pairs in total), the hash loss is the mean of the 10 mined triple hash losses, the mean of the Hadamard classification loss of the 2 image samples, and the mean of the Hadamard classification loss of the 2 image samples. The hash representation is consistent with the regression loss (L2 loss) of the Hadamard representation. The classification loss is the average of the classification losses of the two images. Please refer to the following formula (4) for details:

Among them, w ₁ =0.1, w ₃ =0.2, w ₂ =1-w ₁ -w ₃ . The effect of adding multi-label semantic space learning based on Hadamard vector in this embodiment is to add uniform multi-label semantic space learning on top of the hash feature. The activated Hadamard vector representation can constrain the distribution of hash features to tend to be uniform. Except for the first row of the Hadamard vector, the sums of the other rows are all 0, that is, the number of -1 and 1 is the same. It can be understood that compared with the one-hot representation and the representation of always 1, the representation of the category activation bit is uniform. . The degree of activation can be determined based on the number of activation label vector components under the same feature bit among multiple semantic label vectors. For details, reference may be made to the descriptions in the above embodiments.

In this embodiment, considering that metric learning converges slowly - it is much more difficult to find similar and dissimilar parts on two images than it is to find commonalities in the same category in multiple images of the same category, so semantic learning can be accelerated. To converge and avoid missing recall of semantically related images due to improper local representation in metric learning, set W ₁ =0.1. However, fast semantic convergence can easily bring overfitting to quantitative features. The above constraints are achieved by setting the quantitative learning rate to 0.1 times that of semantic learning, and the semantic multi-label loss weight w ₁ to 1/9 of metric learning w ₂ .

Setting W3 = 0.2 can effectively ensure that the metric learning sample pairs are close to each other, and then use smaller weights to adjust the sample pairs closer to the semantic representation to avoid losing or reducing the metric learning effect when adjusting the semantic loss.

The following is a detailed introduction to each loss:

1)L _hash : composed of L _coding and L _triplet . Please refer to the following formula (5) for details:

L _hash =w ₂₁ L _triplet +w ₂₂ L _coding (5)

Among them, L _triplet can be calculated with reference to the above formula (1), and L _coding can be calculated with reference to the formulas (2) and (3) in the above embodiment. w ₂₁ can be set to 1 and w ₂₂ can be set to 0.5. Since L _reg converges faster than L _triplet , in order to ensure that L _triplet is in a dominant position in the overall loss and thus ensure that embedding always has the ability to measure similarity, w ₂₂ here is set to 0.5 (or other values less than 1, which can be seen situation adjustment). It can be understood that in practical applications, the sign function can be directly used to generate a quantized binary vector for the output Q, which can be used for image retrieval.

2) L _class : For the probability vector output by the classification model, calculate the multi-label loss between it and the expected semantic vector of the image. You can refer to the following formula (6), where t[i] is the 0 and 1 vector of 1*n (where Just replace -1 in the expected semantic vector obtained by the Hadamard vector with 0), and o[i] is the predicted probability of each of the 1*n Hadamard bits (that is, the sorting position in the Hadamard vector).

3) L _reg : The regression loss Lreg for semantic representation modeling in the hash space of the sample pair is as follows, where a and p represent the two samples in the sample pair, Hash _a represents the hash output of the hash sample a, and Hada _a represents the sample For the Hadamard multi-label target of a, you can refer to the following formula (7):

L _reg = (Hash _a -Hada _a ) ² + (Hash _p -Hada _p ) ² (7)

When P=1, Nh=n, at this time, the expected semantic vector and the training feature vector have the same dimension, and the minimum L2 distance can be directly used as a constraint for modeling the semantic space. It can be understood that in this embodiment, the training feature vector is a hash feature, so the sorting position in the training feature vector is a hash bit.

When P>1, that is, when Nh is P times of n, formula (7) needs to be adjusted to: only keep those bits in the hash bits that are mapped to the Hadamard bits, where the hash bits are consistent with the Hadamard bits. The mapping relationship can be referred to Figure 3. You can select one bit from each P hash bit to map to the Hadamard bit, where P is a multiple of the P bit length in Nh = n*P. For each Hadamard bit of image j Bit i (n bits in total), all minimize the regression loss between the Hadamard bit and the corresponding hash bit (Pi+q, q represents the offset). For details, please refer to the following formula (8):

4.3. Model parameter update: Use the stochastic gradient descent algorithm to perform gradient backward calculation on the target model loss value obtained in the previous step to obtain the update value of all model parameters, and use this update value to update the parameters of all models.

3. Quantitative feature retrieval application

Use the trained image feature extraction model to extract all Qs from all inventory images, and store the binary quantified features obtained through sign function activation into the database. Extract Q from the query image through the above image feature extraction model and obtain binary quantified features through binary quantization, and then compare them one by one with the binary quantified features in the inventory. Using Hamming distance calculation for binary quantified features can speed up the calculation efficiency. (Compared to floating-point embedding features), after calculating the distance, the topK most similar ones are returned in order from small to large. In this way, it is possible to retrieve inventory images that are semantically similar to the query image (i.e., the target retrieval result sample above).

The above embodiments have the following beneficial effects:

On the one hand, with the help of sample-rich multi-label semantic space constrained hash features, not only can it represent metric learning features, but it can also store feature information that represents semantics, preventing features from being biased and ineffective in hash feature learning due to biased or insufficient samples. Characterize out-of-domain samples to improve recall accuracy in retrieval scenarios that include out-of-domain samples.

On the other hand, the uniform activation of multiple bits of the multi-label target avoids problems such as the inability of one-hot multi-label to effectively utilize all bits and easy overfitting, and through gradient backpropagation, the hash bits are also concentrated in the specific classification bits. features, resulting in problems such as redundant features.

In addition, with the help of Hadamard-based hash code metric constraints, the overall distribution of samples in the hash space is realized to simulate a uniformly distributed semantic space (simulating the semantic space in the non-semantic space and fine-tuning the learning of hash representations in the non-semantic space). On the one hand, The semantic space greatly improves the multi-bit activation effect of the hash feature, and achieves the effect of hash feature measurement to represent images by fine-tuning the non-semantic bits in the hash feature; on the other hand, it can also make the non-semantic space with less information possible Obtain more meaningful supervision information from the semantic space to obtain a more uniform non-semantic hash distribution, avoiding the uneven distribution of non-semantic hashes, which will cause excessive concentration on certain features during retrieval, resulting in discrimination on these features. The problem of poor search results is not high.

This application also provides another application scenario, which is applicable to the above feature extraction model processing method. In this application scenario, the image feature extraction model and the target multi-label classification model are obtained through the same steps as the above application scenario. The image feature extraction model and the target multi-label classification model are spliced to obtain the target classification model. According to the target classification model, Implement multi-label classification of input images.

It should be understood that although the steps in the flowcharts involved in the above embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated in this article, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least some of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

Based on the same inventive concept, embodiments of the present application also provide a feature extraction model processing device for implementing the above-mentioned feature extraction model processing method and a sample retrieval device for implementing the above-mentioned sample retrieval method. . The problem-solving solutions provided by these devices are similar to the implementation solutions recorded in the above methods. Therefore, the specific limitations in one or more feature extraction model processing device embodiments and sample retrieval device embodiments provided below can be found in The above limitations on the feature extraction model processing method will not be repeated here.

In one embodiment, as shown in Figure 8, a feature extraction model processing device 800 is provided, including:

The feature extraction module 802 is used to extract features of the training samples through the feature extraction model to be trained to obtain training feature vectors;

The classification prediction module 804 is used to classify and predict training samples based on training feature vectors to obtain predicted semantic vectors;

The label vector acquisition module 806 is used to obtain the label semantic vector of the training category label corresponding to the training sample. The label semantic vector includes at least two activation label vector components;

The expected vector determination module 808 is used to determine the expected semantic vector of the training sample based on the label semantic vector, where the expected semantic vector contains the expected activation vector component corresponding to the position distribution of at least two activation label vector components;

The classification loss determination module 810 is used to determine the classification loss based on the difference between each expected activation vector component in the expected semantic vector and the predicted vector component at the corresponding position in the predicted semantic vector;

The model training module 812 is used to train the feature extraction model based on the classification loss. When the training stop condition is met, the target feature extraction model is obtained; the target feature extraction model is used to extract the sample feature vector of the input sample.

The above-mentioned feature extraction model processing device extracts features from training samples through the feature extraction model to be trained to obtain training feature vectors, classifies and predicts training samples based on the training feature vectors, obtains predicted semantic vectors, and obtains training category labels corresponding to the training samples. The label semantic vector of , the feature extraction model is trained through this classification loss, and the final target feature extraction model can learn features that tend to the label semantic vector, so that the semantic space where the label semantic vector is located can be simulated in the vector space where the training feature vector is located. The feature information representing semantics can be stored in the extracted feature vector, which improves the accuracy of feature extraction, and because the label semantic vector includes at least two activation label vector components, the training category label can be more accurately characterized, and the expected semantics The vector is determined based on the label semantic vector and contains the expected activation vector component corresponding to the position distribution of at least two activation label vector components, which can make the calculated classification loss more accurate and further improve the accuracy of feature extraction.

In one embodiment, the expected vector determination module is also used to form a feature component set by label vector components with the same sorting position in each label semantic vector; for a feature component set that contains an activation label vector component, sort the feature component set to which it belongs. The expected vector component at the position is set to the activation value; for the feature component set that does not contain the activation label vector component, the expected vector component at the sorting position to which the feature component set belongs is set to the inactive value; each expected vector component is sorted according to its corresponding The positions are combined to form the expected semantic vector of the training samples.

In one embodiment, the classification loss determination module is configured to, for each expected vector component in the expected semantic vector, determine the expected vector component based on the difference between the expected vector component and the predicted vector component corresponding to the sorted position in the predicted semantic vector. The initial value of the classification loss component at the sorting position; count the activation label vector components in the feature component set at the sorting position where the expected vector component is located, and determine the activation degree at the sorting position where the expected vector component is located based on the statistical results; based on the activation degree The initial value of the classification loss component is weighted to obtain the target value of the classification loss component; the target value of each classification loss component is counted to determine the classification loss.

In one embodiment, the above device further includes: a semantic constraint loss determination module, configured to determine the semantic constraint loss based on the difference between the training feature vector of the training sample and the expected semantic vector. The semantic constraint loss is used to determine the semantic constraint loss required by the feature extraction model. The extracted training feature vectors are semantically constrained; the classification loss determination module is also used to determine the target loss based on the semantic constraint loss and classification loss; the feature extraction model is trained based on the target loss, and when the training stop condition is met, the target feature extraction is obtained Model.

In one embodiment, the semantic constraint loss determination module is also used to calculate the vector distance between the training feature vector and the expected semantic vector if the dimension of the training feature vector is consistent with the dimension of the expected semantic vector, and divide the vector distance into As the semantic constraint loss corresponding to the training sample.

In one embodiment, the semantic constraint loss determination module is also configured to select from the training feature vector for each expected vector component in the expected semantic vector if the dimension of the training feature vector is greater than the dimension of the expected semantic vector. A training vector component, using the selected training vector component as the mapping vector component of the desired vector component; calculating the difference between the desired vector component and the mapping vector component to obtain the semantic constraint loss component corresponding to the desired vector component; counting each semantic constraint loss component to obtain the semantic constraint loss corresponding to the training sample.

In one embodiment, the semantic constraint loss determination module is also used to group the training vector components in the training feature vector according to the dimension of the desired semantic vector, and obtain multiple vectors whose number is consistent with the dimension of the desired semantic vector. Component group; for each expected vector component in the expected semantic vector, select a vector component group from multiple feature component groups as the target vector component group; select a training vector component from the target vector component group as the mapping corresponding to the expected vector component vector components.

In one embodiment, the training samples belong to the training sample set, and the training category labels corresponding to each training sample in the training sample set form a label set. The above device also includes a label vector generation module for constructing the target semantic space; the target of the target semantic space The vector dimension does not exceed the vector dimension of the training feature vector; each training category label in the label set is mapped to the target semantic space respectively, and the respective label semantic vector of each training category label is obtained.

In one embodiment, the label vector generation module is used to generate a target matrix whose matrix order matches the dimension of the target vector. The target matrix includes multiple candidate representation vectors, and the multiple candidate representation vectors form the target semantic space; the candidate representation vectors includes an equal number of first and second values; for each semantic category label in the semantic category label set, a candidate representation vector is selected from multiple candidate representation vectors as the label semantic vector corresponding to each semantic category label.

In one embodiment, the classification loss determination module is also used to obtain the comparison feature vector corresponding to the training sample, and obtain the feature extraction loss based on the difference between the training feature vector and the comparison feature vector; based on the feature extraction loss and classification Loss, the target loss is obtained; based on the target loss, the parameters of the feature extraction model are adjusted and training continues. When the training stop condition is met, the target feature extraction model is obtained.

In one embodiment, the contrast feature vector includes a positive contrast feature vector corresponding to the positive contrast training sample and a negative contrast feature vector corresponding to the negative contrast training sample; the classification loss determination module is also used to obtain the positive feature difference value , the positive feature difference value is the feature difference value between the training feature vector and the positive comparison feature vector; the negative feature difference value is obtained, and the negative feature difference value is the feature difference between the training feature vector and the negative comparison feature vector value; determine the feature extraction loss based on the positive feature difference value and the negative feature difference value.

In one embodiment, the feature extraction module is also used to extract the initial sample features of the training sample through the feature extraction model, and perform quantitative processing on the initial sample features to obtain the training feature vector of the training sample; the classification loss determination module is also used to The quantification target corresponding to each quantization value in the training feature vector is determined based on the preset symbolic function, and the quantization loss is determined based on the difference between each quantization value and the corresponding quantification target; based on the quantization loss and classification loss, the target loss is obtained; based on the target The loss adjusts the parameters of the feature extraction model and continues training. When the training stop condition is met, the target feature extraction model is obtained.

In one embodiment, as shown in Figure 9, a sample retrieval device 900 is provided, including:

Sample acquisition module 902, used to acquire query samples and candidate recall sample sets;

The feature extraction module 904 is used to input the query sample and the candidate recall sample in the candidate recall sample set into the target feature extraction model to obtain the query feature vector corresponding to the query sample and the candidate recall feature vector corresponding to the candidate recall sample; wherein, the target feature The extraction model is obtained by training the feature extraction model to be trained through classification loss. The classification loss is determined based on the difference between each expected activation vector component in the expected semantic vector and the predicted vector component at the corresponding position in the predicted semantic vector. The predicted semantic vector is obtained by classifying and predicting the training sample based on the training feature vector. The expected semantic vector is determined based on the label semantic vector of the training category label corresponding to the training sample. The label semantic vector includes at least two activation label vector components. The expected semantic vector The vector contains the expected activation vector component corresponding to the position distribution of at least two activation label vector components, and the training feature vector is obtained by feature extraction of the training sample through the feature extraction model to be trained;

The retrieval module 906 is configured to determine the target retrieval sample corresponding to the query sample from the candidate recall sample set based on the query feature vector and the candidate recall feature vector.

In the above sample retrieval device, since the target feature extraction model is trained through the classification loss, the final target feature extraction model can learn features that tend to the label semantic vector, so that the label semantic vector can be compared in the vector space where the training feature vector is located. The semantic space is simulated, so that the extracted feature vector can store feature information representing semantics, which improves the accuracy of feature extraction. And because the label semantic vector includes at least two activation label vector components, the training category labels can be more refined. Accurately characterized, and the expected semantic vector is determined based on the label semantic vector and contains the expected activation vector component corresponding to the position distribution of at least two activation label vector components, which can make the calculated classification loss more accurate and further improve the feature The accuracy of extraction, and then using the feature vector extracted by the target feature extraction model for sample retrieval, can improve the accuracy of sample retrieval.

In one embodiment, the above device further includes: an association relationship establishment module, used to perform feature clustering on the candidate recall feature vectors of each candidate recall sample in the candidate recall sample set to obtain multiple clusters; each cluster cluster There is a corresponding clustering center; for each clustering center, establish an association between the clustering center and each candidate recall feature vector in the same cluster; the retrieval module is used to based on the query feature vector and each clustering center determine the target clustering center from each clustering center; obtain each candidate recall feature vector that is associated with the target clustering center, based on the feature distance between the query feature vector and each obtained candidate recall feature vector , determine the target retrieval sample from each obtained candidate recall feature vector.

Each module in the above feature extraction model processing device and sample retrieval device can be implemented in whole or in part by software, hardware, and combinations thereof. Each of the above modules may be embedded in or independent of the processor of the computer device in the form of hardware, or may be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in Figure 10. The computer device includes a processor, a memory, an input/output interface (Input/Output, referred to as I/O), and a communication interface. Among them, the processor, memory and input/output interface are connected through the system bus, and the communication interface is connected to the system bus through the input/output interface. Wherein, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes non-volatile storage media and internal memory. The non-volatile storage medium stores operating systems, computer programs and databases. This internal memory provides an environment for the execution of operating systems and computer programs in non-volatile storage media. The database of the computer device is used to store training sample data. The input/output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with an external terminal through a network connection. The computer program, when executed by the processor, implements a feature extraction model processing method or a sample retrieval method.

In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in Figure 11. The computer device includes a processor, memory, input/output interface, communication interface, display unit and input device. Among them, the processor, memory and input/output interface are connected through the system bus, and the communication interface, display unit and input device are connected to the system bus through the input/output interface. Wherein, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes non-volatile storage media and internal memory. The non-volatile storage medium stores operating systems and computer programs. This internal memory provides an environment for the execution of operating systems and computer programs in non-volatile storage media. The input/output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used for wired or wireless communication with external terminals. The wireless mode can be implemented through WIFI, mobile cellular network, NFC (Near Field Communication) or other technologies. The computer program, when executed by the processor, implements a feature extraction model processing method or a sample retrieval method. The display unit of the computer device is used to form a visually visible picture and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a display screen. The touch layer covered above can also be buttons, trackballs or touch pads provided on the computer equipment shell, or it can also be an external keyboard, touch pad or mouse, etc.

Those skilled in the art can understand that the structures shown in Figures 10 and 11 are only block diagrams of partial structures related to the solution of the present application, and do not constitute a limitation on the computer equipment to which the solution of the present application is applied. Specifically, Computer equipment may include more or fewer components than shown in the figures, or some combinations of components, or have different arrangements of components.

In one embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, it implements the above feature extraction model method or sample retrieval steps.

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned feature extraction model method or sample retrieval steps are implemented.

In one embodiment, a computer program product is provided, including a computer program that implements the above feature extraction model method or sample retrieval steps when executed by a processor.

It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all It is information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of relevant countries and regions.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage. In the media, when executed, the computer program may include the processes of the above method embodiments. Any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetic variable memory (Magnetoresistive Random) Access Memory (MRAM), Ferroelectric Random Access Memory (FRAM), Phase Change Memory (PCM), graphene memory, etc. Volatile memory may include random access memory (Random Access Memory, RAM) or external cache memory. By way of illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM). The databases involved in the various embodiments provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include blockchain-based distributed databases, etc., but are not limited thereto. The processors involved in the various embodiments provided in this application may be general-purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to this.

The technical features of the above embodiments can be combined in any way. To simplify the description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, all possible combinations should be used. It is considered to be within the scope of this manual.

The above-described embodiments only express several implementation modes of the present application, and their descriptions are relatively specific and detailed, but should not be construed as limiting the patent scope of the present application. It should be noted that, for those of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all fall within the protection scope of the present application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims (19)

1. A feature extraction model processing method, characterized in that the method includes: Extract features from training samples through the feature extraction model to be trained to obtain training feature vectors; Classify and predict the training samples based on the training feature vectors to obtain predicted semantic vectors; Obtain the label semantic vector of the training category label corresponding to the training sample, where the label semantic vector includes at least two activation label vector components; Based on the label semantic vector, determine an expected semantic vector of the training sample, where the expected semantic vector includes an expected activation vector component corresponding to the position distribution of the at least two activation label vector components; Determine a classification loss based on the difference between each of the expected activation vector components in the expected semantic vector and the predicted vector components at respective corresponding positions in the predicted semantic vector; The feature extraction model is trained based on the classification loss, and when the training stop condition is met, a target feature extraction model is obtained; the target feature extraction model is used to extract the sample feature vector of the input sample. 2. The method according to claim 1, wherein the training sample corresponds to multiple training category labels, and determining the expected semantic vector of the training sample based on the label semantic vector includes: The label vector components with the same sorted position in each label semantic vector form a feature component set; For a feature component set that contains an activation label vector component, set the expected vector component at the sorted position to which the feature component set belongs as the activation value; For a feature component set that does not contain an activation label vector component, set the expected vector component at the sorting position to which the feature component set belongs to a non-activation value; Each expected vector component is combined according to its corresponding sorting position to form the expected semantic vector of the training sample. 3. The method according to claim 2, characterized in that the difference between each of the expected activation vector components based on the expected semantic vector and the predicted vector component of the corresponding position in the predicted semantic vector , determine the classification loss, including: For each expected vector component in the expected semantic vector, based on the difference between the expected vector component and the predicted vector component corresponding to the sorted position in the predicted semantic vector, determine the classification loss at the sorted position where the expected vector component is located. Component initial value; Perform statistics on the activation label vector components in the feature component set at the sorted position where the expected vector component is located, and determine the activation degree at the sorted position where the expected vector component is located based on the statistical results; Perform weighting processing on the initial value of the classification loss component based on the activation degree to obtain a target value of the classification loss component; Count the target values of each classification loss component to determine the classification loss. 4. The method according to claim 1, characterized in that, before training the feature extraction model based on the classification loss and obtaining the target feature extraction model when the training stop condition is met, the method further include: Determine a semantic constraint loss based on the difference between the training feature vector of the training sample and the expected semantic vector, where the semantic constraint loss is used to semantically constrain the training feature vector extracted by the feature extraction model; The feature extraction model is trained based on the classification loss. When the training stop condition is met, the target feature extraction model is obtained, including: determining a target loss based on the semantic constraint loss and the classification loss; The feature extraction model is trained based on the target loss, and when the training stop condition is met, the target feature extraction model is obtained. 5. The method of claim 4, wherein determining the semantic constraint loss based on the difference between the training feature vector of the training sample and the expected semantic vector includes: If the dimension of the training feature vector is consistent with the dimension of the expected semantic vector, then Calculate the vector distance between the training feature vector and the expected semantic vector, and use the vector distance as the semantic constraint loss corresponding to the training sample. 6. The method of claim 4, wherein determining the semantic constraint loss based on the difference between the training feature vector of the training sample and the expected semantic vector includes: If the dimension of the training feature vector is greater than the dimension of the expected semantic vector, then For each expected vector component in the expected semantic vector, select a training vector component from the training feature vector, and use the selected training vector component as the mapping vector component of the expected vector component; Calculate the difference between the expected vector component and the mapping vector component to obtain the semantic constraint loss component corresponding to the expected vector component; Each semantic constraint loss component is counted to obtain the semantic constraint loss corresponding to the training sample. 7. The method according to claim 6, characterized in that selecting a training vector component from the training feature vector and using the selected training vector component as the mapping vector component of the expected vector component includes: Grouping the training vector components in the training feature vector according to the dimension of the desired semantic vector to obtain a plurality of vector component groups whose number is consistent with the dimension of the desired semantic vector; For each expected vector component in the expected semantic vector, select a vector component group from the plurality of feature component groups as a target vector component group; Select a training vector component from the target vector component group as the mapping vector component corresponding to the desired vector component. 8. The method according to claim 1, wherein the training sample belongs to a training sample set, and the training category labels corresponding to each training sample in the training sample set form a label set, and the method further includes: Construct a target semantic space; the target vector dimension of the target semantic space does not exceed the vector dimension of the training feature vector; Each training category label in the label set is mapped to the target semantic space to obtain a label semantic vector of each training category label. 9. The method according to claim 8, characterized in that said constructing the target semantic space includes: Generate a target matrix whose matrix order matches the dimension of the target vector, the target matrix includes a plurality of candidate representation vectors, the plurality of candidate representation vectors form the target semantic space; the candidate representation vector includes a quantity equal first and second values; The step of mapping each training category label in the label set to the target semantic space to obtain the respective label semantic vector of each training category label includes: For each training category label in the label set, one candidate representation vector is selected from the plurality of candidate representation vectors as the label semantic vector corresponding to each training category label. 10. The method according to claim 1, characterized in that the feature extraction model is trained based on the classification loss, and when the training stop condition is met, the target feature extraction model is obtained, including: Obtain a comparison feature vector corresponding to the comparison training sample of the training sample, and obtain a feature extraction loss based on the difference between the training feature vector and the comparison feature vector; Based on the feature extraction loss and the classification loss, a target loss is obtained; Based on the target loss, parameters of the feature extraction model are adjusted and training is continued. When the training stop condition is met, the target feature extraction model is obtained. 11. The method according to claim 10, wherein the comparison feature vector includes a positive contrast feature vector corresponding to a positive contrast training sample and a negative contrast feature vector corresponding to a negative contrast training sample; The feature extraction loss obtained from the difference between the training feature vector and the comparison feature vector includes: Obtain a forward feature difference value, which is a feature difference value between the training feature vector and the forward comparison feature vector; Obtain a negative feature difference value, which is a feature difference value between the training feature vector and the negative comparison feature vector; The feature extraction loss is determined based on the positive feature difference value and the negative feature difference value. 12. The method according to claim 1, wherein the feature extraction of the training samples through the feature extraction model to obtain a training feature vector includes: Extract the initial sample features of the training sample through the feature extraction model, and perform quantification processing on the initial sample features to obtain the training feature vector of the training sample; The feature extraction model is trained based on the classification loss. When the training stop condition is met, the target feature extraction model is obtained, including: Determine the quantization target corresponding to each quantization value in the training feature vector based on the preset symbolic function, and determine the quantization loss based on the difference between each quantization value and the respective corresponding quantization target; Based on the quantification loss and the classification loss, a target loss is obtained; Based on the target loss, parameters of the feature extraction model are adjusted and training is continued. When the training stop condition is met, the target feature extraction model is obtained. 13. A sample retrieval method, characterized in that the method includes: Obtain query samples and candidate recall sample sets; Respectively input the query sample and the candidate recall sample in the candidate recall sample set into the target feature extraction model to obtain the query feature vector corresponding to the query sample and the candidate recall feature vector corresponding to the candidate recall sample; Wherein, the target feature extraction model is obtained by training the feature extraction model to be trained through classification loss. The classification loss is based on each expected activation vector component in the expected semantic vector and the prediction vector of the corresponding position in the predicted semantic vector. The difference between the components is determined, the predicted semantic vector is obtained by classifying and predicting the training sample based on the training feature vector, and the expected semantic vector is determined based on the label semantic vector of the training category label corresponding to the training sample, The label semantic vector includes at least two activation label vector components, the expected semantic vector includes an expected activation vector component corresponding to the position distribution of the at least two activation label vector components, and the training feature vector is obtained by The feature extraction model to be trained is obtained by extracting features from the training samples; Based on the query feature vector and the candidate recall feature vector, a target retrieval sample corresponding to the query sample is determined from the candidate recall sample set. 14. The method according to claim 13, characterized in that the method further comprises: Perform feature clustering on the candidate recall feature vectors of each candidate recall sample in the candidate recall sample set to obtain multiple clusters; each cluster cluster has a corresponding cluster center; For each clustering center, establish an association between the clustering center and each candidate recall feature vector in the same clustering cluster; Determining the target retrieval sample corresponding to the query sample from the candidate recall sample set based on the query feature vector and the candidate recall feature vector includes: Based on the feature distance between the query feature vector and each cluster center, determine the target cluster center from each of the cluster centers; Each candidate recall feature vector associated with the target clustering center is obtained, and the target retrieval sample is determined based on the feature distance between the query feature vector and each obtained candidate recall feature vector. 15. A feature extraction model processing device, characterized in that the device includes: The feature extraction module is used to extract features from training samples through the feature extraction model to be trained to obtain training feature vectors; A classification prediction module, used to perform classification prediction on the training sample based on the training feature vector to obtain a predicted semantic vector; A label vector acquisition module, configured to obtain a label semantic vector of a training category label corresponding to the training sample, where the label semantic vector includes at least two activation label vector components; An expected vector determination module, configured to determine an expected semantic vector of the training sample based on the label semantic vector, where the expected semantic vector includes an expected activation vector corresponding to the position distribution of the at least two activation label vector components. weight; weight A classification loss determination module, configured to determine the classification loss based on the difference between each of the expected activation vector components in the expected semantic vector and the predicted vector components at respective corresponding positions in the predicted semantic vector; A model training module is used to train the feature extraction model based on the classification loss. When the training stop condition is met, a target feature extraction model is obtained; the target feature extraction model is used to extract the sample feature vector of the input sample. 16. A sample retrieval device, characterized in that the device includes: The sample acquisition module is used to obtain query samples and candidate recall sample sets; The feature extraction module is used to input the query sample and the candidate recall sample in the candidate recall sample set into the target feature extraction model to obtain the query feature vector corresponding to the query sample and the candidate recall feature vector corresponding to the candidate recall sample; wherein , the target feature extraction model is obtained by training the feature extraction model to be trained through classification loss. The classification loss is based on each expected activation vector component in the expected semantic vector and the predicted vector component at the corresponding position in the predicted semantic vector. The difference between them is determined, the predicted semantic vector is obtained by classifying and predicting the training sample based on the training feature vector, and the expected semantic vector is determined based on the label semantic vector of the training category label corresponding to the training sample, so The label semantic vector includes at least two activation label vector components, the expected semantic vector includes an expected activation vector component corresponding to the position distribution of the at least two activation label vector components, and the training feature vector is obtained by The feature extraction model to be trained is obtained by extracting features from the training samples; A retrieval module, configured to determine a target retrieval sample corresponding to the query sample from the candidate recall sample set based on the query feature vector and the candidate recall feature vector. 17. A computer device, including a memory and a processor, the memory stores a computer program, characterized in that when the processor executes the computer program, the processor implements any one of claims 1 to 12 or 13 to 14. steps of the method described. 18. A computer-readable storage medium with a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 12 or 13 to 14 are implemented. . 19. A computer program product, comprising a computer program, characterized in that, when executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 12 or 13 to 14.