CN109740743A - Hierarchical neural network query recommendation method and device - Google Patents

Abstract

The invention discloses a hierarchical neural network query recommendation method and device. The hierarchical neural network query recommendation method includes: establishing a session layer neural network for modeling user short-term query records and a user layer neural network for modeling user long-term query records, wherein the current The state vector of the session layer neural network at the moment is used as the input of the user layer neural network at the current moment, and the state vector of the user layer neural network at the current moment is used as the input of the session layer neural network at the next moment; According to the session layer neural network and the user layer neural network, the query recommendation content is output for the query session. The solution provided by the present invention can improve the efficiency and accuracy of query recommendation.

Description

A kind of hierarchical neural network query recommendation method and device

technical field

The present invention relates to the technical field of computer networks, in particular to a method and device for querying and recommending a hierarchical neural network.

Background technique

In the prior art, there are various query recommendation methods. Query recommendation is that in the process of user information retrieval, when the user only enters part of the query keywords, the system can predict the user's query intention and recommend a set of query candidate words to the user for selection. Query recommendation can help users re-modify the query keywords they input, so as to find the content they need more accurately and improve user satisfaction. At present, traditional query recommendation methods are mostly based on feature mining, and the features used include co-occurrence and semantic similarity between queries. However, the mining of such features is often artificially defined and operated, and there may be some hidden features that have not been mined and discovered.

The query recommendation method based on RNN (Recurrent Neural Networks, Recurrent Neural Network) in the prior art can well model time series data, so as to predict the next possible query item. However, this method only focuses on the current short-term query session and does not consider the user's query records, so the efficiency and accuracy of query recommendation need to be improved.

SUMMARY OF THE INVENTION

In view of this, the purpose of the present invention is to propose a hierarchical neural network query recommendation method and device, which can improve the query recommendation efficiency and accuracy.

According to an aspect of the present invention, there is provided a hierarchical neural network query recommendation method, comprising:

Two neural networks are established, a session layer neural network for modeling the user's short-term query records and a user layer neural network for modeling the user's long-term query records, wherein the state of the session layer neural network at the current moment is calculated. The vector is used as the input of the user layer neural network at the current moment, and the state vector of the user layer neural network at the current moment is used as the input of the session layer neural network at the next moment;

According to the session layer neural network and the user layer neural network, the query recommendation content is output for the query session.

Preferably, the method further includes:

Combining the values of the state vectors in the session layer neural network with different weights to synthesize the state vectors of the new session layer neural network;

The outputting query recommendation content for the query session according to the session layer neural network and the user layer neural network, including:

According to the session layer neural network and the user layer neural network after updating the session layer neural network state vector, the query recommendation content is output for the query session.

Preferably, the state vector of the session layer neural network at the current moment is used as the input of the user layer neural network at the current moment, and the state vector of the user layer neural network at the current moment is used as the state vector of the user layer neural network at the next moment. The input of the session layer neural network, including:

The state vector of the session layer neural network at time t is used as the input of the user layer neural network at time t, the state vector of the user layer neural network at time t is used as the input of the session layer neural network at time t+1, and the session layer at time t+1 is used. The state vector of the neural network is used as the input of the user layer neural network at time t+1, where t is greater than or equal to 0.

Preferably, the state vector of the user layer neural network at time t is used as the input of the session layer neural network at time t+1, and the state vector of the session layer neural network at time t+1 is used as the input of the user layer neural network at time t+1. Hidden layers, including:

The state vector of the user layer neural network at time t is used to initialize the first hidden layer state vector of the session layer neural network at time t+1; the last hidden layer state vector of the session layer neural network at time t+1 is used as t +1 for the input of the user layer neural network at the moment.

Preferably, the state vector of the user layer neural network at time t is used to initialize the first hidden layer state vector of the session layer neural network at time t+1, including:

The state vector of the last hidden layer of the user layer neural network at time t is used to initialize the state vector of the first hidden layer of the session layer neural network at time t+1.

Preferably, the value of the state vector in the session layer neural network is combined with different weights to synthesize the state vector of the new session layer neural network, including:

The values of the state vectors of all hidden layers in the session layer neural network are combined with different weights to synthesize the state vectors of the new session layer neural network.

According to another aspect of the present invention, a hierarchical neural network query and recommendation device is provided, comprising:

A layered neural network establishment module is used to establish two neural networks, a session layer neural network and a user layer neural network, wherein the state vector of the session layer neural network at the current moment is used as the input of the user layer neural network at the current moment, Taking the state vector of the user layer neural network at the current moment as the input of the session layer neural network at the next moment;

A recommendation output module is configured to output query recommendation content for a query session according to the session layer neural network and the user layer neural network established by the layered neural network establishment module.

Preferably, the device further includes:

The attention mechanism processing module is used for synthesizing the state vector of the new session layer neural network with the values of the state vector in the session layer neural network established by the layered neural network establishment module with different weights;

The recommendation output module outputs the query recommendation content for the query session according to the session layer neural network and the user layer neural network after updating the session layer neural network state vector.

Preferably, the layered neural network establishment module uses the state vector of the session layer neural network at time t as the input of the user layer neural network at time t, and uses the state vector of the user layer neural network at time t as the session layer neural network at time t+1. For the input of the network, the state vector of the session layer neural network at time t+1 is used as the input of the user layer neural network at time t+1, where t is greater than or equal to 0.

Preferably, the attention mechanism processing module combines the values of the state vectors of all hidden layers in the session layer neural network with different weights to synthesize the state vectors of the new session layer neural network.

It can be found that the technical solutions of the embodiments of the present invention respectively establish two neural networks, a session layer neural network for modeling the user's short-term query records and a user layer neural network for modeling the user's long-term query records, The state vector of the session layer neural network at the current moment is used as the input of the user layer neural network at the current moment, and the state vector of the user layer neural network at the current moment is used as the session layer neural network at the next moment. When the query session is received, the query recommendation content can be output for the query session according to the session layer neural network and the user layer neural network. The session layer neural network can model the user's short-term query records, the user layer neural network can model the user's long-term query records, and the input and output of the session layer neural network and the user layer neural network can be established. connection relation. In this way, not only the current short-term query records, but also the user's long-term query records are considered, and the state vectors of the two neural networks of the session layer neural network and the user layer neural network are closely linked for comprehensive consideration, which improves the query recommendation. efficiency and accuracy.

Further, in this embodiment of the present invention, an attention mechanism may be introduced, and the values of the state vectors in the session layer neural network are combined with different weights to synthesize the state vectors of a new session layer neural network, so that the attention mechanism is used to improve the performance of the neural network. It captures the user's preference information.

Description of drawings

The above and other objects, features and advantages of the present disclosure will become more apparent from the more detailed description of the exemplary embodiments of the present disclosure taken in conjunction with the accompanying drawings, wherein the same reference numerals generally refer to the exemplary embodiments of the present disclosure. same parts.

FIG. 1 is a schematic flow chart of a method for query recommendation in a hierarchical neural network according to an embodiment of the present invention;

FIG. 2 is another schematic flowchart of a method for query recommendation in a hierarchical neural network according to an embodiment of the present invention;

3 is a schematic diagram of an NQS model according to an embodiment of the present invention;

4 is a schematic diagram of a HNQS model according to an embodiment of the present invention;

5 is a schematic diagram of an AHNQS model according to an embodiment of the present invention;

Fig. 6 is a kind of model detection effect diagram of different query session lengths under a Recall indicator according to an embodiment of the present invention;

7 is a diagram showing the detection effects of each model under different query session lengths under an MRR indicator according to an embodiment of the present invention;

FIG. 8 is a schematic block diagram of a hierarchical neural network query recommendation apparatus according to an embodiment of the present invention;

Fig. 9 is a schematic block diagram of a hierarchical neural network query and recommendation device according to an embodiment of the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

While preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

The invention provides a hierarchical neural network query recommendation method, in particular a hierarchical neural network query recommendation method based on an attention mechanism, which can improve the query recommendation efficiency and accuracy.

The technical solutions of the embodiments of the present invention are described in detail below with reference to the accompanying drawings.

FIG. 1 is a schematic flowchart of a method for query recommendation in a hierarchical neural network according to an embodiment of the present invention. The method can be applied to a hierarchical neural network query recommendation device.

Referring to Figure 1, the method includes:

In step 101, two neural networks are established, a session layer neural network for modeling the user's short-term query records and a user layer neural network for modeling the user's long-term query records, wherein the session at the current moment is The state vector of the layer neural network is used as the input of the user layer neural network at the current moment, and the state vector of the user layer neural network at the current moment is used as the input of the session layer neural network at the next moment.

In this step, the state vector of the session layer neural network at time t can be used as the input of the user layer neural network at time t, the state vector of the user layer neural network at time t can be used as the input of the session layer neural network at time t+1, and t The state vector of the session layer neural network at time +1 is used as the input of the user layer neural network at time t+1, where t is greater than or equal to 0.

Among them, the state vector of the user layer neural network at time t can be used to initialize the first hidden layer state vector of the session layer neural network at time t+1; the last hidden layer of the session layer neural network at time t+1 can be used. The state vector is used as the input of the user layer neural network at time t+1.

Wherein, the state vector of the last hidden layer of the user layer neural network at time t may be used to initialize the first hidden layer state vector of the session layer neural network at time t+1.

In step 102, according to the session layer neural network and the user layer neural network, output query recommendation content for the query session.

It can be found that the technical solutions of the embodiments of the present invention respectively establish two neural networks, a session layer neural network for modeling the user's short-term query records and a user layer neural network for modeling the user's long-term query records, The state vector of the session layer neural network at the current moment is used as the input of the user layer neural network at the current moment, and the state vector of the user layer neural network at the current moment is used as the session layer neural network at the next moment. When the query session is received, the query recommendation content can be output for the query session according to the session layer neural network and the user layer neural network. The session layer neural network can model the user's short-term query records, the user layer neural network can model the user's long-term query records, and the input and output of the session layer neural network and the user layer neural network can be established. connection relation. In this way, the solution of the present invention not only considers the current short-term query record, but also considers the user's long-term query record, and closely links the state vectors of the two neural networks of the session layer neural network and the user layer neural network for comprehensive consideration, improving the The efficiency and accuracy of query recommendation.

Fig. 2 is another schematic flowchart of a method for query recommendation in a hierarchical neural network according to an embodiment of the present invention. FIG. 2 describes the solution of the present invention in more detail with respect to FIG. 1 . The method can be applied to a hierarchical neural network query recommendation device.

The present invention proposes a hierarchical neural network query recommendation method based on an attention mechanism. In this method, two neural networks are included, one is called the session layer neural network (which can be referred to as the session layer for short), and the other is called the user layer neural network (which can be referred to as the user layer for short). Among them, the session layer neural network is mainly aimed at the user's short-term query records and can be used to model the user's short-term query records, and the user layer neural network is mainly aimed at the user's long-term query records and can be used to model the user's long-term query records. A general session layer neural network may include an input layer, a hidden layer, an output layer, etc.; the input layer is the encoded representation of each query in the query session; the state vector of the hidden layer is calculated by the input layer.

Among them, the state vector of the session layer neural network (the state vector used to represent the short-term query session) can be used as the input of the user layer neural network. Similarly, the state vector of the user layer neural network (used to represent the user's state vector) can be Used as the input of the session layer neural network. Specifically, the state vector of the session layer neural network at the current moment can be used as the input of the user layer neural network at the current moment, and then the state vector of the user layer neural network at the current moment can be used as the input of the session layer neural network at the next moment. For example, the state vector of the session layer neural network at time t can be used as the input of the user layer neural network at time t, and then the state vector of the user layer neural network at time t can be used as the input of the session layer neural network at time t+1. The state vector of the session layer neural network at time 1 is used as the input of the user layer neural network at time t+1. The state vector of the user layer neural network at time t can be used to initialize the first hidden layer state vector of the session layer neural network at time t+1; the last hidden layer state vector of the session layer neural network at time t+1 can be used to initialize As the input of the user layer neural network at time t+1. where t is greater than or equal to 0. Among them, the state vector is the value of the state variable at a certain moment, which is called the state of the system at the moment. The value of the state variable at time t=0 is called the initial state or initial state of the system, that is, it is also called the initial state vector or initial state vector. For example, the initial state of the system of the present invention can refer to the first The vector value of the hidden layer (that is, the state vector at time t=0).

In addition, the present invention also uses an attention mechanism to better capture the user's preference information. With the rise of deep learning, neural network based on attention mechanism has become a hotspot in recent neural network research. Attention mechanism is a strategy that was first proposed in the field of visual images. The idea of the attention mechanism is to increase the weight of useful information, so that the task processing system can focus more on finding useful information in the input data that is significantly related to the current output, thereby improving the quality of the output. The attention mechanism is also a resource allocation model, which simulates the work of the human brain and focuses more resources on important content. According to the research of the present invention, in a query session, different queries express different user intentions, and also have different weights when expressing user interests. For example, a query clicked by a user may better express the user's query intention. The present invention uses the attention mechanism to assign different weights to different query sessions, so that the hidden vectors of different query sessions can be combined with different weights to synthesize the state vector of a new query session.

According to the experiments conducted on AOL (America Online Log, search record data set), the experimental results show that the method of the present invention is better than the existing query recommendation method based on RNN (Recurrent Neural Network), in MRR (Mean Reciprocal Rank, average inverse ranking, It is to take the reciprocal of the ranking of the standard answer in the results given by the evaluated system as its accuracy, and then take the average of all questions) and Recall (recall rate) indicators have increased by 21.86% and 22.99%, especially for short query session, the effect is more obvious.

Referring to Figure 2, the method includes:

In step 201, two neural networks, a session layer neural network and a user layer neural network are established, wherein the state vector of the session layer neural network at the current moment is used as the input of the user layer neural network at the current moment, and the user layer neural network at the current moment is used as the input of the user layer neural network at the current moment. The state vector of is used as the input of the session layer neural network at the next moment.

The query recommendation method based on RNN (Recurrent Neural Networks, Recurrent Neural Network) in the prior art is mainly a query recommendation method based on the session layer neural network. The general session layer neural network can include input layer, hidden layer, output layer, etc.; the input layer is the coded representation of each query in the query session; the state vector of the hidden layer is calculated by the input layer, and the calculation process is as formula (1) The output layer is calculated by the hidden layer, and the calculation process is shown in formula (5). The session may refer to a sequence containing queries over a period of time, such as query 1, query 2, ..., query n, and the session layer refers to a neural network. Generally speaking, if the query time of the two query words before and after does not exceed a set time, such as 30 minutes, it can be considered that they are in the same query session.

Referring to FIG. 3 , it is a schematic diagram of an NQS (Neural Query Suggestion, neural network-based query recommendation) model according to an embodiment of the present invention. The session layer neural network includes an input layer, a hidden layer, and an output layer from bottom to top. Among them, suppose a query session has query, expressed as a 1-of-N encoding for each query session in order to generate the state vector input to the RNN. The state vector of the hidden layer can be calculated by formula (1):

in:

in, represents the state vector of the hidden layer, represents the state vector of the previous hidden layer, To update the weights, is the activation state vector, is the forgetting weight, represents the nth input query, means from the weight of, means from the weight of, represents the sigmod activation function. The Sigmoid activation function is also known as the S-shaped growth curve. In information science, due to its single increase and inverse function single increase, the sigmoid function is often used as the threshold function of the neural network to map variables between 0 and 1. . where tanh( ) is the hyperbolic tangent function.

Among them, RNN_session and RNN_user can be used to represent the GRU (Gated Recurrent Unit, gated recurrent unit) function respectively, and the state vector of the session layer neural network can be expressed as . The output of this neural network is the predicted score for the next query recommendation:

in Indicates the value of the output layer in the session layer neural network, indicating that the output is the score of the next query recommendation item. Among them, g() represents the tanh activation function, and hn, t represents the state vector of the nth hidden layer of the session layer neural network at time t.

Among them, the loss function of pairwise comparison can be selected, because it can make the recommended items with higher scores and more likely to be accurate queries to the front of the list, thereby improving user satisfaction. Generally, in the recommendation system, the loss function of cross entropy or TOP1 is mostly used, and the effect of TOP1 is better. Therefore, the following loss function Loss can be used:

in, represents the number of negative samples, represent the score of negative samples and the score of correct values, respectively.

Compared with the RNN-based query recommendation method in the prior art, the present invention proposes a user-session RNN query recommendation method.

The present invention can use the state vector of the last hidden layer of each session layer neural network at the current moment as the input of the user layer neural network at the current moment; the state vector of the last hidden layer in the user layer neural network at the current moment can also be used to initialize the state vector of the first hidden layer of the session layer neural network at the next moment. For example, the state vector of the last hidden layer of the user layer neural network at time t is used to initialize the state vector of the first hidden layer of the session layer neural network at time t+1.

Among them, the state vector of the hidden layer in the user layer neural network is calculated, and the calculation process is shown in formula (7); the calculation of the state vector of the first hidden layer of the session layer neural network is shown in formula (8).

The query recommendation method based on session RNN in the prior art only models the user's short-term query records, and does not model the user's long-term retrieval records. Therefore, the present invention proposes a hierarchical NQS method, namely HNQS (Hierarchical Neural Query Suggestion, a query recommendation model based on hierarchical neural networks). Referring to FIG. 4, it is a schematic diagram of an HNQS model according to an embodiment of the present invention. In Figure 4, the bottom is the session-level neural network (Session-level RNN), and the top is the user-level neural network (User-level RNN). In Figure 4, there are two neural networks, the session layer neural network and the user layer neural network. The lower part of the figure includes the session layer neural network at t, t+1 and t+2, and the upper part includes the user layer at t and t+1. Neural Networks. Among them, the state vector of the session layer neural network at time t is used as the input of the user layer neural network at time t, and then the state vector of the user layer neural network at time t is used as the input of the session layer neural network at time t+1, and t+1 The state vector of the session layer neural network at time is used as the input of the user layer neural network at time t+1. Among them, the state vector of the user layer neural network at time t can be used to initialize the first hidden layer state vector of the session layer neural network at time t+1; the last hidden layer of the session layer neural network at time t+1 can be used. The state vector is used as the input of the user layer neural network at time t+1; in the same way, the state vector of the session layer neural network at time t+1 is used as the input of the user layer neural network at time t+1, and then the user layer at time t+1 is used. The state vector of the neural network is used as the input of the session layer neural network at time t+2, and so on.

For the session layer neural network, the query words in the query session are encoded into the session layer neural network as the input of the neural network for calculation. For the user layer neural network, the last hidden layer vector of the session layer neural network at time t, t+1 and t+2 is used as the input, and the output is the last hidden layer vector of the user layer neural network. That is, the state vector of the user layer neural network. Regarding the connection between the session layer neural network and the user layer neural network, the state vector of the session layer neural network at time t can be used as the input of the user layer neural network at time t, and then the state vector of the user layer neural network at time t can be used as t+ 1 is the input of the session layer neural network.

Referring to Figure 4, for the query session at time t+1, the modeling process of the two neural networks can be as follows: use the state vector Ut of the user layer neural network at time t (that is, the last hidden layer vector ht of the user layer neural network at time t) , u=Ut) and the first query word q1 in the query session at time t+1, the code of t+1 is used as input, the first hidden layer vector in the session layer neural network is calculated, and then combined with the query session in turn Query word encoding, calculate the other hidden layer vectors behind the first hidden layer vector; until the calculation is performed in combination with the last query word of the query session, at this time, it corresponds to the last hidden layer vector of the session layer neural network, which can be called a session. The state vector St+1 of the layer neural network, and then this state vector St+1 is input into the user layer neural network at time t+1. Combined with the state vector Ut of the user layer neural network at time t (that is, the last hidden layer vector ht, u=Ut of the user layer neural network at time t), the calculation produces the user layer neural network state vector Ut+1 at time t+1 (ie t+ At time 1, the last hidden layer vector of the user layer neural network ht+1, u=Ut+1). In this way, the input of a query session and the processing modeling process of two neural networks are completed, and the query session received later is also the same processing process.

Among them, the input of the user layer neural network is the state vector of the session layer neural network, and the update of the user layer neural network is:

represents the value of the state vector of the nth hidden layer in the user layer neural network, Represents the state vector value of the nth session layer neural network. Among them, u represents the user u, and RNNuser( ) represents the user layer neural network.

Among them, the present invention uses the state vector of the user layer neural network to input the session layer neural network, which can be used to introduce the user's long-term retrieval preference information:

in, represents the state vector value of the first hidden layer of the session layer neural network at time t+1, represents the state vector of the user layer neural network, W represents the weight vector, represents the bias vector.

In step 202, the values of the state vectors of all hidden layers in the session layer neural network are combined with different weights to synthesize the state vectors of the new session layer neural network.

The present invention further establishes the HNQS model based on the attention mechanism. The HNQS model based on the attention mechanism proposed by the present invention, namely AHNQS (Attention-based Hierarchical Neural Query Suggestion, the query recommendation model based on the attention mechanism and the hierarchical neural network), the state of all hidden layers in the session layer neural network The values of the vector are grouped with different weights to synthesize the state vector of the new session layer neural network, which is then used as the input of the user layer neural network.

In the solution of the present invention, it is considered that in a query session, different queries express different user intentions, and also have different weights when expressing user interests. For example, a query clicked by a user may better express the user's query intention. The present invention can assign different weights to different query sessions in the session layer neural network through the attention mechanism, so that the state vectors of all hidden layers in the session layer neural network can be combined with different weights to synthesize the state vectors of a new session layer neural network . Referring to FIG. 5, it is a schematic diagram of an AHNQS model according to an embodiment of the present invention. In Figure 5, the lower part is the session level neural network (Session level RNN), and the upper part is the user layer neural network (User-level RNN), in which the session level neural network (Session level) introduces user attention (User Attention).

As shown in Figure 5:

In the user layer neural network, the user state information is updated as follows:

in:

in, represents the state vector value of the session layer neural network at time t, which is calculated by formula (10), Represents the state value of the jth hidden layer of the session layer neural network at time t. represents the weight value of the jth hidden layer of the session layer neural network at time t, is an intermediate parameter. In this way, the parameters in it can be continuously updated through the loss function. Among them, ht, u represents the t-th hidden layer state vector of user u in the user layer neural network, u represents user u, h ^T t-1, u represents the hidden layer state vector of user u at time t-1, hj, t Represents the jth hidden layer state vector value of the session layer neural network at time t, and exp ( ) represents the exponential function. The present invention passes the weight value Combine the state vectors of all hidden layers in the session layer neural network into the state vector of the new session layer neural network.

In step 203, according to the session layer neural network and the user layer neural network after updating the session layer neural network state vector, the query recommendation content is output for the query session.

After assigning different weights to the value of the state vector of all hidden layers in the session layer neural network and synthesizing the state vector of a new session layer neural network with different weights, it is re-used as the input of the user layer neural network, so that the session layer neural network The state vectors of the network and the user layer neural network will change accordingly.

In this step, according to the session layer neural network and the user layer neural network after updating the session layer neural network state vector, the query recommendation content is output for the query session, so that more accurate query recommendation content can be obtained.

The solution of the present invention checks the HNQS model and the AHNQS model through experiments to determine whether the two can improve the effect of query recommendation, that is, whether the layering mechanism and the attention mechanism are helpful to the effect of query recommendation. In addition, as the length of the query session changes, the recommendation effect sometimes changes.

In combination with the foregoing, the query recommendation model described in the present invention includes: (1) ADJ (the original co-occurrence based ranking, original co-occurrence based query recommendation) model; (2) NQS model; (3) HNQS model; (4) AHNQS model.

The present invention shows the data set and specific parameter settings in Table 1 and Table 2.

Table 1

Model batch size rejection rate learning rate momentum NQS 50 0.5 0.01 0.0 HNQS 50 0.1 0.1 0.0 AHNQS 50 0.1 0.1 0.0

Table 2

The present invention uses MRR and Recall indicators to measure the effects of all models. According to the experimental results, the following conclusions are drawn:

1) Overall performance

The present invention displays the effects of all models in Table 3, wherein the data significance test is performed on the best model effects, as shown in Table 3 below:

Model Recall@10 MRR@10 ADJ <u>.7072</u> <u>.6922</u> NQS .6444 .6148 HNQS .8138^▲ .7874^▲ AHNQS .8618^▲ .8514^▲

table 3

According to the above experimental results, it can be seen that AHNQS has the largest value, which also represents the best result is AHNQS, followed by HNQS and ADJ, and the worst is NQS. Among them, AHNQS is 21.86% higher than ADJ in terms of MRR and Recall indicators. 22.99%, and HNQS is only 5.9% and 8.13% higher, respectively, which further shows that the attention mechanism of the present invention is helpful for improving the effect of query recommendation.

In order to further study the influence of the session length on the method of the present invention, the present invention divides the length of the query session into short (short), medium (medium), and long (long), and detects different model methods at different query session lengths, The results are shown in FIGS. 6 and 7 . Wherein, Fig. 6 is a model detection effect diagram of different query session lengths under a Recall indicator according to an embodiment of the present invention; Fig. 7 is a graph of different query session lengths under an MRR indicator according to an embodiment of the present invention The results of each model detection. In Figures 6 and 7, for each session length case of short, medium, and long, the four columns correspond to ADJ, NQS, HNQS, and AHNQS, respectively.

It can be seen that in each length, the value of AHNQS is the highest, and the representative effect is the best, followed by HNQS, but in the MRR index, the improvement of AHNQS is more than that of HNQS than the Recall index, which is further verified. The attention mechanism proposed by the method of the present invention has obvious help effect on the accuracy of query recommendation.

The above describes in detail the hierarchical neural network query recommendation method based on the attention mechanism of the present invention, and the corresponding query recommendation apparatus and equipment of the present invention are described below.

FIG. 8 is a schematic block diagram of a hierarchical neural network query recommendation apparatus according to an embodiment of the present invention.

Referring to FIG. 8 , a hierarchical neural network query and recommendation apparatus 80 includes: a hierarchical neural network establishment module 81 , a recommendation output module 82 , and an attention mechanism processing module 83 .

The hierarchical neural network establishment module 81 is used to establish two neural networks, a session layer neural network for modeling the user's short-term query records and a user layer neural network for modeling the user's long-term query records, wherein the current The state vector of the session layer neural network at the current moment is used as the input of the user layer neural network at the current moment, and the state vector of the user layer neural network at the current moment is used as the input of the session layer neural network at the next moment.

The recommendation output module 82 is configured to output the query recommendation content for the query session according to the session layer neural network and the user layer neural network established by the layered neural network establishment module 81 .

The attention mechanism processing module 83 is used to combine the values of the state vector in the session layer neural network established by the layered neural network establishment module 81 with different weights to synthesize the state vector of the new session layer neural network; the recommendation The output module 82 outputs the query recommendation content for the query session according to the session layer neural network and the user layer neural network after updating the session layer neural network state vector.

The layered neural network establishment module 81 can use the state vector of the session layer neural network at time t as the input of the user layer neural network at time t, and use the state vector of the user layer neural network at time t as the session layer at time t+1 For the input of the neural network, the state vector of the session layer neural network at time t+1 is used as the input of the user layer neural network at time t+1, where t is greater than or equal to 0.

Among them, the state vector of the user layer neural network at time t can be used to initialize the first hidden layer state vector of the session layer neural network at time t+1; the last hidden layer of the session layer neural network at time t+1 can be used. The state vector is used as the input of the user layer neural network at time t+1.

Among them, the state vector of the last hidden layer of the user layer neural network at time t can be used to initialize the first hidden layer state vector of the session layer neural network at time t+1

The attention mechanism processing module 83 may combine the values of the state vectors of all hidden layers in the session layer neural network with different weights to synthesize the state vectors of the new session layer neural network.

Fig. 9 is a schematic block diagram of a hierarchical neural network query and recommendation device according to an embodiment of the present invention.

Referring to FIG. 9 , a hierarchical neural network query and recommendation device 90 includes: a processor 91 and a memory 92 .

The processor 91 establishes two neural networks, a session layer neural network for modeling the user's short-term query records and a user layer neural network for modeling the user's long-term query records, wherein the session layer at the current moment is used for modeling. The state vector of the neural network is used as the input of the user layer neural network at the current moment, and the state vector of the user layer neural network at the current moment is used as the input of the session layer neural network at the next moment; The network and the user layer neural network output query recommendation content for the query session.

The memory 92 stores the query recommendation content output by the processor 91 .

Embodiments of the present invention further provide a non-transitory machine-readable storage medium, on which executable codes are stored, and when the executable codes are executed by a processor of an electronic device, the processor is caused to execute the following method:

Two neural networks are established, a session layer neural network for modeling the user's short-term query records and a user layer neural network for modeling the user's long-term query records, wherein the state of the session layer neural network at the current moment is calculated. The vector is used as the input of the user layer neural network at the current moment, and the state vector of the user layer neural network at the current moment is used as the input of the session layer neural network at the next moment;

According to the session layer neural network and the user layer neural network, the query recommendation content is output for the query session.

To sum up, the present invention provides a hierarchical neural network query recommendation method based on an attention mechanism, wherein the hierarchical mechanism includes a neural network of a session layer neural network and a neural network of a user layer neural network. Neural networks can be used to model short-term query records of users, and user-layer neural networks can be used to model long-term queries of users. The attention mechanism proposed by the present invention can better capture the user's preference information. The present invention has carried out experiments on AOL data set, and the experimental results show that the method of the present invention is better than the existing query recommendation method based on neural network, and the MRR index and Recall index are improved by 21.86% and 22.99% respectively, especially for short Query session, the effect is more obvious.

The technical solution according to the present invention has been described in detail above with reference to the accompanying drawings.

Furthermore, the method according to the invention can also be implemented as a computer program or computer program product comprising computer program code instructions for carrying out the above-mentioned steps defined in the above-mentioned method of the invention.

Alternatively, the present invention can also be implemented as a non-transitory machine-readable storage medium (or computer-readable storage medium, or machine-readable storage medium) on which executable code (or computer program, or computer instruction code is stored) ), when the executable code (or computer program, or computer instruction code) is executed by the processor of the electronic device (or computing device, server, etc.), the processor is caused to perform the various steps of the above method according to the present invention .

Those skilled in the art will also appreciate that the various exemplary logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both.

Those of ordinary skill in the art should understand: the above are only specific embodiments of the present invention, and are not intended to limit the present invention. Any modification, equivalent replacement, improvement made within the spirit and principle of the present invention etc., should be included within the protection scope of the present invention.

Claims (10)

1. a hierarchical neural network query recommendation method is characterized in that, comprising: Two neural networks are established, a session layer neural network for modeling the user's short-term query records and a user layer neural network for modeling the user's long-term query records, wherein the state of the session layer neural network at the current moment is calculated. The vector is used as the input of the user layer neural network at the current moment, and the state vector of the user layer neural network at the current moment is used as the input of the session layer neural network at the next moment; According to the session layer neural network and the user layer neural network, the query recommendation content is output for the query session. 2. The method according to claim 1, wherein the method further comprises: Combining the values of the state vectors in the session layer neural network with different weights to synthesize the state vectors of the new session layer neural network; The outputting query recommendation content for the query session according to the session layer neural network and the user layer neural network, including: According to the session layer neural network and the user layer neural network after updating the session layer neural network state vector, the query recommendation content is output for the query session. 3. The method according to claim 1, wherein the state vector of the session layer neural network at the current moment is used as the input of the user layer neural network at the current moment, and the user layer neural network at the current moment is used. The state vector of the neural network is used as the input of the session layer neural network at the next moment, including: The state vector of the session layer neural network at time t is used as the input of the user layer neural network at time t, the state vector of the user layer neural network at time t is used as the input of the session layer neural network at time t+1, and the session layer at time t+1 is used. The state vector of the neural network is used as the input of the user layer neural network at time t+1, where t is greater than or equal to 0. 4. The method according to claim 3, wherein the state vector of the user layer neural network at time t is used as the input of the session layer neural network at time t+1, and the state vector of the session layer neural network at time t+1 is used. The vector is used as the input of the user layer neural network at time t+1, including: The state vector of the user layer neural network at time t is used to initialize the first hidden layer state vector of the session layer neural network at time t+1; the last hidden layer state vector of the session layer neural network at time t+1 is used as t +1 for the input of the user layer neural network at the moment. 5. The method according to claim 4, wherein the state vector of the user layer neural network at time t is used to initialize the first hidden layer state vector of the session layer neural network at time t+1, comprising: The state vector of the last hidden layer of the user layer neural network at time t is used to initialize the state vector of the first hidden layer of the session layer neural network at time t+1. 6. The method according to claim 2, wherein the state vector of the new session layer neural network is synthesized by combining the values of the state vectors in the session layer neural network with different weights, comprising: The values of the state vectors of all hidden layers in the session layer neural network are combined with different weights to synthesize the state vectors of the new session layer neural network. 7. A hierarchical neural network query and recommendation device, characterized in that, comprising: A hierarchical neural network building module is used to establish two neural networks, a session layer neural network for modeling the user's short-term query records and a user layer neural network for modeling the user's long-term query records. The state vector of the described session layer neural network is used as the input of the user layer neural network at the current moment, and the state vector of the user layer neural network at the current moment is used as the input of the session layer neural network at the next moment; A recommendation output module is configured to output query recommendation content for a query session according to the session layer neural network and the user layer neural network established by the layered neural network establishment module. 8. The apparatus according to claim 7, wherein the apparatus further comprises: The attention mechanism processing module is used for synthesizing the state vector of the new session layer neural network with the values of the state vector in the session layer neural network established by the layered neural network establishment module with different weights; The recommendation output module outputs the query recommendation content for the query session according to the session layer neural network and the user layer neural network after updating the session layer neural network state vector. 9. The device according to claim 7, wherein: The layered neural network building module uses the state vector of the session layer neural network at time t as the input of the user layer neural network at time t, and uses the state vector of the user layer neural network at time t as the input of the session layer neural network at time t+1. , take the state vector of the session layer neural network at time t+1 as the input of the user layer neural network at time t+1, where t is greater than or equal to 0. 10. The device of claim 8, wherein: The attention mechanism processing module combines the values of state vectors of all hidden layers in the session layer neural network with different weights to synthesize the state vectors of a new session layer neural network.