CN114282528A

CN114282528A - Keyword extraction method, device, equipment and storage medium

Info

Publication number: CN114282528A
Application number: CN202110961476.1A
Authority: CN
Inventors: 黄剑辉
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2021-08-20
Filing date: 2021-08-20
Publication date: 2022-04-05

Abstract

The embodiment of the application provides a keyword extraction method, a keyword extraction device, equipment and a storage medium, which relate to the technical field of artificial intelligence, and the method comprises the following steps: and extracting the characteristics of the target text to obtain a text characteristic vector corresponding to the target text and a word segmentation grammar vector of the target word included in the target text. And extracting the characteristics of the target participle to obtain a participle semantic vector of the target participle, and then splicing the participle semantic vector and the participle grammar vector to obtain a first combined vector. Because the first combined vector contains the semantic information and the grammatical information of the target word, the text feature vector of the target text and the first combined vector are fused to obtain a fused feature vector, and the core components in the target text can be better represented. And determining the weight value of the target participle in the target text based on the fusion characteristic vector, and effectively improving the accuracy of extracting the keyword in the target text when determining whether the target participle is the keyword in the target text based on the weight value.

Description

Keyword extraction method, device, equipment and storage medium

Technical Field

The embodiment of the invention relates to the technical field of artificial intelligence, in particular to a keyword extraction method, a keyword extraction device, keyword extraction equipment and a storage medium.

Background

With the development of information technology, a large amount of data is constantly generated on the internet, and in the face of the large amount of data, it is difficult for users to quickly find important and critical contents from the data, so that term-weights tasks (term-weights) are developed at the same time, and refer to a main mode of extracting core components in texts and eliminating the influence of redundant components.

At present, word frequency statistics is mostly performed on the word weight task based on global corpus to extract keywords in a text, such as term frequency-inverse text frequency index (term frequency-inverse document frequency). However, when the keywords in the text are extracted based on the statistical method, the accuracy is low.

Disclosure of Invention

The embodiment of the application provides a keyword extraction method, a keyword extraction device, equipment and a storage medium, which are used for improving the accuracy of extracting keywords in a text.

In one aspect, an embodiment of the present application provides a keyword extraction method, including:

performing feature extraction on a target text to obtain a text feature vector corresponding to the target text and a participle grammar vector corresponding to a target participle contained in the target text, and performing feature extraction on the target participle to obtain a participle semantic vector corresponding to the target participle;

splicing the word segmentation semantic vector and the word segmentation grammar vector to obtain a first combination vector corresponding to the target word segmentation; fusing the text feature vector corresponding to the target text with the first combined vector to obtain a fused feature vector;

determining a weight value of the target participle in the target text based on the fusion feature vector, wherein the weight value is used for representing the influence degree of the target participle on the semantic understanding of the target text;

determining whether the target participle is a keyword in the target text or not based on the weight value of the target participle in the target text.

In one aspect, an embodiment of the present application provides a keyword extraction apparatus, where the apparatus includes:

the feature extraction module is used for extracting features of a target text, obtaining a text feature vector corresponding to the target text and a participle grammar vector corresponding to a target participle contained in the target text, and extracting the features of the target participle to obtain a participle semantic vector corresponding to the target participle;

the splicing module is used for splicing the word segmentation semantic vector and the word segmentation grammar vector to obtain a first combination vector corresponding to the target word segmentation;

the fusion module is used for fusing the text feature vector corresponding to the target text with the first combined vector to obtain a fusion feature vector;

the prediction module is used for determining a weight value of the target participle in the target text based on the fusion feature vector, wherein the weight value is used for representing the influence degree of the target participle on semantic understanding of the target text;

and the judging module is used for determining whether the target participle is a keyword in the target text or not based on the weight value of the target participle in the target text.

Optionally, the feature extraction module is specifically configured to:

respectively extracting a participle grammar vector, a position vector and a segmentation vector corresponding to each participle in the target text; each participle corresponds to a participle grammar vector, a position vector and a segmentation vector, each participle grammar vector is used for representing grammar information of a corresponding participle in the target text, each position vector is used for representing a relative position relation between the corresponding participle and other participles in the target text, and each segmentation vector is used for representing a statement type of a statement to which the corresponding participle belongs;

respectively obtaining a second combination vector corresponding to the corresponding participle based on the respective corresponding participle grammar vector, the position vector and the segmentation vector of each participle;

extracting features of the obtained second combined vectors to obtain text feature vectors corresponding to the target text;

and acquiring a participle grammar vector corresponding to the target participle from the respective corresponding participle grammar vector of each participle.

Optionally, the feature extraction module is specifically configured to:

respectively aiming at the participles, the following operations are executed: and superposing the participle grammar vector, the position vector and the segmentation vector corresponding to one participle to obtain a second combination vector corresponding to the participle.

Optionally, the feature extraction module is specifically configured to:

obtaining attention weight vectors corresponding to the participles respectively according to the second combination vectors and the corresponding attention weight matrixes, wherein each value contained in the attention weight vector corresponding to one participle respectively represents the attention weight of each participle relative to the participle;

and obtaining a text feature vector corresponding to the target text according to the attention weight vector corresponding to each participle and each second combination vector, wherein the text feature vector comprises the participle feature vector corresponding to each participle, and each participle feature vector is obtained by performing weighted summation with the corresponding second combination vector according to each attention weight in the corresponding attention weight vector.

Optionally, the feature extraction module is specifically configured to:

obtaining at least one attention vector corresponding to each participle according to each second combination vector and the corresponding attention weight matrix, wherein the at least one attention vector comprises a request vector and a key vector;

and acquiring an attention weight vector corresponding to each participle based on at least one attention vector corresponding to each participle, wherein the attention weight of each participle relative to the participle is the similarity between the key vector corresponding to each participle and the request vector of the participle.

Optionally, the fusion module is specifically configured to:

and performing dot multiplication on the text feature vector corresponding to the target text and the first combined vector corresponding to the target word segmentation to obtain a corresponding fusion feature vector.

Optionally, the determining module is specifically configured to:

if the weight value corresponding to the target word segmentation is larger than or equal to a preset threshold value, determining the target word segmentation as a keyword in the target text;

and if the weight value corresponding to the target word segmentation is smaller than the preset threshold value, determining that the target word segmentation is not the keyword in the target text.

Optionally, the system further comprises a keyword matching module;

the keyword matching module is specifically configured to:

determining whether the target participle is a keyword in the target text or not based on the weight value of the target participle in the target text, and then acquiring a target keyword to be matched;

performing keyword matching on the target keyword and each video title in a video title library to obtain at least one candidate video title;

sorting the at least one candidate video title according to the weight value of each keyword in the at least one candidate video title;

and determining the matched video title of the target keyword according to the sequencing result.

In one aspect, an embodiment of the present application provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the keyword extraction method when executing the program.

In one aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program executable by a computer device, and when the program runs on the computer device, the computer device is caused to execute the steps of the keyword extraction method.

In the embodiment of the application, the participle semantic vector and the participle grammar vector corresponding to the target participle included in the target text are spliced to obtain the first combination vector corresponding to the target participle, and because the semantic information and the grammar information of the target participle are included in the first combination vector, the text feature vector corresponding to the target text and the first combination vector are fused to obtain the fusion feature vector, so that the core components in the target text can be well represented. And determining the weight value of the target word in the target text based on the fusion characteristic vector, and effectively improving the accuracy of extracting the keywords in the target text when determining whether the target word is the keyword in the target text based on the weight value in the target text.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.

FIG. 1 is a diagram illustrating a system architecture suitable for use in embodiments of the present application;

FIG. 2 is a schematic diagram of a search result interface of a search application suitable for use in embodiments of the present application;

FIG. 3 is a schematic diagram of a network structure of a word weight model according to an embodiment of the present application;

FIG. 4 is a schematic flow chart illustrating a word weight model training method according to an embodiment of the present disclosure;

fig. 5 is a schematic flowchart of a keyword extraction method applicable to the embodiment of the present application;

FIG. 6 is a flowchart illustrating a method for predicting a weight value of a word segmentation in accordance with an embodiment of the present disclosure;

FIG. 7 is a flowchart illustrating a method for predicting a weight value of a word segmentation algorithm according to an embodiment of the present disclosure;

FIG. 8 is a diagram illustrating a method for obtaining a second combined vector according to an embodiment of the present disclosure;

FIG. 9 is a diagram illustrating a method for obtaining a second combined vector according to an embodiment of the present disclosure;

FIG. 10 is a diagram illustrating a method for obtaining a second combined vector according to an embodiment of the present disclosure;

FIG. 11 is a diagram illustrating a method for obtaining a second combined vector according to an embodiment of the present disclosure;

FIG. 12 is a search results interface diagram of a video application suitable for use in accordance with an embodiment of the present application;

fig. 13 is a schematic structural diagram of a keyword extraction apparatus applicable to the embodiment of the present application;

fig. 14 is a schematic structural diagram of a computer device to which the embodiment of the present application is applied.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more clearly apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

For convenience of understanding, terms referred to in the embodiments of the present invention are explained below.

Artificial Intelligence (AI) is a theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human Intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a manner similar to human intelligence. Artificial intelligence is the research of the design principle and the realization method of various intelligent machines, so that the machines have the functions of perception, reasoning and decision making.

The artificial intelligence technology is a comprehensive subject and relates to the field of extensive technology, namely the technology of a hardware level and the technology of a software level. The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like.

Natural Language Processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that enable efficient communication between humans and computers using natural language. Natural language processing is a science integrating linguistics, computer science and mathematics. Therefore, the research in this field will involve natural language, i.e. the language that people use everyday, so it is closely related to the research of linguistics. Natural language processing techniques typically include text processing, semantic understanding, machine translation, robotic question and answer, knowledge mapping, and the like.

Machine Learning (ML) is a multi-domain cross discipline, and relates to a plurality of disciplines such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and the like. The special research on how a computer simulates or realizes the learning behavior of human beings so as to acquire new knowledge or skills and reorganize the existing knowledge structure to continuously improve the performance of the computer. Machine learning is the core of artificial intelligence, is the fundamental approach for computers to have intelligence, and is applied to all fields of artificial intelligence. Machine learning and deep learning generally include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and formal education learning. The scheme provided by the embodiment of the application relates to technologies such as artificial intelligence (NLP) and ML.

BERT: (Bidirectional Encoder retrieval from Transformers), namely the Encoder of Bidirectional Transformer, increases the generalization capability of word vector model, and fully describes the character-level, word-level, sentence-level and even sentence-level relational characteristics. The Transformer model is proposed in 5 months in 2018, can replace a new architecture of a traditional Recurrent Neural Network (RNN) and a Convolutional Neural Network (CNN) and is used for realizing machine translation. The Transformer model includes an Encoder and a Decoder.

The following is a description of the design concept of the embodiments of the present application.

In the natural language processing technology, the task of the weight of the entry word is a main mode for extracting the core semantic components of the sentence and eliminating the influence of redundant components. At present, most word weight tasks are completed in a statistical mode, and the mode based on statistics is only based on global corpus to perform word frequency statistics, and the specific semantics of words and sentences cannot be linked, so that the accuracy of extracting keywords in the sentences is low.

Through analysis, when the characteristics of the whole sentence are extracted, not only can the semantic information of each participle in the sentence be extracted, but also the grammatical information of each participle in the sentence can be obtained, if the weight value of each participle in the sentence is determined by combining the semantic information and the grammatical information of each participle, and the keywords in the sentence are extracted based on the obtained weight values, the accuracy of extracting the keywords in the sentence is effectively improved.

In view of this, an embodiment of the present application provides a keyword extraction method, in which a target text is subjected to feature extraction to obtain a text feature vector corresponding to the target text and a segmentation grammar vector corresponding to a target segmentation included in the target text, and the target segmentation is subjected to feature extraction to obtain a segmentation semantic vector corresponding to the target segmentation. And then splicing the word segmentation semantic vector and the word segmentation grammar vector to obtain a first combination vector corresponding to the target word segmentation, and fusing the text feature vector corresponding to the target text with the first combination vector to obtain a fused feature vector. And determining a weight value of the target participle in the target text based on the fusion feature vector, wherein the weight value is used for representing the influence degree of the target participle on the semantic understanding of the target text. And then determining whether the target participle is a keyword in the target text or not based on the weight value of the target participle in the target text.

Referring to fig. 1, a system architecture diagram for a keyword extraction method provided in the embodiment of the present application is shown, where the architecture includes at least a terminal device 101 and a keyword extraction device 102.

The terminal device 101 may have installed therein a target application with a keyword extraction function, where the target application may be a client application, a web application, an applet application, or the like. The terminal device 101 may be, but is not limited to, a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, and the like.

The keyword extraction device 102 may be a background server of the target application, and provides a corresponding service for the target application, and the keyword extraction device 102 may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing services such as a cloud service, a cloud database, a cloud computing, a cloud function, a cloud storage, a Network service, a cloud communication, a middleware service, a domain name service, a security service, a Content Delivery Network (CDN), a big data and artificial intelligence platform, and the like. The terminal device 101 and the keyword extraction device 102 may be directly or indirectly connected through wired or wireless communication, and the present application is not limited thereto.

The keyword extraction method in the embodiment of the present application may be executed by the terminal device 101, and may also be executed by the keyword extraction device 102.

In the case of the above-described keyword extraction method, the keyword extraction device 102 performs the development:

the terminal device 101 obtains a target text and sends the target text to the keyword extraction device 102, and the keyword extraction device 102 performs feature extraction on the target text to obtain a text feature vector corresponding to the target text and a word segmentation grammar vector corresponding to a target word contained in the target text. And extracting the characteristics of the target participle to obtain a participle semantic vector corresponding to the target participle. And then splicing the word segmentation semantic vector and the word segmentation grammar vector to obtain a first combination vector corresponding to the target word segmentation, and fusing the text feature vector corresponding to the target text with the first combination vector to obtain a fused feature vector. And determining a weight value of the target participle in the target text based on the fusion feature vector, wherein the weight value is used for representing the influence degree of the target participle on the semantic understanding of the target text. And then determining whether the target participle is a keyword in the target text or not based on the weight value of the target participle in the target text. The keyword extraction device 102 determines whether other participles in the target text are keywords in the target text in the same manner, and finally extracts all keywords included in the target text.

In practical applications, the scheme provided by the embodiment of the application can be applied to all scenes needing to understand the text core words and the word weight, such as scenes of title understanding, discourse sentence understanding, text searching, video searching, content recommendation and the like.

For example, in a text retrieval scenario, the keyword extraction device extracts keywords in each article title in the article title library and a weight value corresponding to each keyword by using the method in the embodiment of the present application, and then stores the keywords in the article title and the weight values corresponding to each keyword in the article title library in correspondence with the article title.

The terminal equipment responds to article retrieval operation triggered by a user in a search application and sends a keyword 'XX park' to be matched, which is input by the user, to the keyword extraction equipment. The keyword extraction equipment carries out keyword matching on the keywords to be matched and each article title in the article title library, and the matched article titles of the keywords to be matched are determined to be an article title A 'XX park official net' and an article title B 'cherry blossom pictures in XX parks'. The keyword extraction device sends the article title A and the cover image corresponding to the article title A, and sends the article title B and the cover image corresponding to the article title B to the terminal device. The terminal device displays the article title A and the article title B and corresponding cover images on a search result interface of a search application.

Referring to fig. 2, a first area 201 of a search result interface displays an article title a and a cover image corresponding to the article title a, and a second area 202 of the search result interface displays an article title B and a cover image corresponding to the article title B.

In addition, the user may input a sentence in the search application, and the terminal device transmits the sentence input by the user to the keyword extraction device in response to an article retrieval operation triggered by the user in the search application. The keyword extraction device may extract keywords from the sentence input by the user, and then perform keyword matching with each article title in the title library based on the extracted keywords to determine a matching article title of the sentence input by the user.

The keyword extraction method provided by the embodiment of the present application may be executed by the terminal device 101 or the keyword extraction device 102 in fig. 1, and a specific flow of the method is introduced as follows:

in the embodiment of the application, the keyword extraction process may include an entry word weight task and a keyword extraction task, where the entry word weight task is to obtain a weight value of each entry in the target text, and the keyword extraction task is to extract the keyword according to the obtained weight value, where the weight value may be used to represent an influence degree of each entry on semantic understanding of the target text.

In the embodiment of the application, in order to improve the accuracy of the task of weighting the entry words, a neural network model based on deep learning can be used to obtain the weighting value of each word in the target text. In the following, a possible neural network model is taken as an example to describe the technical solution of the embodiment of the present application.

Referring to fig. 3, a schematic network structure diagram of a word weight model provided in the embodiment of the present application is shown, where the word weight model may include a first encoder (encoder), a second encoder, a concatenation layer, a fusion layer, and a classification layer, where the first encoder is configured to perform feature coding on a target text to obtain a text feature vector of the target text and a word segmentation grammar vector corresponding to n words included in the target text, where n is greater than or equal to 1. The second encoder is used for carrying out feature encoding on the participle i contained in the target text to obtain a participle semantic vector x of the participle i, wherein x is larger than or equal to 1 and is smaller than or equal to n, and i is larger than or equal to 1 and is smaller than or equal to n.

The splicing layer is used for splicing the participle grammar vector y of the participle i output by the first encoder and the participle semantic vector of the participle i output by the second encoder to obtain a first combined vector. The fusion layer is used for fusing the text feature vector of the target text output by the first encoder and the first combination vector output by the splicing layer to obtain a fusion feature vector. And the classification layer performs classification based on the fusion feature vector so as to obtain the weight value of the participle i in the target text. Since the processes performed by the respective layers will be described in detail later, they will not be described in great detail.

Before the word weight model is put into use, the model needs to be trained first, so the training process of the word weight model is described below. Please refer to fig. 4, which is a schematic diagram of a training process of the word weight model.

Step 401: a plurality of training samples are obtained.

In this embodiment of the application, each training sample may include a title and a participle in the title, and each training sample is labeled with a label (label) whether the participle in the training sample is a keyword, where the label may be characterized by two different values, for example, whether the participle is the keyword or not, and may be represented by 0 and 1, when the participle is the keyword, the label of the training sample is 1, and when the participle is not the keyword, the label of the training sample is 0, or, when the participle is the keyword, the label of the training sample is 0, otherwise, when the participle is not the keyword, the label of the training sample is 1, and of course, other possible values may also be used for representing, and this is not limited in this embodiment of the application.

As shown in table 1, an illustration of a data format of a training sample is shown, wherein the data format is entitled "one hop by one hop, teach you 600 cents of aggression" as an example, which can constitute a plurality of training samples. Through semantic understanding of the title, the core content of the title lies in the one-hop-skip strategy, so when the "one-hop-skip" is taken as a participle of a training sample, because the effect of the "one-hop-skip" on semantic understanding of the title is great, that is, the "one-hop-skip" is taken as a core word, the label of the corresponding training sample is 1, and similarly, the label of the training sample corresponding to the "attack" is also 1, so { "one-hop-skip", "600-up-minute attack", "one-hop-skip" } and { "one-hop-skip", and you up-600-minute attack and "attack" } are taken as a marked positive sample.

And for the participle of "teach you" or "in the title, relatively speaking, the effect of the participle on semantic understanding of the title is small, namely, the word" teach you "or" is a non-core word, and thus the label of the training sample corresponding to the word is 0, then { "jump one jump, teach you up to 600 minutes of attack", "teach you" } and { "jump one jump, teach you up to 600 minutes of attack", and the "of" as the labeled negative sample.

Table 1.

Title	Current word	Label (R)
			Jump to oneJump and teach you to go 600 minutes of strategy	Jump from one jump to another	1
Jump one jump and teach you to go 600 minutes of strategy	Strategy for tapping	1
			Jump one jump and teach you to go 600 minutes of strategy	Teach you	0
Jump one jump and teach you to go 600 minutes of strategy	Is/are as follows	0

Through the above process, a plurality of positive samples and negative samples can be obtained, so that the word weight model is trained through the constructed training samples. The format of the training samples may be as follows:

positive sample: { "title": "jump one jump, teach you up to 600 minutes of strategy", "current word": one hop by one hop, and label: 1}

Negative sample: { "title": "jump one jump, teach you up to 600 minutes of strategy", "current word": teach you "," label ": 0}

Of course, other possible sample formats may be adopted, and the embodiment of the present application does not limit this.

Step 402: and determining the prediction weight value of the participle in each training sample by using the word weight model.

In the embodiment of the application, each training sample comprises a title and a participle in the title, and then the weight value of the participle in each training sample can be determined through the word weight model, wherein the process of determining the weight value by the word weight model is specifically described in the following, so that redundant description is omitted here.

Step 403: and determining a loss function of the word weight model according to the obtained prediction weight values and the weight values indicated by the labels in the training samples.

Specifically, negative log cross entropy may be employed as a loss function of the word weight model, as shown in equation (1) below:

where loss represents the loss value of the word weight model, y_iWeight value of label indication representing ith training sample, a_iRepresenting the predicted weight value of the ith training sample, and n represents the number of training samples.

In general, when the degree of difference between the obtained weight value and the weight value indicated by the label is smaller, for example, the label is 1, and the predicted weight value obtained by the word weight model is 0.95, or, when the label is 0, and the predicted weight value obtained by the word weight model is 0.02, the smaller the loss value of the word weight model obtained by the above formula (1), the closer the weight value predicted by the term weight model is to the true value, and therefore the higher the accuracy thereof is.

Step 404: determining whether the word weight model converges according to a loss function.

Step 405: and when the word weight model is determined not to be converged, adjusting the model parameters of the word weight model according to the loss value.

Step 406: when it is determined that the word weight model converges, the training is ended.

In the embodiment of the application, when the loss value is less than the set loss threshold, it indicates that the accuracy of the word weight model can meet the requirement, so that the convergence of the word weight model can be determined, and conversely, when the loss value is not less than the set loss threshold, it indicates that the accuracy of the word weight model cannot meet the requirement, so that further parameter adjustment of the word weight model is required, and a subsequent training process is performed on the word weight model after parameter adjustment, that is, the processes of steps 402 to 404 are repeated.

In the embodiment of the present application, after the word weight model is obtained through training, keyword extraction may be performed by using the trained word weight model, please refer to fig. 5, which includes the following steps:

step S501, feature extraction is carried out on the target text, and a text feature vector corresponding to the target text and a word segmentation grammar vector corresponding to a target word included in the target text are obtained.

Specifically, the target text may be a title of multimedia content such as an article or a video, or a sentence in the multimedia content such as the article or the video. The text feature vector is used for representing semantic information of the target text, and the participle grammar vector corresponding to the target participle is used for representing grammar information of the target participle in the target text.

After the target text is obtained, performing word segmentation operation on the target text to obtain each word segmentation in the target text. The process of Word Segmentation operation refers to segmenting a sentence into individual words, and the Word Segmentation operation may be performed by any possible Word Segmentation method, such as a character matching method, an understanding method or a statistical method, or may be performed by using a corresponding Word Segmentation tool, such as a jieba (jieba) Word Segmentation.

In the embodiment of the present application, feature extraction may be performed on a target text by using a first Encoder to obtain a text feature vector corresponding to the target text and a participle grammar vector corresponding to each participle included in the target text, where the first Encoder may perform feature coding in any possible semantic coding manner to obtain the text feature vector of the target text and the participle grammar vector corresponding to each participle, and may be completed in various manners, such as bert (bidirectional Encoder Representation from transformations), Convolutional Neural Networks (CNN), Long-Short Term Memory artificial Neural Networks (Long-Short Term Memory, LSTM), or LSTM combined with Attention (Attention) mechanisms.

Step S502, extracting the characteristics of the target participle to obtain a participle semantic vector corresponding to the target participle.

Specifically, the participle semantic vector corresponding to the target participle is used for representing semantic information of the target participle. Feature coding can be performed on each participle included in the target text through a second encoder to obtain a participle semantic vector corresponding to the target participle, wherein the second encoder performs feature coding by adopting any possible word coding mode, for example, a Deep Neural Network (DNN) mode and the like can be adopted to complete mapping transformation of a feature space.

The first encoder and the second encoder may be obtained by co-training or may be trained separately. Meanwhile, the process of step 501 and the process of step 502 may be performed simultaneously or sequentially, which is not specifically limited in this application.

Step S503, the word segmentation semantic vector and the word segmentation grammar vector are spliced to obtain a first combination vector corresponding to the target word segmentation.

In the embodiment of the application, a splicing layer in a word weight model is adopted to splice the word segmentation semantic vector and the word segmentation grammar vector, and a first combination vector corresponding to a target word segmentation is output.

Specifically, the participle semantic vector may be spliced at the end of the participle normal vector, or the participle syntax vector may be spliced at the end of the participle semantic vector.

For example, the format of the first combined vector is:

Lword＝[tokenj：wordemb]

wherein Lword represents the first combined vector, tokeni represents a participle grammar vector corresponding to the participle i, and wordemb represents a participle semantic vector corresponding to the participle i.

Step S504, the text feature vector corresponding to the target text is fused with the first combined vector to obtain a fused feature vector.

In the embodiment of the application, a fusion layer in a word weight model is adopted to fuse the text feature vector corresponding to the target text with the first combination vector to obtain a fusion feature vector.

Specifically, dot product processing may be performed on a text feature vector corresponding to the target text and a first combined vector corresponding to the target word segmentation to obtain a corresponding fusion feature vector; or fusing the text feature vector corresponding to the target text with the first combined vector corresponding to the target word segmentation by adopting an attention mechanism to obtain a corresponding fused feature vector; and a bilinear pooling mode can be adopted to fuse the text feature vector corresponding to the target text with the first combined vector corresponding to the target word segmentation so as to obtain a corresponding fused feature vector.

And step S505, determining the weight value of the target participle in the target text based on the fusion feature vector.

In the embodiment of the application, a classification layer in a word weight model is adopted, and the weight value of the target word in the target text is determined based on the fusion feature vector. The classification layer may be implemented by any possible classification algorithm, for example, the classification may be performed by a softmax algorithm, Logistic regression (Logistic), or full link layer, so as to obtain a classification result corresponding to each participle, where the classification result is a weight value of each participle.

Specifically, the weighted value is used for representing the influence degree of the target participle on the semantic understanding of the target text, the larger the weighted value is, the higher the influence degree of the target participle on the semantic understanding of the target text is, and the smaller the weighted value is, the lower the influence degree of the target participle on the semantic understanding of the target text is.

The word weight model can also be used for determining the weight values of other participles in the target text, and details are not repeated here.

Step S506, whether the target participle is a keyword in the target text or not is determined based on the weight value of the target participle in the target text.

In a possible implementation manner, if the weight value corresponding to the target word segmentation is greater than or equal to a preset threshold, the target word segmentation is determined to be a keyword in the target text. And if the weight value corresponding to the target word segmentation is smaller than a preset threshold value, determining that the target word segmentation is not the keyword in the target text. Whether other participles in the target text are keywords in the target text can be determined in the same mode, and then all keywords in the target text can be obtained.

In another possible implementation manner, a word weight model is adopted to obtain a weight value corresponding to each participle in a target participle, then the participles are sorted from large to small according to the weight values, and the participles corresponding to the top N weight values are used as keywords in a target text, wherein N is a preset positive integer.

For example, as shown in fig. 6, in the article search scene, the article title "cherry blossom in the first park is full, and you are waiting to view" is taken as the target title. After the target title is input to the word weight model, the weight value of each participle can be obtained, wherein the weight value of "first park" is 0.91, the weight value of "cherry blossom" is 0.81, the weight value of "full bloom" is 0.7, the weight value of "wait you" is 0.3, the weight value of "view you" is 0.2, the weight value of "come" is 0.2, and the weight value of "yet" is 0.1.

And sequencing all the participles based on the weight values, taking the top 3-digit participles of 'first park', 'cherry blossom' and 'blooming' as keywords of the target title, wherein the extracted keywords can be applied to the article searching process. For example, as shown in fig. 7, after a video title "baby with warm heart" pacifies that a dog going home from home is given high praise "as a target title is input to the word weight model, the weight values of the respective participles can be obtained, wherein the weight value of" baby "is 0.85, the weight value of" dog "is 0.83, the weight value of" baby "is 0.82, the weight value of" praise "is 0.51, the weight value of" warm heart "is 0.3, the weight value of" praise "is 0.1, and the weight value of" going home "is 0.01. Based on the weight values, all the participles are sorted, the top 3-ranked participles of baby, dog and comfort are used as keywords of the target title, and the extracted keywords can be applied to the video searching process.

Optionally, in step S501, the process of extracting features of the target text is described by taking one of the manners as an example, and includes the following steps:

step S5011, respectively extracting a participle grammar vector, a position vector, and a segmentation vector corresponding to each participle in the target text.

Specifically, each participle corresponds to a participle grammar vector, a position vector and a segmentation vector, and each participle grammar vector is used for representing grammar information of a corresponding participle in the target text. Each position vector is used for representing the relative position relation between a corresponding participle and other participles in the target text, and can be represented by the serial number of the participle in the target text or by the word vectors existing before and after the participle. Each segmentation vector is used for representing the sentence type of the sentence to which the corresponding word segmentation belongs.

Step S5012, obtaining a second combined vector corresponding to the corresponding participle based on the respective corresponding participle grammar vector, the position vector, and the segmentation vector of each participle.

Referring to fig. 8, a schematic diagram is obtained for the second combined vector of each participle, where the target text includes n participles, which are participle 1, participle 2, and participle …, and the position vector, segmentation vector, and participle syntax vector corresponding to participle 1 are denoted as Ec1, Eb1, and Ea1, respectively. The position vector, segmentation vector and segmentation grammar vector corresponding to the segmentation 2 are respectively expressed as Ec2, Eb2 and Ea2, and so on.

After the position vector, the segmentation vector, and the segmentation grammar vector of each segmented word are obtained, a second combination vector of the corresponding segmented word may be obtained based on the position vector, the segmentation vector, and the segmentation grammar vector, where the second combination vector corresponding to segmented word 1 is denoted as E1, the second combination vector corresponding to segmented word 2 is denoted as E2, and the second combination vector corresponding to segmented word n is denoted as En. The second combined vector is a vector capable of simultaneously representing information contained in the position vector, the segmentation vector and the word segmentation grammar vector.

Specifically, for each participle, the position vector, the segmentation vector, and the participle grammar vector may be superimposed to obtain a second combined vector of the corresponding participle.

Taking the participle 1 as an example, as shown in fig. 9, the values of the position vector Ec1, the segmentation vector Eb1 and the participle grammar vector Ea1 of the participle 1 at the same position are superimposed to obtain a second combination vector E1 of the participle 1.

Specifically, the normal vector, the position vector and the segmentation vector of the segmented word can be spliced, so that a second combined vector of the corresponding segmented word is obtained.

Taking the participle 1 as an example, as shown in fig. 10, the segmentation vector Eb1 of the participle 1 may be spliced behind the position vector Ec1, and then the participle grammar vector Ea1 may be spliced behind the segmentation vector Eb1, so as to obtain the second combined vector E1 of the participle 1.

Specifically, the word normal vector, the position vector and the segmentation vector may be pooled to obtain a second combined vector of the corresponding word.

As shown in fig. 11, also taking the participle 1 as an example, when performing the maximum pooling process, the values of the position vector Ec1, the segmentation vector Eb1, and the participle syntax vector Ea1 of the participle 1 at the same position are maximized, thereby obtaining a second combined vector E1 of the participle 1.

And step S5013, performing feature extraction on each obtained second combined vector to obtain a text feature vector corresponding to the target text.

In the embodiment of the application, the second combination vectors of each word segmentation of the target text can be combined, so that the text feature vector corresponding to the target text is obtained.

Or, the semantic relation between the participles may not be reflected only by combining, and therefore, feature extraction may be performed on each second combined vector, so that the extracted vectors are combined, and thus the text feature vector corresponding to the target text is obtained.

In a possible implementation manner, a self-attention mechanism may be adopted to perform feature extraction on each second combined vector, so as to obtain a text feature vector corresponding to the target text.

Specifically, according to each second combination vector and the corresponding attention weight matrix, the attention weight vector corresponding to each participle is obtained, wherein each value included in the attention weight vector corresponding to one participle represents the attention weight of each participle relative to one participle.

In a specific implementation, the attention weight matrix may include a request (query) vector matrix and a key (key) vector matrix, and at least one attention vector corresponding to each participle is obtained according to each second combination vector and the corresponding attention weight matrix, where the at least one attention vector includes a query vector and a key vector.

Furthermore, the attention weight vector corresponding to each participle may be obtained based on at least one attention vector corresponding to each participle, where each value included in the attention weight vector corresponding to one participle represents the attention weight of each participle relative to one participle. For example, the target text contains 4 participles, and for participle 1 therein, the attention weight vector of participle 1 contains 4 values, each value representing the attention weight of a participle contained in the target text for participle 1.

Optionally, the attention weight of each participle relative to one participle is the similarity between the key vector corresponding to each participle and the request vector of one participle.

Specifically, the attention weight of the participle 2 to the participle 1 can be obtained through the similarity between the key vector of the participle 2 and the query vector of the participle 1, and similarly, the same is true for other participles, and the attention weight of the participle 1 to the participle 1 can be obtained through the similarity between the key vector of the participle 1 and the query vector of the participle 1.

Then, a text feature vector corresponding to the target text may be obtained according to the attention weight vector corresponding to each participle and each second combination vector, where the text feature vector includes the participle feature vector corresponding to each participle, and each participle feature vector is obtained by performing weighted summation with a corresponding second combination vector according to each attention weight in a corresponding one of the attention weight vectors.

For example, by using the participle 1, weighting and summing each attention weight in the attention weight vector corresponding to the participle 1 and each value in the second combined vector corresponding to the participle 1 to obtain a participle feature vector of the participle 1.

Optionally, the attention weight matrix further comprises a matrix of value vectors, and correspondingly, the at least one attention vector further comprises a value vector.

The text feature vector corresponding to the target text can be obtained according to the attention weight vector corresponding to each participle and at least one attention vector corresponding to each participle, wherein the text feature vector comprises the participle feature vector corresponding to each participle, and each participle feature vector is obtained by performing weighted summation with the corresponding attention vector according to each attention weight in the corresponding attention weight vector.

For example, with the participle 1, weighting and summing each attention weight in the attention weight vector corresponding to the participle 1 and each value in the value vector of the participle 1 to obtain a participle feature vector of the participle 1.

Step S5014, obtaining a participle grammar vector corresponding to the target participle from the respective participle grammar vectors corresponding to the respective participles.

Specifically, when the word weight model predicts the weight value of a target word in a target text, the first encoder respectively extracts a word segmentation grammar vector, a position vector and a segmentation vector corresponding to each word in the target text, and the second encoder performs feature coding on the target word to obtain a word segmentation semantic vector corresponding to the target word. The participle grammar vector corresponding to each participle extracted from the first encoder comprises the participle grammar vector corresponding to the target participle, so that the participle grammar vector corresponding to the target participle can be obtained from the participle grammar vector corresponding to each participle, and then the weighted value of the target participle in the target text is predicted by combining the participle grammar vector corresponding to the target participle, the participle semantic vector and the text feature vector corresponding to the target text.

In the embodiment of the application, word segmentation grammar vectors, position vectors and segmentation vectors corresponding to the word segmentation are extracted, and the feature vectors of multiple dimensions are spliced to obtain a second combination vector of the word segmentation. And then, extracting features of each second combined vector by adopting a self-attention mechanism to obtain a text feature vector corresponding to the target text, so that the text feature vector can better represent semantic relation and grammatical relation among each participle, and further the accuracy of the determined weight value of the participle is improved.

In the embodiment of the present application, the keyword extraction method described above may be applied to all scenes in which a text core word and a word weight need to be understood, such as scenes of title understanding, discourse sentence understanding, text retrieval, video search, content recommendation, and the like.

For example, video searching is performed, target keywords to be matched are obtained first, and then keyword matching is performed on the target keywords and each video title in a video title library to obtain at least one candidate video title. And sequencing the at least one candidate video title according to the weight value of each keyword in the at least one candidate video title. And determining the matched video title of the target keyword according to the sequencing result.

Specifically, the keywords corresponding to each video title in the video title library and the weight values corresponding to the keywords can be obtained by the keyword extraction method in the embodiment of the present application. And performing keyword matching on the target keywords and each video title in the video title library, and if the video titles contain the target keywords, taking the video titles as candidate video titles and obtaining the weight values of the target keywords in the candidate video titles. And then sequencing all the candidate video titles according to the sequence of the weighted values from large to small, and taking the candidate video titles with the top M in the sequencing result as the matched video titles of the target keywords, wherein M is a preset positive integer. And then, sending the obtained matched video title and the corresponding video cover image to the terminal equipment, wherein the terminal equipment can display the obtained matched video title and the corresponding video cover image in the video application.

For example, after the user inputs the target keyword "football game" on the search interface of the video application, the terminal device sends the target keyword "football game" input by the user to the keyword extraction device in response to the video search operation triggered by the user in the video application. The keyword extraction equipment carries out keyword matching on a target keyword 'football match' and each video title in a video title library to obtain 4 candidate video titles containing the target keyword, wherein the candidate video titles are a candidate video title 1, a candidate video title 2, a candidate video title 3 and a candidate video title 4 respectively, the weight value of the target keyword in the candidate video title 1 is 0.7, the weight value of the target keyword in the candidate video title 2 is 0.8, the weight value of the target keyword in the candidate video title 3 is 0.9, and the weight value of the target keyword in the candidate video title 4 is 0.6.

Sequencing the candidate video titles according to the sequence of the weighted values from large to small, wherein the obtained sequencing result is as follows: candidate video title 3, candidate video title 2, candidate video title 1, and candidate video title 4. And taking the candidate video titles 3 and the candidate video titles 2 which are ranked at the top two positions as the matched video titles of the target keywords.

The keyword extraction equipment sends the candidate video titles 3 and the candidate video titles 2 and corresponding video cover images to the terminal equipment, and the terminal equipment displays the candidate video titles 3 and the candidate video titles 2 and the corresponding video cover images in a search result interface in the video application.

Referring to fig. 12, a first area 1201 of the search result interface shows a candidate video title 3 and a video cover image corresponding to the candidate video title 3, and a second area 1202 of the search result interface shows a candidate video title 2 and a video cover image corresponding to the candidate video title 2, where the specific content of the candidate video title 3 is "football match: team A fighting team B ', and the specific content of the candidate video title 2 is ' football match collection '.

In the embodiment of the application, the semantic vector of the sentence and the grammar information of the words in the sentence are simultaneously obtained, and the core information in the sentence can be better represented in an experiment by combining the models of the grammar and the semantic information.

The semantic vector of the target text and the participle semantic vector and the participle grammar vector of each participle in the target text are simultaneously obtained by adopting the word weight model, and the semantic vector of the target text and the fused feature vector obtained by fusing the participle semantic vector and the participle grammar vector can better represent the core information in the target text, so that when the weight value of the participle in the target text is predicted based on the fused feature vector, the accuracy of the obtained participle weight value is improved, the accuracy of extracting the keyword in the target text based on the participle weight value is further improved, and the accuracy of semantic understanding of the text under various scenes is also improved.

Based on the same technical concept, the embodiment of the present application provides a schematic structural diagram of a keyword extraction apparatus, as shown in fig. 13, the apparatus 1300 includes:

the feature extraction module 1301 is configured to perform feature extraction on a target text, obtain a text feature vector corresponding to the target text and a segmentation grammar vector corresponding to a target segmentation word included in the target text, perform feature extraction on the target segmentation word, and obtain a segmentation semantic vector corresponding to the target segmentation word;

a concatenation module 1302, configured to concatenate the word segmentation semantic vector and the word segmentation grammar vector to obtain a first combined vector corresponding to the target word segmentation;

a fusion module 1303, configured to fuse the text feature vector corresponding to the target text with the first combined vector to obtain a fusion feature vector;

a prediction module 1304, configured to determine, based on the fused feature vector, a weight value of the target participle in the target text, where the weight value is used to represent an influence degree of the target participle on semantic understanding of the target text;

a decision module 1305, configured to determine whether the target participle is a keyword in the target text based on a weight value of the target participle in the target text.

Optionally, the feature extraction module 1301 is specifically configured to:

Optionally, the fusion module 1303 is specifically configured to:

Optionally, the judging module 1305 is specifically configured to:

Optionally, a keyword matching module 1306 is further included;

the keyword matching module 1306 is specifically configured to:

Based on the same technical concept, the embodiment of the present application provides a computer device, as shown in fig. 14, including at least one processor 1401 and a memory 1402 connected to the at least one processor, where a specific connection medium between the processor 1401 and the memory 1402 is not limited in this embodiment of the present application, and the processor 1401 and the memory 1402 are connected through a bus in fig. 14 as an example. The bus may be divided into an address bus, a data bus, a control bus, etc.

In the embodiment of the present application, the memory 1402 stores instructions executable by the at least one processor 1401, and the at least one processor 1401 can execute the steps of the keyword extraction method by executing the instructions stored in the memory 1402.

The processor 1401 is a control center of the computer device, and may connect various parts of the computer device by using various interfaces and lines, and extract keywords in the target text by executing or executing instructions stored in the memory 1402 and calling data stored in the memory 1402. Alternatively, the processor 1401 may include one or more processing units, and the processor 1401 may integrate an application processor, which mainly handles an operating system, a user interface, application programs, and the like, and a modem processor, which mainly handles wireless communication. It will be appreciated that the modem processor described above may not be integrated into processor 1401. In some embodiments, processor 1401 and memory 1402 may be implemented on the same chip, or in some embodiments, they may be implemented separately on separate chips.

The processor 1401 may be a general-purpose processor such as a Central Processing Unit (CPU), a digital signal processor, an Application Specific Integrated Circuit (ASIC), a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, and may implement or perform the methods, steps, and logic blocks disclosed in the embodiments of the present Application. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of a method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware processor, or may be implemented by a combination of hardware and software modules in a processor.

Memory 1402, which is a non-volatile computer-readable storage medium, may be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The Memory 1402 may include at least one type of storage medium, and may include, for example, a flash Memory, a hard disk, a multimedia card, a card-type Memory, a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Programmable Read Only Memory (PROM), a Read Only Memory (ROM), a charge Erasable Programmable Read Only Memory (EEPROM), a magnetic Memory, a magnetic disk, an optical disk, and so on. Memory 1402 is any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to such. The memory 1402 in the embodiments of the present application may also be a circuit or any other device capable of performing a storage function for storing program instructions and/or data.

Based on the same inventive concept, embodiments of the present application provide a computer-readable storage medium storing a computer program executable by a computer device, which, when the program runs on the computer device, causes the computer device to execute the steps of the keyword extraction method described above.

It should be apparent to those skilled in the art that embodiments of the present invention may be provided as a method, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.

It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.

Claims

1. A keyword extraction method is characterized by comprising the following steps:

2. The method of claim 1, wherein the extracting features of the target text to obtain a text feature vector corresponding to the target text and a word segmentation grammar vector corresponding to a target word segmentation included in the target text comprises:

3. The method of claim 2, wherein obtaining a second combined vector corresponding to the corresponding participle based on the participle grammar vector, the position vector and the segmentation vector corresponding to the participle respectively comprises:

4. The method of claim 2, wherein the performing feature extraction on each obtained second combined vector to obtain a text feature vector corresponding to the target text comprises:

5. The method of claim 4, wherein obtaining the attention weight vector corresponding to each participle according to each second combination vector and the corresponding attention weight matrix comprises:

6. The method according to any one of claims 1 to 5, wherein the fusing the text feature vector corresponding to the target text with the first combined vector corresponding to the target word segmentation to obtain a fused feature vector comprises:

7. The method of any one of claims 1 to 5, wherein the determining whether the target participle is a keyword in the target text based on the weight value of the target participle in the target text comprises:

8. The method of claim 7, wherein after determining whether the target participle is a keyword in the target text based on a weight value of the target participle in the target text, the method further comprises:

acquiring a target keyword to be matched;

9. A keyword extraction apparatus, comprising:

10. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the steps of the method of any of claims 1 to 8 are performed when the program is executed by the processor.

11. A computer-readable storage medium, having stored thereon a computer program executable by a computer device, for causing the computer device to perform the steps of the method of any one of claims 1 to 8, when the program is run on the computer device.