CN109408641A - It is a kind of based on have supervision topic model file classification method and system - Google Patents

Abstract

The present disclosure provides a text classification method and system based on a supervised topic model. Among them, a text classification method based on a supervised topic model includes: constructing a SLDA-TC text classification model; in the process of training the SLDA-TC text classification model, according to the SLDA-TC-Gibbs algorithm, the implicit classification of each word is performed. The topic is sampled, and the implicit topic is only sampled from other training texts with the same text category label as the word; after determining the implicit topic of each word, the text-topic probability distribution, topic probability distribution, topic ‑Word probability distribution and topic‑category probability distribution; establish accurate mapping between topics and categories; input the text to be tested into the SLDA‑TC text classification model generated by training, infer the topic of the text to be tested, and then predict the category of the text .

Description

A text classification method and system based on supervised topic model

technical field

The present disclosure relates to the field of data classification, and in particular, to a text classification method and system based on a supervised topic model.

Background technique

The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

Text representation is an important step in text mining, and the most widely used text representation method is Bag-of-word (BOW). The bag-of-words method regards a text as a collection of words, and assumes that the occurrence of each word is independent, does not depend on other words, and ignores information such as word order and syntax. Based on BOW, a text is represented by an n-dimensional vector, each dimension corresponds to a word, usually the frequency-related weight of the word, which is the most commonly used vector space model (VSM). Due to the complexity of natural language, text representation has many problems such as "curse of dimensionality", "sparseness", and "semantic loss". The bag-of-words method ignores word order, syntax and other information, making it difficult to extract and quantify the semantic information of words, and the semantic representation of text is still very difficult.

The word2vec model proposed by Mikolov et al. is a word vector training method, which uses the context information of the word to convert a word into a low-dimensional real number vector. The more similar words are in the vector space, the closer they are. The word2vec model training output is the word vector of each word, and the word vector of all words of the text forms the text vector. The word vector text based on the word2vec model training is input into the deep neural network, which has been successfully used in Chinese word segmentation, POS tagging, sentiment classification, syntactic dependencies, etc. The word2vec model can solve the "sparseness" problem. Although word2vec can quantify the similarity between words, it cannot solve the "semantic loss" and "dimension disaster" problems of text.

The topic model is a method that can be used to solve the "curse of dimensionality" and "sparseness", and can extract the semantic information of words to a certain extent. Topic models originated from Latent Semantic Indexing (LSI) and probabilistic Latent Semantic Indexing (pLSI) proposed by Hofmann. On the basis of pLSI, Blei et al. proposed the LDA (Latent DirichletAllocation) topic model. The topic in LDA is regarded as the probability distribution of words. Words with similar semantics can be associated through implicit topics, and semantic information can be extracted from the text, and the text representation can be transformed from a high-dimensional word space to a low-dimensional topic space. Topic models are used directly or by extension in the field of natural language processing, such as clustering and classification, word sense disambiguation, sentiment analysis, etc., target discovery and localization in the field of image processing, image segmentation and other tasks.

The LDA topic model transforms the text representation from a high-dimensional word space to a low-dimensional topic space, and then uses KNN, Naive Bayesian, SVM and other algorithms for direct classification, and the effect is not good. The reason is that the LDA topic model is unsupervised learning, does not consider the category of the text, and does not take advantage of the important information of the category that the training text has annotated.

The existing improved methods, such as Li et al. proposed the Labeled-LDA model, the inventor found that this model trains an LDA model for each type of document, and the parameters that need to be estimated increase many times, increasing the complexity of the model.

SUMMARY OF THE INVENTION

According to an aspect of one or more embodiments of the present disclosure, there is provided a text classification method based on a supervised topic model, which can identify the semantic relationship between topics and categories, and establish an accurate mapping between topics and categories.

One or more embodiments of the present disclosure provide a text classification method based on a supervised topic model, including:

Build the SLDA-TC text classification model. Each document in the training document set of the SLDA-TC text classification model has a class label; the parameters that need to be estimated in the SLDA-TC text classification model include not only text-topic probability distribution, topic-word probability distributions, including topic-category probability distributions;

The SLDA-TC text classification model is trained, and the parameters of the SLDA-TC model are estimated according to the SLDA-TC-Gibbs algorithm. The topic is sampled, and only the implicit topic is sampled from other training texts with the same text category label as the word; after determining the implicit topic of each word, by statistical topic-word, document-topic, topic-category The frequency of text-topic, topic-word probability distribution and topic-category probability distribution are calculated, and then an accurate mapping between topics and categories is established;

Inference and classification of the subject of the text to be tested; input the text to be tested into the trained SLDA-TC text classification model, first perform implicit topic sampling for each word of the document to be tested; then infer the subject probability distribution of the text to be tested; The topic distribution of the document and the topic-category distribution of the SLDA-TC model, output the category label of the text to be tested.

In one or more embodiments, the text-topic probability distribution, topic-word probability distribution, and topic-category probability distribution all obey the Dirichlet distribution.

In one or more embodiments, the SLDA-TC model for classification is generated through multiple iterations of training, and the iteration is over, and the similarity between topics is evaluated by JS divergence, and the topic-category distribution parameter of SLDA-TC is evaluated. Semantic relevance between topics and categories.

In one or more embodiments, the evaluation indicators of the classification results of the SLDA-TC text classification model include macro-average classification precision (Macro-precision), macro-average recall rate (Macro-Recall), and macro-average F1 value (Macro-F1 ).

One or more embodiments of the present disclosure further provide a text classification system, including a text input device, a controller and a display device, the controller includes a memory and a processor, the memory stores a computer program, the A program can perform the following steps when executed by a processor:

Build the SLDA-TC text classification model. Each document in the training document set of the SLDA-TC text classification model has a class label; the parameters that need to be estimated in the SLDA-TC text classification model include not only text-topic probability distribution, topic-word probability distributions, including topic-category probability distributions;

The SLDA-TC text classification model is trained, and the parameters of the SLDA-TC model are estimated according to the SLDA-TC-Gibbs algorithm. The topic is sampled, and only the implicit topic is sampled from other training texts with the same text category label as the word; after determining the implicit topic of each word, by statistical topic-word, document-topic, topic-category The frequency of text-topic, topic-word probability distribution and topic-category probability distribution are calculated, and then an accurate mapping between topics and categories is established;

Inference and classification of the subject of the text to be tested; input the text to be tested into the trained SLDA-TC text classification model, first perform implicit topic sampling for each word of the document to be tested; then infer the subject probability distribution of the text to be tested; The topic distribution of the document and the topic-category distribution of the SLDA-TC model, output the category label of the text to be tested.

The beneficial effects of the present disclosure are:

The text classification method and system of the present disclosure, by constructing and training the completed SLDA-TC text classification model, utilizes text-topic probability distribution, topic-word probability distribution and topic-category probability distribution to extract words and topics, documents and topics, The implicit semantic information mapping between topics and categories, and the number of topics K only needs to be slightly larger than the number of categories C, which not only improves the accuracy of text classification, but also improves time efficiency.

Description of drawings

The accompanying drawings forming a part of the present disclosure are used to provide a further understanding of the present disclosure, and the exemplary embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation of the present disclosure.

FIG. 1 is a flowchart of an SLDA-TC text classification method disclosed in the present disclosure.

Figure 2 shows the LDA topic model.

Figure 3 shows the SLDA-TC text classification model.

Figure 4(a) shows the Macro-Precision comparison of the classification results of SLDA-TC, LDA-TC, and SVM when the 20news-rec data set C=4, K=8.

Figure 4(b) shows the Macro-Recall comparison of the classification results of SLDA-TC, LDA-TC, and SVM when the 20news-rec data set C=4, K=8.

Figure 4(c) is the Macro-F1 comparison of the classification results of SLDA-TC, LDA-TC, and SVM when the 20news-rec data set C=4, K=8.

Figure 5(a) shows the Macro-Precision comparison of the classification results of SLDA-TC, LDA-TC, and SVM when the sogou dataset is C=5 and K=8.

Figure 5(b) shows the Macro-Recall comparison of the classification results of SLDA-TC, LDA-TC and SVM when the sogou data set C=5 and K=8.

Figure 5(c) shows the Macro-F1 comparison of the classification results of SLDA-TC, LDA-TC and SVM when the sogou dataset C=5 and K=8.

Figure 6(a) shows the Macro-Precision comparison of the classification results of SLDA-TC, LDA-TC, and SVM for the 20news-sci dataset C=4, when K=90.

Figure 6(b) shows the Macro-Recall comparison of the classification results of SLDA-TC, LDA-TC, and SVM for the 20news-sci dataset C=4, when K=90.

Figure 6(c) shows the Macro-F1 comparison of the classification results of SLDA-TC, LDA-TC and SVM when K=90 for the 20news-sci dataset C=4.

Figure 7(a) shows the Macro-Precision comparison of the classification results of SLDA-TC, LDA-TC and SVM when the 20news-talk dataset C=3, K=90.

Figure 7(b) is the Macro-Recall comparison of the classification results of SLDA-TC, LDA-TC and SVM when the 20news-talk dataset C=3, K=90.

Figure 7(c) is the Macro-F1 comparison of the classification results of SLDA-TC, LDA-TC and SVM when the 20news-talk dataset C=3, K=90.

Detailed ways

It should be noted that the following detailed description is exemplary and intended to provide further explanation of the present disclosure. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.

It should be noted that the terminology used herein is for the purpose of describing particular embodiments only, and is not intended to limit the exemplary embodiments in accordance with the present disclosure. As used herein, unless the context clearly dictates otherwise, the singular is intended to include the plural as well, and it should also be understood that when the terms "comprising" and/or "including" are used in this specification, it indicates that There are features, steps, operations, devices, components and/or combinations thereof.

Terminology Explanation:

Dirichlet distribution: Dirichlet distribution is a group of continuous multivariate probability distributions, and is a multivariate generalized β distribution. Dirichlet distribution is often used as the prior probability of Bayesian statistics.

Gibbs Sampling: Gibbs sampling, an algorithm based on Markov Monte Carlo (MCMC), is used to approximate sample sequences from a multivariate probability distribution when direct sampling is difficult.

LDA-TC: refers to the method of first extracting a certain number of topics through the LDA topic model, and then classifying them according to the LDA model.

SVM: Support Vector Machine, which refers to the support vector machine, is a common discrimination method. In the field of machine learning, it is a supervised learning model, which is usually used for pattern recognition, classification and regression analysis.

Macro-Precision: Macro average classification accuracy.

Macro-Recall: Macro average recall.

Macro-F1: Macro average F1 value.

FIG. 1 is a flowchart of a text classification method based on a supervised topic model according to the present disclosure.

As shown in FIG. 1, a text classification method of this embodiment includes:

S110: Build a SLDA-TC text classification model; unlike the unsupervised LDA model, each document in the training document set of the SLDA-TC text classification model has a class label; the parameters to be estimated in the SLDA-TC text classification model include text - Topic probability distribution, topic-word probability distribution and topic-category probability distribution.

As shown in Figure 2, the text collection of the LDA topic model M represents the number of texts, and K represents the number of topics. The model has two parameters θ _m and where θm represents the topic probability distribution of the _mth text, Represents the word probability distribution of topic k, w _m is the word bag vector of the m-th text, N _m represents the length of the m-th text, w _m,n represents the n-th word in the m-th text, z _m,n is Topic assigned to w _m,n . _θm and Subject to Dirichlet distribution, topics and words are generated as multinomial parameters, α and β are the prior parameters of the corresponding Dirichlet distribution. The LDA topic model does not consider the category of each document.

As shown in Figure 3, unlike LDA, the training text set of SLDA-TC model Each text w _m has an observable category number y _m ∈ [1, C], C represents the number of categories, assuming that the category number obeys a multinomial distribution related to the text topic probability. w _m,n and y _m are observables and z _m,n are implicit subjects. SLDA-TC model in addition to parameters and θ _m , the present disclosure introduces a new parameter δ _k , representing the class probability distribution of the k-th topic. θ _m , and _δk obey the Dirichlet distribution as multinomial parameters to generate topics, words and categories, respectively, α, β and γ are the prior parameters of the corresponding Dirichlet distribution.

S120: Estimate the parameters of the SLDA-TC topic model, train the SLDA-TC text classification model, and establish the mapping between topics and categories.

The parameters estimated by LDA using Gibbs Sampling algorithm are and θ, the parameters that the SLDA-TC model needs to estimate are theta and delta. The present disclosure proposes the SLDA-TC-Gibbs algorithm on the basis of the Gibbs Sampling method, and does not directly calculate θ _m , and δ _k , first sample the implicit topic of each word, after determining the implicit topic of each word, θ _m , and δ _k can be calculated by statistical frequency.

The Gibbs Sampling algorithm samples a latent variable component _zi = k of the word w each time, and keeps the other word components of z, namely unchanged, the calculation is as in formula (1).

Each time the SLDA-TC-Gibbs algorithm samples a latent variable component _zi = k of the word w _i =t, it keeps the other word components of z, namely unchanged, while maintaining unchanged, the calculation is as in formula (2).

where w=(w ₁ ,...,w _m ) is the word vector for all documents, z=(z ₁ ,...,z _V ) is the topic vector (V is the word dictionary length), y=(y ₁ ,...y _M ) is the category vector. Assuming that the ith word w _i =t, _zi represents the topic variable corresponding to the ith word, Represents the ith item of the culling vector z. At the same time, it is assumed that the document where the i-th word w _i =t is located is the m-th document w _m , and its class label is y _m =j,j∈[1,C], then Represents the mth item of the culling vector y. represents the number of times topic k is assigned to word v, and β _v represents the Dirichlet prior of word v. represents the number of times document m is assigned to topic z, α _z is the Dirichlet prior of topic z. represents the number of times the topic k is assigned to the word t, excluding the ith item of z (ie the ith word _wi = t), and β _v is the Dirichlet prior of the word t. Indicates that the m-th item is removed (the class of the m-th document w _m is marked as y _m =j), the topic k is assigned as the number of documents in the category j, and γ _j represents the Dirichlet prior parameter of the category j.

Compared with formula (1), it is introduced on the left side of formula (2) Add an item to the right The implicit topic sampling of words in the mth document is restricted, that is, the implicit topic sampling is only performed from other training documents of the same category as the mth document, because the topic distributions of the documents of the same category are similar, so Formula (2) is more reasonable when there are category labels. The present disclosure generates only one SLDA-TC model and only needs to estimate a set of parameters theta and delta. The derivation of formula (2) is proved as follows:

Proof: Given a set of training documents Let w=(w ₁ ,...,w _m ), y=(y ₁ ,... y _M ), z=(z ₁ ,...,z _V ), the joint distribution of the SLDA-TC probability model Such as (3) formula.

p(w,z,y|α,β,γ)=p(w|z,β)p(z|α)p(y|z,γ) (3)

According to the Dirichlet distribution:

in, Γ(·) is a Gamma function.

According to the SLDA-TC-Gibbs algorithm,

For the word w _i =t of the mth document, is constant, so

From (3)-(6) and (8), we can get

Therefore, (2) can be proved.

After obtaining the topic z label of each word w, the parameters of the SLDA-TC model θ _m and δ _k are calculated as follows.

in represents the probability that topic k is assigned to word t, θ _m,k represents the probability that document m is assigned to topic k, δ _k,j represents the probability that topic k belongs to category j, to represent the number of times that topic k is assigned to word t, represents the number of times document m is assigned to topic k, Denotes the number of documents whose topic is k assigned to category j, k=1..K, t=1..V, m=1..M.

From this the parameters of the SLDA-TC model θ and δ are estimated.

SLDATC-Gibbs algorithm description:

algorithm:

Input: document vector Hyperparameters α, β, γ, number of topics K, number of iterations T.

Output: topic z distribution, parameters theta and delta.

Initialize variables Initialized to 0.

After training and generating the SLDA-TC model, the implicit topic inference formula for each word in the d-th new document is shown in (13).

in, represents the word vector of the d-th new document, represents its topic vector, Indicates that the new document removes the i-th item (ie the i-th word ), the number of times topic k is assigned to word t, Represents the number of times that the i-th item is removed and the d-th new document is assigned to the topic k. Refer to formula (2) for the meaning of other symbols.

Calculate the implicit topic label of each word in the new document d by equation (13), and then calculate the probability that d belongs to each topic as in equation (14).

From this, the topic probability distribution of the new document d is obtained

S130: Input the text to be tested into the trained SLDA-TC text classification model, infer the subject of the text to be tested, and then predict the text classification.

Given a trained SLDA-TC model, let is the word vector of the d-th new document, is its theme vector, is the class prediction for the new document d, and the classification of the new document is calculated as follows.

It is assumed here that the test sample set and the training sample set obey the same distribution, that is, the two are consistent in the implicit topic-category distribution. Therefore, can be replaced by p(y|z), revealed by the parameter δ of the SLDA-TC model. is the topic probability distribution of the new document d From equations (12), (14) and (15), we can get:

The similarity between topics is evaluated by JS divergence (Jensen-Shannon divergence). JS divergence, also called JS distance, is a variant of KL divergence (Kullback–Leibler divergence), calculated as formula (17). Unlike the KL divergence, the JS divergence satisfies the symmetry and triangular distance formulas.

JS(pi ||p _j )=0.5*KL(pi ||( _pi +p _j )/2) _{+0.5*KL(p j ||(pi +p j ) / 2 )} (17)

Among them, p _i and p _j represent the word probability distribution of topic i and topic j respectively, the range of JS divergence is [0, 1], 0 means p _i and p _j have the same distribution, and 1 means opposite.

The semantic relevance between topics and categories is measured by the parameter δ of the SLDA-TC model, which is calculated as shown in equation (12), where δ _k,j represents the probability that topic k belongs to category j.

In one or more embodiments, the SLDA-TC model for classification is generated through multiple iterations of training, and the iteration ends, the similarity between topics is evaluated by JS divergence, and the topic-category distribution parameter of SLDA-TC is evaluated Semantic relevance between topics and categories; SLDA-TC model text classification results evaluation metrics Macro-precision, macro-recall (Macro-Recall), and macro-average F1 value (Macro-F1).

Experimental analysis and verification:

Select three data subsets of rec, sci and talk from the English data set 20newsgroup, as well as the Sogou Chinese corpus data subsets containing 5 categories of IT, military, education, tourism and finance, and the proportion of training samples and test samples in each data subset is 8:2, and the data subsets are described in Table 1. The Chinese and English datasets are segmented by jieba, and the English stem is extracted by nltk.stem. After removing stop words, TF-IDF is used for feature selection, and 60% of the feature words are retained in the experiment.

Table 1. Dataset description

data set number of training texts Number of categories number of features 20news-rec 3979 4 47067 20news-sci 2373 4 57943 20news-talk 1676 3 40945 sogou 2445 5 70819

In order to verify the effectiveness of the proposed method, the three algorithms of SLDA-TC, LDA-TC and SVM are compared experimentally. SLDA-TC is an algorithm proposed in this disclosure, LDA-TC is a direct classification in traditional LDA, and SVM is an SVM classification algorithm that uses K topics of the LDA model as features.

The goal of the topic inference of the SLDA-TC model is to establish a mapping between topics and categories. The number of topics K is related to the number of categories C in the labeled training set. Experiments show that K only needs to take a value slightly larger than C, and SLDA-TC can achieve good classification accuracy. In the experiment, we analyzed the JS distance between topics generated by the SLDA-TC topic model, the probability distribution of the top 10 feature words of each topic, and the similarity between classes and topics. Tables 2 to 4 describe the experimental results of the SLDA-TC topic model generated when the sogou data subset C=5 and K=8, and Tables 6 to 8 describe the experimental results on the 20news-talk data subset , where α, β and γ of the SLDA-TC topic model are 0.01.

Table 2 JS divergence between subjects (SLDA-TC, sogou, C=5, K=8)

Table 3 Relevance between topics and classes (SLDA-TC, sogou, C=5, K=8)

Table 4 Probability distribution of the top 10 words of the topic (SLDA-TC, sogou, C=5, K=8)

As shown in Table 2, the JS divergence between topics 2, 5, and 7 is 0, indicating that they are topics with the same distribution. It can be seen from Table 4 that the probability distribution of the top 10 feature words of these three topics is also the same of. The JS divergence between the other 5 topics is between 0.45 and 0.58, indicating that there are 5 different topics scattered. As can be seen from Tables 3 and 4, topic 0 maps to the category "IT", topic 1 maps the category "tourism", topic 3 maps the category "Finance", topic 4 maps the category "military", topic 6 maps the category "education", The similarity is above 99%, and topics 2, 5, and 7 are irrelevant to any category, and are called "useless" topics.

Table 5. Inter-topic JS divergence (SLDA-TC 20news-talk, C=3, K=6)

Table 6. Relevance between categories and topics (SLDA-TC, 20news-talk, C=3, K=6)

Table 7 Probability distribution of the top 10 words of the topic (SLDA-TC, 20news-talk, C=3, K=6)

Tables 5 to 7 show the experimental results on 20news-talk. Topics 1, 3 and 4 are "useless" topics with the same distribution, meaningful topic 0 corresponds to the category talk.politics.guns, and topic 2 corresponds to talk. politics.misc, topic 5 corresponds to talk.politics.mideast.

After a lot of experiments, we found that when K-C≥2, K-C "useless" topics of topics with JS divergence of 0 will be generated, so K only needs to choose a value slightly larger than C, so the K of the SLDA-TC topic model is easily determined. The SLDA-TC topic model can filter out K-C "useless" topics and establish an accurate mapping between topics and categories. At the same time, K is only slightly larger than C, much lower than the K value of LDA, which can significantly reduce the training time of the model.

Figures 4(a) to 7(c) describe the three sub-data sets of sogou Chinese and 20newsgroup. After TF-IDF feature selection, 60% of the feature words are retained, and the number of topics K takes different values to iterate 10 times to generate Comparison of classification results between SLDA model and LDA model, SLDA-TC, LDA-TC and SVM.

In FIGS. 4( a ) to 7 ( c ), the abscissas all represent the number of iterations.

Figures 4(a) to 7(c) describe the comparison of the classification results of SLDA-TC, LDA-TC and SVM when the number of topics K on the four datasets takes different values.

(1) The subject K of SLDA-TC only needs to be slightly larger than the number of categories C, and the classification result is better than that of LDA-TC and SVM.

As shown in Fig. 4(a)-Fig. 4(c), when the 20news-rec dataset C=4, K=8, the SLDA-TC is 95.10% for the Macro-Precision, Macro-Recall and Macro-F1 classification metrics , 94.99%, and 94.98%, while LDA-TC was 63.76%, 60.91%, and 60.33%, and SVM was 68.82%, 68.33%, and 68.08%. With the increase of K, LDA-TC and SVM are improved. When K=80, LDA-TC reaches 71.85%, 71.38% and 71.41%, and SVM reaches 83.90%, 83.70% and 83.62%, which are also lower than SLDA-TC. In addition, K=80, the training time of its topic model is much higher than that of the SLDA-TC model K=6.

As shown in Figure 5(a)-Figure 5(c), when the sogou dataset C=5, K=8, the SLDA-TC is 92.80% for the Macro-Precision, Macro-Recall and Macro-F1 classification metrics , 92.73%, and 92.70%, while LDA-TC was 72.67%, 68.89%, and 67.48%, and SVM was 80.69%, 80.40%, and 80.28%. With the increase of K, the classification index of SVM gradually improves. When K is greater than 60, the three classification indexes have been improved to 89.26%, 89.95% and 89.24%, but they are still lower than 92.80% of SLDA-TC when K=8 , 92.73% and 92.70%, and SVM with K=60 (LDA topic as feature), pays more time cost than SLDA-TC with K=8.

(2) The K value of SLDA-TC is not as large as possible. When K is very large, the three classification indicators of SLDA-TC decrease, which shows that K only needs to be slightly larger than C.

As shown in Figure 6(a)-Figure 6(c), the 20news-sci data set C=4, when K=90, the SLDA-TC classification result is very poor. The reason is that the number of categories C=4, only 4 of the generated 90 topics are related to the category, and the remaining 86 are "useless" topics, which do not help the classification, but cause interference, resulting in poor classification results. The same is true for the 20news-talk dataset C=3, K=90 as shown in Fig. 7(a)-Fig. 7(c). A large number of experimental results show that, for the SLDA-TC algorithm, if K is slightly larger than C, high classification accuracy can be obtained.

The time performance and classification results of SLDA-TC, LDA-TC and SVM on different datasets are shown in Table 8.

Table 8. Comparison of SLDA-TC and LDA-TC, SVM time performance and classification results

The generation time of the topic model is proportional to the number of topics K. The larger the value of K, the higher the time cost. For the SLDA-TC model, the number of topics K only needs to be slightly larger than the number of categories C to obtain very good classification results, while LDA-TC and the SVM algorithm using LDA topics as features require the K value to reach tens or hundreds of times. In order to obtain better classification results, it can also be seen from the experimental results shown in Figure 4(a)-Figure 7(d).

As shown in Table 8, on 20news-rec, the LDA and SLDA-TC model generation time ratio is 4.86, that is, SLDA is 4.86 times faster than LDA, because the K of the two are 200 and 8, respectively. On the 20news-sci, 20news-talk and sogou datasets, SLDA is 4.78, 5.16 and 4.90 times faster than LDA, respectively. At the same time, from the perspective of Macro-Precision, Macro-Recall and Macro-F1 classification indicators, the SLDA-TC algorithm is significantly better than the LDA-TC and SVM algorithms. On the four data sets, SLDA-TC is 3.10% higher than SVM. 9.30% %, 7.10%～34.08% higher than LDA-TC.

To sum up, the number of topics K of the SLDA-TC model only needs to take a value slightly larger than the number of categories C, which can identify topics closely related to categories, and is significantly better than LDA-TC and LDA in both classification accuracy and time performance. Subject-based feature SVM algorithm.

Aiming at the problems existing in LDA in text classification, the present disclosure proposes a SLDA-TC text classification model based on a supervised topic model, and proposes an _SLDA -TC-Gibbs parameter estimation algorithm. When the latent variable component _zi = k is sampled, keep the other word components of z, namely unchanged, while maintaining Invariant, that is, implicit topic sampling is only performed from other training documents with the same document class label as the word, because the topic distribution of documents of the same category is similar, and a theoretical proof is given. The SLDA-TC model introduces the parameter δ of the topic-category probability distribution, via Theta and delta probability distributions, extract the implicit semantic information mapping between words and topics, documents and topics, topics and categories. In addition, the number K of SLDA-TC topics only needs to be slightly larger than the number of categories. Experiments show that the SLDA-TC model can significantly improve the classification accuracy and time efficiency.

One or more embodiments of the present disclosure further provide a text classification system, including a text input device, a controller and a display device, the controller includes a memory and a processor, the memory stores a computer program, the When the program is executed by the processor, it can implement the following steps as shown in Figure 1:

(1) Build the SLDA-TC text classification model; unlike the unsupervised LDA model, each document in the training document set of the SLDA-TC text classification model has a class label; the parameters to be estimated in the SLDA-TC text classification model not only include Text-topic probability distribution, topic-word probability distribution, and topic-category probability distribution; among them, text topic probability distribution, topic word probability distribution, and topic category probability distribution all obey Dirichlet distribution.

(2) Train the SLDA-TC text classification model and estimate the parameters of the SLDA-TC model.

Specifically, in the process of training the SLDA-TC text classification model, the number of topics K is set first, and a value slightly larger than the number of categories C is set; then the implicit topics of each word are sampled according to the SLDA-TC-Gibbs algorithm, and Only perform implicit topic sampling from other training texts with the same text category label as the word; after determining the implicit topic of each word, by counting the frequency of topic-word, document-topic, and topic-category, calculate Text-topic probability distribution, topic-word probability distribution and topic-category probability distribution; establish accurate mapping between topics and categories;

(3) Topic inference and classification of the text to be tested. Input the text to be tested into the trained SLDA-TC text classification model, firstly sample the implicit topic of each word in the document to be tested; then infer the topic probability distribution of the text to be tested; according to the topic distribution of the document to be tested and SLDA-TC The topic-category distribution of the model, which outputs the category labels of the text to be tested.

(4) SLDA-TC model and classification result evaluation. For the SLDA-TC model generated by multiple iterative training, the similarity between topics is evaluated by JS divergence, and the semantic correlation between topics and categories is evaluated by the topic-category distribution parameter of SLDA-TC; The evaluation of text classification results is evaluated by the metrics macro-average classification precision (Macro-precision), macro-average recall (Macro-Recall), and macro-average F1 value (Macro-F1).

The text classification method and system of the present disclosure, by constructing and training the completed SLDA-TC text classification model, utilizes the text topic probability distribution, the word probability distribution of the topic and the category probability distribution of the topic to extract words and topics, documents and topics, and topics. The implicit semantic information mapping between categories improves text classification accuracy and time efficiency.

As will be appreciated by one skilled in the art, embodiments of the present disclosure may be provided as a method, system, or computer program product. Accordingly, the present disclosure may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present disclosure may take the form of a computer program product embodied on one or more computer-usable storage media having computer-usable program code embodied therein, including but not limited to disk storage, optical storage, and the like.

The present disclosure is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. During execution, the processes of the embodiments of the above-mentioned methods may be included. Wherein, the storage medium may be a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM) or a random access memory (Random Access Memory, RAM) or the like.

Although the specific embodiments of the present disclosure have been described above in conjunction with the accompanying drawings, they do not limit the protection scope of the present disclosure. Those skilled in the art should understand that on the basis of the technical solutions of the present disclosure, those skilled in the art do not need to pay creative efforts. Various modifications or variations that can be made are still within the protection scope of the present disclosure.

Claims (8)

1. a text classification method based on a supervised topic model, is characterized in that, comprises: Build the SLDA-TC text classification model. Each document in the training document set of the SLDA-TC text classification model has a class label; the parameters that need to be estimated in the SLDA-TC text classification model include not only text-topic probability distribution, topic-word probability distributions, including topic-category probability distributions; The SLDA-TC text classification model is trained, and the SLDA-TC model parameters are estimated according to the SLDA-TC-Gibbs algorithm. The topic is sampled, and the implicit topic is only sampled from other training texts with the same text category label as the word; after determining the implicit topic of each word, by statistical topic-word, document-topic, topic-category The frequency of text-topic, topic-word probability distribution and topic-category probability distribution are calculated, and then an accurate mapping between topics and categories is established; Inference and classification of the subject of the text to be tested; input the text to be tested into the trained SLDA-TC text classification model, first perform implicit topic sampling for each word of the document to be tested; then infer the subject probability distribution of the text to be tested; The topic distribution of the document and the topic-category distribution of the SLDA-TC model, output the category label of the text to be tested. 2. a kind of text classification method based on supervised topic model as claimed in claim 1, is characterized in that, the parameter that needs to be estimated in SLDA-TC text classification model not only comprises text-topic probability distribution, topic-word probability distribution, Also includes topic-category probability distributions with parameters obeying Dirichlet distributions. 3. a kind of text classification method based on supervised topic model as claimed in claim 1, it is characterised in that the SLDA-TC model that is used for text classification is generated by repeatedly iterative training, the iteration ends, and the subject is assessed by JS divergence The similarity between topics and categories is evaluated by the topic-category distribution parameter of SLDA-TC. 4 . The method for text classification based on a supervised topic model according to claim 3 , wherein the evaluation indicators for the text classification results include macro-average classification precision, macro-average recall rate, and macro-average F1 value. 5 . 5. A text classification system based on a supervised topic model, comprising a text input device, a controller and a display device, the controller comprising a memory and a processor, wherein the memory stores a computer program, the program The following steps can be implemented when executed by the processor: Build the SLDA-TC text classification model. Each document in the training document set of the SLDA-TC text classification model has a class label; the parameters that need to be estimated in the SLDA-TC text classification model include not only text-topic probability distribution, topic-word probability distributions, including topic-category probability distributions; The SLDA-TC text classification model is trained, and the parameters of the SLDA-TC model are estimated according to the SLDA-TC-Gibbs algorithm. The topic is sampled, and only the implicit topic is sampled from other training texts with the same text category label as the word; after determining the implicit topic of each word, by statistical topic-word, document-topic, topic-category The frequency of text-topic, topic-word probability distribution and topic-category probability distribution are calculated, and then an accurate mapping between topics and categories is established; Inference and classification of the subject of the text to be tested; input the text to be tested into the trained SLDA-TC text classification model, first perform implicit topic sampling for each word of the document to be tested; then infer the subject probability distribution of the text to be tested; The topic distribution of the document and the topic-category distribution of the SLDA-TC model, output the category label of the text to be tested. 6 . The text classification system based on a supervised topic model according to claim 5 , wherein the text topic probability distribution, the topic word probability distribution and the topic category probability distribution all obey the Dirichlet distribution. 7 . 7. A kind of text classification system based on supervised topic model as claimed in claim 5, it is characterized in that, generate the SLDA-TC model that is used for text classification through repeated iterative training, the iteration ends, and evaluates the topic by JS divergence The similarity between topics and categories is evaluated by the topic-category distribution parameter of SLDA-TC. 8 . The text classification system based on a supervised topic model according to claim 7 , wherein the evaluation indicators of the text classification results include macro average classification accuracy, macro average recall rate and macro average F1 value. 9 .