CN101887460A - A Document Quality Evaluation Method and Its Application - Google Patents

Abstract

The invention provides a document quality evaluation algorithm applied in a document sharing platform, the algorithm comprising the following steps: constructing an academic network graph using the relationship between document-document, document-journal meeting and author; The transition relationship among them is modeled to obtain the transition probability matrix; the user's collection behavior of documents is used to build a model, and the document quality value based on user analysis is calculated; the random walk iterative algorithm with restart is performed on the graph to obtain the document quality, Information on journal conference quality and author academic reputation. The present invention combines user behavior information with document quality evaluation for the first time, and can also give the analysis results of author's academic reputation and academic quality of journal conferences when giving document quality analysis results. The ranking effect of this method is compared with other methods There is a significant improvement.

Description

A Document Quality Evaluation Method and Its Application

technical field

The invention relates to a document quality assessment method, in particular to a document quality assessment method on a document sharing platform, and belongs to the technical field of knowledge mining.

Background technique

In recent years, with the rapid development of scientific research, the publication speed of scientific and technological literature has increased year by year, and the number has become very large. For example, there are more than 1.5 million scientific and technological literature on CiteSeerX, a digital library only for the field of computer and information science. Scientific researchers need to read and refer to a large number of scientific and technological documents in the process of conducting research work. The value of high-quality documents and low-quality documents to scientific researchers is very different. Obtaining high-value scientific literature has become a very difficult task. Therefore, the research topic of how to effectively and automatically evaluate the quality of scientific and technological literature has attracted more and more researchers.

On the social literature sharing and exchange website in the field of academic research, users can collect scientific and technological literature that they think are more valuable, mark tags, make comments, and share these literature with other users. User's collection behavior should be an important reference when analyzing the quality of scientific and technological literature, but currently there are very few studies that use user behavior to analyze the quality of scientific and technological literature. Therefore, in the Web 2.0 environment, how to effectively apply user behavior to the quality evaluation system of scientific and technological literature is worthy of further study.

To evaluate the quality of academic papers, the existing evaluation methods in academia mainly include peer review, citation analysis and methods based on link analysis. Peer review is usually used for early evaluation of papers, such as reviewing submitted papers at conferences or journals; citation evaluation is used for later evaluation, such as evaluating the academic level of published papers by researchers.

Peer review is a comprehensive evaluation by experts and scholars in the same research field from the significance and innovation of the selected topic, research methods, the quality of research completion, and the level of paper writing. The advantage of peer review is that the evaluation of research quality by experts is meticulous and accurate. Experts can see the level of academic research by virtue of their profound academic attainments in related fields. Laxity is likely to lead to some "frauds", and peer evaluation of a large number of academic papers is time-consuming and laborious, which is not realistic.

Citation analysis is to use the citation and cited relationship between academic papers to evaluate the quality of papers by using a specific method and evaluation standard. Researchers of citation analysis methods have proposed a series of quantitative quality evaluation indicators, such as cited frequency, impact factor, etc. Compared with peer review, the evaluation method of citation analysis is simpler, and it is easy to use computer to complete automatically; at the same time, the results of citation analysis are rougher, and the relationship between citations and citations between papers must be used. There are few citations, the evaluation is often given low, and the limitations are strong.

In 1998, Brin and Page proposed the PageRank algorithm based on the link relationship between web pages to sort web pages according to their importance, and based on this, they created the Google search engine. Kleinberg proposed another link analysis algorithm HITS algorithm. Later, considering the link structure naturally formed by the citation relationship between scientific and technical literature, many researchers have solved the problem of literature quality evaluation based on the ideas of these methods.

Contents of the invention

The purpose of the present invention is to use the relationship between user behavior and document quality in the Web 2.0 environment to assist in the analysis of document quality by modeling and analyzing the relationship between documents, authors and journal conferences. The invention unifies the two analysis methods of peer review and citation analysis under the frame of random walk algorithm with restart, and gives the final analysis result.

The scheme adopted by the present invention to solve its technical problems is (flow process as shown in Figure 1):

The present invention proposes a method for evaluating the quality of documents, which is applied to a sharing platform for scientific and technological documents. On the platform, users can collect, add tags, comment on and share documents with other users. The method is characterized in that the method includes The following steps:

A. Using the citation relationship of the literature, the relationship between the literature and journal conferences and authors, and the publication time of the literature, construct a directed graph with weights, which is called an academic network graph;

B. Quantify the citation relationship of literature, the relationship between literature and journal conferences and authors into the transfer relationship between vertices on the graph, and model the transfer probability matrix on the academic network graph;

C. Use the user's collection behavior to establish a model, consider the collection time, and use the HITS algorithm to calculate a document quality value based on user analysis;

D. According to the model established in step B and step C, perform random walk iterations with restart until the result converges, and obtain the probability value of each vertex on the academic network graph. This probability value is the document quality, journal conference quality and Information about the author's academic reputation.

The method provided by the present invention can be used not only for scientific and technological literature sharing platforms, but also for paper sharing platforms or websites (documents therein refer to papers), and picture sharing platforms or websites (documents therein refer to pictures) and the like.

Beneficial effects of the present invention:

The graph-based quality assessment method applied to scientific and technological documents proposed by the present invention combines user behavior information with document quality evaluation for the first time, and can also provide the author's academic reputation and journal conference academic results when the document quality analysis results are given. quality analysis results. For example, if the present invention is applied to a scientific and technological literature retrieval website, the quality values of the results retrieved by users according to keywords can be sorted, which can help users find high-quality scientific and technological literature faster, learn about high-quality journals and conferences, and academic Author of high reputation. Experiments show that the sorting effect of this method is significantly improved compared with other methods.

Description of drawings

Fig. 1 is the general flowchart of the method for evaluating the quality of scientific and technological documents based on graphs according to the present invention;

Fig. 2 is an academic network diagram constructed according to the present invention;

Fig. 3 is a transfer relationship diagram between vertices on the academic network graph constructed according to the present invention;

Fig. 4 is a user-document collection relationship diagram constructed according to the present invention.

Detailed ways

Below in conjunction with accompanying drawing and specific embodiment the present invention is described in further detail:

Step 1: Construct a weighted directed graph, called the academic network graph, using the citation relationship of the literature, the relationship between the literature and journal conferences and authors, and the publication time of the literature.

The academic network diagram designed and constructed by the present invention is composed of three parts, and models the relationships among three entities: documents, authors, and periodical conferences. The three parts are:

● Literature citation interconnection subgraph G _dd = (V _d , E _dd ),

G _dd is a directed graph, which represents the citation relationship between documents, where V _d is a document vertex set, E _dd is an edge set, and the directed edge <d _i , d _j >∈E _dd means that document d _i cites document d _j ;

● Author-document subgraph G _ad = (V _a ∪ V _d , E _ad ),

G _ad is a bipartite graph, which represents the authorship relationship between the author and the document, where V _a is the author vertex set, E _ad is the edge set, and the undirected edge (a _i , d _j ) ∈ _{E ad} represents the author a _i writing Document d _j ;

●Journal conference-document subgraph G _cd ＝(V _c ∪V _d ，E _cd ),

G _cd is a bipartite graph, which represents the publishing relationship between journal conferences and documents, where V _c is the journal and conference vertex set, E _cd is the edge set, and the undirected edge (c _i , d _j ) ∈ E _cd represents the document d _j published in journals or conferences c _i ;

The combination of these three subgraphs is the academic network diagram, as shown in Figure 2.

Define the academic network graph as a directed graph G=(V, E). Where V is the vertex set, V=V _a ∪V _d ∪V _c , E is the edge set, E=E _dd ∪E _ad ∪E _cd . Considering that the random walk needs to be performed on a directed graph, each undirected edge in the author-document subgraph and journal meeting-document subgraph is represented here as two directed edges connecting these two vertices. For example: (c _i , d _j )→<c _i , d _j >∪<d _j , c _i >.

Step 2. Quantify the citation relationship of the literature, the relationship between the literature and the journal conference, and the author into the transition relationship between the vertices on the graph, and model the transition probability matrix on the academic network graph.

Each vertex in the academic network graph G represents an author, a document, or a journal/conference, so graph G is a heterogeneous graph containing three different types of entities. The present invention defines different transition probabilities α for transitions between different types of vertices (entities), as shown in FIG. 3 . For these transition probability parameters, define:

α _ad =α _cd =1

α _da +α _dc +α _dd =1

Among them: α _ad is the transition probability from the author vertex to the document vertex, α _cd is the transition probability from the publication site vertex to the document vertex, α _da is the transition probability from the document vertex to the author vertex, α _dc is the transition probability from the document vertex to the publication vertex The transition probability of the location vertex, α _dd is the transition probability from document vertex to document vertex.

Define W(G) as the weighted adjacency matrix of graph G, which corresponds to the weight of the relationship between different vertices in the academic network graph. According to the previous definition of academic network graph, W(G) can be decomposed into the following table A series of sub-matrices of . First, the present invention assigns initial values to each sub-matrix to obtain an initial weighted adjacency matrix; then, a weighted scoring function is applied to the initial value of the matrix to obtain the final weighted adjacency matrix; finally, the weighted adjacency matrix is used as Based on the calculation, the transition probability matrix is obtained.

The initial definitions of these sub-matrices are given below:

● Weighted adjacency matrix from document vertex to document vertex

Where t(d) represents the publication time of document d, and Γ _dd (d _i ) represents the collection of documents cited by document d _i .

●Weighted adjacency matrix from author location vertex to document vertex

作者a是文献d的第k作者。Where Γ _ad (a _i ) represents the collection of published documents by author a _i ,

Author a is the kth author of document d.

●The weighted adjacency matrix W _da (j, i)＝|Γ _da (d _j )|-k+1 from document vertex to author vertex

Where Γ _da (d _j ) represents the set of authors of document d _j , and k represents that author a _i is the kth author of document d _j .

●The weighted adjacency matrix from the document vertex to the publication site vertex

●The weighted adjacency matrix from the publishing site vertex to the document vertex

Among them, c _ik represents a certain session of conference _ci , or a certain volume of journal _ci , Γ _cd (c _im ) represents the collection of documents published on c _im , t(c _im ) represents the corresponding time of c _im (year ).

Γ _cd (ci _ik )={d|t(d)=t(ci _ik )∧d∈Γ _cd ( _ci )}

$\&ForAll;k,l,t\left(c_{\mathrm{ik}}\right)\&NotEqual;t\left(c_{\mathrm{il}}\right).$ Obviously, $\underset{k}{Y}{\mathrm{\&Gamma;}}_{\mathrm{cd}}\left(c_{\mathrm{ik}}\right)={\mathrm{\&Gamma;}}_{\mathrm{cd}}\left(c_i\right),$ and

$\&ForAll;k,l,t\left(c_{\mathrm{ik}}\right)\&NotEqual;t\left(c_{\mathrm{il}}\right).$

Next apply a weight scoring function Ф to the initial weights in the matrix:

W(i,j)=Ф(W(i,j))

The standard for a suitable weight scoring function is: this function should be a monotonically increasing function, but as the value of the independent variable increases, the growth rate of the function value gradually decreases, that is: Ф′(x)>0 and Ф″(x)<0, this method takes

Next, first define the transition probability matrix corresponding to the three sub-graphs, and finally calculate the transition probability matrix of the entire academic network graph.

● Literature citation subgraph G _dd

The document-to-document transition probability matrix is:

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; dd</span>}}<span\; class="notranslate">\; =</span>{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; dd</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; \&Element;</span>{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}}$

in,

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; dd</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; P</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; dd</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span>}{{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; dd</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span>}$

●Author-document subgraph G _ad

The transition probability matrix from author to document is:

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; ad</span>}}<span\; class="notranslate">\; =</span>{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; ad</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; \&Element;</span>{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; a</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; \&Element;</span>{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}}$

in,

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; ad</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; P</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; ad</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span>}{{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; ad</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span>}$

The transition probability matrix from document to author is:

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; da</span>}}<span\; class="notranslate">\; =</span>{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; da</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; \&Element;</span>{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; a</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; \&Element;</span>{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}}$

in,

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; da</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; P</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; da</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span>}{{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; da</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span>}$

●Journal conference - literature subgraph G _cd

The transition probability matrix from a document to a journal conference is:

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; dc</span>}}<span\; class="notranslate">\; =</span>{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; dc</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; \&Element;</span>{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; \&Element;</span>{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}}$

in,

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; dc</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; P</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; c</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; dc</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span>}{{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; dc</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span>}$

The transition probability matrix from journal conference to literature is:

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; cd</span>}}<span\; class="notranslate">\; =</span>{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; cd</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; \&Element;</span>{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; \&Element;</span>{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}}$

in,

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; cd</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; P</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; c</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; cd</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span>}{{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; cd</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span>}$

Through the transition probability matrix of the subgraph, the transition probability matrix on the academic network graph is obtained:

$\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; P</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; V</span>}}<span\; class="notranslate">\; =</span>\left[\begin{array}{ccc}{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; dd</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; dd</span>}}& {\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; da</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; da</span>}}& {\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; dc</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; dc</span>}}\\ {\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; ad</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ {\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; cd</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\end{array}\right]$

Step 3, use the user's collection behavior to establish a model, consider the collection time, and use the HITS algorithm to calculate a document quality value based on user analysis.

表示收藏关系的集合。(u，d，t)∈R，表示用户u在时刻t收藏了文献d。The present invention connects documents and users through collection behaviors to construct a user-document collection relationship graph. Users and documents are vertices in the graph, and collection behaviors are edges, as shown in FIG. 4 . The present invention defines the user-document collection system as B=(U, D, T, R), wherein U is a collection of users, D is a collection of documents, and T is a collection of a series of time points,

Represents a collection of favorites. (u, d, t) ∈ R, means that user u bookmarked document d at time t.

The quality value vector defining the document set is: q=(q ₁ , q ₂ , Λ, q _m ), where m=|D|; the expert degree vector defining the user set is: e=(e ₁ , e ₂ , Λ , e _n ), where n=|U|. Define the adjacency matrix A of the user-document collection relationship graph:

Calculating the document quality value and user expert degree is to repeat the following iterative process until the result converges:

q=e×A

e=q× ^AT

Step 4, according to the model established in step 2 and step 3, perform random walk iterations with restart until the result converges, and obtain the probability value of each vertex on the academic network graph, which is the quality of literature and the quality of journal conferences and information on the author's academic reputation.

Let d be the vector of document quality value, a be the vector of author's academic reputation, and c be the vector of journal conference quality value. Connect the vectors corresponding to the three entities into one vector: π=[d ^T , a ^T , c ^T ] ^T . The random walk algorithm with restart can be expressed by the following formula:

π _t+1 ＝cM ^T π _t +(1-c)Q, 0≤c≤1

Q is constructed in the following way:

Normalize Q(i) such that $\underset{i\&Element;V}{\mathrm{\&Sigma;}}Q\left(i\right)=\left|V\right|.$

When judging whether it is converged, subtract the π vectors obtained by two adjacent iterations, and if the difference is less than 10 ^-6 , it is judged to be converged. Assuming that the final vector is π _n , the values in it are literature quality value, author's academic reputation value and journal conference quality value.

performance evaluation

The method for evaluating the quality of scientific and technological documents of the present invention provides a quality scoring value for documents, periodical conferences and authors, and uses the sorting results obtained by this score for experimental evaluation.

Firstly, evaluate the results of the literature quality evaluation, and select the literature in three fields: "Opinion Mining", "Topic Model" and "Social Network" for evaluation. The experimental manual evaluation of literature evaluation mainly uses the method of manually scoring the quality sorting results combined with the DCG (Discounted Cumulative Gain) evaluation algorithm to evaluate. Evaluators assign different scores to different documents according to their quality, and the documents with higher scores should be ranked at the front of the sorting results. After that, use the DCG evaluation algorithm to evaluate the results. The higher the DCG value, the more the sorting results output by the algorithm meet the actual needs. The calculation formula of DCG evaluation value is:

${\mathrm{<span\; class="notranslate">\; DCG</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; score</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 2</span>}}{\overset{\mathrm{<span\; class="notranslate">\; p</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\frac{{\mathrm{<span\; class="notranslate">\; score</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}{{\mathrm{<span\; class="notranslate">\; log</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}\mathrm{<span\; class="notranslate">\; i</span>}}$

Where score _i is the score given by the evaluator to the i-th item in the ranking results.

The comparison methods used to evaluate the quality of literature are as follows:

●The literature part of the PageRank algorithm results

●The literature part of the results of the PopRank algorithm

●The literature part in the results of the Random Walk algorithm (RW) on the academic network graph

●Citation Count: The number of times the document is cited in the collection of papers used in this experiment.

The following is the evaluation result (for ease of expression, the method of the present invention is denoted as RW+U):

Opinion Mining TopicModel Social Network PageRank 13.66547 15.78191 12.03448 PopRank 17.16117 17.33343 16.13546 CitationCount 17.40092 17.21429 14.63348 RW 17.68033 17.90367 16.81558 RW+U 18.16559 18.41081 17.28261

The second is to evaluate the results of the author’s academic prestige evaluation experiment. The method is the same as that of the literature quality evaluation experiment. The comparison method is as follows:

Author section in PageRank algorithm results

Author section in PopRank algorithm results

The author part in the results of the Random Walk algorithm (RW) on the academic network graph

Publication Count: The total number of documents published by the author in the field literature collection of the experiment

Citation Count: The total number of citations of the literature published by the author in the experimental field literature collection

The evaluation results are as follows:

Opinion Mining TopicModel Social Network PubNum 12.79365 13.29129 10.64821 CitationCount 16.86091 14.12744 11.1679 PageRank 15.53911 14.77489 13.77117 PopRank 17.33779 15.87551 16.48075 RW 17.81661 16.5786 16.87568 RW+U 17.99627 16.61291 16.8852

Finally, evaluate the academic quality evaluation results of the journals. Considering that the impact factor is a commonly used journal quality evaluation method in academia, the reference standard for evaluation is the result of the modified version of the impact factor analysis method. The calculation method of the revised impact factor is as follows:

${\mathrm{<span\; class="notranslate">\; mIF</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; C</span>}}{\mathrm{<span\; class="notranslate">\; D.</span>}}$

Among them, D is the total number of documents published in journal X, and C is the sum of the number of citations of these documents.

The method for journal evaluation is the accuracy rate of the top N results, which is calculated as follows:

The following are the evaluation results:

P@50 P@80 P@100 PageRank 0.24 0.3375 0.4 PopRank 0.42 0.425 0.47 RW 0.42 0.425 0.47 RW+U 0.44 0.4375 0.48

The table above shows the yearly distribution of the average document quality values in the results of several algorithms. Listed here are the average values from 1971 to 2009, and the average value for each year is the sum of the quality values of published documents in that year divided by the number of published documents. As can be seen from the figure, the method RW and RW+U of the present invention have generally higher quality values for new documents than the other two methods, indicating that the method of the present invention solves the problem that the evaluation results of new documents in traditional methods are generally low .

It should be noted that the purpose of the published embodiments is to help further understand the present invention, and those skilled in the art can understand that various replacements and modifications are possible without departing from the spirit and scope of the present invention and the appended claims . For example, the present invention can also be applied to paper sharing platforms or websites (only need to replace documents with papers), and picture sharing platforms or websites (only need to replace documents with pictures). Therefore, the present invention should not be limited to the content disclosed in the embodiments, and the protection scope of the present invention is subject to the scope defined in the claims.

Claims (8)

1. A method for evaluating quality of literature is applied to a scientific and technical literature sharing platform, on which a user can collect, add tags, comment and share the literature to other users, and is characterized by comprising the following steps:

A. constructing an authorized directed graph called an academic network graph by using the citation relationship of the literature, the relationship between the literature and the periodical conference and the author and the publication time of the literature;

B. quantifying the citation relation of the literature, the relation of the literature and the periodical conference and the relation of the literature and the author into the transfer relation between vertexes on the graph, and modeling to obtain a transfer probability matrix on the academic network graph;

C. establishing a model by using the collection behavior of the user on the document, considering the collection time, and calculating by using a HITS algorithm to obtain a document quality value based on user analysis;

D. and C, according to the models established in the steps B and C, carrying out random walk iteration with restarting until the result is converged to obtain a probability value of each vertex on the academic network diagram, wherein the probability value is the information of the literature quality, the periodical conference quality and the academic reputation of the author.

2. The method of claim 1, wherein the academic network graph in step a is composed of three subgraphs, respectively:

● citation interconnection subfigure G_dd＝(V_d，E_dd)，

G_ddIs a directed graph representing the citation relationship between documents, where V_dIs a set of document vertices, E_ddIs a set of edges, directed edges<d_i，d_j>∈E_ddExpression document d_iReference d is made to_j；

● Author-bibliographic subfigure G_ad＝(V_a∪V_d，E_ad)，

G_adIs a bipartite graph representing the written relationship between the author and the literature, where V_aIs a set of author vertices, E_adIs an edge set, a non-directional edge (a)_i，d_j)∈E_adRepresents author a_iWrite out document d_j；

● conference of periodical-literature subgraph G_cd＝(V_c∪V_d，E_cd)，

G_cdIs a bipartite graph representing the publication relationship between a periodical conference and a document, wherein V_cIs a set of periodicals and meeting vertices, E_cdIs an edge set, no directional edge (c)_i，d_j)∈E_cdExpression document d_jPublished in periodicals or conferences c_iThe above step (1); the academic network graph is a directed graph G(V, E), wherein the set of vertices V ═ V_a∪V_d∪V_cEdge set E ═ E_dd∪E_ad∪E_cd(ii) a Author-literature subgraph G_adAnd journal conference-literature subgraph G_cdEach undirected edge in the set of undirected edges is replaced with two directed edges connecting the two vertices of the edge.

3. The method of claim 2, wherein the step B is implemented by:

B1. different transition probabilities a are defined for transitions between different types of vertices,

α_ad＝α_cd＝1

α_da+α_dc+α_dd＝1

α_adfor transition probability from author vertex to document vertex, α_cdFor transition probability from published site vertex to document vertex, α_daFor transition probability from document vertex to author vertex, α_dcFor transition probabilities from document vertices to published site vertices, α_ddIs the transition probability from document vertex to document vertex;

B2. defining a weighted adjacency matrix W (G) of the graph G, corresponding to the weights of the relations between different vertexes in the academic network graph, and decomposing W (G) into a series of sub-matrixes according to the definition of the academic network graph: w_dd，W_ad，W_da，W_dc，W_cdWherein W is_ddFor weighted adjacency matrices from document vertex to document vertex, W_adFor weighted adjacency matrices from author site vertex to document vertex, W_daFor weighted adjacency matrices from document vertex to author vertex, W_dcFor weighted adjacency matrices from document vertex to publishing point vertex, W_cdIs a weighted adjacency matrix from publication location vertex to document vertex;

B3. and (3) assigning initial values to each submatrix to obtain an initial weighted adjacency matrix:

wherein t (d) represents publication time of document d, Γ_dd(di) shows document d_iA collection of cited documents;

wherein gamma is_ad(a_i) Represents author a_iThe collection of published documents is presented in a number of publications,

author a is the kth author of document d;

c)W_da(j，i)＝|Γ_da(d_j)|-k+1

wherein gamma is_da(d_j) Expression document d_jK denotes author a_iIs document d_jThe kth author of (1);

wherein c is_ikRepresenting a meeting c_iA certain time, or periodical c_iOf a certain volume, gamma_cd(c_im) Presentation is given in c_imDocument set above, t (c)_im) Denotes c_imThe corresponding time of (d);

B4. applying a weight scoring function to the initial value of the matrix to obtain a final weighted adjacent matrix;

B5. and calculating to obtain a transition probability matrix based on the weighted adjacent matrix.

4. The method of claim 3,the weight scoring function adopted in the step B4 is a monotone increasing function, but as the value of the independent variable increases, the increase range of the function value gradually decreases, that is: phi '(x) > 0 and phi' (x) < 0, in the method taking

5. The method of claim 4, wherein the step B5 is implemented by:

i. defining a transition probability matrix for three subgraphs

Document citation subgraph G_dd

Document-to-document transition probability matrix

Wherein

<math><mrow><msub><mi>M</mi><mi>dd</mi></msub><mrow><mo>(</mo><mi>i</mi><mo>,</mo><mi>j</mi><mo>)</mo></mrow><mo>=</mo><mi>P</mi><mrow><mo>(</mo><msub><mi>d</mi><mi>j</mi></msub><mo>|</mo><msub><mi>d</mi><mi>i</mi></msub><mo>)</mo></mrow><mo>=</mo><mfrac><mrow><msub><mi>W</mi><mi>dd</mi></msub><mrow><mo>(</mo><mi>i</mi><mo>,</mo><mi>j</mi><mo>)</mo></mrow></mrow><mrow><msub><mi>&Sigma;</mi><mi>k</mi></msub><msub><mi>W</mi><mi>dd</mi></msub><mrow><mo>(</mo><mi>i</mi><mo>,</mo><mi>k</mi><mo>)</mo></mrow></mrow></mfrac><mo>;</mo></mrow></math>

Author-literature subfigure G_ad

Author to document transition probability matrix

Wherein

<math><mrow><msub><mi>M</mi><mi>ad</mi></msub><mrow><mo>(</mo><mi>i</mi><mo>,</mo><mi>j</mi><mo>)</mo></mrow><mo>=</mo><mi>P</mi><mrow><mo>(</mo><msub><mi>d</mi><mi>j</mi></msub><mo>|</mo><msub><mi>a</mi><mi>i</mi></msub><mo>)</mo></mrow><mo>=</mo><mfrac><mrow><msub><mi>W</mi><mi>ad</mi></msub><mrow><mo>(</mo><mi>i</mi><mo>,</mo><mi>j</mi><mo>)</mo></mrow></mrow><mrow><msub><mi>&Sigma;</mi><mi>k</mi></msub><msub><mi>W</mi><mi>ad</mi></msub><mrow><mo>(</mo><mi>i</mi><mo>,</mo><mi>k</mi><mo>)</mo></mrow></mrow></mfrac><mo>;</mo></mrow></math>

Document to author transition probability matrix

Wherein

<math><mrow><msub><mi>M</mi><mi>da</mi></msub><mrow><mo>(</mo><mi>j</mi><mo>,</mo><mi>i</mi><mo>)</mo></mrow><mo>=</mo><mi>P</mi><mrow><mo>(</mo><msub><mi>a</mi><mi>i</mi></msub><mo>|</mo><msub><mi>d</mi><mi>j</mi></msub><mo>)</mo></mrow><mo>=</mo><mfrac><mrow><msub><mi>W</mi><mi>da</mi></msub><mrow><mo>(</mo><mi>j</mi><mo>,</mo><mi>i</mi><mo>)</mo></mrow></mrow><mrow><msub><mi>&Sigma;</mi><mi>k</mi></msub><msub><mi>W</mi><mi>da</mi></msub><mrow><mo>(</mo><mi>j</mi><mo>,</mo><mi>k</mi><mo>)</mo></mrow></mrow></mfrac><mo>;</mo></mrow></math>

Journal conference-literature subgraph G_cd

Transition probability matrix from literature to periodical conference

Wherein

<math><mrow><msub><mi>M</mi><mi>dc</mi></msub><mrow><mo>(</mo><mi>i</mi><mo>,</mo><mi>j</mi><mo>)</mo></mrow><mo>=</mo><mi>P</mi><mrow><mo>(</mo><msub><mi>c</mi><mi>j</mi></msub><mo>|</mo><msub><mi>d</mi><mi>i</mi></msub><mo>)</mo></mrow><mo>=</mo><mfrac><mrow><msub><mi>W</mi><mi>dc</mi></msub><mrow><mo>(</mo><mi>i</mi><mo>,</mo><mi>j</mi><mo>)</mo></mrow></mrow><mrow><msub><mi>&Sigma;</mi><mi>k</mi></msub><msub><mi>W</mi><mi>dc</mi></msub><mrow><mo>(</mo><mi>i</mi><mo>,</mo><mi>k</mi><mo>)</mo></mrow></mrow></mfrac><mo>;</mo></mrow></math>

Transition probability matrix from periodical conference to literature

Wherein

<math><mrow><msub><mi>M</mi><mi>cd</mi></msub><mrow><mo>(</mo><mi>j</mi><mo>,</mo><mi>i</mi><mo>)</mo></mrow><mo>=</mo><mi>P</mi><mrow><mo>(</mo><msub><mi>d</mi><mi>i</mi></msub><mo>|</mo><msub><mi>c</mi><mi>j</mi></msub><mo>)</mo></mrow><mo>=</mo><mfrac><mrow><msub><mi>W</mi><mi>cd</mi></msub><mrow><mo>(</mo><mi>j</mi><mo>,</mo><mi>i</mi><mo>)</mo></mrow></mrow><mrow><msub><mi>&Sigma;</mi><mi>k</mi></msub><msub><mi>W</mi><mi>cd</mi></msub><mrow><mo>(</mo><mi>j</mi><mo>,</mo><mi>k</mi><mo>)</mo></mrow></mrow></mfrac><mo>;</mo></mrow></math>

ii, obtaining a transition probability matrix on the academic network diagram through the transition probability matrix of the subgraph:

<math><mrow><mi>M</mi><mrow><mo>(</mo><mi>G</mi><mo>)</mo></mrow><mo>=</mo><msub><mrow><mo>(</mo><mi>P</mi><mrow><mo>(</mo><mi>j</mi><mo>|</mo><mi>i</mi><mo>)</mo></mrow><mo>)</mo></mrow><mrow><mi>i</mi><mo>,</mo><mi>j</mi><mo>&Element;</mo><mi>V</mi></mrow></msub><mo>=</mo><mfenced open='[' close=']'><mtable><mtr><mtd><msub><mi>&alpha;</mi><mi>dd</mi></msub><msub><mi>M</mi><mi>dd</mi></msub></mtd><mtd><msub><mi>&alpha;</mi><mi>da</mi></msub><msub><mi>M</mi><mi>da</mi></msub></mtd><mtd><msub><mi>&alpha;</mi><mi>dc</mi></msub><msub><mi>M</mi><mi>dc</mi></msub></mtd></mtr><mtr><mtd><msub><mi>M</mi><mi>ad</mi></msub></mtd><mtd><mn>0</mn></mtd><mtd><mn>0</mn></mtd></mtr><mtr><mtd><msub><mi>M</mi><mi>cd</mi></msub></mtd><mtd><mn>0</mn></mtd><mtd><mn>0</mn></mtd></mtr></mtable></mfenced><mo>.</mo></mrow></math>

6. the method of claim 5, wherein the step C is implemented by:

C1. a user-document collection relationship graph is constructed,

the top point is the user and the literature, and the edge is the collection behavior; defining a user-document collection system as B ═ U, D, T, R, where U is a set of users, D is a set of documents, T is a set of a series of time points,

representing a collection relation set, (u, d, t) belongs to R, and representing that a user u collects a document d at a moment t;

C2. an adjacency matrix a of the user-document collection graph is defined,

first, a quality value vector q ═ for a document collection is defined (q)₁，q₂，Λ，q_m) Wherein m ═ D |; defining an expert degree vector e ═ (e) for a set of users₁，e₂，Λ，e_n) Wherein n ═ U |; then the adjacency matrix of the user-document collection graph

C3. Calculating the document quality value and the user expertise by repeating the following iterative process until the result converges

q＝e×A

e＝q×A^T。

7. The method of claim 6, wherein the step D is implemented by:

D1. let d be the document quality value vector, a be the author academic reputation vector, and c be the periodical conferenceThe quality value vector is formed by connecting the vectors corresponding to the three entities into a vector pi ═ d^T，a^T，c^T]^T；

D2. Using random walk algorithm with restart, using formula pi_t+1＝cM^Tπ_tQ is more than or equal to 0 and less than or equal to 1, wherein

Normalizing Q (i) such that <math><mrow><munder><mi>&Sigma;</mi><mrow><mi>i</mi><mo>&Element;</mo><mi>V</mi></mrow></munder><mi>Q</mi><mrow><mo>(</mo><mi>i</mi><mo>)</mo></mrow><mo>=</mo><mo>|</mo><mi>V</mi><mo>|</mo><mo>;</mo></mrow></math>

D3. Subtracting pi vectors obtained by two adjacent iterations, if the difference is less than 10^-6If yes, judging the convergence; suppose the resulting vector is pi_nValues therein are document quality value, author academic reputation value and periodical conference quality value.

8. Applying the method of claim 1 to: a paper sharing platform or website, a picture sharing platform or website.

Images