CN105488209A - Method and device for analyzing word weight - Google Patents

Abstract

The invention discloses a word weight analysis method and device, relates to the technical field of the Internet, and solves the problem that the existing method for determining the term weight cannot accurately determine the term weight in the query under the Internet search engine environment. The method of the present invention includes: obtaining the pair of <query, title>; counting the occurrence information of each word in the word segment of the query in the pair of <query, title>; calculating the occurrence information of each word in the same word segment according to the occurrence information. Occurrence probability of a word; determining the weight of each word in the same word segment according to the occurrence probability of each word in the same word segment. The invention is mainly used for determining the term weight of the query in the search engine and improving the search quality of the search engine.

Description

A word weight analysis method and device

technical field

The invention relates to the technical field of the Internet, in particular to a word weight analysis method and device.

Background technique

With the development of the Internet, the total amount of stored data in the Internet is very large. Therefore, in order to enable users to quickly and accurately find the required data content, manufacturers that provide Internet search services need to optimize the search quality of search engines. Among them, the weight is the evaluation value given by the search engine to a webpage, and this weight can reflect the importance of the webpage. The higher the weight, the more trust and recognition the webpage has gained from the search engines. In the process of using a search engine, users will submit query content in the search box. These query content is usually called query, and the search engine needs to obtain useful information from massive data according to the query. Since there are different terms in the query, each term has different importance for obtaining useful query results, so if you want to accurately obtain the target query results according to the query, you need to refer to the importance of each term in the query, and also It is necessary to use the weight of the term in the query to query the target results.

In the existing methods for determining term weights, common clicks, parts of speech, and named entities are usually used to determine term weights, but these methods are not based on users using search engines to obtain content in the Internet environment, resulting in the determination of term weights by the above methods. The reference value of the term weight in the Internet search field is not high. Therefore, how to determine the term weight in the Internet search engine environment becomes an urgent problem to be solved when using the Internet search engine.

Contents of the invention

In view of this, the present invention proposes a word weight analysis method and device, the main purpose of which is to solve the problem that the existing method for determining the term weight cannot accurately determine the term weight in the query under the Internet search engine environment.

According to the first aspect of the present invention, the present invention provides a kind of analysis method of word weight, comprising:

Get <query, title> pairs;

Count the occurrence information of each word in the word segment of the query described in the <query, title> pair;

Calculate the probability of occurrence of each word in the same word segment according to the occurrence information;

The weight of each word in the same word segment is determined according to the occurrence probability of each word in the same word segment.

Further, the obtaining <query, title> pair includes:

Obtain user click logs, which include all queries submitted by users and all titles obtained;

The click log is sorted out, and the query submitted by the user is matched with the title obtained by clicking on the url of the query to form a <query, title> pair.

Further, the occurrence information of each word in the word segment of the query in the statistics <query, title> pair includes:

Obtain all word fragments of the query in the <query, title> pair, where the word fragments include each word in the query and a phrase composed of two or more adjacent words;

The occurrence information of each word in all the word segments of the query is counted.

Further, counting the occurrence information of each word in all word segments of the query includes:

Judging whether each word in the word segment of the query appears in the corresponding title in the <query, title> pair of the query;

The appearance information of each word in the queried word segment is counted according to the judgment result, and the appearance information is represented by a preset appearance symbol and a non-appearance symbol.

Further, calculating the occurrence probability of each word in the same word segment according to the occurrence information includes:

Obtain the total number of all titles corresponding to the same word segment;

Obtain the number of occurrences of each word in the same word segment in all the corresponding titles;

The occurrence probability of each word in the same word segment in all the corresponding titles is obtained by dividing the number of times by the total number of all corresponding titles.

Further, determining the weight of each word in the same word segment according to the occurrence probability of each word in the same word segment includes:

The occurrence probability of each word in the same word segment in all corresponding titles is used as the weight of each word in the same word segment.

According to a second aspect of the present invention, the present invention provides a word weight analysis device, including:

Acquisition unit, used to obtain the pair of <query, title>;

A statistical unit, configured to count the occurrence information of each word in the word segment of the query in the <query, title> pair obtained by the acquisition unit;

A computing unit, configured to calculate the occurrence probability of each word in the same word segment according to the occurrence information;

A determining unit, configured to determine the weight of each word in the same word segment according to the occurrence probability of each word in the same word segment calculated by the calculation unit.

Further, the acquisition unit includes:

An acquisition module, configured to acquire user click logs, which include all queries submitted by users and all titles obtained;

A collating module, configured to sort out the click logs acquired by the acquisition module, and make a one-to-one correspondence between the query submitted by the user and the title obtained by clicking on the url of the query to form a <query, title> pair.

Further, the statistical unit includes:

Segmentation module, used to obtain all word fragments of the query in the <query, title> pair, the word fragments include each word in the query and a phrase composed of two or more adjacent words;

A statistical module, configured to count occurrence information of each word in all word segments of the query obtained by the segmentation module.

Further, the statistical unit is also used to judge whether each word in the word segment of the query appears in the title corresponding to the <query, title> pair of the query, and count the words of the query according to the judgment result The appearance information of each word in the segment, the appearance information is represented by preset appearance symbols and non-appearance symbols.

Further, the calculation unit includes:

The counting module is used to obtain the total number of all titles corresponding to the same word segment;

The counting module is also used to obtain the number of times each word in the same word segment appears in all corresponding titles;

A calculation module, configured to divide the number of times by the total number of all the corresponding titles to obtain the occurrence probability of each word in the same word segment in all the corresponding titles.

Further, the determining unit is configured to use the occurrence probability of each word in the same word segment in all corresponding titles as the weight of each word in the same word segment.

By virtue of the above-mentioned technical solution, a word weight analysis method and device provided by the embodiments of the present invention can obtain <query, title> pairs in the process of large-scale use of Internet search engines by users, and count the word fragments in the query Occurrence information of each word in, calculate the occurrence probability of each word in the same word segment according to the occurrence information of each word, determine the occurrence probability of each word in the same word segment according to the occurrence probability of each word in the same word segment weight of words. However, in the prior art, when determining the weight of a word in a search query, it cannot be based on content obtained by using a search engine in an Internet environment, resulting in an inaccurate determination of the weight of a search word, thereby affecting the accuracy of the search result. Compared with this defect in the prior art, the present invention can accurately determine the weight of words in the search query under the Internet search engine environment based on the log formed by the user's large-scale use of search engine clicks, thereby effectively improving the weight of the search results. accuracy.

The above description is only an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention, it can be implemented according to the contents of the description, and in order to make the above and other purposes, features and advantages of the present invention more obvious and understandable , the specific embodiments of the present invention are enumerated below.

Description of drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiment. The drawings are only for the purpose of illustrating a preferred embodiment and are not to be considered as limiting the invention. Also throughout the drawings, the same reference numerals are used to designate the same components. In the attached picture:

Fig. 1 shows the flow chart of the analysis method of a kind of word weight that the embodiment of the present invention provides;

FIG. 2 shows a block diagram of a word weight analysis device provided by an embodiment of the present invention;

FIG. 3 shows a block diagram of another word weight analysis device provided by an embodiment of the present invention;

FIG. 4 shows a block diagram of another word weight analysis device provided by an embodiment of the present invention;

FIG. 5 shows a block diagram of another word weight analysis device provided by an embodiment of the present invention.

detailed description

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided for more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

When users use a search engine, they need to submit a query query. There are different terms in the query query, and each term has different importance for obtaining useful query results. Therefore, it is necessary to accurately obtain the target query results according to the query. You need to refer to the importance of each term in the query, that is, you need to use the weight of the term in the query to query the target results. In the existing methods for determining term weights, common clicks, parts of speech, and named entities are usually used to determine term weights, but these methods are not based on users using search engines to obtain content in the Internet environment, resulting in the determination of term weights by the above methods. The reference value of the term weight in the Internet search field is not high.

In order to solve the above problems, the embodiment of the present invention provides a method for analyzing word weights, which can accurately determine the weight of each keyword term in the query submitted by the user based on the Internet search engine environment, as shown in Figure 1, the method includes:

101. Obtain the <query, title> pair.

When the user uses a search engine to query the desired content, he needs to submit a query query containing the keyword term. The search engine matches the query submitted by the user to some relevant title titles for the user to click and watch. When the user clicks on the relevant title, this In the embodiment of the invention, the query submitted by the user and the clicked title can be combined to form a <query, title> pair, which can also be recorded as a <query, title> pair.

102. Count the occurrence information of each word in the word segment in the query in the <query, title> pair.

Because the search engine needs to adjust the search strategy according to the importance of each word term in the query when searching for corresponding content on the Internet according to the query submitted by the user, and the more times the term in the query appears in the corresponding title of the query, it means The term in the query is more important, so the embodiment of the present invention needs to perform step 102 to count the occurrence information of each word in the word segment of the query in the large-scale <query, title> pair, and determine the occurrence information of each word in the word segment according to the occurrence information. The importance of words.

103. Calculate the occurrence probability of each word in the same word segment according to the occurrence information.

Because the embodiment of the present invention needs to count large-scale <query, title> pairs, therefore contain a large amount of identical word fragments in the query of all statistics, for the same word fragment ABC, in all queries that comprise word fragment ABC, each query In the corresponding title, some titles contain term-A, some titles do not contain term-A; some titles contain term-B, some titles do not contain term-B; some titles contain term-C, and some titles do not contain term -C. That is to say, each word in the same word segment has different occurrence probabilities in the titles corresponding to all queries containing the same word segment, so the importance of each word in the same word segment is also different. Therefore, the embodiment of the present invention needs to perform step 103 to calculate the occurrence probability of each word in the same word segment according to the occurrence information of each word in the title corresponding to all queries containing the same word segment.

104. Determine the weight of each word in the same word segment according to the occurrence probability of each word in the same word segment.

Since weight is a relative concept, for a certain indicator, the weight of the indicator refers to the relative importance of the indicator in the overall evaluation. For the embodiment of the present invention, the weight of a certain term refers to the relative importance of the term in the word segment of the query where it is located, and the term with higher importance appears in the title corresponding to the query where the word segment is located The higher the probability, so after calculating the occurrence probability of each word in the same word segment in the title corresponding to all queries containing the same word segment in step 103, it can be based on each word in the same word segment The occurrence probability of determines the weight of each word in the same word fragment, so that the search engine can adjust the search strategy according to the term weight determined by the large-scale statistical <query, title> pair, and improve the accuracy of the search results.

The method for analyzing word weights provided by the embodiments of the present invention can obtain <query, title> pairs during the large-scale use of Internet search engines by users, and count the occurrence information of each word in the word fragments in the query , calculating the occurrence probability of each word in the same word segment according to the occurrence information of each word, and determining the weight of each word in the same word segment according to the occurrence probability of each word in the same word segment. However, in the prior art, when determining the weight of a word in a search query, it cannot be based on content obtained by using a search engine in an Internet environment, resulting in an inaccurate determination of the weight of a search word, thereby affecting the accuracy of the search result. Compared with this defect in the prior art, the present invention can accurately determine the weight of words in the search query under the Internet search engine environment based on the log formed by the user's large-scale use of search engine clicks, thereby effectively improving the weight of the search results. accuracy.

In order to better understand the above-mentioned method shown in FIG. 1 , as a refinement and extension of the above-mentioned implementation manner, the embodiment of the present invention will describe the steps in FIG. 1 in detail.

Usually, a user will generate a large number of click logs during the use of the Internet, and the click log information includes data such as a query submitted by the user in a search engine, a URL clicked by the query, and a title corresponding to the URL. Since the query submitted by the user and the title obtained by clicking the url of the query usually have a corresponding relationship, the data basis for determining the weight of the search keyword term under the Internet search engine environment can be obtained through large-scale statistics of click log information. When a user submits a query, he sometimes clicks multiple urls to obtain multiple related titles, and the quality of these titles, that is, the degree of matching with the query, also varies. Therefore, the embodiment of the present invention needs to sort out the obtained click logs , correspond the query and title in the click log one by one, and get the pair of <query, title>. Among them, since the user may click multiple urls to obtain multiple corresponding titles when submitting a query, the same query will also have multiple <query, title> pairs in the obtained large-scale <query, title> pairs right.

After the user submits a query in the search engine, the search engine needs to adjust the search strategy according to the relative importance of each term (keyword) in the query, that is, the weight, so as to obtain accurate search results. The importance of each term in the query can be expressed by the appearance of the term in the corresponding title of the query. If a term in a large number of queries appears in the corresponding title, the more important the term is. Since each query will contain a variety of word fragments, the word fragments include each term in the query and a phrase composed of two or more adjacent terms, and each query will also contain the same word fragments, just the same For a word segment, the more times a term in the same word segment appears in titles corresponding to all queries containing the word segment, the more important the term is in the word segment. Therefore, the embodiment of the present invention needs to count the occurrence information of each term in all query word segments. In order to count the occurrence information of each term in the word fragments of all queries, the embodiment of the present invention needs to carry out word segmentation to all queries, that is, process all <query, title> pairs, carry out word segmentation to each query, and obtain the A phrase composed of each term and two or more adjacent terms is the above-mentioned word segment, and the occurrence information of each term in its corresponding title in the word segment is counted.

When counting the occurrence information of each term in all word segments of each query, it can be represented by preset occurrence symbols and non-appearance symbols. That is to judge whether each term in the word segment of the query appears in the title corresponding to the pair of <query, title> of the query, if it appears, it will be represented by the default appearance symbol, if it does not appear, it will be represented by the default non-existing symbol. Symbols appear. For example, for <query: ABCD, title: CDEFG>, a word segment in its query is ABC, and term-A in this word segment ABC does not appear in title: CDEFG, then it is represented by the symbol 0 that does not appear; term If -B does not appear in title: CDEFG, it will be represented by the non-appearing symbol 0; if term-C appears in title: CDEFG, it will be represented by the appearing symbol 1. Therefore, it is enough to count the occurrence information of each term in the word segment ABC Expressed with ABC:001.

After the occurrence information of each term in the word segment of the query in the <query, title> pair is determined through the above method, the occurrence probability of each term in the same word segment can be calculated. Specifically, when calculating the occurrence probability of each term in the same word segment, it is necessary to obtain the total number of all titles corresponding to the same word segment. For the same word segment, it is the total number of all <query, title> pairs that contain the same word segment in the query. Among the total numbers of all these <query, title> pairs, some <query, The title in the title> pair contains the term of the same word segment, and the title in some <query, title> pairs does not contain the term of the same word segment, so when obtaining all the terms corresponding to the same word segment After the total number of titles, it is also necessary to obtain the number of times each term in the same word segment appears in all the titles, that is, the number of titles containing a certain term in all titles. Divide the number of occurrences of each term in all titles in the same word segment by the total number of corresponding titles to obtain the probability of occurrence of each term in all corresponding titles in the same word segment.

For the same word segment, the higher the frequency of occurrence of a term in the title corresponding to its query, the more important the term is, so according to the calculated occurrence probability of each term in the same word segment, it can be determined that the same The weight of each term in the word segment. As an optional implementation manner, in this embodiment of the present invention, the occurrence probability of each term in all corresponding titles in the same word segment may be used as the weight of each term in the same word segment.

In order to better understand the above method, the embodiment of the present invention will take two <query, title> pairs as an example to describe the above process in detail. The two <query, title> pairs are <query: ABC, title: CDEF>, <query: ABCDE, title: FGACDHJ> respectively. Wherein, if the term in the query appears in the corresponding title, it is represented by the occurrence symbol 1, and if the term in the query does not appear in the corresponding title, it is represented by the non-appearance symbol 0.

When counting the appearance of each term in the word segment of the query in the <query, title> pair, first, it is necessary to segment the query in the <query, title> pair to obtain all the word segments, and then count the occurrence of each term in the word segment Occurrence, that is, the word fragment in the query is used as the key, and the occurrence of the term contained in the word fragment in the corresponding title is output as the value. The processing results are as follows:

1) In the <query: ABC, title: CDEF> pair,

Contains 1 term: A:0, B:0, C:1

Contains 2 terms: AB:00, BC:01

Contains 3 terms: ABC:001

2) In the <query: ABCDE, title: FGACDHJ> pair,

Contains 1 term: A:1, B:0, C:1, D:1, E:0

Contains 2 terms: AB:10, BC:01, CD:11, DE:10

Contains 3 terms: ABC:101, BCD:011, CDE:110

Contains 4 terms: ABCD:1011, BCDE:0110

Contains 5 terms: ABCDE:10110

After processing all <query, title> pairs, it is necessary to merge the same word segments according to the occurrence information of each term in the word segment, that is, to calculate the occurrence probability of each term in the same word segment. Taking the word segment ABC as an example, in the pair of <query: ABC, title: CDEF>, the value of the word segment ABC is 001; in the pair of <query: ABCDE, title: FGACDHJ>, the value of the word segment ABC is 101 , where term-A does not appear in the title of the <query: ABC, title: CDEF> pair, but appears in the title of the <query: ABCDE, title: FGACDHJ> pair, so term-A appears in the title The probability of term-B in the title is 0.5; similarly, the probability of term-B appearing in the title is 0, and the probability of term-C appearing in the title is 1. Therefore, for the word segment ABC, the probability of each term appearing in the search results is ABC: 0.5, 0, 1. According to the above statistical results, when a user submits a query containing ABC in a search engine, the importance of the terms to be referred to when searching is term-C>term-A>term-B.

Of course, the above is just an illustration using two <query, title> pairs as an example, and the obtained probabilities are not yet representative, just to clearly illustrate the specific analysis process. In the process of actual analysis, it is necessary to count <query, title> pairs on a large scale according to the above method in order to obtain the reliable occurrence probability of each term in the word segment. For example, if a large number of <query, title> pairs are counted to obtain data similar to the following, word segment ABC: 0.7, 0.3, 0.9, it means the following meaning: in all queries containing word segment ABC, the clicked title contains term- The probability of A is 0.7, the probability of including term-B is 0.3, and the probability of including term-C is 0.9, so it can be considered that the importance of term-A and term-C is relatively high, while the importance of term-B is relatively low.

By the analysis method of the word weight described in the embodiment of the present invention, the word segment under the Internet search environment and the occurrence probability of the term contained in the word segment can be excavated on a large scale, such as the following two word segments: a) tomato Fish soup: 0.75, 0.82; b) How about fish soup: 0.78, 0.51. Among them, in a), it means that among the titles of all query clicks containing "tomato fish soup", 75% contain "tomato", and 82% contain "fish soup". Since "tomato" has a synonym "tomato", the actual The probability of tomatoes included in the title is even higher. In b), it indicates that in the titles of all query clicks containing "how is fish soup", "fish soup" appears more frequently than "how is it", and "fish soup" is more important than "how is it".

The embodiment of the present invention utilizes <query, title> to count whether the term in the query appears in the title, and the occurrence information is output through the value value of the word segment, and further according to the value value of each word segment in the same word segment The probability of occurrence of each term in the title, thereby obtaining the weight information of each term in the word segment, because the weight information of these terms is determined based on the click log information in the large-scale Internet search environment, so it can effectively improve the search engine. search quality.

Further, as an implementation of the method shown in FIG. 1 above, an embodiment of the present invention provides a device for analyzing word weights. As shown in FIG. 2 , the device includes: an acquisition unit 21, a statistics unit 22, a calculation unit 23, and Determining unit 24, wherein,

An acquisition unit 21, configured to acquire a pair of <query, title>;

Statistical unit 22, used to count the occurrence information of each word in the word segment of the query in the <query, title> pair obtained by statistical acquisition unit 21;

Calculation unit 23, is used for calculating the occurrence probability of each word in the same word segment according to the described occurrence information of statistics unit 22 statistics;

The determination unit 24 is configured to determine the weight of each word in the same word segment according to the occurrence probability of each word in the same word segment calculated by the calculation unit 23 .

Further, as shown in Figure 3, the acquisition unit 21 includes:

Obtaining module 211, configured to obtain user click logs, which include all queries submitted by users and all titles obtained;

The sorting module 212 is configured to sort the click logs acquired by the acquiring module 211, and correspond one-to-one between the query submitted by the user and the title obtained by clicking on the url of the query to form a <query, title> pair.

Further, as shown in Figure 4, the statistical unit 22 includes:

Segmentation module 221, is used for obtaining all word fragments of the query in <query, title>, and the word fragments include each word in the query and a phrase composed of two or more adjacent words;

The statistics module 222 is configured to make statistics on the occurrence information of each word in all the word segments of the query acquired by the segmentation module 221 .

Further, the statistical unit 22 is also used to judge whether each word in the word segment of the query appears in the title corresponding to the <query, title> pair of the query, and count the word segments of the query according to the judgment result The occurrence information of each word in , the occurrence information is represented by a preset occurrence symbol and a non-appearance symbol.

Further, as shown in Figure 5, the computing unit 23 includes:

The counting module 231 is used to obtain the total number of all titles corresponding to the same word segment;

The counting module 231 is also used to obtain the number of occurrences of each word in the corresponding all titles in the same word segment;

The calculation module 232 is configured to divide the number of times by the total number of all the corresponding titles to obtain the occurrence probability of each word in the same word segment in all the corresponding titles.

Further, the determining unit 24 is configured to use the occurrence probability of each word in the same word segment in all corresponding titles as the weight of each word in the same word segment.

The word weight analysis device provided by the embodiment of the present invention can obtain <query, title> pairs in the process of large-scale use of Internet search engines by users, and count the occurrence information of each word in the word fragments in the query , calculating the occurrence probability of each word in the same word segment according to the occurrence information of each word, and determining the weight of each word in the same word segment according to the occurrence probability of each word in the same word segment. However, in the prior art, when determining the weight of a word in a search query, it cannot be based on content obtained by using a search engine in an Internet environment, resulting in an inaccurate determination of the weight of a search word, thereby affecting the accuracy of the search result. Compared with this defect in the prior art, the present invention can accurately determine the weight of words in the search query under the Internet search engine environment based on the log formed by the user's large-scale use of search engine clicks, thereby effectively improving the weight of the search results. accuracy.

In addition, the embodiment of the present invention uses <query, title> to count whether the term in the query appears in the title, and the occurrence information is output through the value value of the word segment, and the same word is further counted according to the value value of each word segment The probability of occurrence of each term in the title in the segment, thus obtaining the weight information of each term in the word segment, since the weight information of these terms is determined based on the click log information in the large-scale Internet search environment, it can effectively improve the Search engine search quality.

In the foregoing embodiments, the descriptions of each embodiment have their own emphases, and for parts not described in detail in a certain embodiment, reference may be made to relevant descriptions of other embodiments.

It can be understood that related features in the above methods and devices can refer to each other. In addition, "first", "second" and so on in the above embodiments are used to distinguish each embodiment, and do not represent the advantages and disadvantages of each embodiment.

Those skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

The algorithms and displays presented herein are not inherently related to any particular computer, virtual system, or other device. Various generic systems can also be used with the teachings based on this. The structure required to construct such a system is apparent from the above description. Furthermore, the present invention is not specific to any particular programming language. It should be understood that various programming languages can be used to implement the content of the present invention described herein, and the above description of specific languages is for disclosing the best mode of the present invention.

In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure the understanding of this description.

Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, in order to streamline this disclosure and to facilitate an understanding of one or more of the various inventive aspects, various features of the invention are sometimes grouped together in a single embodiment, figure, or its description. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of this invention.

Those skilled in the art can understand that the modules in the device in the embodiment can be adaptively changed and arranged in one or more devices different from the embodiment. Modules or units or components in the embodiments may be combined into one module or unit or component, and furthermore may be divided into a plurality of sub-modules or sub-units or sub-assemblies. All features disclosed in this specification (including accompanying claims, abstract and drawings) and any method or method so disclosed may be used in any combination, except that at least some of such features and/or processes or units are mutually exclusive. All processes or units of equipment are combined. Each feature disclosed in this specification (including accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Furthermore, those skilled in the art will understand that although some embodiments described herein include some features included in other embodiments but not others, combinations of features from different embodiments are meant to be within the scope of the invention. and form different embodiments. For example, in the following claims, any one of the claimed embodiments may be used in any combination.

The various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or a digital signal processor (DSP) can be used in practice to implement some or all of the components in the title of the invention (such as the device for determining the link level in the website) according to the embodiment of the present invention some or all of the features. The present invention can also be implemented as an apparatus or an apparatus program (for example, a computer program and a computer program product) for performing a part or all of the methods described herein. Such a program for realizing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such a signal may be downloaded from an Internet site, or provided on a carrier signal, or provided in any other form.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means can be embodied by one and the same item of hardware. The use of the words first, second, and third, etc. does not indicate any order. These words can be interpreted as names.

Claims (12)

1. an analysis method of word weight, is characterized in that, described method comprises: Get <query, title> pairs; Count the occurrence information of each word in the word segment of the query described in the <query, title> pair; Calculate the probability of occurrence of each word in the same word segment according to the occurrence information; The weight of each word in the same word segment is determined according to the occurrence probability of each word in the same word segment. 2. The method according to claim 1, wherein said obtaining <query, title> pair comprises: Obtain user click logs, which include all queries submitted by users and all titles obtained; The click log is sorted out, and the query submitted by the user is matched with the title obtained by clicking on the url of the query to form a <query, title> pair. 3. The method according to claim 1, wherein the occurrence information of each word in the word segment of the query in the statistics <query, title> comprises: Obtain all word fragments of the query in the <query, title> pair, where the word fragments include each word in the query and a phrase composed of two or more adjacent words; The occurrence information of each word in all the word segments of the query is counted. 4. The method according to claim 3, wherein the occurrence information of each word in all word segments of the query includes: Judging whether each word in the word segment of the query appears in the corresponding title in the <query, title> pair of the query; The appearance information of each word in the queried word segment is counted according to the judgment result, and the appearance information is represented by a preset appearance symbol and a non-appearance symbol. 5. method according to claim 4, is characterized in that, calculates the probability of occurrence of each word in identical word segment according to described occurrence information and comprises: Obtain the total number of all titles corresponding to the same word segment; Obtain the number of occurrences of each word in the same word segment in all the corresponding titles; The occurrence probability of each word in the same word segment in all the corresponding titles is obtained by dividing the number of times by the total number of all corresponding titles. 6. method according to claim 5, it is characterized in that, according to the probability of occurrence of each word in the same word segment, determine the weight of each word in the same word segment comprises: The occurrence probability of each word in the same word segment in all corresponding titles is used as the weight of each word in the same word segment. 7. An analysis device of word weight, is characterized in that, described device comprises: Acquisition unit, used to obtain the pair of <query, title>; A statistical unit, configured to count the occurrence information of each word in the word segment of the query in the <query, title> pair obtained by the acquisition unit; A calculation unit, configured to calculate the occurrence probability of each word in the same word segment according to the occurrence information counted by the statistics unit; A determining unit, configured to determine the weight of each word in the same word segment according to the occurrence probability of each word in the same word segment calculated by the calculation unit. 8. The device according to claim 7, wherein the acquiring unit comprises: An acquisition module, configured to acquire user click logs, which include all queries submitted by users and all titles obtained; A collating module, configured to sort out the click logs acquired by the acquisition module, and make a one-to-one correspondence between the query submitted by the user and the title obtained by clicking on the url of the query to form a <query, title> pair. 9. The device according to claim 7, wherein the statistical unit comprises: Segmentation module, used to obtain all word fragments of the query in the <query, title> pair, the word fragments include each word in the query and a phrase composed of two or more adjacent words; A statistical module, configured to count occurrence information of each word in all word segments of the query obtained by the segmentation module. 10. The device according to claim 9, wherein the statistical unit is further configured to determine whether each word in the word segment of the query is in the corresponding title of the <query, title> pair of the query Appearance, and counting the occurrence information of each word in the word segment of the query according to the judgment result, the occurrence information is represented by a preset appearance symbol and a non-appearance symbol. 11. The device according to claim 10, wherein the computing unit comprises: The counting module is used to obtain the total number of all titles corresponding to the same word segment; The counting module is also used to obtain the number of times each word in the same word segment appears in all corresponding titles; A calculation module, configured to divide the number of times by the total number of all the corresponding titles to obtain the occurrence probability of each word in the same word segment in all the corresponding titles. 12. The device according to claim 11, wherein the determining unit is used to use the probability of occurrence of each word in the same word segment in all corresponding titles as the probability of each word in the same word segment the weight of.