CN102254014B - Adaptive information extraction method for webpage characteristics - Google Patents

Abstract

The invention discloses a method for extracting information from academic homepages, the steps of which are: (1) discovering academic homepages in the Internet; (2) crawling and analyzing academic homepages, using heuristic strategies to reduce crawling of irrelevant pages (3) Parse the page into a form of DOM tree, and divide it according to the attribute and content of the element to obtain a list of cohesive text units; (4) use the information recognizer to identify the text unit, each A type of information recognizer only recognizes one type of information, and subfield extraction is also required for article information. (5) Carry out association analysis on the extraction results, use the relevance of information to eliminate ambiguity, and complete missing fields; (6) Match the extraction results with the database to eliminate redundant data, and the extraction results are stored in the form of semantic data in the in the semantic database. The invention can efficiently and accurately extract academic information from academic homepages through the combined use of heuristic rules, machine learning methods and conditional probability models.

Description

A Webpage Feature Adaptive Information Extraction Method

technical field

The invention belongs to the field of information extraction systems, and in particular relates to an information extraction method adaptive to web page features. The method is especially suitable for extracting information such as author names, email addresses, institution information, and published articles from academic homepages.

Background technique

With the advent of the information age, the Internet has gradually become the main way for people to share and obtain information. Various information is published on the Internet in the form of web pages for people to read. However, with the explosive growth of Internet information, people find it more and more difficult to find the information they need on the Internet. On the one hand, the amount of information is huge, and on the other hand, the way information is presented is very flexible and free, which increases people's ability to identify targets. The cost of information. Therefore, web page information extraction technology has become a field worthy of research in the information age.

Web page information extraction technology is developed from traditional text information extraction. Different from text information, web page content is expressed in Hypertext Markup Language (HTML), including text, pictures and other multimedia information, and tags are allowed to nest with each other to form a tree structure. The main purpose of webpage information extraction task is to extract target information from semi-structured webpage text. Web page information usually has the following characteristics: (1) discretization, information is not concentrated on a certain site, but is published by different people to different sites. (2) Heterogeneity, even the same kind of information will be presented in different ways on different websites. (3) Redundancy, the same information may appear repeatedly on multiple sites. In view of these characteristics of web page information, the web page information extraction system needs to have strong adaptability and discrimination ability.

Early web page information extraction research focused on regularization methods, from scripted extraction methods based on regular expressions to proprietary extraction languages developed later, the core idea of which is to extract specific patterns containing target information. The method of pattern extraction is the main difference of this type of system. Some systems use manual methods to extract patterns. The advantage of this is that the extracted patterns are more accurate. However, when dealing with complex extraction tasks, there will be a lot of patterns to be extracted, so the labor cost higher. In order to reduce the cost of pattern extraction, people have proposed a pattern learning system based on automatic training. The system needs to accept a set of training samples. The samples are manually marked with the target information blocks, and the learning system automatically summarizes the information from the samples. The possible matching patterns, the patterns are verified and filtered to be used for the actual extraction task. This method has a certain automatic extraction ability, but because the bottom layer still relies on the regularization method, it cannot achieve high accuracy for complex extraction tasks. In recent years, extraction methods have gradually turned to machine learning models, and some methods that were originally used in the process of natural language understanding have been applied to deal with information extraction problems and achieved good results.

The academic homepage is a site used by researchers in the academic field to display their basic personal information and research results. Different authors make different page templates to present personal information according to their preferences. Although the page styles vary, academic home pages usually contain similar information, such as author names, institution information, contact information, projects, article information, etc. It is very valuable to collect this information using an information extraction system.

Contents of the invention

The purpose of the present invention is to provide an adaptive information extraction method for webpage features, which can extract required information from academic homepages of different styles, and has the characteristics of strong adaptability, high accuracy, and strong scalability.

A method for extracting webpage feature self-adaptive information provided by the present invention is characterized in that the method comprises the following steps:

The first step is to search for sites whose type is an academic homepage from the Internet;

The second step is to analyze the searched academic homepage, regard the page of the academic homepage as a set of binary groups (L, C), where L is the URL of the link, and C is the context of the link, and then check whether L and C are Contains keywords, if it contains, go to step 3, otherwise filter out the link;

The third step is to analyze the link to obtain the document tree structure of the page, divide the page according to the attributes and contents of the tree nodes, divide it into text units T, and form a text unit set {T ₁ , T ₂ , ..., T _n }

Step 4 Extract author name N, email M, organization information U and article information set {P ₁ , P ₂ , ..., P from the text unit set {T ₁ , T ₂ , ..., T _n } _n } These four target fields are used as the preliminary extraction results;

The fifth step is to conduct correlation analysis on the preliminary extraction results obtained in step 4, use the correlation of information to eliminate ambiguity, complete the missing fields, obtain the extraction results, and store them in the result database;

Step 6 Match the elements in the article information set {P ₁ , P ₂ , ..., P _n } with the records in the result database to eliminate redundant data;

Step 7 outputs the extraction result.

The present invention provides a web page feature self-adaptive information extraction method, which uses machine learning algorithms, probability models and regularization methods in combination, and can extract the author's name, email address, institution information and Publish articles and other information. Specifically, the present invention has the following effects and advantages:

(1) Strong adaptability

Academic home pages are written by many different researchers, with a wide variety of content and layout. The present invention can well solve the problem of non-uniform page format, and automatically adapt to various changing situations;

(2) High accuracy

The core algorithm of the present invention is based on a machine learning algorithm and a probability model, and combined with the use of heuristic rules, the extraction of each target field can achieve a high accuracy rate;

(3) Strong scalability

The invention can be extended to extract other fields in the page, and its identification process can also be applied to solve other similar problems, the expansion process is simple, and the universality is strong.

Description of drawings

Fig. 1 is the overall flowchart of the extraction process of the present invention;

Fig. 2 is the flow chart that the present invention extracts author's name;

Fig. 3 is the flow chart that the present invention extracts mailboxes;

Fig. 4 is the flow chart that the present invention extracts organization information;

Fig. 5 is a flow chart of extracting article information in the present invention.

Detailed ways

The present invention will be described in detail below in conjunction with the accompanying drawings and examples.

A method for extracting web page feature self-adaptive information provided by the present invention, the steps of which include:

(1) Searching for academic homepage sites from the Internet, the process can be divided into two stages: the search stage and the judgment stage.

In the search phase, firstly, the data set of author’s name is derived from the existing literature data as the seed data, and then each author’s name in the data set is used as a keyword to search in the search engine, and the search engine returns the search results in the form of a list. Each search result usually consists of a title, a link feature and a short summary text. The search engine usually returns multiple pages of results, and stores the link feature and summary text of the first page of the search result in the candidate result list.

In the judging stage, the retrieval results in the candidate result list are firstly filtered according to link features and abstract text. A database is used in the filtering process, which contains obfuscated sites that frequently appear in the search results, called the blocked link database. The filtering strategy includes two steps. First, check whether the search results exist in the shielded link database, and directly exclude the search results located in the database. Then, for the rest of the search results, check whether the link feature is in the pattern of "~" + author's name, if it is, keep it, otherwise it will be directly excluded, and after these two steps of filtering, each of the remaining search results will be searched in turn The operation is as follows: send a page request according to its link characteristics, use the support vector machine classification algorithm to determine whether the returned page is the author’s academic homepage, if so, directly save it as the author’s academic homepage, and the judgment is over, otherwise continue to the next search result Do the same.

(2) Analyze the author's academic homepage. The author's academic homepage is usually a complete site, including many subpages, some of which contain the target information required by the system, and some are completely irrelevant. In order to improve the crawling efficiency and prevent too many useless pages from being deeply analyzed by subsequent modules and consume computing resources, the present invention uses a filtering algorithm based on a heuristic strategy. The algorithm regards the page as a set of two-tuples (L, C), where L is the URL of the link, and C is the context of the link. The algorithm checks whether L and C contain keywords such as publication, paper, and research. If If it contains, then further analyze the link (go to step (3)), otherwise filter out the link.

(3) Analyze the page to be parsed to obtain the document tree structure of the web page, divide the page according to the attributes and contents of the document tree nodes, and divide the page into several small units, called text units T, and the division result is a text unit set {T ₁ , T ₂ , ..., T _n }, the steps are as follows.

(a) Firstly, an HTML parser is used to parse the page to obtain a document tree of the page. The nodes of the document tree correspond to the HTML tags in the page, and the document tree shows the relationship between the HTML tags in the page in a tree structure.

(b) The pages are then divided. HTML tags can be divided into block-level elements and inline elements. Common block-level elements such as BR, DIV, H1, H2, LI, UL, TH, TD, TR, TABLE, etc., common inline elements such as SPAN, BOLD, A, FONT, IMG, etc. An HTML page can be regarded as a collection of block-level elements, and there are two relationships between block-level elements: parent-child relationship and sibling relationship. Block-level elements and inline elements can be nested within each other. The document tree presents these relationships in the form of tree nodes. Nodes containing block-level elements in the document tree are called block-level nodes, and other nodes are called non-block-level nodes. The nodes of the document tree are traversed, and the category of the nodes is judged To divide the page, the division steps are as follows:

(b1) Initially, the text unit set is empty;

(b2) Perform depth-first traversal on the document tree, find out all block-level nodes, generate a text unit Ti for each block-level node Ni, and divide the corresponding content of Ni in the page into Ti;

(b3) For each block-level child node Ni, judge whether it has non-block-level child nodes in the document tree, and if so, divide the corresponding content of all non-block-level child nodes in the page into Ti;

(b4) adding Ti to the text unit collection;

(b5) end.

(c) After the traversal is completed, the division of the page is completed, and the text unit set {T ₁ , T ₂ , . . . , T _n } is obtained.

(4) Extract the author's name N, mailbox M, institution information U and article information set {P ₁ , P ₂ , ..., P from the text unit set {T ₁ , T ₂ , ..., T _n } _n } These four target fields are used as the preliminary extraction results;

For different types of target fields, the following describes the extraction methods of different fields:

The extraction process of the author’s name N is shown in Figure 2, and the basic steps are as follows:

(a1) Use the support vector machine classification algorithm to classify the text units in the text unit set {T ₁ , T ₂ , ..., T _n }, and keep the text unit set T _name whose category is the author's name;

(a2) Use the author's name database to match the author _'s name part from T _name . The author's name database is a database prepared in advance. This database collects and organizes common English male and female names and some Chinese pinyin. Match the set of candidate author names;

(a3) Extract the text in the title of the author's academic homepage. Most of the time, the title of the author's academic homepage will contain the author's name XXX in the form of "XXX's Hompage", and extract the author's name XXX in the title of the author's academic homepage;

(a4) Use the author name XXX obtained in (a3) to match the candidate author names obtained in (a2), and select the name with the highest degree of matching with XXX as the author name N to output.

The extraction process of mailbox M is shown in Figure 3, and its basic steps are as follows:

(b1) First use a support vector machine classifier to find out a possible email candidate text unit set T _Email from the text unit set {T ₁ , T ₂ , . . . , T _n }. The input features of the support vector machine include common symbols in mailbox information, such as "Email", "", ".", etc. Find these characteristic symbols in T _Email and generate a characteristic vector. The support vector machine algorithm judges the mailbox candidate text unit in T _Email according to the feature vector, if the classification result is affirmative, then proceed to (b2) processing, otherwise directly filter it out.

(b2) Remove the redundant part of the email candidate text unit, such as the suggestive prefix "Email:", which is beneficial to the subsequent steps to obtain legal email information.

(b3) Next, the fuzzy matching state machine algorithm is used to match the mailbox candidate text units. A standard mailbox has the following fields: username (provider domain name.)+.top-level domain name. The algorithm builds a matching node for each field, uses a state machine to enumerate possible matching forms, and generates many different matching results, usually dozens.

(b4) Compare each field of the mailbox candidate text unit with the matching result, select the result with the largest matching degree as the final result, and convert it into a standardized legal mailbox format according to the standard mailbox field for output.

The extraction process of institutional information U is shown in Figure 4, and its basic steps are as follows:

(c1) Firstly, collect the data of global universities and research institutes from the Internet, including the name of the institution and its corresponding homepage link, and establish a database of the institution's homepage. Create an inverted index for the database. The inverted index supports fast keyword search and can quickly determine the entries containing a set of keywords.

(c2) Use the support vector machine classifier to find out the possible organization information text unit set T _U from the text unit set {T ₁ , T ₂ , ..., T _n }, and convert the organization information text unit in T _U In text form, use it as a keyword to search in the index to obtain the top three search results. Perform fuzzy matching on the first three search results and the corresponding organization information text unit, if it can be matched, it is determined that the text corresponds to the organization, and the matching result with the highest matching degree is output, otherwise, if none of them can be matched, go to ( c3) processing.

(c3) Use the URL of the home page to search. Academic sites are usually sub-sites of institutional sites, so match the domain name of the home page with the institutional home page database. If there is a matching record, the author is considered to belong to the institution, and the matching record will be output as a result.

The process of extracting article information {P ₁ , P ₂ , ..., P _n } is shown in Figure 5, and its basic steps are as follows:

(a) First use the support vector machine classification algorithm to classify the text units, and screen out the text units that may contain article information. The accuracy of the classification algorithm is closely related to the final recognition accuracy of the article information. The classification algorithm needs to filter out similar information that is easy to confuse, such as course information, patents, and projects. The accuracy of the classification algorithm mainly depends on two aspects: the selection of training samples and features. The construction of training samples follows an iterative method, by continuously adding error samples to the training set to correct the original model. A feature vector consists of a set of discriminative vocabulary vectors. After screening by the classification algorithm, irrelevant text units are excluded, and candidate article information text units are obtained.

(b) Then perform sequential labeling on the information text unit of the candidate article, and extract each subfield in the candidate text, including: author name, title, conference journal name, and year. The algorithm for sequence labeling is based on a conditional random field model in which the following features are used:

① Text features

a) The term itself, both original and root forms

b) Case features, including initial capitalization, full capitalization, single capitalization

c) Numerical features, all numbers, a mixture of numbers and letters, Roman letters

d) Punctuation features, commas, quotation marks, periods, etc.

e) HTML tag features, tag start, middle part and end part

② Mode characteristics

a) Year characteristics, 19XX or 20XX

b) Page mode, XXX-XXX

③Dictionary features

Author's name, geographic location, publisher, time, conference journal name, institution name

④ Terminology Features

Common vocabulary in literature data, such as pp/editor/volume, etc.

The above-mentioned features are extracted from the information text units of the candidate articles, and the feature functions in the conditional random field model use the true value form, that is, the function output is yes or no. After the calculation of the model, the most probable annotation form of the information text unit of the candidate article is given. Symbols with the same label will be merged into corresponding subfields, such as author name field, title field, conference journal field, year field, etc., and then perform corresponding subsequent processing on these fields respectively.

(c) The author name field contains the entire author list and needs to be split into individual author forms. The segmentation algorithm is based on heuristic rules, mainly based on the length of the name, abbreviation and punctuation. The split results are stored in an array.

The title field needs to be normalized and cropped to be the final result. The main purpose of cropping is to remove illegal characters in the prefix and suffix, such as punctuation marks, boundary errors, etc.

In practice, there are many ways to express the names of conference journals, such as abbreviations of capital letters and common customary names. The directly extracted conference journal fields cannot be used as the final result, and need to be matched with those in the database. The bibliographic journal database collects common conference and journal titles and their corresponding abbreviations. First, extract the uppercase abbreviation part of the field to be recognized, search it in the database, and if it matches, fuzzy match the full name of the match with the input field to prevent errors caused by the conflict of abbreviations. If it matches, output the result directly. Otherwise, build an index for the title of the conference journal, search the field to be matched in the index, and perform fuzzy matching on the search result and the field to be matched. Output the result if a match is found.

The year field uses a regularization method, using regular expressions to find legal year patterns in the input text. There are two forms of legal year patterns: the first starts with 19 or 20 and is a four-digit number; the second starts with the initials of the conference journal name in capital letters, followed by quotation marks and the year. Using these two modes can handle most of the situations in practice, and the recognition accuracy rate exceeds 99%.

( ₅ ) Perform missing _field completion _and Disambiguation is eliminated, the final extraction result is obtained, and stored in the result database.

The information contained in the actual page may be missing and irregular to a certain extent, and multiple results may be identified for the same information item, which requires further judgment. This process uses the correlation between information to complete the extraction results and further judge the ambiguous results. Information association includes the following situations:

(a) the association between the author's name and the email username;

(b) The association between institutional information and the domain name of the homepage;

(c) the association of the author's name with the list of authors in the article information;

According to the above association, the extraction results can be supplemented. For example, when the institution information is missing, the home page can be linked to the database for query to obtain the corresponding institution information. In terms of information disambiguation, when there are multiple mailboxes, the corresponding relationship between author name and user name can be used to eliminate wrong results.

(6) Match the elements in the article information set {P ₁ , P ₂ , . . . , P _n } with the records in the result database to eliminate redundant data.

Although the extraction process has been completed after association analysis, there may be repeated redundant information in the result. This step matches the extracted results with the records in the results database. When a matching result is found, a fuzzy comparison is performed between the two, and if the record in the result database is missing a relevant field, the field is completed. If no matching result is found in the results database, the extracted results are added to the results database.

(7) Output the extraction result.

Example:

Take the process of extracting information from the academic homepage http://www.cs.uiuc.edu/~hanj/ as an example, first use Jiawei Han as the search keyword to search in the search engine, and first exclude the The results of Wikipedia and DBLP, and then select the top three results to send a page request, after the classifier judges, the first search result selected is the author's academic homepage.

Use an HTML parser to parse the page, obtain the sub-links, and select the following sub-pages for further analysis according to the link keywords and context:

http://www.cs.uiuc.edu/homes/hanj/pubs/index.htm

https://agora.cs.illinois.edu/display/cs591han/Research+Publications+-+Data+Mining+Research+Group+at+CS%2C+UIUC

Divide each page to be analyzed into text units. Taking the home page as an example, the following results are obtained:

Use the support vector machine to classify the above text units, and determine them as the author's name, irrelevant data, university information, email address, and article information. According to the determined category, further extraction is carried out according to different extraction processes, and irrelevant data are discarded directly.

The process of extracting the author’s name finds the title part of the homepage (Jiawei Han), the author’s name in the text (Jiawei Han), and the author’s name contained in the article information (Jiawei Han, Xiaofei He, Deng Cai). After cross-matching, Jiawei is determined. Han is the final result.

The extraction of mailbox information first removes the prefix part (E-mail): then uses the fuzzy matching automaton to enumerate all possible mailbox matching results, such as:

Hanj(username)at(delimiter)cs(domain name).(dot)uiuc(domain name).(dot)edu(domain name)

The results are scored according to the matching degree, and the best result is selected as the legal form of the mailbox, and then converted into a legal form for output.

In the process of extracting institutional information, the text units classified as institutional information are searched in the institutional index. In this example, the keyword "Univ. The record is "University of Illinois at Urbana-Champaign". After fuzzy matching, it is determined that the two match, so the result can be output directly.

Article information needs to be annotated with a sequence annotation algorithm to identify the author’s name. For example, for the article information found above, it is annotated in the following form:

<Author>Peixiang Zhao, Xiaolei Li, Dong Xin, and Jiawei Han,</author><title>Gfaph Cube: OnWarehousing and OLAP Multidimensional Networks,</title><Conference>Proc.of 2011 ACM SIGMODInt.Conf.on Management of Data (SIGMOD′11), </conference><location>Athens, Greece,</location><time>June 2011</time>

Identifying each sub-field separately completes the identification process of the article information. Then, according to the correlation between the information, the missing and ambiguous results are completed and judged, and the results are merged with the result database.

The present invention is not limited to the above-mentioned specific embodiments, and those skilled in the art can adopt various other specific embodiments to implement the present invention according to the disclosed content of the present invention. Changes or modified designs all fall within the protection scope of the present invention.

Claims (2)

1. the adaptive information extraction method of web page characteristics, is characterized in that, the method comprises the steps:

The 1st step was searched type from the internet be the website of academic homepage;

The 2nd step was analyzed the academic homepage of searching, regard the page of academic homepage as two tuple (L, C) set, wherein L is the URL that links in academic homepage, C is the context that links in academic homepage, reexamines whether to comprise key word in L and C, if comprise, entered for the 3rd step, otherwise filter out this link;

The 3rd step was analyzed the page of described link, obtained the document tree structure of the page, according to attribute and the content of tree node, the page was divided, and was divided into text unit T, consisted of text unit set { T ₁, T ₂...., T _n, step is as follows:

(a) at first use html parser that the page is resolved, obtain the document tree of the page; The node of document tree is namely corresponding to the html tag in the page, and document tree shows the relation between each html tag in the page with tree structure;

(b) then the page is divided; Html tag is divided into piece level element and inline element, html page is seen as the set of piece level element, have two kinds of relations between piece level element: set membership and brotherhood, can be mutually nested between piece level element and inline element, document tree is exactly that form with tree node presents these relations, the node that contains piece level element in document tree is called piece level node, other nodes are called non-level node, node to document tree travels through, classification by decision node is divided the page, and partiting step is as follows:

(b1) initial, the text unit set is empty;

(b2) document tree is carried out depth-first traversal, finds out all piece level nodes, to each piece level node Ni, generate a text unit Ti, and with Ni in the page corresponding division of teaching contents to Ti;

(b3) to each piece level node Ni, judge whether it has non-level child node in document tree, if had with it all non-level child nodes in the page corresponding division of teaching contents to Ti;

(b4) Ti is added in the text unit set;

(b5) finish;

(c) after traversal finishes, complete the division of the page, obtain text unit set { T ₁, T ₂...., T _n;

The 4th step is from text unit set { T ₁, T ₂...., T _nIn extract author's name N, mailbox M, mechanism information U and article information set P1, P2 ..., these four aiming fields of Pm} are as preliminary extraction result; For dissimilar aiming field, its abstracting method is as follows:

The extraction process of author's name N is as follows:

(a1) use the support vector machine sorting algorithm to text unit set { T ₁, T ₂..., T _nInner text unit classifies, retention class is the text unit set T of author's name _Name

(a2) use the authors' name numerical data base from T _NameIn match the authors' name character segment, use this database from T _NameIn match author's name set of candidate;

(a3) extract word in the academic homepage title of author, extract the author's name XXX in the academic homepage title of author;

(a4) the author's name XXX that obtains with (a3) mates candidate author's name that (a2) obtains, and the name that selection and XXX matching degree are the highest is exported as author's name N;

The extraction process of mailbox M is as follows:

(b1) at first use support vector machine classifier from text unit set { T ₁, T ₂..., T _nIn find out possible mailbox candidate text unit set T _EmailAlgorithm of support vector machine according to proper vector to T _EmailMiddle mailbox candidate text unit is judged, if classification results carries out (b2) and processes, otherwise directly filter out for certainly;

(b2) remove unnecessary part in mailbox candidate text unit;

(b3) employing fuzzy matching state machine algorithms is mated the mailbox of mailbox candidate text unit and standard, and the mailbox of a standard has following field: user name (provider's domain name .)+. TLD generates different matching results;

(b4) each field and the matching result of mailbox candidate text unit are compared, choose the result of matching degree maximum as net result, and be converted into the legal mailbox formatted output of standard according to the mailbox field of standard;

The extraction process of mechanism information U is as follows:

(c1) at first from the data of interconnected online collection whole world university and research institute, comprise name and the link of its corresponding homepage of mechanism, set up mechanism's homepage database; Be the Database inverted index;

(c2) use support vector machine classifier from text unit set { T ₁, T ₂...., T _nIn find out possible mechanism information text unit set T _U, with T _UIn the mechanism information text unit be converted to textual form, it is searched in index as key word, obtain first three result for retrieval of rank; First three result for retrieval and corresponding mechanism information text unit are carried out fuzzy matching, if can mate determine that the text is to should mechanism, the matching result output that matching degree is the highest all can't be mated else if, turns (c3) and processes;

(c3) utilize the URL of academic homepage to seek, academic website is the substation point of mechanism's website normally, and therefore domain name and the mechanism's homepage database with academic homepage mates, if there is the record of coupling, think that the author belongs to this mechanism, the record of coupling is as a result of exported;

Article information P1, P2 ..., the process of the extraction of Pm} is as follows:

(a) at first use the support vector machine sorting algorithm that text unit is classified, filter out the text unit that may comprise article information;

(b) then candidate article information text unit is carried out sequence labelling, extract each son field in candidate's text: extract text category feature, pattern feature, dictionary feature and term characteristics from candidate article information text unit, be used for conditional random field models, fundamental function in conditional random field models uses truth expression, i.e. function output is or is no; Through the calculating of conditional random field models, provide the most probable labeling form of candidate article information text unit; Symbol with same label can be merged into corresponding son field, then respectively these fields is carried out corresponding subsequent treatment;

(c) the authors' name field has comprised whole list of authors, need to be divided into single author's form; Partitioning algorithm is based on heuristic rule, according to the length of name, and abbreviated form and punctuation mark; Result after cutting apart is stored in array;

Header field process standardization cutting is as final result;

Mating in meeting journal title and literature periodical database; At first extract in field to be identified capitalization abbreviation part, search in database, if coupling full name and the field to be identified of coupling are carried out fuzzy matching prevents the mistake that the situation of abbreviated form conflict causes; If coupling is direct Output rusults; Otherwise set up index for the meeting journal title, field to be identified is retrieved in index, result for retrieval and field to be identified are done fuzzy matching; Mate Output rusults if find;

Time field service regeulations method uses regular expression to seek legal time pattern in input text;

The preliminary extraction result that the 5th step obtained the 4th step is carried out association analysis, utilizes the relevance disambiguation of information, and the disappearance field is carried out completion, obtains extracting result, deposits to result database;

The 6th the step with the article information set P1, P2 ..., the element in Pm} and the record in result database mate, and eliminate redundant data;

Result is extracted in the output of the 7th step.

2. information extraction method according to claim 1, is characterized in that, the 1st step was divided into two stages: searching stage and decision stage;

In the searching stage, at first derive the data set of author's name as seed data from existing data in literature, then retrieve in search engine as key word with each authors' name of data centralization, search engine returns to result for retrieval with tabular form, each result for retrieval is by title, chain feature and summary texts form, and the chain feature of the result for retrieval of the first page in returning results and summary texts leave in the candidate result list;

in decision stage, at first according to the chain feature of result for retrieval and summary texts to candidate result list filter in the following manner, at first check whether link is present in the shielding linked database, the result that will be arranged in this database is directly got rid of, then, to remaining result for retrieval, check whether its chain feature is rendered as the pattern of "～"+author name, if it is keep, otherwise directly get rid of, successively remaining each result for retrieval being proceeded as follows again of these two step filtrations of process: send page request according to its chain feature, use whether the page that the judgement of support vector machine sorting algorithm is returned is the academic homepage of author, if, directly it is saved as the academic homepage of author, judge and finish, otherwise continue next result for retrieval is carried out identical operation.

Images