CN102693237B - Webpage content adaptation and encapsulation system and method - Google Patents

Abstract

The present invention discloses a system and method for adapting and encapsulating webpage content. The method includes: generating a set of related AST (Abstract Syntax Tree, Abstract Syntax Tree) trees from a DOM (Document Object Model, Document Object Model) tree of a source webpage, Different adaptation strategies are used to adapt the text, pictures, voice and video in the AST tree, and at the same time, the content of the AST tree is classified by subject, and application extensions are added to the AST tree according to the classification results and business value-added strategies. Finally, the AST tree is used as the basic granularity, and the default or user-customized display template is used to encapsulate the AST tree to generate a web page that conforms to various characteristics of the terminal, user personal preferences and application extensions, and returns it to the terminal for display.

Description

A system and method for adapting and encapsulating webpage content

technical field

The invention relates to display adaptation technology of WEB web pages on different terminals, in particular to a system and method for adapting and packaging web page content.

Background technique

The current general state of the Internet is to use PC clients to access WEB pages. Under the background of the integration of the three networks, it will become a development trend to use mobile phones and TV terminals to access Internet information. At the same time, as smart phones are more and more widely used, it is becoming more and more convenient for mobile terminals to access WEB information. However, since most of the web pages on the Internet are specially designed for PC terminals, there is a problem of adaptation and conversion when other terminals access WEB sites.

Chinese Patent No. CN101815093A discloses a method for webpage content adaptation on a mobile terminal. The method extracts valid data from source webpages or other data sources and then organizes them into webpages and sends them to the terminal for display. This method directly uses the adaptation strategy based on the DOM tree to modify the structure and content of the webpage, reduces the coupling between the source webpage and the terminal performance, and is more convenient for mobile terminals such as mobile phones. However, it cannot meet the differentiated requirements of different user terminals. At the same time, due to the complex web page structure of the WEB site, a large amount of information is directly deleted from the DOM tree, and the basic structure of the original child nodes is easily lost.

Contents of the invention

The object of the present invention is to provide a system and method for adapting and packaging webpage content to solve the above problems.

In order to achieve the above purpose, the present invention provides a method for adapting and encapsulating webpage content, comprising:

(1) The step of obtaining the source webpage and parameters, obtaining the source webpage and terminal characteristic parameters;

(2) The step of abstract syntax tree generation, utilize source web page DOM (document object model) tree, and generate relevant abstract syntax tree set according to terminal characteristic parameter;

(3) Adaptation step, carrying out adaptation transformation processing to each content in the abstract syntax tree respectively;

(4) The step of adding a new application is to classify the content of the abstract syntax tree by topic, and add new nodes and content in the abstract syntax tree according to the classification results;

(5) The step of encapsulating the webpage, taking the abstract syntax tree as the basic granularity, using the display template to reorganize and encapsulate each abstract syntax tree into a webpage, and displaying the content of the webpage on the display terminal.

In the above technical solution, the step of obtaining the source web page and terminal characteristic parameters further includes:

(1-1) The terminal initiates a request including the destination url and product-related parameters to the proxy server; (1-2) The proxy server obtains the user's request and parses out the requested url and the product model of the terminal device, and obtains the requested URL from the profile library or Search the corresponding profile document in the database, if found, analyze the document to obtain the relevant parameter information of the terminal equipment, if there is no profile document or database table record of the product model, the terminal is required to send its detailed parameters and product model to the proxy server again, Generate the corresponding profile document and store it in the library.

The terminal characteristic parameters include: terminal product model, screen size, screen resolution, color quality, terminal storage capacity and processor parameters. The classification in step (4) adopts a machine learning method to train a text classifier, and uses the classifier to classify the content of the AST tree according to categories and levels.

The present invention also provides a webpage content adaptation packaging system, the system includes:

The source webpage and parameter acquisition module is used to analyze the request sent by the user to the terminal server, and acquire the source webpage and terminal characteristic parameters;

The abstract syntax tree generation module is used to utilize the DOM (Document Object Model) tree of the source webpage, and generate a set of related abstract syntax trees according to the terminal characteristic parameters;

An adaptation module is used to perform adaptation and transformation processing on the contents in the abstract syntax tree;

The new application adding module is used to classify the content of the abstract syntax tree, and add new nodes and content to the abstract syntax tree according to the classification results;

The webpage encapsulation module is used for reorganizing and encapsulating each abstract syntax tree into a webpage by using the display template with the abstract syntax tree as the basic granularity, and displaying the content of the webpage on the display terminal.

In the above technical solution, the new application adding module further includes: a classifier unit, configured to classify the content of the AST tree; and an application adding unit, configured to add an extended application in the corresponding AST tree according to the classification result of the classifier.

The abstract syntax tree adapting module further includes: a text content adapting unit, a picture voice and video adapting unit, an index segment adapting unit and a new application adding unit.

The system also includes a caching module for caching the AST tree corresponding to the visited webpage.

Among them, the present invention maintains a profile document library or a performance parameter database, and saves terminal device performance parameters. Terminal characteristics include terminal product model, screen size, screen resolution, color quality, terminal storage capacity, processor parameters, etc.; these parameters are acquired for the first time Afterwards, it is stored in the profile document library or database in the form of XML documents or data records, and is used to parse and obtain the calibrated parameters from the profile library or database according to different terminal product models. According to the terminal capability characteristics and adaptation strategy, the relevant AST tree set is generated from the DOM tree of the source webpage requested by the user, and the different contents in the tree, such as text, picture, voice, video, etc., are adapted and converted; the supervised learning method is adopted Method training to obtain a text classifier, use the classifier to classify the content of the AST tree, and add new business content to the AST tree according to the classification results and business value-added strategies; use the AST tree as the basic granularity, and use the default or user-defined display The template reorganizes and packages each AST tree into a web page.

The advantage of the present invention is that the present invention adopts the AST tree set to adapt and package the WEB web pages according to the terminal capability characteristics, and uses the AST tree as the basic granularity to adopt the default display strategy for different characteristics of different terminals, so as to solve the problem of inappropriate display when accessing Internet web pages. At the same time, machine learning technology can be used in the AST tree adaptation process to identify the subject category of the AST tree, providing a method for adding extended applications to web page content.

Description of drawings

1 is a schematic diagram of the composition of the webpage content adaptation and packaging system of the present invention;

Fig. 2 is a functional composition block diagram of the web page content adaptation packaging system of the present invention;

Fig. 3 is the function composition block diagram of abstract syntax tree (AST) adaptation module of the present invention;

FIG. 4 is a flow chart of a method for adapting and encapsulating webpage content based on the above-mentioned system provided by the present invention;

FIG. 5 is a schematic flow diagram of steps for obtaining source webpages and terminal characteristic parameters of the present invention;

Fig. 6 is a flow chart of the method for adapting and encapsulating webpage content after acquiring the source webpage according to the present invention;

Fig. 7 is a schematic diagram of expressing web page content using AST tree provided by the present invention.

Detailed ways

The present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments.

The present invention adapts and encapsulates webpage content between various display terminals and WEB servers, uses machine learning methods in combination with terminal capability characteristics, adaptation strategies, and value-added strategies to generate related AST tree sets from the source webpage DOM tree, and Each content in the website, such as text, pictures, voice, video, etc., is adapted and converted; with the AST tree as the basic granularity, different default display strategies are adopted for different display characteristics of mobile phones or TV terminals to reorganize and package the AST tree into web pages .

As shown in Figure 1, this figure is a schematic diagram of the composition of the webpage content adaptation packaging system of the present invention, the system includes:

The source webpage and parameter acquisition module 1 acquires the source webpage and terminal characteristic parameters in an appropriate manner.

The abstract syntax tree generating module 3 is configured to generate a set of related abstract syntax trees according to the DOM tree of the source webpage and according to the terminal characteristic parameters.

Adaptation module 4, used for adapting the AST tree according to the content of the regional elements; and

The webpage encapsulation module 6 is used for encapsulating the webpage according to the relevant content of the template library and the AST tree adaptation module.

As shown in Figure 2, this figure is a specific implementation block diagram of the functional modules of the web page content adaptation packaging system of the present invention, the system includes:

Obtaining the source webpage and terminal characteristic parameter module 1, which is used to analyze the terminal request and obtain the source webpage and terminal characteristic parameter.

The webpage parsing and filtering module 2 is configured to parse the obtained webpage to obtain the DOM tree of the source webpage.

The abstract syntax tree set generating module 3 is used to generate a related abstract syntax tree set according to the DOM tree of the source webpage and according to the terminal characteristic parameters.

The abstract syntax tree adaptation module 4 is used to adapt the AST tree according to the content of the area element.

In addition, modules added by the new application can be implemented before repackaging the web page, which in turn includes:

The classifier 8 is used to classify the content of the AST tree; and the new application adding module 5 is used to add an extended application in the corresponding AST tree according to the classification result of the classifier.

The webpage encapsulation module 6 is used for encapsulating the webpage according to the related content of the template library module 7 and the AST tree adaptation module, specifically corresponding to the webpage encapsulation and transfer module 104 .

The system of the present invention also includes a cache module for storing abstract syntax trees formed by several source webpages. This module can speed up the access speed of the user terminal to the webpage.

As shown in FIG. 3 , this figure is a specific structural block diagram of an abstract syntax tree (AST) adaptation module, and a schematic structural diagram of further included modules. Among them, the AST tree adaptation strategy is described in detail as follows:

(1). Text content adaptation module

For the area where the text is relatively concentrated, or the proportion of text in the page is large, the first sentence conversion or keyword conversion is adopted, that is, the first sentence or keywords are displayed on the web page as a hypertext link, and the link is given in the form of a list. Other content Only returns to the terminal if the user clicks. The entire page is neat and tidy, and can be operated quickly by using the remote control or the arrow keys on the keyboard of the mobile phone.

If the text has a lot of content and has a hierarchical structure, you can adopt the strategy of linking the hierarchical table of contents in the first sentence.

(2). Picture, voice and video adaptation module

The image uses the default compression display thumbnail or mark link method. The first method compresses the original image by 25% to 75% by default, and only gives the compressed thumbnail in the final page. The second method uses the picture title or alt content as a hypertext link to point to the picture, and only after the user clicks on the picture, it is sent to the terminal device for display. Audio and video are given as text links via corresponding titles.

(3). Index segment adaptation module

When the content structure of the source page is complex, logically divide it according to the sequence of list tags, paragraphs, or tables. On this basis, the first sentence link method is used to point the hypertext to the specific content of the segmented area.

If these sub-blocks cannot be fully presented on a terminal page, they can be further subdivided, and the content in the sub-blocks is divided and linked according to the same principle. When the number of sub-blocks is large, the previous and next navigation strategies are used to display the links of the previous sub-block and the next sub-block.

(4). Application add module

According to the classification result of the content of the AST tree, add related extended services to the AST tree. For example, if the content of the AST tree is educational, relevant advertisements such as college entrance examinations and postgraduate examinations can be inserted into the tree.

As shown in Figure 4, this figure is a method for adapting and encapsulating webpage content based on the above-mentioned system provided by the present invention, and the method includes:

Step 101: Obtain the source webpage and terminal characteristic parameters;

Step 102: Utilize the DOM (Document Object Model) tree of the source web page, and generate a set of related abstract syntax trees according to the terminal characteristic parameters;

Step 103: Carry out adaptation and transformation processing on each content in the abstract syntax tree;

Step 104: Carry out subject classification on the content of the abstract syntax tree, and add new nodes and content in the abstract syntax tree according to the classification result;

Step 105: Taking the abstract syntax tree as the basic granularity, using display templates to reorganize and package each abstract syntax tree into a web page, and display the content of the web page on the display terminal.

Wherein, cache processing is performed on the generated abstract syntax tree at the same time to speed up the access speed of subsequent user terminals to the same source webpage.

As shown in Figure 5, this figure is a number of sub-steps further included in the step of obtaining the source web page and terminal characteristic parameters, described as follows:

The terminal user initiates a request to the proxy server, including the destination url and the performance parameters of the terminal device.

Step 201: After obtaining the user's request, the proxy server parses out the requested url and the product model of the terminal device;

Step 202: Find the corresponding profile document from the profile document library.

Step 203: If found, analyze the document to obtain relevant parameter information of the terminal device. If there is no profile document of the product model, the terminal is required to send back its detailed parameters and product model, generate a corresponding profile document and store it in the database.

After obtaining the user's access url, at first query the cache module 9, if there is already an AST tree corresponding to the webpage, then directly encapsulate it and send it back to the terminal, otherwise access the WEB server to obtain the specified source webpage document.

As shown in Figure 6, this figure is a detailed flowchart of a method for adapting and encapsulating webpage content after obtaining the source webpage.

Step 301: Obtain the AST tree of different sub-blocks according to the processing capability, storage capability and display capability parameters of the terminal device provided by the profile document, and adapt the AST tree according to the different content of regional elements such as text, picture, voice and video, etc. ;

Step 302: Use the trained classifier to classify the content of the AST tree;

Step 303: For the classification result, an extended application such as adding advertisement content can be added to the corresponding AST tree;

Step 304: Check whether the terminal browser supports the HTML5.0 standard according to the information provided by the profile document;

Step 305: Encapsulate the webpage with HTML5.0 standard if supported;

Step 306: Otherwise, use the default standard to package the webpage, and finally use the default template or user-defined template to package the adaptation result to obtain the final webpage.

Wherein, step 304, step 305 and step 306 are a specific implementation manner of webpage encapsulation.

As shown in FIG. 7 , which is a schematic diagram of a specific embodiment of an AST tree, wherein the left side is a common page content, and the right side is the corresponding AST tree. Regarding the content classification of the AST tree, according to the different content categories, the AST tree is divided into news, education, finance, sports, entertainment, technology, life and other categories. Each category is divided into different levels. For example, education is divided into college entrance examination, Senior high school entrance examination, adult education, postgraduate entrance examination, overseas examination, life can be divided into travel, shopping, marriage, childcare, etc. A text classifier is trained by a supervised learning method, and its content is classified according to the classifier in the AST tree adaptation process to support extended applications.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than limit them. Although the present invention has been described in detail with reference to the embodiments, those skilled in the art should understand that modifications or equivalent replacements to the technical solutions of the present invention do not depart from the spirit and scope of the technical solutions of the present invention, and all of them should be included in the scope of the present invention. within the scope of the claims.

Claims (7)

1. the adaptive method for packing of web page contents, comprises:

(1) obtain source web page content and termination property parameter;

(2) AST tree generates step, utilizes source web page document object model tree, generates the set of related abstractions syntax tree according to termination property parameter;

(3) adaptation step, carries out respectively adaptive conversion process to the every content in AST tree;

(4) step is added in new application, and abstract syntax tree content is carried out to subject classification, adds new node and content according to classification results in abstract syntax tree;

(5) step of webpage encapsulation, taking abstract syntax tree as basic granularity, adopts indicating template that each abstract syntax tree is reorganized and is packaged into webpage, display web page content on display terminal.

2. the adaptive method for packing of web page contents according to claim 1, it is characterized in that, the generative process of AST tree described in step (2), adopts and obtains in conjunction with source web page dom tree, terminal capabilities parameter, the visual characteristic of webpage and the method for page-tag.

3. according to the adaptive method for packing of a kind of web page contents described in claim 1, it is characterized in that, adaptation procedure described in step (3), for video, audio frequency, picture and content of text in AST tree feature separately, carries out respectively adaptation processing.

4. according to the adaptive method for packing of a kind of web page contents described in claim 1, it is characterized in that, the described classification of step (4) adopts machine learning method training to obtain text classifier, and adopts this sorter that AST is set to content to carry out subject classification according to classification and level.

5. the adaptive package system of web page contents, this system comprises:

Source web page and parameter acquisition module, for obtaining source web page and termination property parameter;

Abstract syntax tree generation module, for utilizing source web page document object model tree, and generates the set of related abstractions syntax tree according to termination property parameter;

Adaptation module, for carrying out respectively adaptive conversion process to every content of abstract syntax tree;

Module is added in new application, for abstract syntax tree content is carried out to subject classification, adds new node and content according to classification results in abstract syntax tree;

Webpage package module, for taking abstract syntax tree as basic granularity, adopts indicating template that each abstract syntax tree is reorganized and is packaged into webpage, display web page content on display terminal.

6. the adaptive package system of web page contents according to claim 5, is characterized in that, described new application is added module and further comprised:

Sorter unit, for classifying to AST tree content; With

Application adding device, for adding expanded application for sorter classification results in corresponding AST tree.

7. the adaptive package system of web page contents according to claim 5, is characterized in that, described adaptation module further comprises: content of text adaptation unit, picture voice and video adaptation unit, index segmentation adaptation unit and application adding device.