CN112328246A

CN112328246A - Page component generation method and device, computer equipment and storage medium

Info

Publication number: CN112328246A
Application number: CN202011208407.5A
Authority: CN
Inventors: 王继辉; 王陶; 周虎
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2020-11-03
Filing date: 2020-11-03
Publication date: 2021-02-05

Abstract

The application relates to a page component generation method, a page component generation device, computer equipment and a storage medium, and relates to the technical field of front ends. The method comprises the following steps: acquiring page contents of at least two pages; analyzing the page contents of at least two pages, and constructing a Document Object Model (DOM) tree of the at least two pages; extracting node characteristic information of each node in DOM trees of at least two pages; extracting the same structural parts in the DOM trees of the at least two pages based on the node characteristic information of each node in the DOM trees of the at least two pages; generating page components corresponding to the same structural parts based on the node characteristic information of the same structural parts; by the method, the automation of the generation of the page components which can be multiplexed in a plurality of pages is realized, and the generation efficiency and the accuracy of the page components are improved.

Description

Page component generation method and device, computer equipment and storage medium

Technical Field

The embodiment of the application relates to the technical field of front ends, in particular to a page component generation method and device, computer equipment and a storage medium.

Background

In the front-end project development process, a component development mode is adopted to improve the efficiency of project development.

In the related art, a developer manually extracts the same parts of different web pages according to the similarity of the page structures of a plurality of independent pages provided by reconstruction, and encapsulates the same parts to generate a page component.

However, in the above scheme, the generation process of the page component is limited by manual work, and in a scene where the same content of the web pages is large and the number of the web pages is large, the manual extraction efficiency is low, and due to the fact that manual operation often has certain risks and errors, the accuracy is low.

Disclosure of Invention

The embodiment of the application provides a page component generation method, a page component generation device, computer equipment and a storage medium, which can realize automatic generation of a page component and improve the generation efficiency and accuracy of the page component, and the technical scheme is as follows:

in one aspect, a method for generating a page component is provided, where the method includes:

acquiring page contents of at least two pages;

analyzing the page contents of the at least two pages, and constructing a Document Object Model (DOM) tree of the at least two pages;

extracting node characteristic information of each node in the DOM trees of the at least two pages;

extracting the same structural parts in the DOM trees of the at least two pages based on the node characteristic information of each node in the DOM trees of the at least two pages;

and generating a page component corresponding to the same structural part based on the node characteristic information of the same structural part.

In another aspect, an apparatus for generating a page component is provided, the apparatus including:

the page content acquisition module is used for acquiring the page contents of at least two pages;

the page content analyzing module is used for analyzing the page contents of the at least two pages and constructing Document Object Model (DOM) trees of the at least two pages;

the node characteristic information extraction module is used for extracting the node characteristic information of each node in the DOM trees of the at least two pages;

the same structure extraction module is used for extracting the same structure parts in the DOM trees of the at least two pages based on the node characteristic information of each node in the DOM trees of the at least two pages;

and the page component generating module is used for generating the page component corresponding to the same structural part based on the node characteristic information of the same structural part.

In a possible implementation manner, the node feature information includes first feature information, where the first feature information includes a class name and a node name of a corresponding node;

the same structure extraction module comprises:

the Hash attribute obtaining submodule is used for carrying out Hash calculation on the basis of the first characteristic information of each node in the DOM trees in the at least two pages to obtain the Hash attribute of each node;

and the same structure extraction sub-module is used for extracting the same structure parts in the DOM trees of the at least two pages based on the hash attributes of all nodes in the DOM trees of the at least two pages.

In a possible implementation manner, the hash attribute obtaining sub-module includes:

the first splicing result acquisition unit is used for splicing the class name of the target node and the node name of the target node to acquire a splicing result of the target node in response to the fact that the target node does not have an effective child node; the target node is any one of nodes in DOM trees in the at least two pages;

and the first hash value calculation unit is used for calculating the hash value of the splicing result of the target node and obtaining the hash attribute of the target node.

the second splicing result acquisition unit is used for splicing the class name of the effective child node and the node name of the effective child node in response to the target node having the effective child node, and acquiring the splicing result of the effective child node; the target node is any one of nodes in DOM trees in the at least two pages;

the second hash value calculation unit is used for calculating the hash value of the splicing result of the effective child node;

the first splicing result obtaining unit is used for splicing the class name of the target node and the node name of the target node;

the first hash value calculation unit is used for calculating the hash value of the splicing result of the target node;

and the hash attribute acquisition unit is used for taking the hash value of the splicing result of the target node and the hash value of the splicing result of the effective child node as the hash attribute of the target node.

In a possible implementation manner, the second splicing result obtaining unit is configured to, in response to that the valid child node includes at least two duplicate nodes, splice the class name and the node name of one node of the at least two duplicate nodes, and obtain the splicing result of the valid child node.

In one possible implementation, the class name is composed of at least two element class names, and the node name is composed of at least two element node names, and the apparatus further includes:

the first ordering module is used for ordering the at least two element class names of the nodes according to a first specified order before the same structure extraction module extracts the same structure parts in the DOM trees of the at least two pages based on the node characteristic information of the nodes in the DOM trees of the at least two pages, so as to obtain the class names of the nodes;

and the second sequencing module is used for sequencing the at least two element node names of each node according to a second designated sequence to obtain the node names of each node.

In a possible implementation manner, the node characteristic information further includes hierarchy information of the corresponding node, and the hierarchy information is used for corresponding to the hierarchy of the node in the DOM tree;

the same structure extraction module comprises:

the obtaining submodule is used for obtaining the hash attribute of one node contained in each DOM tree in the at least two pages to obtain the hash attribute of the at least two nodes; the at least two nodes are nodes having the same hierarchical information;

and the determining submodule is used for determining the at least two nodes as the nodes belonging to the same structural part in response to the fact that the hash attribute between every two of the at least two nodes meets a specified condition.

In one possible implementation, the specified condition includes:

and the coincidence proportion of the hash attributes between every two nodes reaches a specified threshold value.

In a possible implementation manner, the obtaining sub-module is configured to obtain, according to an order from a root node to a child node of the hierarchy information, hash attributes of a node included in DOM trees in the at least two pages, and obtain the hash attributes of the at least two nodes.

In a possible implementation manner, the page component generating module is configured to construct a DOM tree based on the node feature information of the nodes in the same structural portion according to an order from a child node to a root node of the hierarchy information, and obtain the page components corresponding to the same structural portion.

In a possible implementation manner, the node feature information further includes a node attribute and a sub-level attribute of the corresponding node;

the page component generation module comprises:

a component node label generation submodule for generating component node labels based on node names of nodes in the same structural part according to the sequence of the hierarchy information from child nodes to root nodes;

the component node attribute generation submodule is used for generating component node attributes based on the node attributes of the nodes in the same structural part;

the component internal structure building submodule is used for building a component internal structure based on the sublevel attributes of the nodes in the same structural part;

and the page component generation sub-module is used for generating the page component corresponding to the same structural part based on the component node label, the component node attribute and the component internal structure.

In another aspect, a computer device is provided, the computer device comprising a processor and a memory, the memory having stored therein at least one instruction, at least one program, set of codes, or set of instructions, which is loaded and executed by the processor to implement the page component generation method provided in the various alternative implementations described above.

In another aspect, a computer-readable storage medium is provided, in which at least one computer program is stored, the computer program being loaded and executed by a processor to implement the page component generating method provided in the various optional implementations described above.

In another aspect, a computer program product or computer program is provided, the computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the page component generation method provided in the above-mentioned various optional implementation modes.

The technical scheme provided by the application can comprise the following beneficial effects:

the method comprises the steps of analyzing the obtained page contents of a plurality of pages, obtaining DOM tree structures corresponding to the pages respectively, comparing the node characteristic information in each DOM tree structure, obtaining the same structure part in each DOM tree, and generating a page assembly with a plurality of pages capable of being reused based on the obtained node characteristic information of the same structure part, so that the automation of the generation of the page assembly capable of being reused in the pages is realized, and the generation efficiency and the accuracy of the page assembly are improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.

FIG. 1 is a diagram illustrating a DOM tree corresponding to an HTML web page provided by an exemplary embodiment of the present application;

FIG. 2 is a flowchart illustrating a method for generating a page component according to an exemplary embodiment of the present application;

FIG. 3 illustrates a flow chart of a page component generation method provided by an exemplary embodiment of the present application;

FIG. 4 illustrates a diagram of a DOM tree shown in an exemplary embodiment of the present application;

FIG. 5 illustrates a schematic diagram of a json structure shown in an exemplary embodiment of the present application;

FIG. 6 is a diagram illustrating a page component generation method according to an exemplary embodiment of the present application;

FIG. 7 is a block diagram of a page component generation apparatus provided by an exemplary embodiment of the present application;

FIG. 8 illustrates a block diagram of a computer device provided in an exemplary embodiment of the present application;

fig. 9 shows a block diagram of a computer device according to an exemplary embodiment of the present application.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

It is to be understood that reference herein to "a number" means one or more and "a plurality" means two or more. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.

The embodiment of the application provides a page component generation method, which can realize automatic generation of a page component and improve the generation efficiency and accuracy of the page component. For ease of understanding, the terms referred to in this application are explained below.

1) DOM (Document Object Model)

The DOM is a set of Web (global Wide area network) standards that describe how scripts interact with and access structured documents, and functions to parse documents supported by browsers, including HTML (hypertext Markup Language), XML (Extensible Markup Language), XHTML (Extensible hypertext Markup Language), as an object, which is in effect a programmed API (Application Programming Interface) that manipulates the content contained within the document, allowing developers to read, search, modify, and delete data from the document.

A structured document can be converted into a DOM tree by virtue of the DOM model, and a program can access and modify nodes in the tree and can add and delete nodes in the tree so as to achieve the purposes of reading, searching, modifying and deleting data from the document.

2) DOM tree

In the DOM specification, there is no limitation on the method of representing a document, and therefore, a DOM tree is a common implementation manner in various document structures, and the basic elements of the DOM structure are "nodes", and the document structure is composed of hierarchical nodes. In the DOM model, the concept of nodes is broad, and an entire Document (Document) is a node called a Document node, and in addition, an Element (Element) node, an attribute node, an Entity node, a Comment (Comment) node and the like are provided. Fig. 1 is a schematic diagram illustrating a DOM tree corresponding to an HTML web page provided by an exemplary embodiment of the present application, where, as shown in fig. 1, the HTML page is a text with a structured characteristic, and each node in the corresponding DOM tree corresponds to each component content in the HTML page, for example, a node 110 represents a document node HTML, as a root node of the DOM tree, and its child nodes are a node 120 and a node 130, the node 120 represents Head, the node 130 represents Body, the child node corresponding to the node 130 is a node 140 and a node 150, the node 140 represents a P element, and the node 150 represents a DIV (Division) element, so as to represent a hierarchical structure of the HTML by the DOM tree.

Fig. 2 shows a flowchart of a page component generation method provided in an exemplary embodiment of the present application, where the page component generation method may be executed by a computer device, and the computer device may be implemented as a server or a terminal, and as shown in fig. 2, the page component generation method includes the following steps:

step 210, obtaining page contents of at least two pages.

The page contents of the at least two pages are finished independent pages provided for front-end personnel after the pattern is written by reconstruction personnel, different pages with the same pattern are generated based on the same DOM structure and the same pattern, therefore, the front-end personnel need to analyze the similarity of the page structures, extract common parts in different pages, package the common parts into a component for use when the at least two pages are built, and different page contents can be displayed in different pages by setting different parameters in the component.

In one possible implementation, the at least two pages are html pages or wxml pages, or pages having html web page structures or wxml file structures.

Step 220, analyzing the page contents of the at least two pages, and constructing a Document Object Model (DOM) tree of the at least two pages.

In a possible implementation manner, the computer device constructs a DOM tree corresponding to the page file according to the structure identifier in the page file, wherein the structure identifier in the page file is marked in advance when a reconstruction person completes each independent page.

In step 230, node feature information of each node in the DOM trees of at least two pages is extracted.

Step 240, extracting the same structural parts in the DOM trees of the at least two pages based on the node characteristic information of each node in the DOM trees of the at least two pages.

And 250, generating a page component corresponding to the same structural part based on the node characteristic information of the same structural part.

In summary, according to the page component generation method provided by the embodiment of the application, the acquired page contents of the multiple pages are analyzed to obtain DOM tree structures corresponding to the multiple pages respectively, the node feature information in each DOM tree structure is compared to obtain the same structure part in each DOM tree, and the page component reusable for the multiple pages is generated based on the obtained node feature information of the same structure part, so that the generation automation of the page component reusable in the multiple pages is realized, and the generation efficiency and accuracy of the page component are improved.

In a possible implementation manner, the feature information of each node includes a class name of each node and a node name of each node, and in the above case, the extraction of the same structural part in the DOM tree of at least two pages may be implemented based on the class name of each node and the node name of each node; fig. 3 shows a flowchart of a page component generation method provided in an exemplary embodiment of the present application, where the page component generation method may be executed by a computer device, and the computer device may be implemented as a server or a terminal, and as shown in fig. 3, the page component generation method includes the following steps:

in step 310, page contents of at least two pages are obtained.

In one possible implementation, the computer device reads the page content of the page to be processed by calling the fs module in the node environment. The fs module is a core module of node.js, and is used for operating files in the system, a common operation mode is reading and writing the files, and node.js refers to JavaScript running on a server.

And 320, analyzing the page contents of the at least two pages, and constructing a DOM tree of the at least two pages.

In one possible implementation, the contents of the pages are analyzed by creating HTML Hash classes, simulating browser environment operation DOM by using a jsdom library, and generating DOM trees corresponding to the pages. Fig. 4 shows a schematic diagram of a DOM tree shown in an exemplary embodiment of the present application, and as shown in fig. 4, each element in a document can be regarded as a node of the DOM tree, and an html document can be said to be a set of nodes, which includes three common DOM nodes: element nodes, text nodes, and attribute nodes, wherein the node 410 (page, html), the node 420 (body), the node 430 (view), and the node 440(a) in fig. 4 are all element nodes, i.e., tags; node 450 (text a) and node 460 (text b) are the content presented to the user, and node 470 (attribute href) is the attribute node to represent the attributes of the corresponding element such as the connection attribute, the loop attribute, the selectable attribute, the replaceable attribute, etc.

Step 330, extracting node characteristic information of each node in the DOM trees of at least two pages.

In a possible implementation manner, the node feature information nodes of each node have attributes such as nodeType (node class name), nodeName (node name), attributes of attributes, and the like, where nodeName is a read-only attribute and cannot be written and modified, and nodeName may be a tag name of an element node, an attribute name of an attribute node, a # text of a text node, and a # document of a document node.

In a possible implementation manner, when node feature information of each node in a DOM tree of at least two pages is extracted, a json structure corresponding to each node may be constructed, fig. 5 shows a schematic diagram of the json structure shown in an exemplary embodiment of the present application, and as shown in fig. 5, the node feature information of a node 510 includes node content (content), node identifier (flag), class name (nodeType), node name (nodeName) node attributes (attributes), and level information (level); the hierarchical information is node characteristic information which is added to the nodes according to the hierarchical number of the traversal DOM tree when the node characteristic information of the nodes is extracted.

In one possible implementation manner, in response to that a node contains a child node, that is, the node is a parent node, the corresponding node characteristic information further includes child level information (child attribute).

In one possible implementation, when creating the json structure, a createSimpleDom class may be defined to pass in the node name and hierarchy information of the node.

In a possible implementation manner, text nodes and empty nodes need to be excluded when generating the json structure, and the format of the nodeName is specified, for example, the format of the english alphabet capitalization is uniformly adopted, so that the comparison result is prevented from being influenced due to the non-specification format.

In a possible implementation manner, node attributes of element nodes are obtained, the node attribute of each element node includes at least two element node attributes, the node attributes are traversed, and if an element node attribute includes a "-" identifier, a flag is marked on the element node attribute to indicate that the element node attribute is selectable.

In a possible implementation mode, acquiring text contents of text nodes, because some contents in the text contents are fixed and some contents are replaceable, segmenting the fixed text contents and the replaceable text contents by using "-" identifiers to generate corresponding arrays, and if the length of the array is greater than 1, marking variable marks on the replaceable text contents; for example, for the text content "name: xiaoming "the name: "is fixed text content and" small brightness "is replaceable text, so a variable mark is marked on the array corresponding to the" small brightness ". And simultaneously, the text type of the text content is acquired and judged as a character string or a number, so that different parameters can be input according to the text type when the generated page component is used.

Step 340, performing hash calculation based on the first characteristic information of each node in the DOM trees in the at least two pages to obtain the hash attribute of each node.

The first characteristic information comprises a class name and a node name of the corresponding node.

In a possible implementation manner, the node characteristic information of each node further includes an identifier of the corresponding node, where the identifier is used to indicate whether the node is an optional node.

In one possible implementation, the target node has valid child nodes, where a valid child node refers to a child node that does not have the optional node identifier among the child nodes of the target node.

Splicing the class name of the target node and the node name of the target node to obtain a splicing result of the target node in response to the fact that the target node does not have an effective child node, wherein the target node is any one of nodes in a DOM tree of at least two pages;

and calculating the hash value of the splicing result of the target node to obtain the hash attribute of the target node.

For example, the node class name of the target node is "test 1test 2", and the node name is "NameNodeHttpServer", so that the splicing result of the target node is "test 1test2 NameNodeHttpServer", that is, the character string representing the node class name and the character string identifying the node name are connected into one character string.

In a possible implementation manner, the target node has valid child nodes, and the obtaining of the hash attribute of each node by performing hash calculation based on the first feature information of each node in the DOM tree in the at least two pages may be implemented as:

splicing the class name of the effective child node and the node name of the effective child node in response to that the target node has the effective child node, and acquiring a splicing result of the effective child node, wherein the target node is any one of nodes in a DOM tree in at least two pages;

calculating the hash value of the splicing result of the effective child node;

splicing the class name of the target node and the node name of the target node;

calculating the hash value of the splicing result of the target node;

and taking the hash value of the splicing result of the target node and the hash value of the splicing result of the effective child node as the hash attribute of the target node.

That is to say, for a target node having a valid child node, the hash attribute of the target node is determined by the first characteristic information of the target node and the first characteristic value of the valid child node, and the hash attribute of the target node can represent the structural attribute of the child node, and if the hash attributes of two target nodes are completely the same, it means that the target node is the same as another target node, and the valid child node of the target node is the same as the child node of another target node.

In a possible implementation manner, in response to that the valid child nodes include at least two duplicate nodes, when the class names and the node names of the valid nodes are spliced, the class name and the node name of one node of the at least two duplicate nodes are spliced to obtain a splicing result of the valid child nodes, that is, for the duplicate nodes in the child nodes, only one of the duplicate nodes is obtained as a valid child node, the class name and the node name are spliced, and after a hash value is obtained by performing hash calculation, the obtained hash value and a hash value of a target node are used as a hash attribute of the target node.

In one possible implementation, the concatenation result of the nodes is hashed by using MD5 Message Digest Algorithm (MD5 Message-Digest Algorithm 5).

In a possible implementation manner, the class name of the same target node may include a plurality of element class names, the node name of the same target node is composed of at least two element node names, the element class names and the element node names are not in sequence, but when hash calculation is performed, the ordering of different element class names or the ordering of different element node names may affect the splicing result of the target node, so that hash values calculated based on the splicing result are different, that is, hash attributes of the target node are different; taking node class names as an example, the class name of one node is "test 1test 2", the class name of the other node is "test 2 test 1", and for the structural style, the class names of the two nodes are the same, but for the hash calculation, the hash results calculated based on the two class names are different due to different arrangement orders of the element class names.

To avoid the foregoing, in one possible implementation, when the class name is composed of at least two element class names and the node name is composed of at least two element node names, before extracting the same structural part in the DOM trees of the at least two pages based on the node feature information of each node in the DOM trees of the at least two pages, the method further includes:

sequencing at least two element class names of each node according to a first designated sequence to obtain the class names of the nodes;

and sequencing at least two element node names of each node according to a second specified sequence to obtain the node names of the nodes.

The first designated order and the second designated order may be the same or different, and in one possible implementation, the first designated order/the second designated order may be implemented as being sequentially arranged in an arrangement order of the first letters of the element class names/element node names.

The above description of the arrangement of the first arrangement order and the second arrangement order is illustrative, and the front-end staff may set the ordering rule according to the actual requirement, which is not limited in this application.

In one possible implementation, before sorting the element class names in the class names, the method further includes:

preprocessing the class name of the node; the method comprises the following steps:

according to the 'mark' defined in each element class name in the node class name, marking a flag mark on the element class name, and marking that each element class name is selectable or variable, wherein when the class name of the node is obtained, the element class name with the flag mark and other class names can form the class name of the node, or the element class name with the flag mark can be used as an invalid class name, and when the class name of the node is formed, the element class name with the flag mark is excluded;

and removing head and tail empty strings from the node class names, and separating the node class names into arrays according to spaces for sorting, such as 'test 1test 2'.

Step 350, extracting the same structural parts in the DOM trees of the at least two pages based on the hash attributes of the nodes in the DOM trees of the at least two pages.

In one possible implementation, the node characteristic information further includes hierarchy information of the corresponding node, and the hierarchy information is used for corresponding to the hierarchy of the node in the DOM tree. The extracting the same structural parts in the DOM trees of the at least two pages based on the hash attributes of the nodes in the DOM trees of the at least two pages includes:

the method comprises the steps of obtaining the hash attribute of a node contained in each DOM tree in at least two pages to obtain the hash attribute of at least two nodes; the at least two nodes are nodes having the same hierarchical information;

and determining at least two nodes as nodes belonging to the same structural part in response to the hash attribute between every two of the at least two nodes meeting a specified condition.

Taking three pages as an example, acquiring the hash attribute of a first node in a DOM tree of a first page, acquiring the hash attribute of a second node in the DOM tree of a second page, which has the same hierarchical information as the first node, and the hash attribute of a third node in the DOM tree of a third page, which has the same hierarchical information as the first node, comparing the acquired hash attributes of the three nodes in pairs, judging whether the designated conditions are met, and determining the three nodes as nodes belonging to the same structural part when the hash attributes between the three nodes in pairs meet the designated conditions, namely determining the three nodes to be the same; when the hash attributes between every two three nodes do not meet the specified conditions, the three nodes are determined to be nodes with different structural parts, namely, the three nodes are determined to have different nodes.

In one possible implementation, the specified condition includes: the coincidence proportion of the hash attributes between every two nodes reaches a specified threshold value.

When the hash attributes of the nodes are acquired, the hash values of the nodes and the hash values of the child nodes are used as the hash attributes of the nodes, so that when the hash attributes are compared, a specified threshold value can be set to judge the similarity between the node structures.

In a possible implementation manner, the hash attributes of a node included in each DOM tree in at least two pages are obtained according to the sequence of the hierarchy information from the root node to the child nodes, so as to obtain the hash attributes of at least two nodes. That is to say, when the hash attributes of the nodes in the DOM tree are compared, the hash attributes are compared in the order from the root node to the child nodes, so that when the hash attributes of the parent node are determined to be different from the hash attributes of the nodes corresponding to other pages, it is unnecessary to determine whether the child nodes under the parent node are the same based on the relevance of the DOM tree hierarchy structure, and the child nodes corresponding to the parent node are skipped when the next hierarchy pass is performed, thereby reducing unnecessary waste of computer resources.

For example, when the hash attribute of the first node is compared with the hash attribute of the second node, only the hash value of the first node is the same as the hash value of the second node, and the other hash values in the hash attribute of the first node are different from the other hash values in the hash attribute of the second node, so that it is meaningless to perform subsequent comparison at the branch corresponding to the first node and the second node for the DOM tree where the first node is located and the DOM tree where the second node is located, and therefore, in response to that the hash attribute of the first node and the hash attribute of the second node do not satisfy the specified condition, the matching of the hash attribute of the node in the branch corresponding to the first node in the DOM tree and the hash attribute of the node in the branch corresponding to the second node in the DOM tree is stopped.

In a possible implementation manner, for a node having an optional node identifier in the DOM tree, the node may be acquired as an optional node of the same structural part, and when generating a page component, the node is added or deleted according to actual conditions.

And step 360, generating a page component corresponding to the same structural part based on the node characteristic information of the same structural part.

In a possible implementation manner, the node feature information of the same structural part can be converted into a json file, and a dom node is generated reversely by traversing the json file, so that the page component corresponding to the same structural part is generated.

In a possible implementation manner, when generating page components of the same structural part, according to the sequence of the hierarchy information from the child node to the root node, a DOM tree is constructed based on the node feature information of the nodes in the same structural part, and the page components corresponding to the same structural part are obtained.

In a possible implementation manner, the node characteristic information further includes a node attribute and a sub-level attribute of the corresponding node.

When page components corresponding to the same structural parts are obtained, generating component node labels based on node names of nodes in the same structural parts according to the sequence of the hierarchy information from child nodes to root nodes;

generating component node attributes based on node attributes of nodes in the same structural part;

building an internal structure of the component based on the sub-level attributes of the nodes in the same structural part;

and generating the page component corresponding to the same structural part based on the component node label, the component node attribute and the component internal structure.

The method can be applied to the generation environment of the page component of the mobile data display page or the management background, and the page structure in the environment is single, so that the page is simple, and the generation efficiency and accuracy of the page component can be greatly improved by adopting the scheme.

Fig. 6 is a schematic diagram illustrating a page component generation method according to an exemplary embodiment of the present application, as shown in fig. 6, the page component generation method may extract page content 610 from a plurality of pages, respectively construct a DOM tree structure 620 corresponding to each page, extract node feature information of each node in the DOM tree structure corresponding to each page, form a json structure corresponding to each node, in the diagram, use node 1 in page 611, node 2 in page 612, and node 3 in page 613 as examples to illustrate, obtain a corresponding json structure 630, include node feature information of corresponding nodes in the json structure of each node, generate hash attributes of corresponding nodes based on class names and node names in the node feature information, determine whether hash attributes corresponding to nodes 1, 2, and 3 with the same level satisfy a specified condition between each other, if the answer is satisfied, determining the nodes of the same structure parts of the node 1, the node 2 and the node 3, converting the leaf node characteristic information of the nodes into json files, then performing subsequent judgment (not shown in the figure) on the child nodes until all the nodes are traversed, and reversely generating the page component according to the json files corresponding to the same structure parts.

Fig. 7 is a block diagram illustrating a page component generating apparatus according to an exemplary embodiment of the present application, where the page component generating apparatus may be applied to a computer device, and the computer device may be implemented as a server or a terminal, and as shown in fig. 7, the page component generating apparatus includes:

a page content obtaining module 710, configured to obtain page contents of at least two pages;

the page content analyzing module 720 is configured to analyze the page content of at least two pages and construct a Document Object Model (DOM) tree of the at least two pages;

a node characteristic information extraction module 730, configured to extract node characteristic information of each node in a DOM tree of at least two pages;

the same structure extracting module 740 is configured to extract the same structure portions in the DOM trees of the at least two pages based on the node feature information of each node in the DOM trees of the at least two pages;

and a page component generating module 750, configured to generate a page component corresponding to the same structural part based on the node feature information of the same structural part.

In one possible implementation manner, the node feature information includes first feature information, and the first feature information includes a class name and a node name of a corresponding node;

the same structure extraction module 740 includes:

the Hash attribute acquisition submodule is used for carrying out Hash calculation based on the first characteristic information of each node in the DOM trees in at least two pages to obtain the Hash attribute of each node;

the first splicing result acquisition unit is used for splicing the class name of the target node and the node name of the target node to acquire a splicing result of the target node in response to the fact that the target node does not have an effective child node; the target node is any one of nodes in a DOM tree in at least two pages;

and the first hash value calculation unit is used for calculating the hash value of the splicing result of the target node to obtain the hash attribute of the target node.

the second splicing result acquisition unit is used for splicing the class name of the effective child node and the node name of the effective child node in response to the target node having the effective child node, and acquiring the splicing result of the effective child node; the target node is any one of nodes in a DOM tree in at least two pages;

the first splicing result acquisition unit is used for splicing the class name of the target node and the node name of the target node;

the first ordering module is configured to order the at least two element class names of the nodes according to a first specified order to obtain the class names of the nodes before the identical structure extraction module 740 extracts the identical structure parts in the DOM trees of the at least two pages based on the node feature information of the nodes in the DOM trees of the at least two pages;

and the second sequencing module is used for sequencing at least two element node names of each node according to a second designated sequence to obtain the node names of the nodes.

In a possible implementation manner, the node feature information further includes hierarchy information of the corresponding node, and the hierarchy information is used for corresponding to a hierarchy of the node in the DOM tree;

the same structure extraction module 740 includes:

the obtaining submodule is used for obtaining the hash attribute of one node contained in each DOM tree in at least two pages to obtain the hash attribute of at least two nodes; at least two nodes are nodes having the same hierarchical information;

and the determining submodule is used for determining at least two nodes as the nodes belonging to the same structural part in response to the fact that the hash attribute between every two of the at least two nodes meets the specified condition.

In one possible implementation, the specified condition includes:

the coincidence proportion of the hash attributes between every two nodes reaches a specified threshold value.

In a possible implementation manner, the obtaining sub-module is configured to obtain, according to an order of the hierarchy information from the root node to the child node, hash attributes of a node included in each of DOM trees in the at least two pages, so as to obtain the hash attributes of the at least two nodes.

In a possible implementation manner, the page component generation module 750 is configured to construct a DOM tree based on node feature information of nodes in the same structural portion according to an order of the hierarchy information from the child node to the root node, and obtain page components corresponding to the same structural portion.

the page component generation module 750 includes:

the component node label generation submodule is used for generating component node labels based on node names of nodes in the same structural part according to the sequence of the hierarchy information from the child nodes to the root nodes;

and the page component generation submodule is used for generating the page component corresponding to the same structural part based on the component node label, the component node attribute and the component internal structure.

To sum up, the page component generation device provided by the embodiment of the application is applied to a computer device, and is used for analyzing the page contents of a plurality of acquired pages to obtain DOM tree structures corresponding to the plurality of pages respectively, comparing the node characteristic information in each DOM tree structure to obtain the same structure part in each DOM tree, and generating a page component with a plurality of pages capable of being reused based on the obtained node characteristic information of the same structure part, so that the generation automation of the page component capable of being reused in the plurality of pages is realized, and the generation efficiency and the accuracy of the page component are improved.

Fig. 8 shows a block diagram of a computer device 800 provided in an exemplary embodiment of the present application. The computer device 800 may be a portable mobile terminal, such as: a smartphone, a tablet, a laptop, or a desktop computer. Computer device 800 may also be referred to by other names such as user equipment, portable terminals, laptop terminals, desktop terminals, and the like.

Generally, the computer device 800 includes: a processor 801 and a memory 802.

The processor 801 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and so forth. The processor 801 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 801 may also include a main processor and a coprocessor, where the main processor is a processor for Processing data in an awake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 801 may be integrated with a GPU (Graphics Processing Unit) which is responsible for rendering and drawing the content required to be displayed by the display screen. In some embodiments, the processor 801 may further include an AI (Artificial Intelligence) processor for processing computing operations related to machine learning.

Memory 802 may include one or more computer-readable storage media, which may be non-transitory. Memory 802 may also include high speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory 802 is used to store at least one instruction for execution by processor 801 to implement the page component generation method provided by method embodiments herein.

In some embodiments, the computer device 800 may further optionally include: a peripheral interface 803 and at least one peripheral. The processor 801, memory 802 and peripheral interface 803 may be connected by bus or signal lines. Various peripheral devices may be connected to peripheral interface 803 by a bus, signal line, or circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 804, a display screen 805, a camera assembly 806, an audio circuit 707, a positioning assembly 808, and a power supply 809.

The peripheral interface 803 may be used to connect at least one peripheral related to I/O (Input/Output) to the processor 801 and the memory 802. In some embodiments, the processor 801, memory 802, and peripheral interface 803 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 801, the memory 802, and the peripheral interface 803 may be implemented on separate chips or circuit boards, which are not limited by this embodiment.

In some embodiments, the computer device 800 also includes one or more sensors 810. The one or more sensors 810 include, but are not limited to: acceleration sensor 811, gyro sensor 812, pressure sensor 813, fingerprint sensor 814, optical sensor 815 and proximity sensor 816.

Those skilled in the art will appreciate that the configuration illustrated in FIG. 8 is not intended to be limiting of the computer device 800 and may include more or fewer components than those illustrated, or some components may be combined, or a different arrangement of components may be employed.

Fig. 9 shows a block diagram of a computer device 900 according to an exemplary embodiment of the present application. The computer device may be implemented as a protection blocking device in the above-mentioned aspect of the present application. The computer apparatus 900 includes a Central Processing Unit (CPU) 901, a system Memory 904 including a Random Access Memory (RAM) 902 and a Read-Only Memory (ROM) 903, and a system bus 905 connecting the system Memory 904 and the CPU 901. The computer device 900 also includes a basic Input/Output system (I/O system) 906, which facilitates the transfer of information between devices within the computer, and a mass storage device 907 for storing an operating system 913, application programs 914, and other program modules 915.

The basic input/output system 906 includes a display 908 for displaying information and an input device 909 such as a mouse, keyboard, etc. for user input of information. Wherein the display 908 and the input device 909 are connected to the central processing unit 901 through an input output controller 910 connected to the system bus 905. The basic input/output system 906 may also include an input/output controller 910 for receiving and processing input from a number of other devices, such as a keyboard, mouse, or electronic stylus. Similarly, input-output controller 910 also provides output to a display screen, a printer, or other type of output device.

The mass storage device 907 is connected to the central processing unit 901 through a mass storage controller (not shown) connected to the system bus 905. The mass storage device 907 and its associated computer-readable media provide non-volatile storage for the computer device 900. That is, the mass storage device 907 may include a computer-readable medium (not shown) such as a hard disk or Compact Disc-Only Memory (CD-ROM) drive.

Without loss of generality, the computer-readable media may comprise computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM, ROM, Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), flash Memory or other solid state Memory technology, CD-ROM, Digital Versatile Disks (DVD), or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage, or other magnetic storage devices. Of course, those skilled in the art will appreciate that the computer storage media is not limited to the foregoing. The system memory 904 and mass storage device 907 described above may be collectively referred to as memory.

The computer device 900 may also operate as a remote computer connected to a network via a network, such as the internet, in accordance with various embodiments of the present disclosure. That is, the computer device 900 may be connected to the network 912 through the network interface unit 911 coupled to the system bus 905, or the network interface unit 911 may be used to connect to other types of networks or remote computer systems (not shown).

The memory further includes at least one instruction, at least one program, a code set, or a set of instructions, which is stored in the memory, and the central processor 901 implements all or part of the steps in the page component generating method shown in the foregoing embodiments by executing the at least one instruction, the at least one program, the code set, or the set of instructions.

In an exemplary embodiment, a computer readable storage medium is further provided, and at least one computer program is stored in the computer readable storage medium, and the computer program is loaded by a processor and executed to implement all or part of the steps of the method shown in any one of the above-mentioned fig. 2 or fig. 3. For example, the non-transitory computer readable storage medium may be a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

In an exemplary embodiment, a computer program product or a computer program is also provided, which comprises computer instructions, which are stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform all or part of the steps of the method described in any of the embodiments of fig. 2 or fig. 3.

Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.

It will be understood that the present application is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims

1. A method for generating page components, the method comprising:

acquiring page contents of at least two pages;

2. The method according to claim 1, wherein the node characteristic information includes first characteristic information including a class name and a node name of a corresponding node;

the extracting the same structural parts in the DOM trees of the at least two pages based on the node characteristic information of each node in the DOM trees of the at least two pages includes:

performing hash calculation based on the first characteristic information of each node in DOM trees in the at least two pages to obtain a hash attribute of each node;

extracting identical structural portions in DOM trees of the at least two pages based on the hash-attributes of the respective nodes in the DOM trees of the at least two pages.

3. The method according to claim 2, wherein performing a hash calculation based on the first feature information of each node in the DOM trees in the at least two pages to obtain a hash attribute of each node comprises:

responding to the fact that a target node does not have an effective child node, splicing the class name of the target node and the node name of the target node to obtain a splicing result of the target node; the target node is any one of nodes in DOM trees in the at least two pages;

4. The method according to claim 2, wherein performing a hash calculation based on the first feature information of each node in the DOM trees in the at least two pages to obtain a hash attribute of each node comprises:

responding to that a target node has an effective child node, splicing the class name of the effective child node and the node name of the effective child node to obtain a splicing result of the effective child node; the target node is any one of nodes in DOM trees in the at least two pages;

calculating the hash value of the splicing result of the effective child node;

calculating the hash value of the splicing result of the target node;

5. The method according to claim 4, wherein the obtaining the splicing result of the valid child node by splicing the class name and the node name of the valid child node comprises:

and responding to the fact that the effective child nodes comprise at least two repeated nodes, splicing the class name and the node name of one node of the at least two repeated nodes, and obtaining a splicing result of the effective child nodes.

6. The method according to any one of claims 2 to 5, wherein the class name is composed of at least two element class names, the node name is composed of at least two element node names, and before extracting the same structural part in the DOM trees of the at least two pages based on the node characteristic information of each node in the DOM trees of the at least two pages, the method further comprises:

sequencing the at least two element class names of each node according to a first specified sequence to obtain the class names of each node;

and sequencing the at least two element node names of each node according to a second specified sequence to obtain the node names of each node.

7. The method according to claim 2, wherein the node characteristic information further comprises hierarchy information of the corresponding node, the hierarchy information being used to correspond to a hierarchy of the node in a DOM tree;

the extracting identical structural portions in the DOM trees of the at least two pages based on the hash-attribute of each node in the DOM trees of the at least two pages comprises:

obtaining the hash attribute of a node contained in each DOM tree in the at least two pages to obtain the hash attributes of the at least two nodes; the at least two nodes are nodes having the same hierarchical information;

and determining the at least two nodes as the nodes belonging to the same structural part in response to the Hash attribute between every two of the at least two nodes meeting a specified condition.

8. The method of claim 7, wherein the specified condition comprises:

9. The method according to claim 7, wherein the obtaining the hash attribute of one node included in each DOM tree in the at least two pages to obtain the hash attributes of the at least two nodes comprises:

and acquiring the hash attribute of a node contained in each DOM tree in the at least two pages according to the sequence of the hierarchy information from the root node to the child node, so as to obtain the hash attributes of the at least two nodes.

10. The method according to claim 7, wherein the generating the page component corresponding to the same structural portion based on the node characteristic information of the same structural portion comprises:

and constructing a DOM tree based on the node characteristic information of the nodes in the same structural part according to the sequence of the hierarchical information from the child node to the root node, and obtaining page components corresponding to the same structural part.

11. The method of claim 10, wherein the node characteristic information further comprises node attributes and sub-level attributes of the corresponding node;

constructing a DOM tree based on the node characteristic information of the nodes in the same structural part according to the sequence of the hierarchy information from the child node to the root node, and obtaining page components corresponding to the same structural part, wherein the page components comprise:

generating component node labels based on node names of nodes in the same structural part according to the sequence of the hierarchy information from child nodes to root nodes;

building a component internal structure based on the sub-level attributes of the nodes in the same structural part;

12. An apparatus for generating a page component, the apparatus comprising:

13. The apparatus according to claim 12, wherein the node characteristic information includes first characteristic information, the first characteristic information including a class name and a node name of a corresponding node;

the same structure extraction module comprises:

14. A computer device comprising a processor and a memory, said memory having stored therein at least one instruction, at least one program, a set of codes, or a set of instructions; the at least one instruction, the at least one program, the set of codes, or the set of instructions being loaded and executed by the processor to implement the page component generation method of any of claims 1 to 11.

15. A computer-readable storage medium, in which at least one computer program is stored, the computer program being loaded and executed by a processor to implement the page component generating method according to any one of claims 1 to 11.