CN115033635A - Data extraction method and device, processor and electronic equipment - Google Patents

Abstract

The application discloses a data extraction method, a data extraction device, a processor and electronic equipment. Relating to the field of big data and other fields, the method comprises the following steps: acquiring structured data from a structured database to obtain a plurality of first data; acquiring structured data from a document to obtain a plurality of second data, wherein the document is an unstructured document; taking the first data and the second data as initial data, establishing a preset incidence relation between all the initial data according to the incidence fields, and establishing a target database according to the initial data; and determining a first target field and a second target field, and extracting target data in a target database through a preset incidence relation, the first target field and the second target field. By the method and the device, the problem that in the related technology, due to the fact that the data source has both structured data and unstructured data and the unstructured data cannot be retrieved, all data needing to be extracted cannot be extracted from the data source directly is solved.

Description

Data extraction method, device, processor and electronic device

technical field

The present application relates to the field of big data and other fields, and in particular, to a data extraction method, apparatus, processor and electronic device.

Background technique

Enterprises accumulate massive amounts of data based on daily operational data, and enterprises store massive amounts of structured data through the establishment of structured database systems. However, enterprises also have a large amount of unstructured data such as document reports, which cannot be uniformly stored in a database. In addition, the same type of documents are often similar in structure and different in content, and are often different for the same specific project. The emphasis is on repetitive generation of multiple documents.

In the related art, business personnel review documents one by one through manual processing, extract useful information, search for required fields according to the database system, and summarize and organize into a new document. This manual processing method has a large workload and low efficiency. In addition, due to information asymmetry among business personnel, the reporting content overlaps, which wastes a lot of labor costs. In the prior art, elements of required information are extracted by performing natural language processing on a certain type of document. The specific method is to extract relevant metadata from unstructured text data and convert it into structured data according to constraints.

However, the existing technology can only extract the content of a certain type of document, but in actual business work, there are a large number of various types of documents related to the project. Facing the growing demand for unstructured text extraction, despite accumulating massive amounts of information, they are relatively independent of each other, and the existing structured data has not been fully utilized.

In the related art, there are both structured data and unstructured data in the data source. Since the unstructured data cannot be retrieved, it is difficult to directly extract all the data that needs to be extracted from the data source. Effective solution.

SUMMARY OF THE INVENTION

The main purpose of the present application is to provide a data extraction method, device, processor and electronic equipment, so as to solve the problem that there are both structured data and unstructured data in the data source in the related art. Therefore, it is difficult to directly extract all the data that needs to be extracted from the data source.

In order to achieve the above object, according to an aspect of the present application, a data extraction method is provided. The method includes: acquiring structured data from a structured database to obtain a plurality of first data, wherein the structured database is used for storing structured data; acquiring structured data from a document to obtain a plurality of second data, wherein, The document is an unstructured document; the first data and the second data are used as the initial data, the preset association relationship between all the initial data is established according to the associated fields, and the target database is established according to the initial data; the first target field and the second data are determined. The target field is to extract target data from the target database through the preset association relationship, the first target field and the second target field.

Optionally, establishing a preset association relationship between all the initial data according to the association field includes: establishing an association relationship between each pair of initial data with the same field, obtaining multiple association relationships, and combining the multiple association relationships into a preset association relationship. Set up a relationship.

Optionally, extracting the target data from the target database by using the preset association relationship, the first target field and the second target field includes: traversing the target database, determining that initial data of the first target field exists, obtaining the first initial data, and determining. There is initial data of the second target field, and the second initial data is obtained; the first initial data is determined as the start node, the second initial data is determined as the end node, and all nodes from the start node to the end node are determined by the preset association relationship path, and obtain multiple paths, wherein each path contains at least two nodes corresponding to the initial data; determine the target path from the multiple paths; extract the data corresponding to each node in the target path in the target database to obtain the target path data.

Optionally, determining the target path from the multiple paths includes: in each path, determining the weight of the node according to the degree of accuracy of the initial data corresponding to the node, wherein the degree of accuracy is used to represent the accuracy of extracting the initial data from the data source. degree; calculate the sum of the weights of all nodes on each path to obtain multiple path weights; determine the path corresponding to the minimum path weight among the multiple paths as the target path.

Optionally, determining the weight of the node according to the degree of accuracy of the initial data corresponding to the node includes: in the case that the initial data is the second data, obtaining the accuracy rate of extracting the second data through the data extraction model, and determining the inverse of the accuracy rate. is the weight of the node; when the initial data is the first data, the preset value is determined as the weight of the node.

Optionally, acquiring a plurality of second data from a plurality of documents includes: determining a data extraction model of a corresponding type from a plurality of data extraction models according to the type of each document; extracting data from each document according to the data extraction model of the corresponding type. Extract the second data from the second data to obtain a plurality of second data.

Optionally, after extracting the target data from the target database through the preset association relationship, the first target field and the second target field, the method further includes: determining a preset condition, and removing data that does not meet the preset condition from the target data to get the updated target data.

In order to achieve the above object, according to another aspect of the present application, a data extraction apparatus is provided. The device includes: a first acquisition unit, used for acquiring structured data from a structured database, to obtain a plurality of first data, wherein the structured database is used for storing structured data; a second acquiring unit, used for extracting data from documents Obtaining structured data to obtain a plurality of second data, wherein the document is an unstructured document; a determining unit, used for using the first data and the second data as initial data, and establishing presets between all initial data according to the associated fields The association relationship is established, and the target database is established according to the initial data; the extraction unit is used to determine the first target field and the second target field, and extract the target data from the target database through the preset association relationship, the first target field and the second target field.

Through the present application, the following steps are adopted: obtaining structured data from a structured database to obtain a plurality of first data, wherein the structured database is used to store structured data; obtaining structured data from a document, obtaining a plurality of second data data, wherein the document is an unstructured document; the first data and the second data are used as initial data, a preset association relationship between all initial data is established according to the associated fields, and a target database is established according to the initial data; the first target is determined field and the second target field, extract target data in the target database through the preset association relationship, the first target field and the second target field, which solves the problem that there are both structured data and unstructured data in the data source in the related art, Since unstructured data cannot be retrieved, it is difficult to directly extract all the data that needs to be extracted from the data source. By extracting structured data from unstructured documents and establishing the relationship between the structured data in the existing database and the extracted structured data, the effect of retrieving data from unstructured documents is achieved.

Description of drawings

The accompanying drawings constituting a part of the present application are used to provide further understanding of the present application, and the schematic embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation of the present application. In the attached image:

1 is a flowchart of a data extraction method provided according to an embodiment of the present application;

2 is a schematic diagram of a path provided according to an embodiment of the present application;

3 is a flowchart of a document generation method provided according to an embodiment of the present application;

4 is a schematic diagram of a data extraction apparatus provided according to an embodiment of the present application;

FIG. 5 is a schematic diagram of an electronic device provided according to an embodiment of the present application.

Detailed ways

It should be noted that the embodiments in the present application and the features of the embodiments may be combined with each other in the case of no conflict. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

In order to make those skilled in the art better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only The embodiments are part of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the scope of protection of the present application.

It should be noted that the terms "first", "second", etc. in the description and claims of the present application and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances for the embodiments of the application described herein. Furthermore, the terms "comprising" and "having" and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those expressly listed Rather, those steps or units may include other steps or units not expressly listed or inherent to these processes, methods, products or devices.

It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to display data, analysis data, etc.) involved in this disclosure are all authorized by the user or information and data fully authorized by the parties.

The present invention will be described below in conjunction with the preferred implementation steps. FIG. 1 is a flowchart of a data extraction method provided according to an embodiment of the present application. As shown in FIG. 1 , the method includes the following steps:

Step S101: Acquire structured data from a structured database to obtain a plurality of first data, wherein the structured database is used for storing structured data.

Specifically, structured data is data that can obtain corresponding information through inherent key values, and the format of the data is fixed, such as numbers, symbols, and the like.

Step S102: Obtain structured data from the document to obtain a plurality of second data, wherein the document is an unstructured document.

Specifically, an unstructured document refers to a document that stores various texts, pictures, and other unstructured data, and extracts structured data from multiple documents through various types of data extraction models to obtain multiple pieces of second data. For example, the second data is extracted from documents of the invoice document type according to the invoice document extraction model.

Step S103, taking the first data and the second data as initial data, establishing a preset association relationship between all initial data according to the association field, and establishing a target database according to the initial data.

Specifically, the initial data includes the first data and the second data. In order to facilitate retrieval, the first data and the second data are stored in the same target database. The associated field refers to the same field existing between the initial data, for example, two If the initial data all contain the same company name, a relationship is established between the two initial data. There can be multiple associations for each initial data. The association relationship between all the initial data together constitutes a preset association relationship.

Step S104, determining the first target field and the second target field, and extracting target data from the target database through the preset association relationship, the first target field and the second target field.

Specifically, the first target field and the second target field may be fields selected by the user for retrieving data. For example, the name of company m is used as the first target field, and the transfer record of company m is used as the second target field. A target field determines the start node in the preset association relationship, the second target field determines the end node in the preset association relationship, the target path is determined through the preset association relationship, and all nodes on the target path are corresponding to the target path. The initial data is extracted from the target database to obtain the target data.

In the data extraction method provided by the embodiments of the present application, a plurality of first data are obtained by obtaining structured data from a structured database, wherein the structured database is used to store structured data; the structured data is obtained from a document, and multiple first data are obtained. a second data, wherein the document is an unstructured document; take the first data and the second data as initial data, establish a preset association relationship between all initial data according to the associated field, and establish a target database according to the initial data; determine The first target field and the second target field extract the target data from the target database through the preset association relationship, the first target field and the second target field, which solves the problem that there are both structured data and unstructured data in the data source in the related art. Due to the inability to retrieve unstructured data, it is difficult to directly extract all the data that needs to be extracted from the data source. By extracting structured data from unstructured documents and establishing the relationship between the structured data in the existing database and the extracted structured data, the effect of retrieving data from unstructured documents is achieved.

In order to facilitate data extraction from the target database, a preset association relationship is established between the initial data. Optionally, in the data extraction method provided in this embodiment of the present application, a preset association relationship between all initial data is established according to an association field. The method includes: establishing an association relationship between each pair of initial data with the same field, obtaining multiple association relationships, and combining the multiple association relationships into a preset association relationship.

Specifically, the same field can be the same customer code, customer name, certificate number, etc., for example, there is a guest code field m0101 in the field of the initial data A, and the same guest code field m0101 also exists in the field of the initial data B, then the initial An association relationship is established between data A and initial data B. An association relationship is established between all the initial data with the same field, and all the association relationships are combined into a preset association relationship. Interrelated data can be easily extracted by establishing a preset association relationship between the initial data.

After the preset association relationship is established, the target data can be quickly extracted according to the preset association relationship. Optionally, in the data extraction method provided in the embodiment of the present application, the preset association relationship, the first target field and the second target field are Extracting the target data from the target database includes: traversing the target database, determining that the initial data of the first target field exists, obtaining the first initial data, and determining that the initial data of the second target field exists, and obtaining the second initial data; The data is determined as the start node, the second initial data is determined as the end node, and all paths from the start node to the end node are determined through a preset association relationship, and multiple paths are obtained, wherein each path contains at least two initial data The corresponding node; the target path is determined from the multiple paths; the data corresponding to each node in the target path is extracted from the target database to obtain the target data.

Specifically, the first target field can be a field that the user needs to retrieve, such as company name C, and the second target field can be a field related to the first target field, such as the transaction record of company C, and the user needs to extract all the data from the target database. Therefore, it is determined that the first target field is the start node, and the second target field is the end node. According to the association relationship between the initial data, all paths from the start node to the end node are obtained, and from One target path is determined among all the paths. The target path is the path with the smallest weight among all the paths, and it is also the path with the most accurate retrieval result. After the target path is determined, the initial data corresponding to all nodes on the target path is extracted, that is, the target data. By extracting the target data according to the preset association relationship, the retrieval efficiency can be improved.

The determination of the target path needs to calculate the path weight of each path. Optionally, in the data extraction method provided by the embodiment of the present application, determining the target path from multiple paths includes: in each path, according to the initial The degree of accuracy of the data determines the weight of the node, where the degree of accuracy is used to characterize the accuracy of extracting the initial data from the data source; calculate the sum of the weights of all nodes on each path to obtain multiple path weights; The path corresponding to the minimum path weight is determined as the target path.

Specifically, the degree of accuracy can be the degree of accuracy when the initial data in the target database is acquired. By calculating the inverse of the degree of accuracy of the initial data corresponding to each node, the weight of the node is obtained, and the sum of the weights of the nodes on all paths is calculated. , determine the path weight, the path corresponding to the smallest path weight is the target path, and the user can extract the most accurate target data by determining the target path.

For example, FIG. 2 is a schematic diagram of a path provided according to an embodiment of the present application. As shown in FIG. 2 , the start node is A, the end node is D, and there are two paths from A to D, namely A-B-D and A-B-C-D, point A The weight of point B is 1.5, the weight of point B is 1.6, the weight of point C is 1.4, and the weight of point D is 1, then the path weight of A-B-D is 1.5+1.6+1 equal to 4.1, and the path weight of A-B-C-D is 1.5+1.6+1.4 +1 equals 5.5, the path with the smallest weight is A-B-D, so the target path is A-B-D.

Before determining the target path, the weight of each node needs to be determined. Optionally, in the data extraction method provided by the embodiment of the present application, determining the weight of the node according to the accuracy of the initial data corresponding to the node includes: when the initial data is the first In the case of the second data, the accuracy rate of the second data extracted by the data extraction model is obtained, and the inverse of the accuracy rate is determined as the weight of the node; in the case that the initial data is the first data, the preset value is determined as the node's weight. Weights.

Specifically, the weight of the node is determined according to the type of the initial data. If the initial data is the second data, since the second data is extracted through the data extraction model, there will be different accuracy rates according to different data extraction models. The reciprocal of the accuracy rate is used as the weight of the second data. If the initial data is the first data, since the first data is derived from the structured data of the database, the accuracy rate of the first data is 1, so the weight of the first data is also 1. , that is, the default value is 1. By setting the weight of the node, the most accurate data extraction path can be selected.

The second data is extracted from the documents according to various types of data extraction models. Optionally, in the data extraction method provided in this embodiment of the present application, acquiring multiple pieces of second data from multiple documents includes: according to each A data extraction model of a corresponding type is determined from a plurality of data extraction models for each document type; second data is extracted from each document according to the data extraction model of the corresponding type to obtain a plurality of second data.

Specifically, the data extraction model may be a contract document extraction model, a credit document extraction model, an invoice document extraction model, etc., according to the contract document extraction model to extract the second data from the documents of the contract document type, and according to the credit document extraction model from the credit document type The second data is extracted from the documents of the invoice document, and the second data is extracted from the documents of the invoice document type according to the invoice document extraction model to obtain a plurality of second data. By extracting the second data from the document according to different data extraction models, the problem that the target data cannot be retrieved from the unstructured data can be avoided.

Since there are still some data that the user does not need to extract in the extracted target data, the target data is filtered to obtain updated target data. After the preset association relationship, the first target field and the second target field are extracted from the target database, the method further includes: determining a preset condition, removing data that does not meet the preset condition from the target data, and obtaining an updated target data.

For example, the preset condition may be the target data with the customer name m, and the data without the customer name m is eliminated from the target data to obtain the updated target data. By further screening the target data, it is ensured that the target data will not be mixed with invalid data.

According to another embodiment of the present application, a method for generating documents is provided. FIG. 3 is a flowchart of the method for generating documents according to an embodiment of the present application. As shown in FIG. 3 , first, the extraction indexes of massive documents of the modeling platform are obtained. , and establish the corresponding database. Then, the acquired indicator library is integrated with the existing database, and knowledge such as document/table-related fields-document/table is extracted, and the association between the database and the document library is established. Next, retrieve the shortest path between the data containing the required fields. Finally, all the extracted fields are automatically generated according to the document template.

With the document generation method provided by the embodiment of the present application, the unstructured data is converted, and the unstructured data and the structured data are fused, so that the data is stored in the same database. Through the extraction of massive documents and the establishment of an indicator library, business personnel can directly generate documents containing the required indicators by defining the indicators required to be output, reducing the work pressure of business personnel.

It should be noted that the steps shown in the flowcharts of the accompanying drawings may be executed in a computer system, such as a set of computer-executable instructions, and, although a logical sequence is shown in the flowcharts, in some cases, Steps shown or described may be performed in an order different from that herein.

The embodiment of the present application further provides a data extraction apparatus. It should be noted that the data extraction apparatus of the embodiment of the present application may be used to execute the method for data extraction provided by the embodiment of the present application. The data extraction apparatus provided by the embodiment of the present application will be introduced below.

FIG. 4 is a schematic diagram of a data extraction apparatus according to an embodiment of the present application. As shown in Figure 4, the device includes:

The first obtaining unit 10 is configured to obtain structured data from a structured database to obtain a plurality of first data, wherein the structured database is used for storing structured data;

The second obtaining unit 20 is configured to obtain structured data from the document to obtain a plurality of second data, wherein the document is an unstructured document;

The determining unit 30 is configured to use the first data and the second data as initial data, establish a preset association relationship between all the initial data according to the associated field, and establish a target database according to the initial data;

The extraction unit 40 is configured to determine the first target field and the second target field, and extract the target data from the target database through the preset association relationship, the first target field and the second target field.

In the data extraction device provided by the embodiment of the present application, the first obtaining unit 10 obtains structured data from a structured database to obtain a plurality of first data, wherein the structured database is used for storing structured data; the second obtaining unit 20. Obtain structured data from the document, and obtain a plurality of second data, wherein the document is an unstructured document; the determining unit 30 uses the first data and the second data as initial data, and establishes a relationship between all initial data according to the associated field. The preset association relationship between them, and the target database is established according to the initial data; the extraction unit 40 determines the first target field and the second target field, and extracts the target database through the preset association relationship, the first target field and the second target field The target data solves the problem in related technologies that there are both structured data and unstructured data in the data source. Since the unstructured data cannot be retrieved, it is difficult to directly extract all the data that needs to be extracted from the data source. , by extracting structured data from unstructured documents, and establishing the connection between the structured data in the existing database and the extracted structured data, so as to achieve the effect of retrieving data from unstructured documents .

Optionally, in the data extraction apparatus provided in this embodiment of the present application, the determining unit 30 includes: a building module, configured to establish an association relationship between pairs of initial data with the same field, obtain a plurality of association relationships, and combine the multiple A combination of association relationships is a preset association relationship.

Optionally, in the data extraction device provided in the embodiment of the present application, the extraction unit 40 includes: a traversal module, configured to traverse the target database, determine that the initial data of the first target field exists, obtain the first initial data, and determine that the first target field exists. The initial data of the two target fields is used to obtain the second initial data; the first determination module is used to determine the first initial data as the starting node, the second initial data as the ending node, and determine all the starting nodes through the preset association relationship. From the path from the start node to the end node, multiple paths are obtained, wherein each path contains at least two nodes corresponding to the initial data; the second determination module is used to determine the target path from the multiple paths; the extraction module is used to The data corresponding to each node in the target path is extracted from the target database to obtain target data.

Optionally, in the data extraction apparatus provided in the embodiment of the present application, the second determination module includes: a first determination sub-module, configured to determine the weight of the node according to the accuracy of the initial data corresponding to the node in each path. , where the degree of accuracy is used to characterize the accuracy of extracting the initial data from the data source; the calculation sub-module is used to calculate the sum of the weights of all nodes on each path to obtain multiple path weights; the second determination sub-module uses It is used to determine the path corresponding to the minimum path weight among the multiple paths as the target path.

Optionally, in the data extraction apparatus provided in the embodiment of the present application, the first determination sub-module includes: an accuracy rate acquisition module, configured to acquire the second data extracted by the data extraction model when the initial data is the second data. and determine the reciprocal of the accuracy as the weight of the node; the weight determination module is used to determine the preset value as the weight of the node when the initial data is the first data.

Optionally, in the data extraction apparatus provided in the embodiment of the present application, the second acquisition unit 20 includes: a third determination module, configured to determine a data extraction model of a corresponding type from a plurality of data extraction models according to the type of each document. The extraction module is used to extract the second data from each document according to the data extraction model of the corresponding type, and obtain a plurality of second data.

Optionally, in the data extraction device provided in the embodiment of the present application, the device further includes: an update unit, configured to determine a preset condition, remove data that does not meet the preset condition from the target data, and obtain updated target data .

The data extraction device includes a processor and a memory. The above-mentioned first acquisition unit 10, second acquisition unit 20, determination unit 30, and extraction unit 40 are all stored in the memory as program units, and the processor executes the above-mentioned programs stored in the memory. unit to achieve the corresponding function.

The processor includes a kernel, and the kernel calls the corresponding program unit from the memory. The kernel can set one or more parameters to retrieve data from unstructured speech documents by adjusting the kernel parameters.

Memory may include non-persistent memory in computer readable media, random access memory (RAM) and/or non-volatile memory, such as read only memory (ROM) or flash memory (flash RAM), the memory including at least one memory chip.

Embodiments of the present invention provide a computer-readable storage medium, on which a program is stored, and when the program is executed by a processor, a data extraction method is implemented.

An embodiment of the present invention provides a processor, where the processor is used for running a program, wherein the data extraction method is executed when the program is running.

As shown in FIG. 5 , an embodiment of the present invention provides an electronic device. The device 501 includes a processor, a memory, and a program stored in the memory and running on the processor. When the processor executes the program, the following steps are implemented: data extraction method . The devices in this article can be servers, PCs, PADs, mobile phones, and so on.

The application also provides a computer program product, when executed on a data processing device, adapted to execute a program initialized with the following method steps: a data extraction method.

As will be appreciated by those skilled in the art, the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

Memory may include non-persistent memory in computer readable media, random access memory (RAM) and/or non-volatile memory in the form of, for example, read only memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media includes both persistent and non-permanent, removable and non-removable media, and storage of information may be implemented by any method or technology. Information may be computer readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Flash Memory or other memory technology, Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.

It should also be noted that the terms "comprising", "comprising" or any other variation thereof are intended to encompass a non-exclusive inclusion such that a process, method, article or device comprising a series of elements includes not only those elements, but also Other elements not expressly listed, or which are inherent to such a process, method, article of manufacture, or apparatus are also included. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article of manufacture or apparatus that includes the element.

It will be appreciated by those skilled in the art that the embodiments of the present application may be provided as a method, a system or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The above are merely examples of the present application, and are not intended to limit the present application. Various modifications and variations of this application are possible for those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included within the scope of the claims of this application.

Claims (10)

1. a data extraction method, is characterized in that, comprises: Obtain structured data from a structured database to obtain a plurality of first data, wherein the structured database is used to store structured data; Obtain structured data from a document to obtain a plurality of second data, wherein the document is an unstructured document; Taking the first data and the second data as initial data, establishing a preset association relationship between all the initial data according to the associated field, and establishing a target database according to the initial data; A first target field and a second target field are determined, and target data is extracted from the target database through the preset association relationship, the first target field and the second target field. 2. The method according to claim 1, wherein establishing the preset association relationship between all the initial data according to the association field comprises: An association relationship between each pair of the initial data with the same field is established, a plurality of the association relationships are obtained, and the plurality of the association relationships are combined into the preset association relationship. 3. The method according to claim 1, wherein extracting target data from the target database by using the preset association relationship, the first target field and the second target field comprises: Traverse the target database, determine that the initial data of the first target field exists, obtain the first initial data, and determine that the initial data of the second target field exists, and obtain the second initial data; Determining the first initial data as a start node, and determining the second initial data as an end node, All paths from the start node to the end node are determined according to the preset association relationship, and multiple paths are obtained, wherein each of the paths includes at least two nodes corresponding to the initial data; determining a target path from a plurality of said paths; Data corresponding to each node in the target path is extracted from the target database to obtain the target data. 4. The method according to claim 3, wherein determining a target path from a plurality of the paths comprises: In each path, the weight of the node is determined according to the degree of accuracy of the initial data corresponding to the node, wherein the degree of accuracy is used to represent the accuracy of extracting the initial data from the data source; Calculate the sum of the weights of all the nodes on each of the paths to obtain multiple path weights; The path corresponding to the smallest path weight among the multiple paths is determined as the target path. 5. The method according to claim 4, wherein determining the weight of the node according to the accuracy of the initial data corresponding to the node comprises: In the case that the initial data is the second data, obtain the accuracy rate of extracting the second data through a data extraction model, and determine the inverse of the accuracy rate as the weight of the node; When the initial data is the first data, a preset value is determined as the weight of the node. 6. The method of claim 1, wherein acquiring a plurality of second data from a plurality of documents comprises: Determine the corresponding type of data extraction model from multiple data extraction models according to the type of each document; The second data is extracted from each of the documents according to the data extraction model of the corresponding type, to obtain a plurality of the second data. 7 . The method according to claim 1 , wherein after extracting target data from the target database through the preset association relationship, the first target field and the second target field, the Methods also include: A preset condition is determined, data that does not meet the preset condition is eliminated from the target data, and updated target data is obtained. 8. A data extraction device, characterized in that, comprising: a first obtaining unit, configured to obtain structured data from a structured database to obtain a plurality of first data, wherein the structured database is used to store structured data; a second obtaining unit, configured to obtain structured data from a document to obtain a plurality of second data, wherein the document is an unstructured document; a determining unit, configured to use the first data and the second data as initial data, establish a preset association relationship between all the initial data according to the associated field, and establish a target database according to the initial data; The extraction unit is configured to determine a first target field and a second target field, and extract target data from the target database through the preset association relationship, the first target field and the second target field. 9 . A processor, wherein the processor is configured to run a program, wherein the data extraction method according to any one of claims 1 to 7 is executed when the program is run. 10. An electronic device, comprising one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more programs When executed by the processor, the one or more processors are caused to implement the data extraction method described in any one of claims 1 to 7 .

Images