CN116502633A - Method and device for executing service, storage medium and electronic equipment - Google Patents

Abstract

This specification discloses a service execution method, device, storage medium and electronic equipment. First, obtain the knowledge graph to be expanded corresponding to the target business. Secondly, input the entity name corresponding to the target node in the knowledge graph to be expanded into the search engine to obtain the text data and image data corresponding to the target node. Then, input the text data corresponding to the target node into the pre-trained first recognition model, determine the triplet information of the text data, and input the image data corresponding to the target node into the pre-trained second recognition model, determine the image The triplet information of the data. Finally, based on the triplet information of the text data and the triplet information of the image data, the entity relationship between the target node and other nodes is expanded in the knowledge graph to be expanded to obtain the target knowledge graph, and through the target knowledge graph, the target business. The method can improve the accuracy of business execution.

Description

Method, device, storage medium and electronic equipment for business execution

technical field

This specification relates to the field of computer technology, and in particular to a method, device, storage medium, and electronic equipment for service execution.

Background technique

At present, the knowledge graph (KG) is essentially a semantic network, a graph-based data structure composed of nodes and edges. In the knowledge graph, each node represents an entity, and each edge represents the relationship between entities. Among them, an entity refers to something that is distinguishable and exists independently, for example, a certain person, a certain city, a certain kind of plant, a certain kind of commodity, etc.

However, in actual application scenarios, there may be less data used to construct knowledge graphs, resulting in lower accuracy of business execution through knowledge graphs.

Therefore, how to expand the knowledge graph to improve the accuracy of business execution is an urgent problem to be solved.

Contents of the invention

This specification provides a service execution method, device, storage medium and electronic equipment to partially solve the above-mentioned problems existing in the prior art.

This manual adopts the following technical solutions:

This specification provides a method for business execution, including:

Obtain the knowledge graph to be expanded corresponding to the target business;

Input the entity name corresponding to the target node in the knowledge map to be expanded into the search engine to obtain text data and image data corresponding to the target node;

Input the text data corresponding to the target node into the pre-trained first recognition model, so as to recognize the entity relationship between the entities involved in the text data through the first recognition model, and according to the Entity relationships between entities involved in the text data, determining the triplet information of the text data, and inputting the image data corresponding to the target node into the pre-trained second recognition model, so as to pass the first 2. a recognition model, identifying the entity relationship between the entities involved in the image data, and determining the triplet information of the image data according to the entity relationship between the entities involved in the image data;

Based on the triplet information of the text data and the triplet information of the image data, expand the entity relationship between the target node and other nodes in the knowledge graph to be expanded to obtain the target knowledge graph, and pass The target knowledge graph executes the target business.

Optionally, determining the target node specifically includes:

Obtain the business requirements corresponding to the target business;

According to the business requirements corresponding to the target business, determine the correlation between the entity names corresponding to the nodes in the knowledge map to be expanded and the business requirements corresponding to the target business, as the correlation of each node;

The nodes whose correlation degree is greater than the set correlation threshold are taken as the target nodes.

Optionally, determining the target node specifically includes:

Determine the number of triples corresponding to each node in the knowledge map to be expanded;

The node whose number of triples is less than the set number threshold is used as the target node.

Optionally, the first recognition model includes: a text recognition network and a text relationship network;

Input the text data corresponding to the target node into the pre-trained first recognition model, so as to recognize the entity relationship between the entities involved in the text data through the first recognition model, and according to the Entity relationship between entities involved in the text data, determine triplet information of the text data, specifically including:

Input the text data corresponding to the target node into the text recognition network, and determine the entities involved in the text data;

inputting the entities involved in the text data into the text relationship network, and determining the entity relationship between the entities involved in the text data;

The triplet information of the text data is determined according to the entities involved in the text data and the entity relationships among the entities involved in the text data.

Optionally, before inputting the text data corresponding to the target node into the text recognition network and determining entities involved in the text data, the method further includes:

performing data cleaning on the text data corresponding to the target node to obtain the cleaned text data;

Input the text data corresponding to the target node into the text recognition network, and determine each entity involved in the text data, specifically including:

The cleaned text data is input into the text recognition network, and entities involved in the cleaned text data are determined.

Optionally, the second recognition model includes: an image recognition network and an image relationship network;

input the image data corresponding to the target node into the pre-trained second recognition model, so as to recognize the entity relationship between the entities involved in the image data through the second recognition model, and according to the Entity relationship between entities involved in the image data, determine the triplet information of the image data, specifically include:

input the image data corresponding to the target node into the image recognition network, and determine the entities involved in the image data;

input the entities involved in the image data into the image relationship network, and determine the entity relationship between the entities involved in the image data;

The triplet information of the image data is determined according to the entities involved in the image data and the entity relationships among the entities involved in the image data.

Optionally, before inputting the image data corresponding to the target node into the image recognition network and determining entities involved in the image data, the method further includes:

Adjusting the image size and image resolution of the image data corresponding to the target node to obtain adjusted image data;

Input the image data corresponding to the target node into the image recognition network, and determine each entity involved in the image data, specifically including:

The adjusted image data is input into the image recognition network, and entities involved in the adjusted image data are determined.

Optionally, based on the triplet information of the text data and the triplet information of the image data, the entity relationship between the target node and other nodes is expanded in the knowledge graph to be expanded to obtain the target knowledge Spectrum, including:

Obtain various open source knowledge graphs;

Query the target node from the open source knowledge graphs, determine other nodes adjacent to the target node as adjacent nodes of the target node, and according to the entity relationship between the target node and the adjacent nodes of the target node , determining the triplet information corresponding to each of the open source knowledge graphs;

Based on the triplet information corresponding to each open source knowledge graph, the triplet information of the text data, and the triplet information of the image data, expand the knowledge graph to be expanded between the target node and other nodes The entity relationship among them to get the target knowledge graph.

Optionally, the open source knowledge graph includes: an open domain knowledge graph and a vertical domain knowledge graph corresponding to the target business.

Optionally, query the target node from each of the open source knowledge graphs, determine other nodes adjacent to the target node as adjacent nodes of the target node, and Entity relationship among entities, determine triplet information corresponding to each open source knowledge graph, specifically including:

Query the entity name corresponding to the target node from each of the open source knowledge graphs, and determine the open source knowledge graph containing the entity name corresponding to the target node as a candidate knowledge graph;

According to the similarity between the adjacent nodes of the target node in each candidate knowledge graph and the adjacent nodes of the target node in the knowledge graph to be expanded, determine the candidate knowledge graph corresponding to the target node;

The entity relationship between the target node and the adjacent nodes of the target node in the candidate knowledge graph corresponding to the target node is converted to obtain triple information of the candidate knowledge graph.

Optionally, training the first recognition model specifically includes:

Obtain text training samples;

Input the text training sample into the first recognition model to be trained, and determine the triplet information corresponding to the text training sample;

The first recognition model is trained with minimizing the deviation between the triplet information corresponding to the text training sample and the label corresponding to the text training sample as an optimization goal.

Optionally, training the second recognition model specifically includes:

Obtain image training samples;

Input the image training sample into the second recognition model to be trained, and determine the triplet information corresponding to the image training sample;

The second identification model is trained with the optimization goal of minimizing the deviation between the triplet information corresponding to the image training samples and the labels corresponding to the image training samples.

This manual provides a device for business execution, including:

The acquisition module is used to acquire the knowledge map to be expanded corresponding to the target business;

The input module is used to input the entity name corresponding to the target node in the knowledge map to be expanded into the search engine, and obtain the text data and image data corresponding to the target node;

A recognition module, configured to input the text data corresponding to the target node into the pre-trained first recognition model, so as to recognize the entity relationship between the entities involved in the text data through the first recognition model , and according to the entity relationship between the entities involved in the text data, determine the triplet information of the text data, and input the image data corresponding to the target node into the pre-trained second recognition model, Through the second recognition model, the entity relationship among the entities involved in the image data is identified, and according to the entity relationship between the entities involved in the image data, three elements of the image data are determined tuple information;

An execution module, configured to expand the entity relationship between the target node and other nodes in the knowledge map to be expanded based on the triple information of the text data and the triple information of the image data, so as to obtain the target knowledge graph, and execute the target business through the target knowledge graph.

This specification provides a computer-readable storage medium, the storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned service execution method is realized.

This specification provides an electronic device, including a memory, a processor, and a computer program stored on the memory and operable on the processor. The processor implements the above-mentioned service execution method when executing the program.

The above-mentioned at least one technical solution adopted in this specification can achieve the following beneficial effects:

In the business execution method provided in this manual, the knowledge map to be expanded corresponding to the target business is obtained. Secondly, input the entity name corresponding to the target node in the knowledge graph to be expanded into the search engine to obtain the text data and image data corresponding to the target node. Then, input the text data corresponding to the target node into the pre-trained first recognition model, so as to recognize the entity relationship between the entities involved in the text data through the first recognition model, and according to the various entities involved in the text data Entity relationship between entities, determine the triplet information of the text data, and input the image data corresponding to the target node into the pre-trained second recognition model, so that through the second recognition model, identify each element involved in the image data Entity relationship between entities, and according to the entity relationship between entities involved in the image data, triplet information of the image data is determined. Finally, based on the triplet information of the text data and the triplet information of the image data, the entity relationship between the target node and other nodes is expanded in the knowledge graph to be expanded to obtain the target knowledge graph, and through the target knowledge graph, the target business.

It can be seen from the above business execution method that this method can input the entity name corresponding to the target node in the knowledge map to be expanded into the search engine, and obtain the text data and image data corresponding to the target node. Then, input the text data corresponding to the target node into the pre-trained first recognition model, so as to determine the triplet information of the text data through the first recognition model, and input the image data corresponding to the target node into the pre-trained first recognition model. In the second recognition model, the triplet information of the image data is determined through the second recognition model. Finally, based on the triplet information of the text data and the triplet information of the image data, the entity relationship between the target node and other nodes is expanded in the knowledge graph to be expanded to obtain the target knowledge graph, and through the target knowledge graph, the target business. The method can improve the accuracy of business execution.

Description of drawings

The drawings described here are used to provide a further understanding of this specification and constitute a part of this specification. The schematic embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation of this specification. In the attached picture:

FIG. 1 is a schematic flowchart of a method for executing a business provided by an embodiment of this specification;

FIG. 2 is a schematic flow diagram of an expanded knowledge map provided by the embodiment of this specification;

FIG. 3 is a schematic structural diagram of a service execution device provided by an embodiment of this specification;

FIG. 4 is a schematic structural diagram of an electronic device provided by an embodiment of this specification.

Detailed ways

In order to make the purpose, technical solution and advantages of this specification clearer, the technical solution of this specification will be clearly and completely described below in conjunction with specific embodiments of this specification and corresponding drawings. Apparently, the described embodiments are only some of the embodiments in this specification, not all of them. Based on the embodiments in this specification, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of this specification.

The technical solutions provided by each embodiment of this specification will be described in detail below in conjunction with the accompanying drawings.

Fig. 1 is a schematic flow chart of the method of service execution provided by the embodiment of this specification, which specifically includes the following steps:

S100: Obtain the knowledge graph to be expanded corresponding to the target business.

In the embodiment of this specification, the execution subject of the business execution method provided in this specification may be an electronic device such as a server, a desktop computer, etc. For the convenience of description, the following only uses the server as the execution subject to describe the business execution method provided in this specification .

In this embodiment of the specification, the server may acquire the knowledge graph to be expanded corresponding to the target business.

S102: Input the entity name corresponding to the target node in the knowledge graph to be expanded into the search engine, and obtain text data and image data corresponding to the target node.

In this embodiment of the specification, the server may input the entity name corresponding to the target node in the knowledge graph to be expanded into the search engine to obtain text data and image data corresponding to the target node. The search engine mentioned here can refer to a system that uses specific computer programs to collect information on the Internet according to a certain strategy, and provides users with retrieval services after organizing and processing the information.

In practical applications, different businesses have different business requirements, and different nodes in the knowledge graph to be expanded have different degrees of correlation with the business requirements corresponding to the target business. Therefore, the server needs to determine a node with a high correlation with the business requirement corresponding to the target business as the target node, and then determine the triplet corresponding to the target node to expand the knowledge map to be expanded.

Specifically, the server may obtain the service requirement corresponding to the target service.

Secondly, according to the business requirements corresponding to the target business, the server can determine the correlation between the entity names corresponding to the nodes in the knowledge map to be expanded and the business requirements corresponding to the target business as the correlation of each node.

Then, the server may use a node whose correlation degree is greater than a set correlation threshold as a target node.

In practical applications, the number of triples corresponding to a node in the knowledge graph is limited. The more triples the node has, the harder it is for the node to expand. The smaller the number of triples in the node, the easier it is to expand the node. Therefore, the server may determine a node with a small number of triples in the knowledge graph to be expanded as a target node, thereby expanding the knowledge graph to be expanded.

In this embodiment of the specification, the server may determine the number of triples corresponding to each node in the knowledge graph to be expanded.

Then, the server may use the node whose number of triples is less than the set number threshold as the target node.

S104: Input the text data corresponding to the target node into the pre-trained first recognition model, so as to recognize the entity relationship between the entities involved in the text data through the first recognition model, and according to Entity relationships between entities involved in the text data, determining triplet information of the text data, and inputting the image data corresponding to the target node into the pre-trained second recognition model, so as to pass the The second identification model, which identifies the entity relationship between the entities involved in the image data, and determines the triplet information of the image data according to the entity relationship between the entities involved in the image data .

In the embodiment of this specification, the server may input the text data corresponding to the target node into the pre-trained first recognition model, so as to recognize the entity relationship between the entities involved in the text data through the first recognition model, and According to the entity relationship between the entities involved in the text data, determine the triplet information of the text data, and input the image data corresponding to the target node into the pre-trained second recognition model, so that through the second recognition model, the recognition The entity relationship among the entities involved in the image data is obtained, and the triple information of the image data is determined according to the entity relationship among the entities involved in the image data.

The triple information mentioned here may refer to (entity 1, relationship, entity 2), for example, (user A, apprentice, user B). It can also refer to (entity, attribute name, attribute value), for example, (Shenmu, scenic spot, Shenhu Lake).

In the embodiment of this specification, the first recognition model includes: a text recognition network and a text relationship network. The server can input the text data corresponding to the target node into the text recognition network to determine the entities involved in the text data. The text recognition network mentioned here can be represented by Bidirectional Encoder Representation from Transformers (BidirectionalEncoder Representation from Transformers, BERT), Bi-directional Long Short-Term Memory (BiLSTM) and conditional random field (conditional randomfield, CRF) composition. Bidirectionally encoded representations from Transformers refer to pre-trained models for determining feature vectors for text data. The bidirectional LSTM network refers to the encoding layer, which is used for feature extraction. The conditional random field refers to the decoding layer, which is used to output each entity in the text data. Of course, the text recognition network may also be other models for named entity recognition (NamedEntity Recognition, NER), and this specification does not limit the text recognition network.

Secondly, the server can input the entities involved in the text data into the text relationship network to determine the entity relationship among the entities involved in the text data.

Then, the server can determine the triplet information of the text data according to the entities involved in the text data and the entity relationships among the entities involved in the text data.

In practical applications, the text data searched by the search engine contains data such as empty characters, useless tags, special characters, etc., which may cause errors in the triplet information of the text data corresponding to the target node output by the first recognition model . Therefore, the server needs to perform data cleaning on the text data, so as to improve the accuracy of the output result of the first recognition model.

In the embodiment of this specification, the server may perform data cleaning on the text data corresponding to the target node to obtain the cleaned text data. For example, the server may remove text such as page labels, null characters, special characters, useless labels, etc. in the text data corresponding to the target node.

Then, the server may input the cleaned text data into the text recognition network to determine entities involved in the cleaned text data.

In the embodiment of this specification, before the server applies the first recognition model, it needs to train the first recognition model.

First, the server can obtain text training samples.

Secondly, the server may input the text training samples into the first recognition model to be trained, and determine triplet information corresponding to the text training samples.

Finally, the server may train the first recognition model with the optimization goal of minimizing the deviation between the triplet information corresponding to the text training sample and the label corresponding to the text training sample.

In this embodiment of the specification, the second recognition model may refer to a scene graph generation model (Scene GraphGeneration, SGG). Of course, the second recognition model may also be other models for recognizing entity relationships among entities in the image, and this specification does not limit the second recognition model.

Wherein, the second recognition model includes: an image recognition network and an image relationship network.

First, the server may input the image data corresponding to the target node into the image recognition network to determine the entities involved in the image data.

Then, the server can input the entities involved in the image data into the image relationship layer, and determine the entity relationship among the entities involved in the image data.

Finally, the server can determine the triplet information of the image data according to the entities involved in the image data and the entity relationships among the entities involved in the image data.

In practical applications, the image size and image resolution of the image data searched by the search engine are not the same, but the images with different image sizes and image resolutions are input to the second recognition model for processing, and the determined output The result is less accurate. Therefore, the server needs to adjust the image size and image resolution of each image data, so as to improve the accuracy of the output result of the second recognition model.

In the embodiment of this specification, the server may adjust the image size and image resolution of the image data corresponding to the target node to obtain the adjusted image data.

The server may then input the adjusted image data into the image recognition network to determine entities involved in the adjusted image data.

In the embodiment of this specification, before the server applies the second recognition model, it needs to train the second recognition model.

First, the server can obtain image training samples.

Secondly, the server may input the image training samples into the second recognition model to be trained, and determine triplet information corresponding to the image training samples.

Finally, the server may train the second recognition model with the optimization goal of minimizing the deviation between the triplet information corresponding to the image training samples and the labels corresponding to the image training samples.

S106: Based on the triplet information of the text data and the triplet information of the image data, expand the entity relationship between the target node and other nodes in the knowledge graph to be expanded to obtain a target knowledge graph.

In this embodiment of the specification, the server may expand the entity relationship between the target node and other nodes in the knowledge graph to be expanded based on the triple information of the text data and the triple information of the image data to obtain the target knowledge graph. Other nodes mentioned here may refer to existing nodes in the knowledge graph to be expanded, or may refer to newly added nodes obtained from triplet information of text data or triplet information of image data.

In practical applications, among the determined triplet information of text data and triplet information of image data, there is triplet information with a low accuracy rate. In order to avoid the low accuracy of the constructed target knowledge graph, the server can select triple information with a higher prediction probability according to the prediction probability when the recognition model predicts the triple information to construct the target knowledge graph.

In this embodiment of the specification, the server may determine the prediction probability when the first recognition model predicts the triplet information corresponding to the text data. And determine the triplet information corresponding to the text data whose prediction probability is greater than the set prediction probability threshold.

Likewise, the server may determine the prediction probability when the second recognition model predicts the triplet information corresponding to the image data. And determine the triplet information corresponding to the image data whose prediction probability is greater than the set prediction probability threshold.

Then, the server can expand the target node and Entity relationships between other nodes to get the target knowledge graph.

In practical applications, in addition to determining triplet information based on the entity relationship between entities in text data and image data, the server can also directly obtain triplet information from other knowledge graphs.

In this embodiment of the specification, the server can obtain each open source knowledge graph. Open source knowledge graphs include: open domain knowledge graphs and vertical domain knowledge graphs corresponding to target businesses. The open domain mentioned here may refer to a collection of all domains. The vertical field mentioned here may refer to a single field, for example, the medical field, the financial field, and so on.

Secondly, the server can query the target node from various open source knowledge graphs, determine other nodes adjacent to the target node as the adjacent nodes of the target node, and determine The triplet information corresponding to each open source knowledge graph.

Specifically, the server may input the target node into the entity link model to determine the location of the target node in each open source knowledge graph.

Finally, based on the triplet information corresponding to each open source knowledge graph, the triplet information of text data, and the triplet information of image data, the server can expand the entity relationship between the target node and other nodes in the knowledge graph to be expanded, so as to Get the target knowledge map.

Specifically, the server may query the entity name corresponding to the target node from each open source knowledge graph, and determine an open source knowledge graph containing the entity name corresponding to the target node as a candidate knowledge graph.

Secondly, the server may determine the candidate knowledge graph corresponding to the target node according to the similarity between the adjacent nodes of the target node in each candidate knowledge graph and the adjacent nodes of the target node in the knowledge graph to be expanded.

Finally, the server can convert the entity relationship between the target node in the candidate knowledge graph corresponding to the target node and the adjacent nodes of the target node to obtain the triplet information of the candidate knowledge graph.

Wherein, the server may also convert entity relationships between adjacent nodes of the target node and other nodes in the candidate knowledge graph corresponding to the target node to obtain triplet information of the candidate knowledge graph.

In practical applications, one entity may have multiple mentions, that is, multiple entity names. Therefore, the server needs to unify the names of multiple entities.

In this embodiment of the specification, the server may unify entity names in triplet information of text data, triplet information of image data, and triplet information of each open source knowledge graph. For example, if a user's surname is A and his name is B, and his aliases include Big B, Little A, etc., they need to be changed to AB uniformly.

Further, there may be repeated triplet information in triplet information of text data, triplet information of image data, and triplet information of each open source knowledge graph. Based on this, the server can deduplicate the information of the above triples to construct the target knowledge graph.

FIG. 2 is a schematic flowchart of an extended knowledge map provided by the embodiment of this specification.

In FIG. 2 , the server determines a node whose correlation degree is greater than a set correlation threshold from the knowledge graph to be expanded as a target node.

Secondly, input the entity name corresponding to the target node in the knowledge map to be expanded into the search engine, and obtain the text data corresponding to the target node, the image data corresponding to the target node, and the open source knowledge map corresponding to the target node.

Then, the server can input the text data corresponding to the target node into the pre-trained first recognition model, so as to recognize the entity relationship between the entities involved in the text data through the first recognition model, and according to the text data involved in The entity relationship between the entities, determine the triplet information of the text data, and input the image data corresponding to the target node into the pre-trained second recognition model, so that through the second recognition model, the image data related to The entity relationship between the various entities in the image data, and according to the entity relationship between the various entities involved in the image data, determine the triplet information of the image data.

Then, the server can input the target node into the entity link model, determine the location of the target node in each open source knowledge graph, and then determine the candidate knowledge graph corresponding to the target node, and combine the target node and the target node in the candidate knowledge graph corresponding to the target node The entity relationship between the adjacent nodes is converted to obtain the triple information of the candidate knowledge graph.

Finally, based on the triplet information corresponding to each open source knowledge graph, the triplet information of text data, and the triplet information of image data, the server can expand the entity relationship between the target node and other nodes in the knowledge graph to be expanded, so as to Get the target knowledge map.

It can be seen from the above process that this method can input the entity name corresponding to the target node in the knowledge map to be expanded into the search engine, and obtain the text data and image data corresponding to the target node. Then, input the text data corresponding to the target node into the pre-trained first recognition model, so as to determine the triplet information of the text data through the first recognition model, and input the image data corresponding to the target node into the pre-trained first recognition model. In the second recognition model, the triplet information of the image data is determined through the second recognition model. Finally, based on the triplet information of the text data and the triplet information of the image data, the entity relationship between the target node and other nodes is expanded in the knowledge graph to be expanded to obtain the target knowledge graph, and through the target knowledge graph, the target business. The method can improve the accuracy of business execution.

The above is the service execution method provided by one or more embodiments of this specification. Based on the same idea, this specification also provides a corresponding service execution device, as shown in FIG. 3 .

Fig. 3 is a schematic structural diagram of a service execution device provided by an embodiment of this specification, specifically including:

An acquisition module 300, configured to acquire the knowledge map to be expanded corresponding to the target business;

The input module 302 is used to input the entity name corresponding to the target node in the knowledge map to be expanded into the search engine to obtain text data and image data corresponding to the target node;

The recognition module 304 is configured to input the text data corresponding to the target node into the pre-trained first recognition model, so as to recognize the entities among the entities involved in the text data through the first recognition model relationship, and according to the entity relationship between the entities involved in the text data, determine the triplet information of the text data, and input the image data corresponding to the target node into the pre-trained second recognition model , so as to identify the entity relationship between the entities involved in the image data through the second recognition model, and determine the entity relationship between the entities involved in the image data according to the entity relationship between the entities involved in the image data triplet information;

The execution module 306 is configured to expand the entity relationship between the target node and other nodes in the knowledge map to be expanded based on the triple information of the text data and the triple information of the image data, so as to obtain A target knowledge graph, and execute the target business through the target knowledge graph.

Optionally, the input module 302 is specifically configured to obtain the business requirements corresponding to the target business, and determine the entity names corresponding to the nodes in the knowledge graph to be expanded and the target business requirements according to the business requirements corresponding to the target business. The correlation between the business requirements corresponding to the business is taken as the correlation of each node, and the node whose correlation is greater than the set correlation threshold is used as the target node.

Optionally, the input module 302 is specifically configured to determine the number of triples corresponding to each node in the to-be-expanded knowledge graph, and use a node whose triple number is less than a set number threshold as a target node.

Optionally, the first recognition model includes: a text recognition network and a text relationship network;

The recognition module 304 is specifically configured to input the text data corresponding to the target node into the text recognition network, determine the entities involved in the text data, and input the entities involved in the text data into The text relationship network determines the entity relationship between the entities involved in the text data, and determines the entity relationship between the entities involved in the text data and the entities involved in the text data. The triplet information describing the text data.

Optionally, the identification module 304 is further configured to perform data cleaning on the text data corresponding to the target node to obtain cleaned text data, and input the cleaned text data into the text recognition network , to determine the entities involved in the cleaned text data.

Optionally, the second recognition model includes: an image recognition network and an image relationship network;

The recognition module 304 is specifically configured to input the image data corresponding to the target node into the image recognition network, determine the entities involved in the image data, and input the entities involved in the image data into the The image relationship layer determines the entity relationship between the entities involved in the image data, and determines the entity relationship between the entities involved in the image data and the entities involved in the image data. The triplet information of the described image data.

Optionally, the identification module 304 is further configured to adjust the image size and image resolution of the image data corresponding to the target node to obtain adjusted image data, and input the adjusted image data to In the image recognition network, entities involved in the adjusted image data are determined.

Optionally, the execution module 306 is specifically configured to acquire each open source knowledge graph, query the target node from each open source knowledge graph, determine other nodes adjacent to the target node as adjacent nodes of the target node, and According to the entity relationship between the target node and the adjacent nodes of the target node, determine triplet information corresponding to each open source knowledge graph, based on the triplet information corresponding to each open source knowledge graph, the The triplet information of the text data and the triplet information of the image data expand the entity relationship between the target node and other nodes in the knowledge graph to be expanded to obtain the target knowledge graph.

Optionally, the open source knowledge graph includes: an open domain knowledge graph and a vertical domain knowledge graph corresponding to the target business.

Optionally, the execution module 306 is specifically configured to query the entity name corresponding to the target node from the open source knowledge graphs, and determine an open source knowledge graph containing the entity name corresponding to the target node as a candidate knowledge graph, According to the similarity between the adjacent nodes of the target node in each candidate knowledge graph and the adjacent nodes of the target node in the knowledge graph to be expanded, the candidate knowledge graph corresponding to the target node is determined, and the The entity relationship between the target node and the adjacent nodes of the target node in the candidate knowledge graph corresponding to the target node is converted to obtain the triplet information of the candidate knowledge graph.

Optionally, the recognition module 304 is specifically configured to obtain text training samples, input the text training samples into the first recognition model to be trained, and determine triplet information corresponding to the text training samples, with a minimum The deviation between the triplet information corresponding to the text training sample and the label corresponding to the text training sample is used as an optimization target, and the first recognition model is trained.

Optionally, the recognition module 304 is specifically configured to acquire image training samples, input the image training samples into the second recognition model to be trained, and determine triplet information corresponding to the image training samples, with a minimum The deviation between the triplet information corresponding to the image training sample and the label corresponding to the image training sample is used as an optimization target, and the second recognition model is trained.

This specification also provides a computer-readable storage medium, which stores a computer program, and the computer program can be used to execute the service execution method provided in FIG. 1 above.

This specification also provides a schematic structural diagram of the electronic device shown in FIG. 4 . As shown in FIG. 4 , at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and of course may also include hardware required by other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, so as to realize the service execution method provided in the above-mentioned FIG. 1 .

Of course, in addition to the software implementation, this specification does not exclude other implementations, such as logic devices or the combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic device.

In the 1990s, the improvement of a technology can be clearly distinguished as an improvement in hardware (for example, improvements in circuit structures such as diodes, transistors, and switches) or improvements in software (improvement in method flow). However, with the development of technology, the improvement of many current method flows can be regarded as the direct improvement of the hardware circuit structure. Designers almost always get the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be realized by hardware physical modules. For example, a Programmable Logic Device (Programmable Logic Device, PLD) (such as a Field Programmable Gate Array (Field Programmable Gate Array, FPGA)) is such an integrated circuit, and its logic function is determined by programming the device by a user. It is programmed by the designer to "integrate" a digital system on a PLD, instead of asking a chip manufacturer to design and make a dedicated integrated circuit chip. Moreover, nowadays, instead of making integrated circuit chips by hand, this kind of programming is mostly realized by "logic compiler (logic compiler)" software, which is similar to the software compiler used when writing programs. The original code of the computer must also be written in a specific programming language, which is called a hardware description language (Hardware Description Language, HDL), and there is not only one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language) , AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., currently the most commonly used is VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. It should also be clear to those skilled in the art that only a little logical programming of the method flow in the above-mentioned hardware description languages and programming into an integrated circuit can easily obtain a hardware circuit for realizing the logic method flow.

The controller may be implemented in any suitable way, for example the controller may take the form of a microprocessor or processor and a computer readable medium storing computer readable program code (such as software or firmware) executable by the (micro)processor , logic gates, switches, Application Specific Integrated Circuit (ASIC), programmable logic controllers, and embedded microcontrollers, examples of controllers include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, the memory controller can also be implemented as part of the memory's control logic. Those skilled in the art also know that, in addition to realizing the controller in a purely computer-readable program code mode, it is entirely possible to make the controller use logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded The same function can be realized in the form of a microcontroller or the like. Therefore, such a controller can be regarded as a hardware component, and the devices included in it for realizing various functions can also be regarded as structures within the hardware component. Or even, means for realizing various functions can be regarded as a structure within both a software module realizing a method and a hardware component.

The systems, devices, modules, or units described in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementing device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or Combinations of any of these devices.

For the convenience of description, when describing the above devices, functions are divided into various units and described separately. Of course, when implementing this specification, the functions of each unit can be implemented in one or more pieces of software and/or hardware.

Those skilled in the art should understand that the embodiments of this specification may be provided as methods, systems, or computer program products. Accordingly, this description may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present description may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The specification is described with reference to flowcharts and/or block diagrams of methods, devices (systems), and computer program products according to embodiments of the specification. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a An apparatus for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions The device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby The instructions provide steps for implementing the functions specified in the flow chart or blocks of the flowchart and/or the block or blocks of the block diagrams.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

Memory may include non-permanent storage in computer readable media, in the form of random access memory (RAM) and/or nonvolatile memory such as read only memory (ROM) or flash RAM. Memory is an example of computer readable media.

Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can be implemented by any method or technology for storage of information. Information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Flash memory or other memory technology, Compact Disc Read-Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cartridge, tape magnetic disk storage or other magnetic storage device or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media excludes transitory computer-readable media, such as modulated data signals and carrier waves.

It should also be noted that the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus comprising a set of elements includes not only those elements, but also includes Other elements not expressly listed, or elements inherent in the process, method, commodity, or apparatus are also included. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element.

Those skilled in the art should understand that the embodiments of this specification may be provided as methods, systems or computer program products. Accordingly, this description may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present description may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The specification may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The present description may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including storage devices.

Each embodiment in this specification is described in a progressive manner, the same and similar parts of each embodiment can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for relevant parts, refer to part of the description of the method embodiment.

The above descriptions are only examples of this specification, and are not intended to limit this specification. For those skilled in the art, various modifications and changes may occur in this description. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this specification shall be included within the scope of the claims of this specification.

Claims (15)

1. A method for business execution, comprising: Obtain the knowledge graph to be expanded corresponding to the target business; Input the entity name corresponding to the target node in the knowledge map to be expanded into the search engine to obtain text data and image data corresponding to the target node; Input the text data corresponding to the target node into the pre-trained first recognition model, so as to recognize the entity relationship between the entities involved in the text data through the first recognition model, and according to the Entity relationships between entities involved in the text data, determining the triplet information of the text data, and inputting the image data corresponding to the target node into the pre-trained second recognition model, so as to pass the first 2. a recognition model, identifying the entity relationship between the entities involved in the image data, and determining the triplet information of the image data according to the entity relationship between the entities involved in the image data; Based on the triplet information of the text data and the triplet information of the image data, expand the entity relationship between the target node and other nodes in the knowledge graph to be expanded to obtain the target knowledge graph, and pass The target knowledge graph executes the target business. 2. The method according to claim 1, wherein determining the target node specifically comprises: Obtain the business requirements corresponding to the target business; According to the business requirements corresponding to the target business, determine the correlation between the entity names corresponding to the nodes in the knowledge map to be expanded and the business requirements corresponding to the target business, as the correlation of each node; The nodes whose correlation degree is greater than the set correlation threshold are taken as the target nodes. 3. The method according to claim 1, wherein determining the target node specifically comprises: Determine the number of triples corresponding to each node in the knowledge map to be expanded; The node whose number of triples is less than the set number threshold is used as the target node. 4. The method according to claim 1, wherein the first recognition model comprises: a text recognition network and a text relational network; Input the text data corresponding to the target node into the pre-trained first recognition model, so as to recognize the entity relationship between the entities involved in the text data through the first recognition model, and according to the Entity relationship between entities involved in the text data, determine triplet information of the text data, specifically including: Input the text data corresponding to the target node into the text recognition network, and determine the entities involved in the text data; inputting the entities involved in the text data into the text relationship network, and determining the entity relationship between the entities involved in the text data; The triplet information of the text data is determined according to the entities involved in the text data and the entity relationships among the entities involved in the text data. 5. The method according to claim 4, wherein, before inputting the text data corresponding to the target node into the text recognition network and determining each entity involved in the text data, the method further includes include: performing data cleaning on the text data corresponding to the target node to obtain the cleaned text data; Input the text data corresponding to the target node into the text recognition network, and determine each entity involved in the text data, specifically including: The cleaned text data is input into the text recognition network, and entities involved in the cleaned text data are determined. 6. The method according to claim 1, wherein the second recognition model comprises: an image recognition network and an image relationship network; input the image data corresponding to the target node into the pre-trained second recognition model, so as to recognize the entity relationship between the entities involved in the image data through the second recognition model, and according to the Entity relationship between entities involved in the image data, determine the triplet information of the image data, specifically include: input the image data corresponding to the target node into the image recognition network, and determine the entities involved in the image data; Input the entities involved in the image data into the image relationship layer, and determine the entity relationship between the entities involved in the image data; The triplet information of the image data is determined according to the entities involved in the image data and the entity relationships among the entities involved in the image data. 7. The method according to claim 6, wherein, before inputting the image data corresponding to the target node into the image recognition network and determining each entity involved in the image data, the method further includes include: Adjusting the image size and image resolution of the image data corresponding to the target node to obtain adjusted image data; Input the image data corresponding to the target node into the image recognition network, and determine each entity involved in the image data, specifically including: The adjusted image data is input into the image recognition network, and entities involved in the adjusted image data are determined. 8. The method according to claim 1, wherein, based on the triplet information of the text data and the triplet information of the image data, the target node and other The entity relationship between nodes to obtain the target knowledge graph, including: Obtain various open source knowledge graphs; Query the target node from the open source knowledge graphs, determine other nodes adjacent to the target node as adjacent nodes of the target node, and according to the entity relationship between the target node and the adjacent nodes of the target node , determining the triplet information corresponding to each of the open source knowledge graphs; Based on the triplet information corresponding to each open source knowledge graph, the triplet information of the text data, and the triplet information of the image data, expand the knowledge graph to be expanded between the target node and other nodes The entity relationship among them to get the target knowledge graph. 9. The method according to claim 8, wherein the open source knowledge graph comprises: an open domain knowledge graph and a vertical domain knowledge graph corresponding to the target business. 10. The method according to claim 8, wherein the target node is queried from the open source knowledge graphs, other nodes adjacent to the target node are determined as adjacent nodes of the target node, and according to the target The entity relationship between the node and the adjacent nodes of the target node determines the triple information corresponding to each open source knowledge graph, specifically including: Query the entity name corresponding to the target node from each of the open source knowledge graphs, and determine the open source knowledge graph containing the entity name corresponding to the target node as a candidate knowledge graph; According to the similarity between the adjacent nodes of the target node in each candidate knowledge graph and the adjacent nodes of the target node in the knowledge graph to be expanded, determine the candidate knowledge graph corresponding to the target node; The entity relationship between the target node and the adjacent nodes of the target node in the candidate knowledge graph corresponding to the target node is converted to obtain triple information of the candidate knowledge graph. 11. The method according to claim 1, wherein training the first recognition model specifically comprises: Obtain text training samples; Input the text training sample into the first recognition model to be trained, and determine the triplet information corresponding to the text training sample; The first recognition model is trained with minimizing the deviation between the triplet information corresponding to the text training sample and the label corresponding to the text training sample as an optimization goal. 12. The method according to claim 1, wherein training the second recognition model specifically comprises: Obtain image training samples; Input the image training sample into the second recognition model to be trained, and determine the triplet information corresponding to the image training sample; The second identification model is trained with the optimization goal of minimizing the deviation between the triplet information corresponding to the image training samples and the labels corresponding to the image training samples. 13. A device for service execution, comprising: The acquisition module is used to acquire the knowledge map to be expanded corresponding to the target business; The input module is used to input the entity name corresponding to the target node in the knowledge map to be expanded into the search engine, and obtain the text data and image data corresponding to the target node; A recognition module, configured to input the text data corresponding to the target node into the pre-trained first recognition model, so as to recognize the entity relationship between the entities involved in the text data through the first recognition model , and according to the entity relationship between the entities involved in the text data, determine the triplet information of the text data, and input the image data corresponding to the target node into the pre-trained second recognition model, Through the second recognition model, the entity relationship among the entities involved in the image data is identified, and according to the entity relationship between the entities involved in the image data, three elements of the image data are determined tuple information; An execution module, configured to expand the entity relationship between the target node and other nodes in the knowledge map to be expanded based on the triple information of the text data and the triple information of the image data, so as to obtain the target knowledge graph, and execute the target business through the target knowledge graph. 14. A computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1-12 is implemented. 15. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and operable on the processor, characterized in that, when the processor executes the program, any one of claims 1 to 12 above is realized method described in the item.