CN118779468A - A method for entity relationship extraction based on large model and dynamic prompts - Google Patents

Abstract

The present invention discloses an entity relationship extraction method based on a large model and dynamic prompts, the method comprising: defining a pattern layer and an entity set of a domain knowledge graph; constructing an entity relationship triplet example set using the pattern layer and the entity set; constructing an entity vector database DB; constructing a prompt template set PR; dividing the text to be processed into paragraphs, and extracting a keyword list of the paragraphs; constructing a paragraph-entity association list T for each paragraph using the keyword list; for each paragraph, constructing a dynamic prompt using the association list T, the prompt template set PR and the entity relationship triplet example set, sending the dynamic prompt to the large model for entity relationship extraction, and verifying the correctness of the result. The present invention integrates the advantages of knowledge graphs, entity vector representation, dynamic prompts and large models, does not need to fine-tune the large model, realizes automatic extraction of entity relationships, reduces the cost of entity relationship extraction, improves efficiency and accuracy, and has a wide range of application and promotion value.

Description

A method for entity relationship extraction based on large model and dynamic prompts

Technical Field

The present invention relates to entity relationship extraction, and in particular to an entity relationship extraction method based on a large model and dynamic prompts, and belongs to the field of artificial intelligence applications.

Background Art

As large language models have shown excellent expressiveness in various tasks, people are actively exploring their wide application possibilities in different fields. Existing large models are trained using knowledge corpora in general fields, and can learn language patterns from a wide range of texts and extract entity relationships from unstructured texts to build general domain knowledge graphs. Due to the lack of training in knowledge corpora in specific fields, and the complexity of terminology, concepts, and entity relationships in specific fields, existing general large models cannot achieve the expected results when used to extract entity relationships in specific fields.

In order to improve the accuracy of entity relationship extraction by large language models, researchers have proposed two types of methods: one is to use specific domain knowledge to build a corpus for fine-tuning large models; the other is to provide large models with prompts to assist in understanding text through prompt engineering technology. Huang Jing et al. constructed dynamic prompts for joint extraction of API entity-relationships in the patent "API entity-relationship joint extraction method and system based on dynamic prompts" and constructed a structured extraction language. Based on the structured extraction language, training sets and test sets were constructed to fine-tune the large model for joint extraction of API entity relationships. This patent targets the API field of software development and improves the accuracy of large models in extracting entity relationships in API description texts through dynamic prompts; Yang Li et al. encoded sentence features in the patent "Entity relationship extraction method, device, medium and equipment based on prompt learning" and optimized based on prompt learning, decomposing the entity relationship extraction task into two stages: entity recognition and relationship classification. Combined with question-answering tasks, the semantic information of the text to be extracted is used to extract triples from a given small amount of annotated text. This patent divides entity relationship extraction into two independent stages, and cannot perform joint extraction. Huang Yihua et al. proposed a small sample nested relationship extraction algorithm framework based on dynamic prompt learning in the patent "A small sample nested relationship extraction algorithm based on dynamic prompt learning", which improves the relationship recognition accuracy of the nested relationship extraction algorithm without involving the joint extraction of entity relationships.

Summary of the invention

The present invention discloses an entity relationship extraction method based on a large model and dynamic prompts, optimizes a dynamic prompt construction method for specific domain knowledge, and uses a large language model to extract entity relationships in unstructured text (hereinafter referred to as text); the method comprises: 1) defining a pattern layer and an entity set of a domain knowledge graph; 2) constructing an entity relationship triple example set using the pattern layer and the entity set; 3) constructing an entity vector database DB; 4) constructing a prompt template set PR; 5) dividing the text to be processed into paragraphs, and extracting a keyword list of the paragraphs; 6) constructing a paragraph-entity association list T for each paragraph using the keyword list; 7) for each paragraph, constructing a dynamic prompt using the association list T, the prompt template set PR and the entity relationship triple example set; 8) for each paragraph, sending the dynamic prompt to the large model for entity relationship extraction, and checking the correctness of the result; specifically, the method of the present invention comprises the following steps:

A. Define the model layer and entity set of the domain knowledge graph. The specific steps are as follows:

A1. Define the set of entity categories C={C ₁ ,…C _i ,…C _N }(i∈[1,N]), the set of relationships between entity categories R={R ₁ ,…R _j ,…R _M }(j∈[1,M]), where N and M represent the number of entity categories and the number of relationships in the knowledge graph to be built;

A2. Extract words belonging to entity category C _i (i∈[1,N]) from the industry terminology dictionary or catalog as entities to form the entity set E _i ={e _i1 ,…e _ij ,…,e _ik } of this category (i∈[1,N], j∈[1,k], k represents the number of entities in the entity set, which varies depending on the entity category C _i );

A3. The entity category _Ch (h∈[1,N]) and entity category C _t (t∈[1,N]) with the relation R _j (j∈[1,M]) are recorded as entity category triples ( _Ch , R _j , C _t ), where _Ch is called the head entity category and its entity set is recorded as E _hj , C _t is called the tail entity category and its entity set is recorded as E _tj ; all entity category triplets constitute the pattern layer of the knowledge graph;

B. Use the schema layer and entity set to construct an example set of entity relationship triples. Take the entity category triple (C _h , R _j , C _t ) as an example. The specific operations are as follows:

B1. Select F entity pairs from entity set E _h and entity set E _t under the condition that they satisfy the R _j relationship;

B2. Combine each entity pair and relation R _j into an entity-relation triple (e _hi ,R _j ,e _ti ), where entity e _hi ∈E _h (i∈[1,F]), e _ti ∈E _t (i∈[1,F]);

B3. Using the F entity-relationship triples constructed in step B2, construct an example set of entity-relationship triples S _j ={(e _hi ,R _j ,e _ti ) for i in [1:F]} of relation R _j , where for i in [1:F] indicates that the value of i is an integer from 1 to F;

C. Construct entity vector database DB. The specific steps are as follows:

C1. For each entity e _ij ∈E _i (i∈[1,N],j∈[1,|Ei|]) in the entity set E _i (i∈[1,N]), encode it into an entity vector ve _ij using the pre-trained language model;

C2. storing each pair of entity e _ij and entity vector ve _ij into the entity vector database DB;

D. Build the prompt template set PR. The specific steps are as follows:

D1. Define the template set PR = {pr ₁ ,…,pr _k ,…,pr _n } (k∈[1,n]), where n represents the number of templates contained in the template set PR;

D2. Construct each prompt template pr _k ∈PR, which contains several template variables. The specific steps are as follows:

D2.1 defines a domain variable as ${field}, where the value range of field is the set of domain identifiers;

D2.2 defines the paragraph variable as ${P _i }, where the value range of P _i is the set of paragraph tags;

D2.3 Define entity variables as ${e ₁ }, …, ${e _j }, …, ${e _m }, where e ₁ ~e _m are the corresponding entities in the paragraph;

D2.4 Define the relationship variables as ${R ₁ }, …, ${R _j }, …, ${R _M }, where R ₁ ~R _M are all the relationships in the knowledge graph to be built;

D2.5 defines the entity-relationship triple example set variables as ${S ₁ }, …, ${S _j }, …, ${S _M }, where S ₁ ~S _m is the entity-relationship triple example set corresponding to relations R ₁ ~R _M ;

E. Divide the text to be processed into paragraphs and extract the keyword list of the paragraphs. The specific steps are as follows:

E1. Divide the unstructured text to be processed into paragraphs. The specific steps are as follows:

E1.1 Clean the text data and remove useless punctuation marks, special characters, etc.; E1.2 Use the pre-trained word vector model to represent the text with word vectors and generate word vector sequences; E1.3 Input the word vector sequence into the neural network sequence labeling model to detect the topic boundaries of the text and obtain the start and end positions of each paragraph;

E1.4 Divide the text into different paragraphs according to the detected topic boundaries, and obtain the paragraph set P = {P ₀ ,…P _i ,…,P _t } (i∈[1,t]), where t is the total number of paragraphs in the set;

E2. Extract the keyword list in each paragraph _Pi (i∈[1,t]). The specific steps are as follows:

E2.1 Extract keywords from each paragraph P _i (i∈[1,t]) and add them to the set K _i ={k _i1 ,k _i2 ,...,k _ip } (p is the total number of keywords in P _i ). Keyword extraction methods include but are not limited to TF-IDF, TextRank, and methods based on pre-trained language models;

E2.2 Combine paragraph _Pi and keyword set _Ki into a keyword list PL _i = [P _i , _Ki ] of paragraph _Pi ;

F. Use the keyword list to construct a paragraph-entity association list T for each paragraph. Taking paragraph _Pi as an example, the specific steps are as follows:

F1. Encode each keyword k _ij ∈K _i (j∈[1,p]) in K _i into a keyword vector vk _ij using the pre-trained language model in step C1;

F2. Perform similarity search on the keyword vector vk _ij in the entity vector library to obtain the Top-N similar entities, denoted as Top-N[k _ij ]={e ₁ ,e ₂ ,…,e _N };

F3. Remove duplicates from all similar entities corresponding to all keywords in K _i to form an entity list L _i ;

F4. Combine paragraph _Pi and entity list _Li into _Pi 's paragraph-entity association list _Ti = [ _Pi , _Li ];

G. For each paragraph, construct a dynamic prompt using the association list T, prompt template set PR and entity relationship triplet example set. Taking the association list _Ti , template pr _k and entity relationship triplet example set of paragraph _Pi as an example, the specific steps are as follows:

G1. Replace the template variables in the prompt template pr _k with the corresponding variable values. The specific steps are as follows:

G1.1 Substitute the paragraph _Pi in the association list _Ti into the paragraph variable ${ _Pi };

G1.2 Substitute each entity of the entity list _Li in the association list _Ti into the entity variables ${e ₁ }~${e _m };

G1.3 Substitute the relations R ₁ ~R _j of the entities in the entity list _Li into the relation variables ${R ₁ }~${R _j };

G1.4 Substitute the entity relationship triple example set in the entity triple example set S ₁ ~S _j corresponding to the relation R ₁ ~R _j into the entity relationship triple example set variable ${S ₁ } ~${S _j };

G2. For the remaining variables in the template, enter the corresponding values according to the definition of the template variables to form a complete dynamic prompt content Q _k P _i (k∈[1,n],i∈[1,t]). For example, enter field (field name) and substitute it into the field variable ${field};

H. Send the dynamic prompt content to the big model for entity relationship extraction and verify the correctness of the results. The specific steps are as follows:

H1. Communicate with the big model by calling the API, and send the dynamic prompt content to multiple big models for entity relationship extraction;

H2. Compare and analyze the entity relationship extraction results of multiple large models and select the optimal extraction result.

Compared with the prior art, the present invention has the following advantages: a complete set of entity relationship extraction methods for unstructured text in specific fields is implemented, starting from the pattern layer and entity set of the domain knowledge graph, the various elements required for prompt are dynamically constructed, and dynamic prompts are generated to guide the large model to extract entity relationships in specific fields; the entity vector database is constructed using a pre-trained language model, which improves the accuracy of entity recognition and relationship extraction in paragraph text. The present invention combines the advantages of knowledge graph, entity vector representation, dynamic prompt and large model, without fine-tuning the large model, to achieve automatic extraction of entity relationships, reduce the cost of entity relationship extraction, improve efficiency and accuracy, and has a wide range of application and promotion value.

BRIEF DESCRIPTION OF THE DRAWINGS

Figure 1: Flowchart of an entity relationship extraction method based on a large model and dynamic prompts

DETAILED DESCRIPTION

The present invention is described in detail below with reference to the accompanying drawings and specific implementation examples.

The present invention discloses an entity relationship extraction method based on a large model and dynamic prompts, the method comprising: 1) defining a pattern layer and an entity set of a domain knowledge graph; 2) constructing an entity relationship triple example set using the pattern layer and the entity set; 3) constructing an entity vector database DB; 4) constructing a prompt template set PR; 5) dividing the text to be processed into paragraphs, and extracting a keyword list of the paragraphs; 6) constructing a paragraph-entity association list T for each paragraph using the keyword list; 7) for each paragraph, constructing a dynamic prompt using the association list T, the prompt template set PR and the entity relationship triple example set; 8) for each paragraph, sending the dynamic prompt to the large model for entity relationship extraction, and checking the correctness of the result. The following steps are taken as an example of a part of the knowledge graph of the Python programming language course, and the extraction of some entities of one entity class and their related partial relations as an example, to clearly and completely describe the technical solution in the present invention.

1. Define the model layer and entity set of the domain knowledge graph. The specific steps are as follows:

1.1 Part of the requirements for the Python programming language course knowledge graph is to extract various knowledge points in the textbook and extract the inclusion and dependency relationships between different knowledge points;

According to this requirement, we define the entity category {knowledge point}, denoted as C ₁ , and the entity category set as C={knowledge point}; the relationship set between knowledge point entities is R={inclusion relationship, dependency relationship}, the inclusion relationship is denoted as R ₁ , and the dependency relationship is denoted as R ₂ . The number of entity categories in the knowledge graph to be built is M=1, and the number of relationship categories is N=2.

1.2 According to the textbook catalog, extract the entity set E ₁ of C ₁ = {basic data type, control structure, digital type, branch structure, integer type, single branch structure, binary, decimal, octal, hexadecimal, Boolean type};

1.3 The inclusion relationship R ₁ and dependency relationship R ₂ between the knowledge point C ₁ categories, the corresponding entity category triples and the corresponding entity sets are specifically recorded as:

1.3.1 The entity category triple corresponding to the inclusion relation _R1 is recorded as (knowledge point, inclusion relation, knowledge point), and the head entity set is recorded as _Eh1 = {basic data type, control structure, digital type, branch structure, integer type}, _Et1 = {digital type, branch structure, integer type, single branch structure, binary, decimal, octal, hexadecimal};

1.3.2 The entity category triple corresponding to the dependency relationship _R2 is recorded as (knowledge point, dependency relationship, knowledge point), and the head entity set is recorded as _Eh2 = {control structure}, _Et2 = {basic data type, Boolean type};

2. Use the pattern layer and entity set to construct an example set of entity relationship triples. Take the entity category triples (knowledge point, inclusion relationship, knowledge point) and (knowledge point, dependency relationship, knowledge point) as examples. The specific operations are as follows:

2.1 Select two entity pairs from entity set E _h1 and entity set E _t1 under the condition of satisfying the inclusion relationship R ₁ ; select two entity pairs from entity set E _h2 and entity set E _t2 under the condition of satisfying the dependency relationship R ₂ ;

2.2 The entity relationship triple instance set S ₁ of the inclusion relation R ₁ ={(control structure, inclusion relation, branch structure), (branch structure, inclusion relation, two-branch structure)}; the entity relationship triple instance set S ₂ of the dependency relation R ₂ ={(control structure, dependency relation, basic numeric type), (control structure, dependency relation, Boolean type)};

3. Build the entity vector database DB. The specific steps are as follows:

3.1 For each entity e _1j ∈E ₁ (j∈[1,11]) in the entity set E ₁ , use BERT to encode it into an entity vector ve _1j (j∈[1,11]);

3.2 Store each pair of entity e _1j and entity vector ve _1j into the entity vector database DB;

4. Build the prompt template set PR. The specific steps are as follows:

4.1 Define the template set PR = {pr ₁ ,pr ₂ ,pr ₃ };

4.2 Construct each prompt template pr _k ∈ PR, i ∈ [1, 3], the template contains several template variables, the specific steps are as follows:

4.2.1 Define the template content of pr ₁ : "You are a senior expert in a specific field ${field}. Given a paragraph ${P _i } and the entities in the paragraph: ${e ₁ }, ..., entity ${e _j }, ..., entity ${e _m }, please extract the entity-relationship triples in this paragraph; the format of the triples is (head entity, relationship, tail entity); only the following relationships are allowed between entities: ${R ₁ }, ${R ₂ }; the R ₁ relationship is ${explainR ₁ }, and the R ₂ relationship is ${explainR ₂ }"; the template contains variables: field (field name), _Pi (paragraph), e ₁ ~e _m (the corresponding entities in paragraph _Pi ), R ₁ ~R ₂ (the relationships in the knowledge graph to be built) and explainR ₁ ~explainR ₂ (according to the example set of entity-relationship triples corresponding to the relationships R ₁ ~R ₂ , construct an example description of the relationships R ₁ ~R ₂ );

4.2.2 Define the template content of pr ₂ : "Please extract entity-relationship triples from paragraph ${Pi} from the perspective of professionals in the ${field} field, with the following requirements: 1. The entities of the triples must be the following: ${e ₁ }, ..., entity ${e _j }, ..., entity ${e _m }; 2. The relations of the triples must be the following: ${R ₁ }, ${R ₂ }; 3. Refer to the following examples: example ${S ₁ } of relation ${R ₁ }, example ${S _{2 } of relation ${R 2 }} "; the template contains variables: field (field name), _Pi (paragraph), e ₁ ~e _m (entities corresponding to paragraph _Pi ), R ₁ ~R ₂ (relations in the knowledge graph to be built) and S ₁ ~S ₂ (example set S ₁ ~S ₂ of entity-relationship triples corresponding to relations R ₁ ~R ₂ );

4.2.3 Define the template content of pr ₃ : Please extract entity-relationship triples for the specified paragraph text based on the following information: paragraph: ${Pi}; entity: ${e ₁ }, ..., entity ${e _j }, ..., entity ${e _m }; relationship: ${R ₁ }, ${R ₂ }; triple instance: ${S ₁ }, ${S ₂ };"; The template contains variables: field (field name), _Pi (paragraph), e ₁ ~e _m (the corresponding entity in paragraph _Pi ), R ₁ ~R ₂ (the relationship in the knowledge graph to be built) and S ₁ ~S ₂ (the entity-relationship triple example set S ₁ ~S ₂ corresponding to the relationship R ₁ ~R ₃ );

5. Divide the text to be processed into paragraphs and extract the keyword list of the paragraphs. The specific steps are as follows:

5.1 Divide the unstructured text to be processed into paragraphs. The specific steps are as follows:

5.1.1 Clean the text data and remove useless punctuation marks, special characters, etc.; 5.1.2 Use BERT to represent the text as word vectors and generate a word vector sequence; 5.1.3 Input the word vector sequence into the BiLSTM-CRF model, perform topic boundary detection on the text, and obtain the start and end positions of each paragraph; 5.1.4 Divide the text into different paragraphs based on the detected topic boundaries, and obtain the paragraph set P={P ₀ ,…P _i ,…,P ₁₀ }(i∈[1,10]);

5.2 Extract keywords from each paragraph _Pi (i∈[1,10]). Taking paragraph _P4 as an example, the specific steps are as follows:

5.2.1 Use the Tongyi Qianwen model to extract the keywords of paragraph P ₄ : "Integer type is consistent with the concept of integer in mathematics. The following are examples of integer types: 1010, 99, -217, 0x9a, -0x89. Integer type has four bases: decimal, binary, octal and hexadecimal. By default, integers use decimal. Other bases require a leading symbol. Binary numbers are led by 0b, octal numbers are led by 0o, and hexadecimal numbers are led by 0x. Both uppercase and lowercase letters can be used." and add the keywords to the set K ₄ ={integer type, integer, integer concept, base representation, decimal, binary, octal, hexadecimal, leading symbol};

5.2.2 Combine paragraph P ₄ and keyword set K ₄ into keyword list PL ₄ =[P ₄ , K ₄ ] of paragraph P ₄ ;

6. Use the keyword list to construct a paragraph-entity association list T for each paragraph. Taking paragraph P ₄ as an example, the specific steps are as follows:

6.1 Encode each keyword k _4j ∈K ₄ (j∈[1,8]) in K ₄ into a keyword vector vk _4j using BERT;

6.2 Perform similarity search on the keyword vector vk _4j in the entity vector library to obtain the top-1 similar entities. Taking the keyword "integer type" as an example, the top-1 similar entity is also "integer type", recorded as Top-1[k _4j ]={integer type};

6.3 Remove duplicates from all similar entities corresponding to all keywords in K ₄ to form an entity list L ₄ = [integer type, decimal, binary, octal, hexadecimal];

6.4 Combine paragraph P ₄ and entity list L ₄ into P ₄ 's paragraph-entity association list T ₄ = [P ₄ , L ₄ ];

7. For each paragraph, construct a dynamic prompt using the association list T, the prompt template set PR, and the entity relationship triplet example set. Take the association list T ₄ , template pr ₁ , and the entity relationship triplet example set of paragraph P ₄ as an example. The specific steps are as follows:

7.1 Replace the template variables in the prompt template pr ₁ with the corresponding variable values. The specific steps are as follows:

7.1.1 Substitute paragraph P ₄ in association list T ₄ into paragraph variable ${P _i };

7.1.2 Substitute each entity of the entity list L ₄ in the association list T ₄ into the entity variables ${e ₁ }~${e ₅ };

7.1.3 Substitute the relations R ₁ ~R ₂ of the entities in the entity list L ₄ into the relation variables ${R ₁ } ~${R ₂ } respectively;

7.1.4 Substitute the entity triples in the entity triple example set S ₁ ~S ₂ corresponding to the relations R ₁ ~R ₂ into the entity relation triple example set variables ${S ₁ } ~${S ₂ };

7.2 For the remaining variables in the template, enter the values corresponding to the template variables as needed to form a complete dynamic prompt content Q _k P ₄ (k∈[1,3]). Taking pr ₁ as an example, enter "Python programming language" and substitute it into the field variable ${field}; enter "refers to the concept that two knowledge points belong to the same type and the scope referred to by the former includes the latter, such as the relationship between the control structure and the branch structure", and substitute it into the template variable ${explainR ₁ }; enter "refers to the fact that when learning or applying a certain knowledge point, one must first master or understand another or more knowledge points, such as before learning the control structure, one must first master the Boolean type knowledge point", and substitute it into the template variable ${explainR ₂ };

Therefore, the dynamic prompt content Q ₁ P ₄ is: You are a senior expert in the Python programming language in a specific field. Given a paragraph "The integer type is consistent with the concept of integers in mathematics. The following are examples of integer types: 1010, 99, -217, 0x9a, -0x89. Integer types have 4 bases: decimal, binary, octal, and hexadecimal. By default, integers use decimal, and other bases require a leading symbol. Binary numbers are led by 0b, octal numbers are led by 0o, and hexadecimal numbers are led by 0x. Both uppercase and lowercase letters can be used.", and given the entity in the paragraph: integer Number, decimal, binary, octal, hexadecimal, please extract the entity relationship triples in this paragraph; the format of the triple is (head entity, relationship, tail entity); only the following relationships are allowed between entities: inclusion relationship, dependency relationship; inclusion relationship means that two knowledge points belong to the same type of concept and the scope of the former includes the latter, such as the relationship between control structure and branch structure; dependency relationship means that when learning or applying a certain knowledge point, you must first master or understand another one or more knowledge points, such as before learning control structure, you must first master the knowledge point of Boolean type;

Similarly, we can get the dynamic prompt content Q ₂ P ₄ : Please use the perspective of professionals in the field of Python programming language to extract entity relationship triples from the paragraph "Integer types are consistent with the concept of integers in mathematics. The following are examples of integer types: 1010, 99, -217, 0x9a, -0x89. Integer types are represented in 4 bases: decimal, binary, octal, and hexadecimal. By default, integers are in decimal, and other bases require a leading symbol. Binary numbers are led by 0b, octal numbers are led by 0o, and hexadecimal numbers are led by 0x. Both uppercase and lowercase letters can be used." The requirements are as follows: 1. The entities of the triples must be the following: integer, decimal, binary, octal, hexadecimal, 2. The relationships of the triples must be the following: inclusion relationship, dependency relationship, 3. Refer to the following examples: Examples of inclusion relationships (control structure, inclusion relationship, branch structure), (branch structure, inclusion relationship, binary branch structure), examples of dependency relationships (control structure, dependency relationship, basic data type), (control structure, dependency relationship, Boolean type);

The dynamic prompt content Q ₃ P ₄ is: Please extract entity relationship triples for the specified paragraph text based on the following information: Paragraph: "The integer type is consistent with the concept of integers in mathematics. The following are examples of integer types: 1010, 99, -217, 0x9a, -0x89. There are 4 bases for integer types: decimal, binary, octal and hexadecimal. By default, integers use decimal, and other bases need to add leading symbols. Binary numbers are led by 0b, octal numbers are led by 0o, and hexadecimal numbers are led by 0x. Both uppercase and lowercase letters can be used."; Entities: integers, decimal, binary, octal, hexadecimal; Relationships: inclusion relations, dependency relations; Triple instances: (control structure, inclusion relations, branch structure), (branch structure, inclusion relations, two-branch structure), (control structure, dependency relations, basic data types), (control structure, dependency relations, Boolean type);

8. Send the dynamic prompt content to the big model for entity relationship extraction. The specific steps are as follows:

8.1 By calling the API to communicate with the big model, the dynamic prompt content generated in step 7 is sent to ChatGPT-4 and Claude-3-opus for entity relationship extraction. Taking the dynamic prompt content Q ₁ P ₄ , Q ₂ P ₄ and Q ₃ P ₄ as examples, the entity relationship extraction results of ChatGPT-4 and Claude-3-opus are shown in Table 1;

Table 1 Entity relationship extraction results

ChatGPT-4 Claude-3-opus Q ₁ P ₄ Based on your requirements, the following are the entity relationship triplets extracted from the paragraph: Inclusion relationship: (integer, inclusion relationship, decimal) (integer, inclusion relationship, binary) (integer, inclusion relationship, octal) (integer, inclusion relationship, hexadecimal) Dependency relationship: (binary, dependency relationship, leading symbol 0b) (octal, dependency relationship, leading symbol 0o) (hexadecimal, dependency relationship, leading symbol 0x) These triplets accurately describe the relationship between the entities involved in the paragraph and meet your requirements. Based on the given paragraphs and entities, I extracted the following entity relationship triples: (integer type, including, decimal) (integer type, including, binary) (integer type, including, octal) (integer type, including, hexadecimal) (binary, dependent, 0b leading) (octal, dependent, 0o leading) (hexadecimal, dependent, 0x leading) These triples indicate that the integer type contains four base representations: decimal, binary, octal, and hexadecimal. Binary numbers depend on the 0b leading symbol. Octal numbers depend on the 0o leading symbol. Hexadecimal numbers depend on the 0x leading symbol. Q ₂ P ₄ Based on the provided paragraph and given requirements, we can extract the following entity-relationship triples from the text: (integer, inclusion, decimal)(integer, inclusion, binary)(integer, inclusion, octal)(integer, inclusion, hexadecimal)These four triples indicate that integers contain different base representations. Each base is a representation of an integer. (binary, dependency, 0b leader)(octal, dependency, 0o leader)(hexadecimal, dependency, 0x leader)These three triples indicate that different bases depend on specific leaders to represent. In Python, 0b, 0o, and 0x are the leader symbols used to represent binary, octal, and hexadecimal. Note that the specific leader of the decimal system is not directly mentioned in the text because, by default, numbers without any prefix are interpreted as decimal, so no triples that depend on specific leaders are created. According to the given paragraphs and requirements, I extracted the following entity-relationship triples: 1. (integer, inclusion, decimal) 2. (integer, inclusion, binary) 3. (integer, inclusion, octal) 4. (integer, inclusion, hexadecimal) 5. (binary, dependency, 0b leading) 6. (octal, dependency, 0o leading) 7. (hexadecimal, dependency, 0x leading) The above triples meet the given entity and relationship constraints. The integer type includes four representations: decimal, binary, octal and hexadecimal. Binary, octal and hexadecimal rely on the leading symbols 0b, 0o and 0x respectively. Q ₃ P ₄ Based on the information you provided, we can extract entity-relationship triples for the specified paragraph. In this paragraph, the key entities are "integer", "decimal", "binary", "octal", and "hexadecimal", while the relationship types include "inclusion relationship", "identity relationship", and "dependency relationship". Here are some examples of how to build these entity and relationship triples:1. Inclusion relationship (integer, inclusion relationship, decimal)(integer, inclusion relationship, binary)(integer, inclusion relationship, octal(integer, inclusion relationship, hexadecimal)2. Dependency relationship (binary, dependency relationship, 0b leader)(octal, dependency relationship, 0o leader)(hexadecimal, dependency relationship, 0x leader)Indicates that non-decimal representations require specific leader symbols, such as 0b, 0o, and 0x. These triples can help describe and understand the structure and characteristics of integer types in different base representations. Each triple accurately reflects the information mentioned in the paragraph, such as different representation methods of integer types and their dependencies. OK, given the paragraph and entity, I try to extract the following triples: (integer, inclusion, decimal) (integer, inclusion, binary) (integer, inclusion, octal) (integer, inclusion, hexadecimal) (binary, dependency, 0b) (octal, dependency, 0o) (hexadecimal, dependency, 0x) These triplets cover the inclusion relationship between the integer types mentioned in the paragraph and different base representations. The integer type contains four base representations, and these four bases have their own corresponding representation methods, and the binary, octal and hexadecimal representations all rely on specific leading symbols.

8.2 Compare and analyze the entity relationship extraction results of the two large models and select the optimal extraction result. The specific operations are as follows:

For inclusion relationships, the entity relationship extraction results of the two large models under three dynamic prompts are basically the same; for dependency relationships, the tail entities extracted by the two large models under three dynamic prompts are slightly different in expression, and it is necessary to test the entity relationship triplets of the dependency relationship and select the optimal extraction result.

The present invention realizes a complete entity relationship extraction method for unstructured text in a specific field. Starting from the pattern layer and entity set of the domain knowledge graph, the various elements required for prompt are dynamically constructed, and dynamic prompts are generated to guide the large model to extract entity relationships in a specific field; the entity vector database is constructed using a pre-trained language model to improve the accuracy of entity recognition and relationship extraction in paragraph text. The present invention combines the advantages of knowledge graph, entity vector representation, dynamic prompt and large model, without fine-tuning the large model, to achieve automatic extraction of entity relationships, reduce the cost of entity relationship extraction, improve efficiency and accuracy, and has a wide range of application and promotion value.

Claims (5)

1. A method for extracting entity relationships based on a large model and dynamic prompts, the steps of which include: A. Define the model layer and entity set of the domain knowledge graph. The specific steps are as follows: A1. Define the set of entity categories C={C ₁ ,…C _i ,…C _N }(i∈[1,N]), the set of relationships between entity categories R={R ₁ ,…R _j ,…R _M }(j∈[1,M]), where N and M represent the number of entity categories and the number of relationships in the knowledge graph to be built; A2. Extract words belonging to entity category C _i (i∈[1,N]) from the industry terminology dictionary or catalog as entities to form the entity set E _i ={e _i1 ,…e _ij ,…,e _ik } of this category (i∈[1,N], j∈[1,k], k represents the number of entities in the entity set, which varies depending on the entity category C _i ); A3. The entity category _Ch (h∈[1,N]) and entity category C _t (t∈[1,N]) with the relation R _j (j∈[1,M]) are recorded as entity category triples ( _Ch , R _j , C _t ), where _Ch is called the head entity category and its entity set is recorded as E _hj , C _t is called the tail entity category and its entity set is recorded as E _tj ; all entity category triplets constitute the pattern layer of the knowledge graph; B. Use the pattern layer and entity set to construct an example set of entity relationship triples. Take the entity category triple (C _h , R _j , C _t ) as an example. The specific operations are as follows: B1. Select F entity pairs from entity set E _h and entity set E _t under the condition that they satisfy the R _j relationship; B2. Combine each entity pair and relation R _j into an entity-relation triple (e _hi ,R _j ,e _ti ), where entity e _hi ∈E _h (i∈[1,F]), e _ti ∈E _t (i∈[1,F]); B3. Using the F entity-relationship triples constructed in step B2, construct an example set of entity-relationship triples S _j ={(e _hi ,R _j ,e _ti ) for i in [1:F]} of relation R _j , where for i in [1:F] indicates that the value of i is an integer from 1 to F; C. Construct entity vector database DB. The specific steps are as follows: C1. For each entity e _ij ∈E _i (i∈[1,N],j∈[1,|Ei|]) in the entity set E _i (i∈[1,N]), encode it into an entity vector ve _ij using the pre-trained language model; C2. storing each pair of entity e _ij and entity vector ve _ij into the entity vector database DB; D. Build the prompt template set PR. The specific steps are as follows: D1. Define the template set PR = {pr ₁ ,…,pr _k ,…,pr _n } (k∈[1,n]), where n represents the number of templates contained in the template set PR; D2. Construct each prompt template pr _k ∈ PR, where the template contains several template variables; E. Divide the text to be processed into paragraphs and extract the keyword list of the paragraphs. The specific steps are as follows: E1. Divide the unstructured text to be processed into paragraphs; E2. Extract the keyword list in each paragraph _Pi (i∈[1,t]); F. Use the keyword list to construct a paragraph-entity association list T for each paragraph. Taking paragraph _Pi as an example, the specific steps are as follows: F1. Encode each keyword k _ij ∈K _i (j∈[1,p]) in K _i into a keyword vector vk _ij using the pre-trained language model in step C1; F2. Perform similarity search on the keyword vector vk _ij in the entity vector library to obtain the Top-N similar entities, denoted as Top-N[k _ij ]={e ₁ ,e ₂ ,…,e _N }; F3. Remove duplicates from all similar entities corresponding to all keywords in K _i to form an entity list L _i ; F4. Combine paragraph _Pi and entity list _Li into _Pi 's paragraph-entity association list _Ti = [ _Pi , _Li ]; G. For each paragraph, construct a dynamic prompt using the association list T, the prompt template set PR, and the entity relationship triple example set; G1. Replace the template variables in the prompt template pr _k with the corresponding variable values; G2. For the remaining variables in the template, enter the corresponding values according to the definition of the template variables to form a complete dynamic prompt content Q _k P _i (k∈[1,n],i∈[1,t]). For example, enter field (field name) and substitute it into the field variable ${field}; H. Send the dynamic prompt content to the big model for entity relationship extraction and verify the correctness of the results. The specific steps are as follows: H1. Communicate with the big model by calling the API, and send the dynamic prompt content to multiple big models for entity relationship extraction; H2. Compare and analyze the entity relationship extraction results of multiple large models and select the optimal extraction result. 2. The entity relationship extraction method based on a large model and dynamic prompts as claimed in claim 1, constructing each prompt template prk∈PR, wherein the template contains a number of template variables, and the specific steps are as follows: D2.1 defines a domain variable as ${field}, where the value range of field is the set of domain identifiers; D2.2 defines the paragraph variable as ${P _i }, where the value range of P _i is the set of paragraph tags; D2.3 Define entity variables as ${e ₁ }, …, ${e _j }, …, ${e _m }, where e ₁ ~e _m are the corresponding entities in the paragraph; D2.4 Define the relationship variables as ${R ₁ }, …, ${R _j }, …, ${R _M }, where R ₁ ~R _M are all the relationships in the knowledge graph to be built; D2.5 defines the entity-relationship triplet example set variables as ${S ₁ }, …, ${S _j }, …, ${S _M }, where S ₁ ~S _m is the entity-relationship triplet example set corresponding to relations R ₁ ~R _M. 3. The entity relationship extraction method based on a large model and dynamic prompts as claimed in claim 1, wherein the unstructured text to be processed is divided into paragraphs, and the specific steps are as follows: E1.1 Clean the text data and remove useless punctuation marks, special characters, etc.; E1.2 Use the pre-trained word vector model to represent the text with word vectors and generate word vector sequences; E1.3 Input the word vector sequence into the neural network sequence labeling model to detect the topic boundaries of the text and obtain the start and end positions of each paragraph; E1.4 Based on the detected topic boundaries, the text is divided into different paragraphs, and the paragraph set P = {P ₀ ,…P _i ,…,P _t } (i∈[1,t]) is obtained, where t is the total number of paragraphs in the set. 4. The entity relationship extraction method based on a large model and dynamic prompts as claimed in claim 1, extracting a keyword list in each paragraph Pi (i∈[1,t]), the specific steps are as follows: E2.1 Extract keywords from each paragraph P _i (i∈[1,t]) and add them to the set K _i ={k _i1 ,k _i2 ,...,k _ip } (p is the total number of keywords in P _i ). Keyword extraction methods include but are not limited to TF-IDF, TextRank, and methods based on pre-trained language models; E2.2 Combine paragraph _Pi and keyword set _Ki into keyword list PL _i = [P _i , _Ki ] of paragraph _Pi . 5. The entity relationship extraction method based on a large model and dynamic prompts as claimed in claim 1, wherein the template variables in the prompt template prk are replaced with corresponding variable values, and the specific steps are as follows: G1.1 Substitute the paragraph _Pi in the association list _Ti into the paragraph variable ${ _Pi }; G1.2 Substitute each entity of the entity list _Li in the association list _Ti into the entity variables ${e ₁ }~${e _m }; G1.3 Substitute the relations R ₁ ~R _j of the entities in the entity list _Li into the relation variables ${R ₁ }~${R _j }; G1.4 Substitute the entity relationship triple example set in the entity triple example set S ₁ ~S _j corresponding to the relation R ₁ ~R _j into the entity relationship triple example set variable ${S ₁ } ~${S _j }.