CN115130989A - Method, device and equipment for auditing service document and storage medium - Google Patents

Abstract

The disclosure provides a method, a device, equipment and a storage medium for auditing a business document, which relate to the technical field of image processing, in particular to the technical fields of artificial intelligence, computer vision and intelligent finance. The specific implementation scheme is as follows: acquiring service document images of at least two service documents; wherein, the at least two service documents are all associated with the target service content and belong to different document types; extracting key information of the target business content from the business document image corresponding to the document type according to information extraction modes corresponding to different document types; and auditing the at least two service documents according to the extracted key information. The scheme can quickly and accurately complete the auditing of the business documents of different document types.

Description

A business document review method, device, device and storage medium

technical field

The present disclosure relates to the technical field of image processing, in particular to the technical fields of artificial intelligence, computer vision and intelligent finance, and in particular to a business document review method, device, device and storage medium.

Background technique

With the development of artificial intelligence technology, a document intelligence application represented by document image review, document image comparison, document image error correction, and report writing has gradually emerged. The document image review may be used in the review process of business documents, and specifically may be a process of performing key information consistency review on business documents of at least two document types associated with the same business content. However, when there are many business contents associated with business documents of at least two document types, how to quickly and accurately complete the review of business documents of different document types is very important.

SUMMARY OF THE INVENTION

The present disclosure provides a business document review method, device, device and storage medium.

According to an aspect of the present disclosure, a method for reviewing business documents is provided, including:

Obtaining business document images of at least two kinds of business documents; wherein, the at least two kinds of business documents are both associated with the target business content and belong to different document types;

Extract the key information of the target business content from the business document image corresponding to the document type according to information extraction methods corresponding to different document types;

The at least two business documents are reviewed according to the extracted key information.

According to another aspect of the present disclosure, there is provided an electronic device comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the business document review method of any embodiment of the present disclosure.

According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the business document review method of any embodiment of the present disclosure.

This solution can quickly and accurately complete the review of business documents of different document types, providing a new solution for the review of business documents.

It should be understood that what is described in this section is not intended to identify key or critical features of embodiments of the disclosure, nor is it intended to limit the scope of the disclosure. Other features of the present disclosure will become readily understood from the following description.

Description of drawings

The accompanying drawings are used for better understanding of the present solution, and do not constitute a limitation to the present disclosure. in:

1 is a flowchart of a method for reviewing business documents provided according to an embodiment of the present disclosure;

2A is a flowchart of a method for reviewing business documents provided according to an embodiment of the present disclosure;

2B is a schematic diagram of the working principle of a text image perception model provided according to an embodiment of the present disclosure;

3 is a schematic block diagram of a process of adjusting parameters and using a text image perception model provided by an embodiment of the present disclosure;

4A is a flowchart of a method for reviewing business documents provided according to an embodiment of the present disclosure;

FIG. 4B is a schematic diagram of an input feature construction principle of a table image perception model provided according to an embodiment of the present disclosure;

5 is a flowchart of a method for reviewing a business document provided according to an embodiment of the present disclosure;

6 is a flowchart of a method for reviewing business documents provided according to an embodiment of the present disclosure;

7 is a flowchart of a method for reviewing business documents provided according to an embodiment of the present disclosure;

8A is a flowchart of a method for reviewing business documents provided according to an embodiment of the present disclosure;

8B is a schematic diagram of a display effect of a visual interface provided according to an embodiment of the present disclosure;

9 is a flowchart of a method for reviewing a business document provided according to an embodiment of the present disclosure;

FIG. 10 is a schematic block diagram of the parameter adjustment and use process of the sorting model provided by the embodiment of the present disclosure;

11 is a flowchart of a method for reviewing business documents provided according to an embodiment of the present disclosure;

12A is a flowchart of a work order document review provided by an embodiment of the present disclosure;

FIG. 12B is a flow chart of the content working principle of the perception engine provided by the embodiment of the present disclosure;

13 is a schematic structural diagram of an apparatus for reviewing business documents provided according to an embodiment of the present disclosure;

FIG. 14 is a block diagram of an electronic device used to implement the business document review method according to an embodiment of the present disclosure.

Detailed ways

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding and should be considered as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted from the following description for clarity and conciseness.

FIG. 1 is a flowchart of a business document review method provided according to an embodiment of the present disclosure. The embodiment of the present disclosure is applicable to the case of performing information consistency review on business documents of at least two document types associated with the same target business content . For example, it is applicable to the case of conducting information consistency review on business forms and business contracts related to the same target business content. The method may be executed by a device for reviewing business documents, and the device may be implemented in software and/or hardware. Specifically, it can be integrated into an electronic device installed with a document intelligence application.

S101, acquiring business document images of at least two business documents.

The business document image in this embodiment is an image obtained by photographing or scanning the document page of the business document. Since there are at least two types of business documents in this embodiment, the types of business document images also include at least two types. Optionally, the document types corresponding to the at least two business documents in this embodiment may include but are not limited to: text type, table type, slide type, and the like. The text-type business documents may be documents represented in the form of paragraphs or chapters, for example, business contracts and business application documents. The form-type business document may be a document represented in the form of a form, for example, a business application form or a business approval form. The business document of the slide type may be a document represented in the form of a slide, for example, it may be a business presentation slide.

The business in this embodiment can generally refer to any business whose business content needs to be recorded through a variety of different types of business documents during the execution process. Correspondingly, the business document is a document generated to record the relevant business content during the execution of the business. Business documents of different document types can record business content in different forms and ways. That is, at least two kinds of business documents acquired in this embodiment are associated with the target business content and belong to different document types. The target service is the service corresponding to the at least two service documents to be reviewed, and the content of the target service may be all relevant content generated during the execution of the target service. In this embodiment, although the corresponding document types of different types of service documents are different, their corresponding target service contents should be the same. The essence of auditing different types of business documents in this solution is to examine whether the information recorded in different types of business documents about the target business content is consistent.

Optionally, this embodiment may acquire business document images corresponding to business documents of at least two document types corresponding to the target business to be reviewed according to the review requirements of the business documents. If business documents of at least two document types are obtained, each document page of each type of business document can be scanned or photographed to obtain a business document image corresponding to each document type. It should be noted that, in this embodiment, for each type of business document, the number of corresponding business document images is at least one.

Optionally, if there is no distinction between the types of business documents in the process of obtaining the business document images of at least two business documents, after obtaining the business document images, it is also necessary to parse the corresponding business document images for each obtained business document image. The document type of the business document. Specifically, there are many ways to parse the document type of the business document corresponding to each business document image, which is not limited in this embodiment. For example, an implementation manner is: combining the format features of each document type, and determining the document type of the business document corresponding to each business document image based on feature comparison. Another possible implementation is: for each business document image, first use text recognition technology, such as optical character recognition (Optical Character Recognition, OCR) technology, to recognize the text information in it, and then based on the classification model (for example, based on logic The multi-classification model constructed by the regression (softmax) function), according to the text information of each business document image, to parse the document type of the business document image corresponding to the business document.

Optionally, for a target business, there may be many types of business documents corresponding to it, but it may only be necessary to review the business documents of some of the document types when reviewing, so this step obtains at least two types of business documents. When obtaining business document images of documents, only the business document images of those document types that need to be reviewed may be obtained.

For example, for the target business, when conducting business document review, if only the content of the table type and text type business documents are consistent, then in order to prevent other types of business documents from affecting the review results, this embodiment executes In this step, business document images of business documents of other document types except table type and text type can be excluded, and only business document images corresponding to business documents of table type and text type can be extracted.

S102: Extract key information of the target business content from the business document image corresponding to the document type according to information extraction methods corresponding to different document types.

The key information may be information corresponding to the content of the target service and used for reviewing the service documents of the target service. Optionally, different target businesses may select different key information based on their business requirements. The quantity of key information extracted from the business document image of the business document of each document type in this embodiment is at least one. For example, if the target service is a project contract service, the key information about the content of the target service may include the contracting parties, the contracting time, and the contracting terms.

It should be noted that, in this embodiment, when selecting key information, it is necessary to ensure that the key information must be recorded in business documents of different document types. Business documents of different document types can record this key information through the same or different fields. Optionally, the key information in this embodiment may be represented by a key-value pair, wherein the key represents the field name of the key information in the corresponding business document, and the value represents the specific content extracted from the corresponding document. .

Optionally, in this embodiment, for business documents of different document types, different information extraction methods may be selected for different document types based on the characteristics of information recording methods of various types of business documents, so as to extract information from business documents of corresponding document types. Extract the key information of the target business content from the image. Specifically, for each acquired business document image, based on text recognition technology, such as OCR technology, identify the text information in the business document image, and then according to the information extraction method corresponding to each business document image, from the The key information of the target business content is extracted from the text information in the document image.

Optionally, for the business document image of the business document of the form type, the target recorded in the business document image can be extracted according to the text information in the business document image through structured information extraction and/or a pre-trained form image perception model. Key information for business content. For a business document image of a text-type business document, the key information of the target business content recorded in the business document image can be extracted according to the text information in the business document image through semantic parsing and/or a pre-trained text image perception model.

It should be noted that the text image perception model and the table image perception model are obtained by fine-tuning the parameters of pre-trained models constructed by different algorithms. The text-image-aware model performs better when performing the task of extracting key information from business document images corresponding to text types; the table-image-aware model performs better when performing the task of extracting key information from business document images corresponding to table types. There is no limitation on the specific process for the two models to perform key information extraction tasks.

S103, review at least two business documents according to the extracted key information.

Optionally, in this embodiment, according to the business document image corresponding to the extracted key information and the document extraction field (that is, the field in which the key information is recorded in the business document corresponding to the key information), the extracted key information is located in a different location. A business document, and at least two key information corresponding to the document extraction field with the same semantics as an information pair. Then, the content consistency audit is performed for at least two key information included in each information pair. If they are consistent, it means that the information pair has passed the audit; otherwise, the audit has failed. If the audit results of all the information pairs are approved, it is said that the audit results of at least two business documents of the target business in this embodiment are passed, otherwise the audit results are not passed.

Optionally, there are many ways to determine the document extraction field corresponding to the key information in this embodiment. For example, if the key information is represented by a key-value pair (key-value), the key information can be The key value is used as its corresponding document extraction field. If the key information is only the content extracted from the business document image, at this time, the key information can be identified by the entity tag of the corresponding field of the target business, and the identification result can be used as the document extraction field corresponding to the key information.

In this embodiment, there are many ways to determine the document extraction fields with the same semantics. For example, the extraction fields of documents with the same semantics can be determined by performing semantic analysis on the extraction fields of different documents, or which documents are preset according to the actual needs of the business. Extract fields are semantically identical document extraction fields.

According to the solution of the embodiment of the present disclosure, after acquiring the business document images of at least two kinds of business documents, different information extraction methods are used for different types of business document images to extract key information from the corresponding business document images, and then according to the extracted key information to review at least two types of business documents. For business documents of different document types, this solution automatically extracts key information in different ways, improves the accuracy and efficiency of key information extraction, and provides a guarantee for quickly and accurately completing the review of business documents of different document types.

2A is a flowchart of a business document review method provided according to an embodiment of the present disclosure, and FIG. 2B is a schematic diagram of a working principle of a text image perception model provided according to an embodiment of the present disclosure; On the basis of , the process of how to extract the key information of the target business content from the business document image corresponding to the text-type business document is further explained in detail. Methods can include:

S201 , acquiring business document images of at least two kinds of business documents.

Wherein, at least two kinds of business documents are associated with the target business content and belong to different document types.

S202, if the document type is a text type, construct an input sequence according to a preset spacer, a document extraction field, and text information in a business document image of the text type.

The preset spacer in this embodiment may be a preset symbol used to separate the start and end of the input sequence and different types of content in the input sequence. In this embodiment, three preset spacers are preferred, ie, a start character (CLS) representing the beginning of the input sequence, an end character (SEP1) representing the end of the input sequence, and a separation (SEP2) separating different kinds of content in the input sequence.

The document extraction field may be a field in the business document corresponding to the key information that records the key information. It should be noted that, in this embodiment, the document extraction fields corresponding to the key information of the same content in different types of business documents may be the same or different. For example, in a table-type business document, the document extraction fields corresponding to project participants (ie, key information) are "deployed business organization" and "transaction counterparty"; in a text-type business document, project participants (ie, key information) correspond to The document extraction fields are "Party A" and "Party B". If in this embodiment, the key information of the same content has different document extraction fields in different types of business documents, this step may use the document extraction fields corresponding to the document type business documents as the required fields for constructing the input sequence in this step. document extraction field.

Optionally, in the case where the document type is a text type, this embodiment may perform text information recognition on a text-type business document image, and the obtained recognition result includes: the recognized characters and the position information of the characters in the business document image. and recognition confidence. Then, word quantization processing is performed on the recognition result to obtain a plurality of continuous quantified word information (token), wherein the quantified word information may include: quantified words and their position information and recognition confidence; next, as shown in FIG. 2B , splicing the start character (CLS), document extraction field (promot) separator (SEP2), multiple consecutive quantized word information (token) and end character (SEP1) in the preset spacer in sequence to obtain the input sequence.

S203: Determine the extraction start position corresponding to the document extraction field and the confidence level of the extraction start position according to the input sequence through the text image perception model.

Wherein, the text-image perception model in this embodiment is preferably a simplified version of automatic modeling, which is constructed based on the coding part of the pre-training model (ERNIE) and a unified information extraction framework (Universal Information Extraction, UIE). The framework realizes the unified modeling of entity extraction, relation extraction, event extraction, sentiment analysis and other tasks, and enables good transfer and generalization capabilities between different tasks.

Specifically, as shown in FIG. 2B , in this embodiment, the input sequence constructed in S202 may be input into a text-image perception model, and the model can learn from the text information in the input sequence (that is, a plurality of continuous quantifications in the text information). word), parse the continuous quantified words corresponding to the document extraction field, and use the position information corresponding to the first quantified word and the last quantified word in the continuous quantified words recorded in the quantified word information as the group of continuous quantified words. The extraction start position (Start) and the extraction end position (End) are the extraction start positions corresponding to the continuous quantized words. And give the corresponding confidence of the extraction starting position. Exemplarily, the two sets of output results shown in FIG. 2B are the two sets of extraction starting positions and their confidence levels.

It should be noted that, in text-type business documents, the same document extraction field may appear multiple times. For example, business participant information appears multiple times. Therefore, for each document extraction field, multiple sets of corresponding fields may be extracted. The continuous quantified words of , and then for each document extraction field, the text image perception model will output multiple sets of corresponding extraction starting positions and their confidence levels.

S204 , according to the extraction start position and the confidence level of the extraction start position, determine the extraction information corresponding to the document extraction field and the confidence level of the extraction information.

Optionally, in this embodiment, for each document extraction field, based on each group of extraction start positions, in the text information of the text-type business document image, the position between the group of extraction start positions can be extracted. Each character is used as a set of extraction information corresponding to the extraction field of the document. Then the confidence of the extraction starting position is taken as the confidence of the extracted information. Specifically, when the confidence level of the extraction start position includes: the confidence level of the extraction start position and the confidence level of the extraction end position, the confidence level of the extraction start position and the confidence level of the extraction end position may be fused (such as summing up). processing or averaging, etc.), and use the processing result as the confidence level of the extracted information.

It should be noted that, if a document extraction field corresponds to multiple sets of extraction starting positions and their confidence levels, this step may determine multiple sets of corresponding extraction information for the document extraction field, and the confidence levels corresponding to each set of extraction information.

S205: Determine the key information of the target business content in the text-type business document image according to the extraction information corresponding to the document extraction field and the confidence level of the extracted information.

Specifically, for each document extraction field, if the corresponding extraction information is one, the extraction information can be directly used as a text-type business document image, and the document extraction field corresponds to key information. If there are multiple pieces of corresponding extraction information, the extraction information with the highest confidence may be used as the text-type business document image, and the document extraction field corresponds to the key information. The key information corresponding to all document extraction fields is integrated to obtain the key information of the target business content in the text-type business document image.

S206, if the document type is other document types except the text type, extract the key information of the target business content from the business document image corresponding to the document type according to the information extraction mode corresponding to each other document type.

S207, at least two kinds of business documents are reviewed according to the extracted key information.

In the solution of the embodiment of the present disclosure, after acquiring the business document images of at least two kinds of business documents, for the text-type business document images, an input sequence is constructed according to the preset spacer, the document extraction field, and the text information of the text-type business document images, According to the text image perception model, according to the input sequence, determine the extraction start position and its confidence level corresponding to the document extraction field, and then determine the key information of the business document image corresponding to the document type based on the extraction start position and its confidence level, and use other information The method also extracts key information from other types of business document images, and then checks at least two business documents according to the extracted key information. This embodiment presents a specific implementation of how the text image perception model based on ERNIE+UIE extracts key information from text-type business document images. The model can more accurately and comprehensively extract text-type business document images corresponding to It provides a guarantee for the subsequent accurate review of business documents.

Optionally, in this embodiment of the present disclosure, a specific implementation process of text information in a text-type business document image according to a preset spacer, a document extraction field, and a text-type business document image may include: performing an operation on the text information in the text-type business document image. Preprocessing to obtain preprocessed text; wherein, the preprocessing includes at least one of cleaning processing, symbol replacement processing, and format conversion processing; constructing an input sequence according to the preprocessing text, preset spacers, and document extraction fields.

Specifically, the purpose of cleaning the text information in the business document image is to remove the space character " ", the character "\t", the newline character "\n" and the underscore "_" in the text. The purpose of performing symbol replacement processing on the text information in the business document image is to uniformly convert all the punctuation marks in the text into Chinese punctuation marks or English punctuation marks. The purpose of the format conversion process is to convert the textual information into a format supported by the textual image perception model. The process of constructing the input sequence according to the preprocessed text, the preset spacer and the document extraction field is similar to the process of constructing the input sequence according to the text information, the preset spacer and the document extraction field described in the above embodiment. No further description is given here.

Before constructing the input sequence, this embodiment introduces a preprocessing operation on the text information, improves the uniformity and standardization of the text information, and facilitates the subsequent text-image perception model to more accurately predict the extraction start position corresponding to the document extraction field, and the extraction process. Confidence of the starting position.

Exemplarily, FIG. 3 is a schematic block diagram of a process of adjusting parameters and using a text image perception model provided by an embodiment of the present disclosure. As shown in FIG. 3 , blocks 301 to 304 correspond to the parameter adjustment process of the text image perception model. Specifically: in the parameter adjustment process, the business document image corresponding to block 301 is a document of a text-type business document. Image, that is, the training sample image (such as a scanned copy of a PDF document), input the training sample image in block 301 into block 302, and perform text recognition (such as OCR recognition) on it and the recognized text The information is input to block 303 for data preprocessing (such as at least one of cleaning processing, symbol replacement processing, and format conversion processing), according to the preprocessed text information (ie input text), preset spacers and document extraction field, constructing the input sequence, and labeling the preprocessed data based on the labeling tool for document extraction fields, wherein, in this embodiment, a labeling tool (doccano) that supports nested entities is preferably used, and the labeling tool can identify the same vocabulary. Multiple entities of different categories can more accurately and comprehensively complete the annotation of document extraction fields. The input sequence obtained by data preprocessing in block 303 and the result of the input text annotation of the input sequence are input into block 304, and the text image perception model is fine-tuned (fine-turning). The fine-tuned text image perception model is It can be published for the subsequent block 305 to perform key information extraction operations.

Block 301 to block 303 and block 305 correspond to the process of using the text image perception model, specifically: in the actual use process, block 301 corresponds to acquiring the business document image of the text-type business document, and block 301 The acquired business document image is input into block 302, and text recognition (such as OCR recognition) is performed on it, and the recognized text information is input into block 303 for data preprocessing (such as cleaning processing, symbol replacement processing, and formatting). at least one of conversion processing), constructs an input sequence according to the preprocessed text information (ie, the input text), the preset spacer and the document extraction field. The input sequence constructed in block 303 is input into block 305, and the key information of the target business content is extracted based on the input sequence through the text-aware model published in block 304, wherein, how the text-aware model specifically extracts the information of the target business content based on the input sequence. For the process of key information, see S203-S205 in the above embodiment.

4A is a flowchart of a business document review method provided according to an embodiment of the present disclosure, and FIG. 4B is a schematic diagram of an input feature construction principle of a table image perception model provided according to an embodiment of the present disclosure; On the basis of the embodiment, the process of how to extract the key information of the target service content from the service document image corresponding to the service document of the form document type is further explained in detail. Document review methods can include:

S401, acquiring business document images of at least two business documents.

Wherein, at least two kinds of business documents are associated with the target business content and belong to different document types.

S402, in the case that the document type is the form type, divide the business document image of the form type into blocks to obtain an image block sequence, and determine the image features of the image blocks in the image block sequence and the block positions of the image blocks in the business document image , and the block number of the image block in the image block sequence.

Specifically, as shown in FIG. 4B , for table-type business document images, each business document image can be divided into multiple image blocks based on a preset segmentation strategy. The segmentation strategy corresponding to FIG. 4B is to divide The image of the business document is divided into 4 equal parts, and the divided 4 image blocks (ie the upper left image block, the upper right image block, the lower left image block and the lower right image block) are arranged in sequence to form an image block sequence.

Visual encoder is used to visually encode each image block in the image block sequence to obtain the image features corresponding to each image block, and the vertex position (such as the upper left corner position) or center position of each image block in the business document image It is used as the block position of each image block in the business document image; the sorting position of each image block in the image block sequence is used as the block sequence number of each image block in the image block sequence. The block sequence number may be a segment of ascending sequence numbers.

Exemplarily, the first 4 feature maps of the line where the visual and text features T1 are located in FIG. 4B are the image features of the image blocks in the image block sequence; that is, the first feature map of the line where T1 is located is equal to the business document image. The feature map corresponding to the upper-left image block after division, the second feature map of the row where T1 is located is the feature map corresponding to the upper-right image block after dividing the business document image into equal parts, and the third feature map of the line where T1 is located is The feature map corresponding to the image block in the lower left corner after the business document image is divided into equal parts. The fourth feature map in the row where T1 is located is the feature map corresponding to the image block in the lower right corner after the image block of the business document image is divided into equal parts.

The first four position coordinates in the row where the coordinate position feature T2 is located in FIG. 4B are the block positions of each image block in the business document image. For example, the first coordinate position of the row where T2 is located is the first feature map in T1 The block position of the corresponding image block in the business document image; the first 4 serial numbers (0-3) in the row where the sorting position feature T3 is located in FIG. 4B are the block serial numbers of each image block in the image block sequence, for example, T3 The serial number 0 of the row is the block serial number of the image block corresponding to the first feature map in T1 in the image block sequence.

S403, perform word quantization processing on the text information in the business document image to obtain a word sequence, and determine the word characteristics of the quantified word in the word sequence, the word position of the quantified word in the business document image, and the word of the quantified word in the word sequence serial number.

Specifically, as shown in FIG. 4B , for table-type business document images, based on text recognition technology (such as OCR technology), the text information of each business document image is identified, and then word quantization processing is performed on the identified text information to obtain multiple Each quantified word (ie, quantified word), and a plurality of quantified words are arranged in sequence to form a word sequence. Preferably, a preset delimiter can be added at the beginning and end of the word sequence, such as a start delimiter <S> and an end delimiter </S>. The character corresponding to each quantified word is used as the word feature of the quantified word, and the word position of each quantified word in the business document image (that is, the position given by the text recognition technology when recognizing the text) is used as each quantified word in the business document image. The word position of each quantified word; sort the position of each quantified word in the word sequence, as the word sequence number of each quantified word in the word sequence. The word sequence number may be a segment of ascending sequence numbers.

Exemplarily, each vocabulary and the head and tail separators (<S> and </S>) of the line where the visual and text features T1 are located in Figure 4B are the word features of the quantified words in the word sequence; the coordinate position feature T2 in Figure 4B is located. The 5th to the last position coordinate in the row is the word position of the quantified word in the business document image; the numerical value (ie 0-9) of the 5th to the last position in the row where the sorting position feature T3 is located in FIG. 4B is The word number of the quantified word in the word sequence.

It should be noted that, in this embodiment, the two processes of S403 and S404 may be performed simultaneously, and there is no order of precedence.

S404: Determine the model input feature according to the image feature, block position and block serial number of the image block in the image block sequence, and the word feature, word position and word serial number of the quantized word in the word sequence.

The model input features are features that need to be input into the table image perception model.

Optionally, as shown in FIG. 4B , in this embodiment, the image features of the image blocks in the image block sequence and the word features of the quantized words in the word sequence can be spliced together as visual and text features T1; The block position of the block and the word position of the quantized word in the word sequence are spliced as the coordinate feature T2; the block sequence number of the image block in the image block sequence and the word sequence number of the quantized word in the word sequence are spliced as the sorting position feature T3; The visual and text features T1, the coordinate position features T2, and the ranking position features T3 constitute features in the form of triples, ie, model input features.

S405 , extract key information of the target business content from the table-type business document image by using the table image perception model and according to the model input feature.

The table image perception model in this embodiment is preferably a model released after fine-tuning a multimodal pre-training model (Multimodal Pre-training for Multilingual Visually-rich Document Understanding, LayoutXLM). The so-called multimodality in this embodiment includes three modalities, the visual and text features T1, the coordinate position feature T2, and the sorting position feature T3 that constitute the input feature.

Specifically, as shown in FIG. 4B , in this embodiment, the model input features including three modalities may be input into the table image perception model, and the table image perception model can parse the business document of the table type according to the input features. In the image, the key information of the target business content. That is, the key information of the target business content corresponding to each document extraction field (specifically, the table document extraction field) in the business document of the form type is given.

S406, if the document type is other document types except the table type, extract the key information of the target business content from the business document image corresponding to the document type according to the information extraction method corresponding to each other document type.

S407, at least two kinds of business documents are reviewed according to the extracted key information.

According to the solution of the embodiment of the present disclosure, after obtaining the business document images of at least two kinds of business documents, the business document images of the form type are divided into multiple image blocks, and the image features, block positions and block serial numbers of the image blocks are obtained, And perform word quantization processing on the text information in the business document image, obtain the word feature, word position and word serial number of the quantified word, and integrate the obtained information to obtain the multi-modal model input feature, and then perceive the model through the table image. , according to the model input features, extract key information from the business document image corresponding to the table type, and then use other information methods to extract key information from other types of business document images, and then perform at least two business documents according to the extracted key information. review. This embodiment presents a table image perception model based on LayoutXLM, which extracts key information from table-type business document images through multi-modal input features. The model can extract table types more accurately and comprehensively. The key information in the corresponding business document image provides a guarantee for the subsequent accurate review of the business document.

FIG. 5 is a flowchart of a business document review method provided according to an embodiment of the present disclosure. On the basis of the above-mentioned embodiment, the embodiment of the present disclosure further describes how to extract a business document image corresponding to a business document of the form document type. The process of the key information of the target business content is explained in detail. As shown in FIG. 5 , the review method of the business document provided in this embodiment may include:

S501: Obtain business document images of at least two business documents.

Wherein, at least two kinds of business documents are associated with the target business content and belong to different document types.

S502 , when the document type is a table type, extract the first candidate information of the target business content from the business document image of the table type by using a table image perception model.

It should be noted that, for the specific implementation manner of this step, refer to the process of S402-S405 in the foregoing embodiment, which will not be repeated here.

S503, according to the structured information extraction logic, extract the second candidate information of the target business content from the table-type business document image.

The structured information extraction logic may be based on a preset document extraction field-key information structured template for information extraction, and the structured template may be set according to a table dedicated to the business document. For example, the structure The document extraction fields in the template can be the business field names in the business document of this form type, such as name, age, and time.

Specifically, the document extraction field in the structured template can be matched with the text information in the business document image corresponding to the table type to obtain the text information corresponding to the document extraction field, that is, the candidate information corresponding to the document extraction field, and then The candidate information corresponding to the document extraction field in the structured template is integrated to obtain the second candidate information of the target business content extracted from the business document image corresponding to the form type.

S504, according to the first candidate information and the second candidate information, determine the key information of the target business content in the table-type business document image.

It should be noted that, in S502, a table image perception model is used, and although the recall rate of the first candidate information extracted from the input feature of the multimodal model is high, the accuracy is low. S503, based on the structured information extraction logic, determines the second candidate information. High accuracy but low recall. Therefore, in this step, the first candidate information can be corrected according to the second candidate information, and the corrected first candidate information can be used as the key information of the target business content in the table-type business document image. Specifically, for each document extraction field in the first candidate information, if the second candidate information also includes the document extraction field, update the document extraction field in the first candidate information with the content of the document extraction field in the second candidate information The content of a candidate message.

S505 , if the document type is other document types except the table type, extract the key information of the target business content from the business document image corresponding to the document type according to the information extraction method corresponding to each other document type.

S506, at least two kinds of business documents are reviewed according to the extracted key information.

According to the solution of the embodiment of the present disclosure, after obtaining the business document images of at least two kinds of business documents, for the business document images corresponding to the form types, a part of candidate information is extracted through the form image perception model and the structured information extraction logic for fusion to obtain the form type For the corresponding business document images, other information methods are used to extract key information from other types of business document images, and then at least two business documents are reviewed according to the extracted key information. In the process of extracting the key information in the business document image corresponding to the form type, this solution combines the neural network extraction method with high recall rate but low accuracy with the structured extraction logic with low recall rate but high accuracy. Taking into account the comprehensiveness and accuracy of the extraction results, it provides a guarantee for the subsequent more accurate review of business documents.

FIG. 6 is a flowchart of a method for reviewing business documents provided according to an embodiment of the present disclosure. On the basis of the above-mentioned embodiment, the embodiment of the present disclosure further specifies how to check the at least two kinds of business documents according to the extracted key information. The process of document auditing is explained in detail. As shown in FIG. 6 , the auditing method for business documents provided in this embodiment may include:

S601, acquiring business document images of at least two business documents.

Wherein, at least two kinds of business documents are associated with the target business content and belong to different document types.

S602: Extract key information of the target business content from the business document image corresponding to the document type according to information extraction methods corresponding to different document types.

S603: Divide the extracted key information into at least one information pair according to the business document image and the document extraction field corresponding to the extracted key information.

The document extraction fields corresponding to the key information in the same information pair have the same semantics and are taken from business document images of different document types.

That is to say, considering that in different types of business documents, the names of the document extraction fields representing the same business content may not necessarily be the same, so this embodiment may compare the key information extracted from the business document images corresponding to different document types, and use The semantics of the document extraction fields are the same, and the key information extracted from the business document images of different document types constitutes an information pair, so that all the key information extracted in S602 is divided into at least one information pair.

S604, extracting fields from documents corresponding to the information pairs, and determining target audit rules for the information pairs.

Optionally, in this embodiment, a corresponding target audit rule may be set for each semantic document extraction field, which is specially used for auditing the key information in the information pair containing this semantic document extraction field. Specifically, this step extracts fields according to the document corresponding to each information pair, and determines the target audit rule for the information pair, which may include at least one of the following situations:

Case 1: In the case where the information expression mode of the corresponding document extraction field of the information pair is unique, the target audit rule for the information pair is character consistency audit. Specifically, for a document extraction field with a unique and fixed information expression method, for example, a document extraction field corresponding to the name of a business participant, the target review rule of the information pair to which it belongs is that the characters of each key information contained in the information pair must be completely Unanimous to be approved.

Case 2: In the case that the information expression mode of the corresponding document extraction field of the information pair is not unique, the target review rule of the information pair is semantic similarity review. Specifically, for document extraction fields with flexible and non-unique information expression methods, for example, the document extraction fields corresponding to the business execution duration can be expressed by the start time and end time, or by the total duration. The target auditing rule for the information pair to which the document extraction field of the type belongs is that the characters of each key information contained in the information pair do not have to be completely consistent, as long as the expressed semantics are the same, the auditing can be passed. For example, the similarity between each key information in the information pair may be calculated, and if the similarity is greater than a similarity threshold (eg, 0.5), it is considered that the consistency review has passed.

Scenario 3: In the case where the corresponding document extraction field of the information pair represents numerical information, the target auditing rule for the information pair is numerical character consistency auditing. Specifically, for a document extraction field representing numerical information, the target review rule for the information pair to which it belongs is to extract the numerical information in each key information contained in the information pair, and then review the numerical consistency of each key information. Consistent to pass the review, there is no need to require all characters of critical information to be consistent.

Situation 4: In the case where the document extraction field corresponding to the information pair represents date information, the target audit rule for the information pair is the date character consistency audit. Specifically, for the document extraction field representing date information, since the expression of date is not unique, for example, it can be represented by pure numbers, or by plain text, or by text plus numbers, so the information to which it belongs is not unique. The correct target audit rule is to convert the key information of the date type contained in the information pair into a unified representation (such as XX-XX-XX), and then conduct a string consistency audit, and if they are consistent, the audit is passed.

Table 1. Audit rule record sheet

Exemplarily, Table 1 is an audit rule record table. The first column in Table 1 represents the document extraction fields in the table-type business document (that is, the table document extraction fields), and the second column in Table 1 represents the document extraction fields in the text-type business documents (that is, the text document extraction fields) ), and the table document extraction fields and text document extraction fields located in the same row are a set of document extraction fields with the same semantics. The third column corresponds to the audit rules for each group of document extraction fields with the same semantics. Among them, the first row is the header, the target audit rules for the information pairs corresponding to the document extraction fields in the second row and the third row are determined by the above situation 1, the fourth row, the tenth row and the twelfth row are due to only one type of business. There is a corresponding document extraction field in the document, so it does not constitute an information pair, and the target audit rule is not determined for it. The target review rules for the information pairs corresponding to the document extraction fields in the fifth row and the sixth row are determined through the above situation 4, and the target review rules for the information pairs corresponding to the document extraction fields in the seventh row value and the ninth row are determined through the above situation 3; The target audit rule of the information pair corresponding to the document extraction field of eleven lines is determined by the above-mentioned second situation.

Based on the uniqueness of the expression mode of the document extraction field and the type of the expression content, this embodiment flexibly sets target review rules for different document extraction fields, which improves the accuracy of subsequent review of business documents.

It should be noted that, in this embodiment, priorities can be set for the above four situations of determining target audit rules. When a certain document extraction field meets the above multiple conditions at the same time, the one with higher priority can be selected as the final target audit rule. determine the way. For example, if a certain information represents a date for the corresponding document extraction field, both the above-mentioned cases 1 and 4 are satisfied, but since the priority of case 4 is higher than that of case 1, the final audit rule for the target determined for the information pair is satisfied. Review the rule for the target corresponding to Case 4.

S605 , according to the corresponding target review rule of the information pair, perform consistency review on the key information in the group information pair, and determine the review result of at least two business documents according to the review result.

Specifically, this embodiment may perform information consistency audit on each key information in the information pair based on the target audit rule determined for each information pair in S604, if the audit results corresponding to all the information pairs are If the content of the key information is consistent, it is determined that the review result of at least two kinds of business documents is passed; otherwise, the review result of at least two kinds of business documents is not passed.

In the solution of the embodiment of the present disclosure, after obtaining the business document images of at least two kinds of business documents, different information extraction methods are used to extract key information from the corresponding business document images for different types of business document images, and the extracted key information is Divide into multiple information pairs, extract fields from the corresponding documents based on each information pair, select the corresponding target audit rules for the information pair to audit, and then determine the auditing of at least two business documents according to the audit results corresponding to each information pair result. This solution flexibly selects suitable target review rules for information pairs corresponding to different document extraction fields. Compared with a single target review rule, this solution has more accurate review results for key information.

FIG. 7 is a flowchart of a method for reviewing business documents according to an embodiment of the present disclosure. On the basis of the above-mentioned embodiment, the embodiment of the present disclosure further examines how to perform verification on at least two business documents according to the extracted key information. The review process is explained in detail. As shown in FIG. 7 , the review method of the business document provided in this embodiment may include:

S701, acquiring business document images of at least two kinds of business documents.

Wherein, at least two kinds of business documents are associated with the target business content and belong to different document types.

S702: Extract key information of the target business content from the business document image corresponding to the document type according to information extraction methods corresponding to different document types.

S703: Perform seal recognition on at least two business document images to obtain a seal recognition result.

Optionally, in this embodiment, there may be many ways of performing seal recognition on at least two business document images, which is not limited. For example, an implementable manner may be: based on a pre-trained seal recognition model, seal recognition is performed on each business document image, and a seal recognition result corresponding to each business document image is obtained. Another achievable method is to search for an area that satisfies the characteristics of the seal image in each business document image based on the feature matching of the seal image, and then further extract the information of the area to obtain the seal recognition result.

Optionally, the seal recognition result recognized in this embodiment may include, but is not limited to, the shape, position, quantity, and text on the seal, and the like.

S704, at least two kinds of business documents are reviewed according to the seal identification result and the extracted key information.

Optionally, when reviewing at least two types of business documents in this embodiment, it may first check whether the document seal meets the requirements according to the seal identification result, for example, whether the text content of the seal is correct, whether the number of seals is sufficient, and whether the location of the seal is not. Correct, and whether the pattern of the seal is complete, etc. In the case that the seals of the at least two business documents meet the requirements, the at least two business documents are further reviewed according to the extracted key information. Specifically, the at least two business documents are reviewed based on the extracted key information. The process has been described in detail in the above embodiments, and will not be repeated here.

In the solution of the embodiment of the present disclosure, after acquiring the business document images of at least two kinds of business documents, different information extraction methods are used to extract key information from the corresponding business document images for different types of business document images, and the at least two kinds of business document images are extracted with different information extraction methods. The business document image is subjected to seal recognition, and at least two kinds of business documents are reviewed according to the seal recognition result and the extracted key information. This solution audits business documents from two aspects: seal detection and key information detection, which not only ensures the accuracy of audit results, but also improves the comprehensiveness of document content.

FIG. 8A is a flowchart of a method for reviewing business documents according to an embodiment of the present disclosure, and FIG. 8B is a schematic diagram of a display effect of a visual interface provided according to an embodiment of the present disclosure. The embodiment of the present disclosure is based on the above-mentioned embodiment. In the above, further optimization is carried out, and the process of displaying the audit result is added. As shown in Figures 8A-8B, the audit method of the business document provided by this embodiment may include:

S801. Obtain business document images of at least two business documents.

Wherein, at least two kinds of business documents are associated with the target business content and belong to different document types.

S802: Extract key information of the target business content from the business document image corresponding to the document type according to information extraction methods corresponding to different document types.

S803, at least two kinds of business documents are reviewed according to the extracted key information.

S804, displaying the extracted key information, business document images of at least two business documents, and audit results through a visual interface.

Optionally, as shown in FIG. 8B , in this embodiment, after the review of at least two types of business documents is completed, the review results can be displayed to the user through a visual interface. Specifically, the business documents of at least two types of business documents can be displayed through a visual interface. image, and display the key information extracted from the business document image next to the business document image, for example, the key information that belongs to the same information pair can be displayed together in association. In addition, the audit results of at least two types of business documents can also be displayed. Optionally, when displaying the review results of at least two kinds of business documents, if the business document review result is not passed, you can also display the information pairs when each group of displayed information pairs contains multiple key information. The corresponding audit results are convenient for users to review the audit results.

According to the solution of the embodiment of the present disclosure, after obtaining the business document images of at least two kinds of business documents, different information extraction methods are used to extract key information from the corresponding business document images for different types of business document images. information to audit at least two types of business documents, and visualize the audit results, the collected business document images, and the extracted key information to the user. This solution provides a visual interface to show users the audit results and relevant data during the audit process, so that users can understand the audit situation more intuitively.

Optionally, the solution of this embodiment may also support human-machine collaboration to review at least two types of business documents. That is, after displaying the extracted key information, at least two types of document images, and the review result of the image set to be reviewed through the visual interface, it may also include: receiving modification information generated by acting on the visual interface; Describe the review results of the image set to be reviewed.

Specifically, after displaying the extracted key information, at least two types of business document images, and the device's review results of the two types of business documents through the visual interface, the user can, based on the content displayed on the visual interface, review the system's The audit results are reviewed. If the system audit results are found to be incorrect, the relevant components of the visual interface can be triggered to input modification information. Correspondingly, the device for performing document audit in this embodiment can correct the audit results according to the modification information input by the user. , and redisplay the corrected audit results in the visual interface.

Optionally, the modification information input by the user may be the key information for modification and extraction, or the modification review result. If the modification information is the extracted key information, the system may re-execute the review process of the business document based on the modified key information. , and update the audit result on the visual interface. If the modified information is the audit result, the system can directly update the audit result in the visual interface based on the modified information.

On the basis of the above-mentioned visual display of audit results and related information, this scheme further provides a scheme for human-machine collaborative auditing of document information, which further ensures the accuracy of audit results through manual review.

Optionally, on the basis of this embodiment, displaying the extracted key information through a visual interface includes: displaying the extracted key information and the confidence level of the key information through the visual interface. Specifically, when the key information in the business document image is extracted through the table image perception model and the text perception model, the table image perception model and the text perception model will also give the confidence level of each key information. The extracted key information is displayed along with its corresponding confidence. The advantage of this setting is that it is convenient for users to quickly locate the key information that needs to be reviewed based on the confidence displayed on the visual interface, which improves the efficiency of the manual review process.

FIG. 9 is a flowchart of a business document review method provided according to an embodiment of the present disclosure. On the basis of the above-mentioned embodiment, the embodiment of the present disclosure further optimizes, and adds a process of sorting and displaying business document images, such as As shown in FIG. 9 , the review method of the business document provided by this embodiment may include:

S901, acquiring business document images of at least two kinds of business documents.

Wherein, at least two kinds of business documents are associated with the target business content and belong to different document types.

S902: Extract key information of the target business content from the business document image corresponding to the document type according to information extraction methods corresponding to different document types.

S903, at least two kinds of business documents are reviewed according to the extracted key information.

S904, determining the display order of the business document images of each business document.

Wherein, in this embodiment, the display order of the business document images is the number of pages corresponding to the business document images in the business document to which they belong,

Optionally, according to the complexity of the business, the number of pages of the corresponding business document is also different. When the number of pages of the business document is at least two pages, the business document images corresponding to the at least two pages of business documents may be acquired in the order in which they are obtained. It is messy, which affects the continuity of browsing in the subsequent visualization display. Therefore, in order to avoid the situation of displaying the business document images out of order, in this embodiment, before displaying the business document images, the business document images of each business document may be sorted, that is, to determine the number of business document images of each business document. display order.

Specifically, the process of determining the display order of the business document images of each business document in this embodiment is: for each business document, first perform text recognition on each business document image to obtain text information of each business document image, and then Based on the pre-trained sorting model, the text information of each business document image is parsed, and the display order corresponding to each business document image is output. Wherein, the ranking model may be a BERT binary classification model.

Preferably, in this embodiment, a context data set can be constructed according to the text information of each business document image corresponding to each type of business document, and the preset head and tail sequences. Specifically, if there are N business document images, N+1 is constructed. A context data set, each context data set includes the above and the following two parts, wherein, the above in the first context data set is the start sequence in the preset head-to-end sequence, and the following is the text information of N business document images; Thereafter, the text information of each business document image is taken as the above, and the text information of other business document images except the business document image and the end sequence in the preset head and tail sequence are taken as the following. Thereby, N+1 context datasets are constructed. Then, the constructed N+1 context data sets are input into the pre-trained sorting model in turn, and the sorting model can output the display order of each business document image corresponding to the business document according to the input context data set.

Optionally, in order to save the power consumption of the system, in this embodiment, before executing this step, it may first determine whether the number of pages of each business document is more than one page, and if so, execute the operation of this step, otherwise, display it by default. The operation of the subsequent S905 is performed sequentially (ie, the first page is displayed).

S905 , according to the display sequence, display the business document image of each business document in the visual interface, and simultaneously display the extracted key information and the audit results of at least two business documents through the visual interface.

Optionally, when the business document images of the business documents are displayed through the visual interface in this embodiment, the business document images may be sorted based on the display order determined in S904, and then displayed on the visual page in sequence according to the sorting order. The extracted key information and the audit results of at least two business documents are displayed together in the visual interface.

According to the solution of the embodiment of the present disclosure, after obtaining the business document images of at least two kinds of business documents, different information extraction methods are used to extract key information from the corresponding business document images for different types of business document images. information to review at least two types of business documents. At the same time, various business document images are sorted, and based on the sorting results, each business document image is displayed on the visual interface in turn. In addition, the extracted key information and audit results are also displayed on the visual interface. Before displaying the business document images in the visual interface, this solution firstly adjusts the display order of the business document images to ensure that the display order is consistent with the number of pages of the corresponding business document before displaying, which greatly improves the visual interface. Browse business document image coherence.

Exemplarily, FIG. 10 is a schematic block diagram of a process of adjusting parameters and using a ranking model according to an embodiment of the present disclosure. As shown in FIG. 10 , blocks 1001 to 1003 correspond to the parameter adjustment process of the sorting model. Specifically: in the parameter adjustment process, the business document image corresponding to block 1001 may be the number of pages used for training the model. A group of images (that is, training sample images) after the multi-page business document image is shot or scanned and the images are scrambled. The process of inputting the training sample images in block 1001 into block 1002 for data preprocessing can be as follows: Perform text information recognition on the input scrambled business document images, and build a context data set according to the recognition results (the specific construction process has been described in detail in the above embodiment), and the real sorting labels of the group of images . Input the context data set and real ranking labels processed in block 1002 into block 1003, fine-turning the ranking model, and the fine-tuned ranking model can be published for subsequent model inference operations in block 1004 .

Blocks 1001-1002 and 1004-1005 correspond to the use process of the sorting model, specifically: in the actual use process, the business document images corresponding to a certain document type corresponding to block 1001 that need to be sorted, Input to block 1002 for preprocessing, that is, after constructing a context data set, input it to block 1004. Block 1004 parses the input context data set based on the ranking model released in block 1003, and determines the content of each set of context data. The corresponding context sorting results are then transmitted to block 1005, where block 1005 summarizes the sorting results of each group of context data to obtain an overall sorting result of the entire business document image.

11 is a flowchart of a method for reviewing business documents according to an embodiment of the present disclosure. On the basis of the above-mentioned embodiments, the embodiment of the present disclosure further describes how to display business document images of at least two kinds of business documents through a visual interface. The process is explained in detail. As shown in FIG. 11 , the review method of the business document provided in this embodiment may include:

S1101 , acquiring business document images of at least two business documents.

Wherein, at least two kinds of business documents are associated with the target business content and belong to different document types.

S1102: Extract key information of the target business content from the business document image corresponding to the document type according to information extraction methods corresponding to different document types.

S1103: Review at least two business documents according to the extracted key information.

S1104, according to the business document image and the starting position corresponding to the extracted key information, mark the extraction position of the key information in the business document images of at least two kinds of business documents.

Optionally, when the solution extracts key information from the text information in the business document image corresponding to the business document, since the text information includes not only the text content, but also the position of the text, each of the extracted key information is The word also corresponds to its position information in the corresponding business document image, so in this embodiment, for each extracted key information, the position information corresponding to the first word and the last word in the key information can be used as the The starting position and ending position of the key information, and combining the two positions to obtain the starting position of the key information. Then, according to the business document image corresponding to the key information, the determined starting position of the key information is mapped to the corresponding business document image. At this time, the mapping location area can be directly used as the extraction position of the key information in the business document image. Label. Exemplarily, annotations can be made by highlighting, adding text boxes, or changing fonts.

It should be noted that, when extracting the key information of the target business content from the business document image of the corresponding document type, this solution may perform preprocessing operations such as cleaning and symbol replacement processing on the text information in the business document image, which may lead to The text positions in the text information before and after preprocessing are inconsistent, which affects the accuracy of subsequent position annotations. In view of this situation, in this embodiment, before preprocessing the data, a copy of the original text information may be cached, and the position information of each character obtained by text recognition may be recorded. In the process of preprocessing the original text information, preprocessing operations such as cleaning and conformity replacement are performed synchronously in the cached text information, so that the cached text information is consistent with the character position information of the preprocessed text information. When labeling key information, you can cache the text information to determine the starting position of the key information for labeling to ensure the accuracy of the labeling results.

S1105 , displaying the marked business document image on the visual interface, and simultaneously displaying the extracted key information and audit results of at least two business documents through the visual interface.

According to the solution of the embodiment of the present disclosure, after obtaining the business document images of at least two kinds of business documents, different information extraction methods are used to extract key information from the corresponding business document images for different types of business document images. information to review at least two types of business documents. And according to the business document image and the starting position corresponding to the extracted key information, the extraction position of the key information is marked in the business document image of at least two business documents and displayed in the visual interface, and the extracted key information is also displayed on the visual interface. information and corresponding audit results. The business document image displayed in the visual interface of this solution is the business document image marked with the extraction location of key information, which is convenient for users to intuitively and quickly find the extraction location of key information, and then review the accuracy of extraction of key information and improve the subsequent human-machine system. Review the efficiency of text messages.

Optionally, in the embodiment of the present disclosure, a certain key information can be distributed in different lines in the business document. At this time, if the labeling is simply based on the starting position of the key information, the labeling range may be too large or inaccurate. . In order to solve this problem, this embodiment may be: according to the starting position corresponding to the extracted key information, determine whether the key information belongs to cross-line extraction; according to the determination result and the text image corresponding to the key information, in at least two types of document images Label the extraction location of key information.

Specifically, this embodiment can determine whether the key information belongs to cross-line extraction according to the starting position corresponding to the extracted key information. The position information of all characters at the end position, and then judge whether there is a sudden decrease in the coordinates of the X-axis (that is, the direction of the line in which the characters in the document are located) coordinates of the obtained position information. If it exists, it means that the key information belongs to the cross-line extraction, and the corresponding position of the mutation point is the cross-line position. In this case, the cross-line position is used as the dividing point, and the key information is divided into at least two paragraphs for labeling. For example, if a certain key information has a cross-line position, the key information is divided into two sections for labeling, that is, the point from the start position of the key information to the previous position of the cross-line position is taken as the first paragraph, and the cross-line position to the key The end position of the information is taken as the second segment, and then for each segment, the extraction location is marked in the corresponding business document image in the manner described in the foregoing embodiment. The advantage of this setting in this embodiment is that the situation that the key information is displayed across lines is considered, and the accuracy of the location of the key information marked in the business document image is further improved.

Exemplarily, FIG. 12A is a flowchart of a work order document review provided by an embodiment of the present disclosure, and FIG. 12B is a flowchart of a content working principle of a perception engine provided by an embodiment of the present disclosure. Next, this embodiment takes the review of the business document in the work order as an example to introduce the review method of the service document in this embodiment. It should be noted that in the field of financial deposits, each deposit transaction can be regarded as a work order, and each work order will generate at least two basic materials, namely the approval form and the deposit transaction contract. The review of the work order in this embodiment refers to whether the specific record contents of the transaction parties, value date, maturity date, deposit period, deposit amount, interest rate, interest calculation method and other fields recorded in the approval form and the transaction contract are consistent.

Specifically, as shown in FIG. 12A, the work order image in block 1201 (that is, including the approval form image and the transaction contract image) is input into the perception engine in block 1202. The specific working principle of the perception engine is shown in FIG. 12B. , which is used to obtain the text information of the work order image (including: character information of each text box, the recognition confidence of each character, and the coordinate information of the characters) by using the general OCR technology, and obtain the seal information of the work order image through seal recognition ( Including: whether the image contains seals, character information of each seal, etc.), obtain the form information of wired or wireless forms in the work order image through form recognition technology (including: wired or wireless, row and column information, cell character information, cell character recognition confidence).

After obtaining the perception result in block 1202, the perception result can be input to the classifier in block 1203, the classifier can be obtained after adjusting the parameters of the pre-training model (ERNIE), and the classifier will be based on the input perception result. , gives the document type corresponding to each perception result, that is, whether the work order image corresponding to each perception result is an image of an approval form, an image of a transaction contract, or an image of other types. Then, according to the classification result, the approval form image and its corresponding related information (such as the relevant information perceived in block 1202) are input into the form image information extraction part of the cognitive engine in block 1204 to perform key extraction from the approval form image. information manipulation. The transaction contract image and its corresponding related information (such as the related information perceived in 1202) are input into the text image sorting part of the cognitive engine in block 1204, and the transaction contract images are sorted by the text image sorting part and then input into the text image information Extraction part, the operation of extracting key information from the sorted transaction contract images is performed by the text image information extraction part. It should be noted that, in the work order document review scenario, the number of pages of the approval form is usually one page, while the number of pages of the transaction contract is usually more, so this embodiment only performs sorting processing on the images of the transaction contract.

Block 1204 will transmit all the key information extracted by it to block 1205, and the review module in block 1205 will review the approval form and transaction in the work order according to all the key information extracted in block 1204. The contract is reviewed for content consistency, and the review results are displayed through a visual interface, and users can collaboratively review on the visual interface, thereby realizing the process of human-machine collaborative review in block 1206 . For the specific review and display process, please refer to the description of the above embodiment.

This embodiment integrates neural network-based classification technology, neural network-based table image information extraction technology, and neural network-based text image information extraction technology, realizes automatic extraction of key information, and integrates human-machine collaboration technology to assist auditors Improve audit efficiency in financial deposit scenarios.

FIG. 13 is a schematic structural diagram of an apparatus for reviewing business documents provided according to an embodiment of the present disclosure. The embodiment of the present disclosure is applicable to the case of performing information consistency review on business documents of at least two document types associated with the same target business content . For example, it is applicable to the case of conducting information consistency review on business forms and business contracts related to the same target business content. The apparatus may be configured in an electronic device installed with a document intelligence application, and implemented by software and/or hardware, and the apparatus may implement the business document review method of any embodiment of the present disclosure. As shown in FIG. 13 , the apparatus 1300 for reviewing the document image includes:

A document image acquisition module 1301, configured to acquire business document images of at least two types of business documents; wherein, the at least two types of business documents are associated with target business content and belong to different document types;

The key information extraction module 1302 is configured to extract the key information of the target business content from the business document image corresponding to the document type according to the information extraction methods corresponding to different document types;

The business document review module 1303 is configured to review the at least two types of business documents according to the extracted key information.

According to the solution of the embodiment of the present disclosure, after acquiring the business document images of at least two kinds of business documents, different information extraction methods are used for different types of business document images to extract key information from the corresponding business document images, and then according to the extracted key information to review at least two types of business documents. For business documents of different document types, this solution automatically extracts key information in different ways, improves the accuracy and efficiency of key information extraction, and provides a guarantee for quickly and accurately completing the review of business documents of different document types.

Further, the key information extraction module 1302 includes:

an input sequence construction unit, configured to construct an input sequence according to a preset spacer, a document extraction field and text information in a business document image of the text type when the document type is a text type;

a model-aware unit, configured to determine, according to the input sequence, an extraction start position corresponding to the document extraction field, and a confidence level of the extraction start position through a text-image-aware model;

an extraction information determination unit, configured to determine the extraction information corresponding to the document extraction field and the confidence level of the extraction information according to the extraction start position and the confidence level of the extraction start position;

A key information determination unit, configured to determine the key information of the target business content in the text-type business document image according to the extraction information corresponding to the document extraction field and the confidence level of the extracted information.

Further, the input sequence construction unit is specifically used for:

Preprocessing the text information in the text-type business document image to obtain preprocessed text; wherein, the preprocessing includes at least one of cleaning processing, symbol replacement processing, and format conversion processing;

An input sequence is constructed from the preprocessed text, preset spacers, and document extraction fields.

Further, the key information extraction module 1302 is specifically used for:

In the case where the document type is a form type, the business document image of the form type is divided into blocks to obtain a sequence of image blocks, and the image features of the image blocks in the sequence of image blocks and the location of the image blocks in the image block sequence are determined. the block position in the business document image, and the block sequence number of the image block in the image block sequence;

Perform word quantization processing on the text information in the business document image to obtain a word sequence, and determine the word feature of the quantified word in the word sequence, the word position of the quantified word in the business document image, and the the word number of the quantified word in the word sequence;

According to the image feature, block position and block serial number of the image block in the image block sequence, and the word feature, word position and word serial number of the quantized word in the word sequence, determine the model input feature;

Through the form image perception model, the key information of the target business content is extracted from the form type business document image according to the model input feature.

Further, the key information extraction module 1302 is also specifically used for:

In the case that the document type is a table type, extract the first candidate information of the target business content from the business document image of the table type by using a table image perception model;

extracting the second candidate information of the target business content from the table-type business document image according to the structured information extraction logic;

According to the first candidate information and the second candidate information, key information of the target business content in the form-type business document image is determined.

Further, the business document review module 1303 includes:

An information pair dividing unit, configured to divide the extracted key information into at least one information pair according to the business document image and document extraction field corresponding to the extracted key information, wherein the document extraction field corresponding to the key information in the same information pair have the same semantics and are taken from business document images of different document types;

an audit rule determination unit, configured to extract fields from the corresponding document according to the information pair, and determine a target audit rule for the information pair;

The information auditing unit is configured to perform consistency auditing on the key information in the information pair according to the corresponding target auditing rules of the information pair, and determine the auditing results of the at least two business documents according to the auditing results.

Further, the audit rule determination unit specifically implements at least one of the following:

In the case that the information expression mode of the corresponding document extraction field of the information pair is unique, the target review rule of the information pair is character consistency review;

In the case that the information expression mode of the corresponding document extraction field of the information pair is not unique, the target review rule of the information pair is semantic similarity review;

In the case that the information pair corresponds to the document extraction field representing numerical information, the target audit rule for the information pair is numerical character consistency audit;

In the case that the document extraction field corresponding to the information pair represents date information, the target review rule of the information pair is the date character consistency review.

Further, the business document review module 1303 is specifically used for:

Perform seal recognition on at least two business document images to obtain seal recognition results;

The at least two business documents are reviewed according to the seal recognition result and the extracted key information.

Further, the verification device 1300 of the document image also includes:

The display module is used to display the extracted key information, the business document images of the at least two business documents, and the audit result through a visual interface.

Further, the display module includes:

a sorting unit, used to determine the display order of the business document images of each business document;

The image display unit is configured to display the business document image of each business document in the visual interface according to the display sequence.

Further, the display module includes:

a location labeling unit, configured to label the extraction location of the key information in the business document images of the at least two business documents according to the business document image and the starting location corresponding to the extracted key information;

The image display unit is used to display the marked business document image in the visual interface.

Further, the position labeling unit is specifically used for:

According to the starting position corresponding to the extracted key information, determine whether the key information belongs to cross-line extraction;

According to the determination result and the business document image corresponding to the key information, the extraction position of the key information is marked in the business document image of the at least two types of business documents.

Further, the display module includes:

The key information display unit is used to display the extracted key information and the confidence level of the key information through a visual interface.

Further, the document image review device 1300 further includes:

The information receiving module is used to receive the modification information generated by acting on the visual interface;

An interface update module, configured to update the audit results of the at least two business documents on the visual interface according to the modification information.

The above product can execute the method provided by any embodiment of the present disclosure, and has corresponding functional modules and beneficial effects for executing the method.

In the technical solution of the present disclosure, the business documents of any document type involved, as well as the acquisition, storage and application of business document images of business documents, etc., all comply with relevant laws and regulations, and do not violate public order and good customs.

In the technical solution of the present disclosure, the approval form and transaction contract involved are not deposit work orders for a specific user, and cannot reflect the personal information of a specific user.

According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

14 shows a schematic block diagram of an example electronic device 1400 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are by way of example only, and are not intended to limit implementations of the disclosure described and/or claimed herein.

As shown in FIG. 14 , the device 1400 includes a computing unit 1401 that can be executed according to a computer program stored in a read only memory (ROM) 1402 or a computer program loaded from a storage unit 1408 into a random access memory (RAM) 1403 Various appropriate actions and handling. In the RAM 1403, various programs and data necessary for the operation of the device 1400 can also be stored. The computing unit 1401 , the ROM 1402 , and the RAM 1403 are connected to each other through a bus 1404 . An input/output (I/O) interface 1405 is also connected to bus 1404 .

Various components in the device 1400 are connected to the I/O interface 1405, including: an input unit 1406, such as a keyboard, mouse, etc.; an output unit 1407, such as various types of displays, speakers, etc.; a storage unit 1408, such as a magnetic disk, an optical disk, etc. ; and a communication unit 1409, such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 1409 allows the device 1400 to exchange information/data with other devices through a computer network such as the Internet and/or various telecommunication networks.

Computing unit 1401 may be various general-purpose and/or special-purpose processing components with processing and computing capabilities. Some examples of computing units 1401 include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various specialized artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processing processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1401 executes the various methods and processes described above, such as the review method of business documents. For example, in some embodiments, the review method of a business document may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 1408 . In some embodiments, part or all of the computer program may be loaded and/or installed on device 1400 via ROM 1402 and/or communication unit 1409 . When the computer program is loaded into RAM 1403 and executed by computing unit 1401, one or more steps of the above-described review method for business documents may be performed. Alternatively, in other embodiments, the computing unit 1401 may be configured to perform the review method of the business document by any other suitable means (eg, by means of firmware).

Various implementations of the systems and techniques described herein above can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips system (SOC), complex programmable logic device (CPLD), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include being implemented in one or more computer programs executable and/or interpretable on a programmable system including at least one programmable processor that The processor, which may be a special purpose or general-purpose programmable processor, may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device an output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer or other programmable data processing apparatus, such that the program code, when executed by the processor or controller, performs the functions/functions specified in the flowcharts and/or block diagrams. Action is implemented. The program code may execute entirely on the machine, partly on the machine, partly on the machine and partly on a remote machine as a stand-alone software package or entirely on the remote machine or server.

In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with the instruction execution system, apparatus or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), fiber optics, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.

To provide interaction with a user, the systems and techniques described herein may be implemented on a computer having a display device (eg, a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user ); and a keyboard and pointing device (eg, a mouse or trackball) through which a user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (eg, visual feedback, auditory feedback, or tactile feedback); and can be in any form (including acoustic input, voice input, or tactile input) to receive input from the user.

The systems and techniques described herein may be implemented on a computing system that includes back-end components (eg, as a data server), or a computing system that includes middleware components (eg, an application server), or a computing system that includes front-end components (eg, a user's computer having a graphical user interface or web browser through which a user may interact with implementations of the systems and techniques described herein), or including such backend components, middleware components, Or any combination of front-end components in a computing system. The components of the system may be interconnected by any form or medium of digital data communication (eg, a communication network). Examples of communication networks include: Local Area Networks (LANs), Wide Area Networks (WANs), blockchain networks, and the Internet.

A computer system can include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server arises by computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or a cloud host. It is a host product in the cloud computing service system to solve the traditional physical host and VPS services, which are difficult to manage and weak in business scalability. defect. The server can also be a server of a distributed system, or a server combined with a blockchain.

Artificial intelligence is the study of making computers to simulate certain thinking processes and intelligent behaviors of people (such as learning, reasoning, thinking, planning, etc.), both hardware-level technology and software-level technology. AI hardware technologies generally include technologies such as sensors, dedicated AI chips, cloud computing, distributed storage, and big data processing; AI software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, and machine learning/depth Learning technology, big data processing technology, knowledge graph technology and other major directions.

Cloud computing refers to accessing elastically scalable shared physical or virtual resource pools through the network. Resources can include servers, operating systems, networks, software, applications and storage devices, etc., and can be self-service on demand and on demand. A technical system for deploying and managing resources in a way. Through cloud computing technology, it can provide efficient and powerful data processing capabilities for artificial intelligence, blockchain and other technical applications and model training.

It should be understood that steps may be reordered, added or deleted using the various forms of flow shown above. For example, the steps described in the present disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, no limitation is imposed herein.

The above-mentioned specific embodiments do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present disclosure should be included within the protection scope of the present disclosure.

Claims (31)

1. A business document auditing method comprises the following steps:

acquiring service document images of at least two service documents; wherein, the at least two service documents are all associated with the target service content and belong to different document types;

extracting key information of the target business content from a business document image corresponding to the document type according to information extraction modes corresponding to different document types;

and auditing the at least two service documents according to the extracted key information.

2. The method according to claim 1, wherein the extracting key information of the target service content from the service document image corresponding to the document type according to the information extraction manner corresponding to the different document types includes:

under the condition that the document type is a text type, constructing an input sequence according to a preset spacer, a document extraction field and text information in a service document image of the text type;

determining an extraction starting position corresponding to the document extraction field and a confidence coefficient of the extraction starting position according to the input sequence through a text image perception model;

according to the extraction starting position and the confidence coefficient of the extraction starting position, determining extraction information corresponding to the document extraction field and the confidence coefficient of the extraction information;

and determining key information of the target business content in the business document image of the text type according to the extraction information corresponding to the document extraction field and the confidence coefficient of the extraction information.

3. The method of claim 2, wherein the constructing an input sequence according to a preset spacer, a document extraction field and text information in the text type service document image comprises:

preprocessing the text information in the service document image of the text type to obtain a preprocessed text; wherein the pre-processing comprises: at least one of a cleaning process, a symbol replacement process, and a format conversion process;

and constructing an input sequence according to the preprocessed text, the preset spacers and the document extraction field.

4. The method according to claim 1, wherein the extracting key information of the target service content from the service document image corresponding to the document type according to the information extraction manner corresponding to the different document types includes:

under the condition that the document type is the table type, partitioning the service document image of the table type to obtain an image block sequence, and determining the image characteristics of the image blocks in the image block sequence, the block positions of the image blocks in the service document image and the block sequence numbers of the image blocks in the image block sequence;

performing word quantization processing on text information in the service document image to obtain a word sequence, and determining word characteristics of quantized words in the word sequence, word positions of the quantized words in the service document image, and word sequence numbers of the quantized words in the word sequence;

determining model input characteristics according to the image characteristics, the block positions and the block serial numbers of the image blocks in the image block sequence and the word characteristics, the word positions and the word serial numbers of the quantization words in the word sequence;

and extracting key information of the target business content from the business document image of the form type according to the input characteristics of the form image perception model.

5. The method according to claim 1, wherein the extracting key information of the target service content from the service document image corresponding to the document type according to the information extraction manner corresponding to the different document types includes:

under the condition that the document type is a table type, extracting first candidate information of the target business content from a business document image of the table type through a table image perception model;

extracting second candidate information of the target business content from the business document image of the table type according to structured information extraction logic;

and determining key information of the target business content in the business document image of the form type according to the first candidate information and the second candidate information.

6. The method of claim 1, wherein the auditing the at least two business documents according to the extracted key information comprises:

dividing the extracted key information into at least one information pair according to the business document image and the document extraction field corresponding to the extracted key information, wherein the semantics of the document extraction fields corresponding to the key information in the same information pair are the same and are taken from business document images of different document types;

extracting fields according to the documents corresponding to the information pairs, and determining target auditing rules for the information pairs;

and performing consistency audit on the key information in the information pair according to the target audit rule corresponding to the information pair, and determining the audit results of the at least two service documents according to the audit results.

7. The method of claim 6, wherein the determining a target audit rule for the information pair according to the document extraction field corresponding to the information pair comprises at least one of:

under the condition that the information expression mode of the document extraction field corresponding to the information pair is unique, the target auditing rule of the information pair is character consistency auditing;

under the condition that the information expression mode of the document extraction field corresponding to the information pair is not unique, the target auditing rule of the information pair is semantic similarity auditing;

under the condition that the document extraction field corresponding to the information represents numerical information, the target auditing rule of the information pair is numerical character consistency auditing;

and under the condition that the document extraction field corresponding to the information represents date information, the target auditing rule of the information pair is date character consistency auditing.

8. The method of claim 1, wherein the auditing the at least two business documents according to the extracted key information comprises:

carrying out seal identification on at least two service document images to obtain seal identification results;

and auditing the at least two service documents according to the seal identification result and the extracted key information.

9. The method according to any one of claims 1-8, further including:

and displaying the extracted key information, the service document images and the auditing results of the at least two service documents through a visual interface.

10. The method of claim 9, wherein presenting the business document image of at least two business documents through a visualization interface comprises:

determining the display sequence of the business document images of each business document;

and displaying the business document image of each business document in a visual interface according to the display sequence.

11. The method of claim 9, wherein presenting the business document image of at least two business documents through a visualization interface comprises:

and marking the extraction position of the key information in the service document images of the at least two service documents according to the service document images and the initial positions corresponding to the extracted key information, and displaying the marked service document images in a visual interface.

12. The method of claim 11, wherein labeling the extraction positions of the key information in the business document images of the at least two business documents according to the business document images and the starting positions corresponding to the extracted key information comprises:

determining whether the key information belongs to cross-row extraction or not according to the initial position corresponding to the extracted key information;

and marking the extraction position of the key information in the service document images of the at least two service documents according to the determination result and the service document images corresponding to the key information.

13. The method of claim 9, wherein presenting the extracted key information through a visualization interface comprises:

displaying the extracted key information and the confidence coefficient of the key information through a visual interface.

14. The method according to any one of claims 9-13, further including:

receiving modification information generated by acting on the visual interface;

and updating the auditing results of the at least two business documents on the visual interface according to the modification information.

15. An auditing device for business documents, comprising:

the document image acquisition module is used for acquiring service document images of at least two service documents; the at least two business documents are associated with the target business content and belong to different document types;

the key information extraction module is used for extracting the key information of the target service content from the service document image of the corresponding document type according to the information extraction modes corresponding to different document types;

and the business document auditing module is used for auditing the at least two business documents according to the extracted key information.

16. The apparatus of claim 15, wherein the key information extraction module comprises:

an input sequence construction unit, configured to construct an input sequence according to a preset spacer, a document extraction field, and text information in a service document image of the text type when the document type is a text type;

the model sensing unit is used for determining an extraction initial position corresponding to the document extraction field and a confidence coefficient of the extraction initial position according to the input sequence through a text image sensing model;

an extraction information determining unit, configured to determine, according to the extraction start position and the confidence level of the extraction start position, extraction information corresponding to the document extraction field and the confidence level of the extraction information;

and the key information determining unit is used for determining the key information of the target business content in the business document image of the text type according to the extraction information corresponding to the document extraction field and the confidence coefficient of the extraction information.

17. The apparatus according to claim 16, wherein the input sequence construction unit is specifically configured to:

preprocessing the text information in the service document image of the text type to obtain a preprocessed text; wherein the pre-processing comprises: at least one of a cleaning process, a symbol replacement process, and a format conversion process;

and constructing an input sequence according to the preprocessed text, the preset spacers and the document extraction field.

18. The apparatus according to claim 15, wherein the key information extraction module is specifically configured to:

under the condition that the document type is the table type, partitioning the service document image of the table type to obtain an image block sequence, and determining the image characteristics of the image blocks in the image block sequence, the block positions of the image blocks in the service document image and the block sequence numbers of the image blocks in the image block sequence;

performing word quantization processing on text information in the service document image to obtain a word sequence, and determining word characteristics of quantized words in the word sequence, word positions of the quantized words in the service document image, and word sequence numbers of the quantized words in the word sequence;

determining model input characteristics according to the image characteristics, the block positions and the block serial numbers of the image blocks in the image block sequence and the word characteristics, the word positions and the word serial numbers of the quantization words in the word sequence;

and extracting key information of the target business content from the business document image of the form type according to the input characteristics of the form image perception model.

19. The apparatus of claim 15, wherein the key information extraction module is further specifically configured to:

under the condition that the document type is a table type, extracting first candidate information of the target business content from a business document image of the table type through a table image perception model;

extracting second candidate information of the target business content from the business document image of the form type according to structured information extraction logic;

and determining key information of the target service content in the form type service document image according to the first candidate information and the second candidate information.

20. The apparatus of claim 15, wherein the business document auditing module comprises:

the information pair dividing unit is used for dividing the extracted key information into at least one information pair according to the business document image and the document extraction field corresponding to the extracted key information, wherein the semantics of the document extraction fields corresponding to the key information in the same information pair are the same and are taken from the business document images of different document types;

the auditing rule determining unit is used for extracting fields from the document corresponding to the information pair and determining a target auditing rule for the information pair;

and the information auditing unit is used for performing consistency auditing on the key information in the information pair according to the target auditing rule corresponding to the information pair, and determining the auditing results of the at least two service documents according to the auditing results.

21. The apparatus according to claim 20, wherein the audit rule determining unit specifically performs at least one of:

under the condition that the information expression mode of the corresponding document extraction field of the information pair is unique, the target auditing rule of the information pair is character consistency auditing;

under the condition that the information expression mode of the document extraction field corresponding to the information pair is not unique, the target auditing rule of the information pair is semantic similarity auditing;

under the condition that the document extraction field corresponding to the information pair represents numerical information, the target auditing rule of the information pair is numerical character consistency auditing;

and under the condition that the document extraction field corresponding to the information represents date information, the target auditing rule of the information pair is date character consistency auditing.

22. The apparatus according to claim 15, wherein the service document auditing module is specifically configured to:

carrying out seal identification on at least two service document images to obtain seal identification results;

and auditing the at least two service documents according to the seal identification result and the extracted key information.

23. The apparatus of any of claims 15-22, further comprising:

and the display module is used for displaying the extracted key information, the business document images of the at least two business documents and the auditing result through a visual interface.

24. The apparatus of claim 23, wherein the presentation module comprises:

the sequencing unit is used for determining the display sequence of the business document images of each business document;

and the image display unit is used for displaying the business document image of each business document in a visual interface according to the display sequence.

25. The apparatus of claim 23, wherein the presentation module comprises:

the position marking unit is used for marking the extraction positions of the key information in the business document images of the at least two business documents according to the business document images and the initial positions corresponding to the extracted key information;

and the image display unit is used for displaying the marked business document image in a visual interface.

26. The apparatus according to claim 25, wherein the location labeling unit is specifically configured to:

determining whether the key information belongs to cross-row extraction or not according to the initial position corresponding to the extracted key information;

and marking the extraction position of the key information in the service document images of the at least two service documents according to the determination result and the service document images corresponding to the key information.

27. The apparatus of claim 23, wherein the presentation module comprises:

and the key information display unit is used for displaying the extracted key information and the confidence coefficient of the key information through a visual interface.

28. The apparatus of any of claims 23-27, further comprising:

the information receiving module is used for receiving modification information generated by acting on the visual interface;

and the interface updating module is used for updating the auditing results of the at least two service documents on the visual interface according to the modification information.

29. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method of auditing a business document according to any of claims 1-14.

30. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform a method of auditing a business document according to any of claims 1-14.

31. A computer program product comprising a computer program which, when executed by a processor, implements a method of auditing business documents according to any of claims 1-14.

Images