RU2597163C2 - Comparing documents using reliable source - Google Patents

Abstract

FIELD: information technology.

SUBSTANCE: group of inventions relates to techniques for recognition of electronic documents. Disclosed is a method for comparing document images, implemented by the computing device containing a processor. Method includes a step of obtaining an image of the first document from the standard document and a corresponding image of the second document from the compared document. Further, according to the method, marking of the obtained images of the first and second documents is determined. Method also includes the first procedure of optical character recognition of the obtained images of the first and second documents and forming a standard dictionary, the standard dictionary contains words from the text block from the image of the first document.

EFFECT: high accuracy of character recognition by converting the compared image of the document based on marking the standard document image.

21 cl, 6 dwg

Description

FIELD OF TECHNOLOGY

[0001] The present invention generally relates to computing devices for processing electronic documents, and in particular to comparison and improving the quality of documents.

BACKGROUND

[0002] In document management, paperwork and in many aspects of doing business, the task is to compare two or more documents containing text or other information in order to determine their identity or to find differences between the analyzed documents. The solution to this problem often involves the production of an electronic document by scanning or obtaining an image of a paper document in another way, as well as performing optical character recognition (OCR) to obtain the text of the document. When comparing documents, false differences often arise, associated not with the actual difference between the documents, but with inaccuracies in the character recognition procedure.

BRIEF DESCRIPTION OF THE DRAWINGS

[0003] The present invention is illustrated by way of examples, and not by limitation. For its full understanding, the following description of preferred embodiments should be considered in conjunction with the drawings, in which:

[0004] FIG. 1 is a block diagram of one embodiment of a computing device operating in accordance with one or more aspects of the present invention;

[0005] FIG. 2 illustrates an example of a reference document and a comparable document in accordance with one or more aspects of the present invention;

[0006] FIG. 3 illustrates examples of improving the quality of documents in accordance with one or more aspects of the present invention.

[0007] In FIG. 4 is a flowchart of an example illustrating a method 400 for improving quality and comparing documents in accordance with one or more aspects of the present invention;

[0008] In FIG. 5 illustrates an example of improving quality and comparing three documents in accordance with one or more aspects of the present invention;

[0009] In FIG. 6 is a more detailed diagram of an illustrative example of a computing device that implements the methods of the present invention.

DESCRIPTION OF PREFERRED EMBODIMENTS

[00010] The present invention relates to methods and systems for comparing documents using a reference document. A further reference to the term “document” in the present description of the invention relates to an electronic document, which can be obtained, for example, by digitizing a paper document, and which contains an electronic image of a paper document. The document processing described below relates to image processing of documents. The reference document can be an initial or earlier version of the document, and it can be considered a reliable source, since it can contain a high-quality text layer with almost no visual artifacts that adversely affect character recognition (OCR). The document comparison system can use a reference document to create special dictionaries and convert the markup of a document associated with it. Each reference dictionary contains a list of words from a specific part of the document (for example, a fragment of text or page). Reference dictionaries and markup conversions can be used to improve the quality of character recognition procedures. This can lead to a reduction in the number of false differences that may arise due to inaccuracies in character recognition, and not due to actual differences between documents.

[00011] The document comparison system may convert the markup of the document to be compared, correlating it with the markup of the reference document. Comparison of layouts may include selecting reference points in each document based on reference points in the image (for example, words or objects) and processing reference points to determine image conversion. The document comparison system can then convert (for example, using a linear transformation) the corresponding document to correct distortions and restore its markup.

[00012] In one example implementation, the reference document may be a legally binding contract that has been compiled in electronic format and subsequently converted to Portable Document Format (PDF). The document to be compared may be a version of a legally binding contract that has been modified, printed, signed and scanned. Scanning a document can change the document by introducing, for example, distortion of perspective. This distortion can adversely affect character recognition and can lead to recognition errors. The method of the present invention introduces an iterative comparison, which may include several character recognition operations and document conversion to reduce false differences.

[00013] Various aspects of the above methods and systems are described in detail later in this document by way of examples, and not for limitation.

[00014] In FIG. 1 is a block diagram of one illustrative example of a computing device 100 operating in accordance with one or more aspects of the present invention. In illustrative examples, computing device 100 may be represented by various computing devices, including a tablet computer, smartphone, laptop, or desktop computer.

[00015] Computing device 100 may include a processor 110 connected to the system bus 120. Other devices connected to the system bus 120 may include a memory 130, a display 135, a keyboard 140, an optical input device 160, and one or more network interfaces 165 The term “connected” in this document means an electrical connection and / or data exchange through one or more interface devices, adapters, and the like.

[00016] In various illustrative examples, the processor 110 may be represented by one or more processing devices, such as universal and / or specialized processors. The memory 130 may be one or more volatile memory devices (e.g., RAM chips), one or more non-volatile memory devices (e.g., ROM or EEPROM chips) and / or one or more memory devices (e.g., optical or magnetic disks). The optical input device 160 may be a scanner or a camera designed to capture light reflected from objects located in its field of view. An example of a computing device that implements aspects of the present invention will be discussed in more detail below in the description of FIG. 6.

[00017] In memory 130, instructions of the comparison module 190 may be stored to perform optical character recognition. In some implementations of the invention, comparison module 190 may perform document comparison and conversion methods in accordance with one or more aspects of the present invention. As an illustrative example, the comparison module 190 may be implemented as a function that is called using the user interface of another application. In addition, the comparison module 190 may be implemented as a standalone application. In one embodiment, the memory 130 may store several separate documents, which are then compared.

[00018] In FIG. 2 illustrates an example of a reference document 210 and a comparison document 220, which can be compared with each other using a comparison module 190 operating in a computing device 100 in accordance with one or more aspects of the present invention. The reference document 210 may include a graphic element 212 and a text element 214, and the compared document may contain similar graphic and text elements that could be distorted when digitizing the document (for example, when scanning or receiving an image). These distortions can, in particular, represent a shift or rotation and / or visual artifacts 222: for example, rare black and white distortions (such as so-called salt-and-pepper noise or Gaussian noise).

[00019] Reference document 210 and reference document 220 may be similar to each other. For example, reference document 220 may be a different version (eg, an earlier or later version) of reference document 210. Reference document 210 may be a reliable source with a text layer and markup of higher quality and with fewer visual artifacts 222 than in the compared document 210. Visual artifacts 222 may include noise (eg, salt-and-pepper noise), distortion, defocus, glare, and / or blur. In one example implementation, the reference document 210 may be a digitized document with high quality and image accuracy with a minimum number of visual artifacts. In another example implementation, the reference document 210 may be a document with a fixed markup (for example, in PDF or TIFF format), converted electronically from a document with editable markup (for example, DOC, ODF, RTF or TXT) without digitization. Electronic conversion may include converting vector graphics into a bitmap (for example, pixels or dots) for subsequent digital image processing, for example, similar to rasterization or ray tracing.

[00020] As shown in FIG. 2, the comparison module may analyze OCR documents and procedures to evaluate reference document 210 and determine reference markup 230 and reference dictionaries 240A-C. The analysis of the document may determine a reference markup 230, which may include locations (eg, coordinates), parameters and structure of document elements, such as a graphic element 212 and a text element 214. The structure of the elements may include text blocks 232A-C, columns 234A- B, as well as tables, headers and footers, and other markup elements. In one example, reference markup 230 and reference dictionaries 240A-C are constructed for each image of a document (for example, for each page of a reference document).

[00021] Reference dictionaries 240A-C may be constructed based on an OCR procedure and / or document analysis. Reference dictionaries 240A-C may include a list of words contained in one or more parts of reference document 210, and can be used to improve the quality and / or validation of the results of optical character recognition in the compared document 220. Each dictionary may contain a list of words that includes words only from a certain part (for example, page, fragment of text) of the reference document. As shown in FIG. 2, the reference dictionaries 240A-C may comprise lists of words from text blocks 232A-C, respectively. In another example implementation, a reference dictionary may be created for a specific page, column, table, paragraph, cell, or other part of the document.

[00022] Alternatively, the reference dictionaries 240A-C may include a set of characters (eg, alphabet, numbers, and symbols). The character set can be obtained as a result of OCR and / or document analysis. The character set can be limited only by the characters present in a certain part of the reference document 210. This restriction can be used for the second optical image recognition procedure of documents that contain, for example, Chinese, Japanese and / or Korean characters (CJK characters), because it is limited a character set can include only a subset of all characters, for example, 200 characters from 40,000 available characters. Characters included in a limited character set can be encoded using any standard text encoding such as Unicode (e.g. UTF-8, UTF-16), ASCII, Japanese Shift-JIS encoding, or other similar text encoding.

[00023] In one example implementation, the reference dictionaries 240A-C may include a limited list of words and a limited set of characters, each of which can be generated based on a specific part of the image of the reference document. They can be used to improve the quality of the subsequent (eg, second) character recognition procedure in the document 220 being compared.

[00024] FIG. 3 is an example illustrating quality improvement and comparison procedures. The example contains a comparable document 220, which is represented by images 324A-C at various stages of improving quality. The quality improvement procedure may involve preprocessing the image 350, comparing 352, converting the image 360, and calculating the difference 362.

[00025] Image pre-processing 350 may modify image 324A to obtain image 324B by performing one or more digital image processing steps to improve image quality. It may include binarization, noise reduction and / or other similar image processing methods. Binarization can change the image and convert it to a binarized image (for example, a two-level image), in which for each pixel, only two color values are used, usually black and white. Image binarization can be performed for grayscale images or color images using a specific algorithm, for example, threshold image binarization.

[00026] Threshold binarization of an image may include setting threshold values for a pixel or group of pixels and evaluating pixel values to determine which of the two values will be assigned to that pixel. In one example implementation, the Otsu method can be used for threshold binarization of an image based on clustering. This algorithm can calculate the optimal threshold value dividing several classes so that their joint spread (dispersion within the class) is minimal. It can also be used to perform multi-level threshold separation using the multi-Otsu method.

[00027] Image pre-processing 350 may also include noise reduction, which may include the removal and / or reduction of visual artifacts 222. Noise reduction can be performed using noise reduction techniques such as linear or non-linear anti-aliasing filters and / or anisotropic diffusion. Some other image pre-processing methods may include removing spots from the image, filtering textures, cropping page margins, aligning, changing page orientation, straightening lines, correcting perspectives, removing motion blur, horizontal separation of adjacent images, splitting a scanned page with multiple slides or business cards on individual images and / or other similar methods.

[00028] When the preliminary processing of the image 350 is completed, the computing device 100 may perform an analysis to determine the markup and / or text of the compared document 220. The analysis may be performed in several steps, one step may include analysis of the document to determine the markup, while the other the step may include performing OCR. In the OCR step, the markup defined in the analysis of the document may be used for text recognition. In one implementation example, document analysis may be performed first, and after the analysis of the document is completed, the OCR step may be performed, and in another implementation example, they may be executed in parallel.

[00029] The analysis can be performed using any document analysis tool or character recognition method. In one implementation example, hypotheses regarding the markup of the source document may be put forward when analyzing documents. Such hypotheses may include one or more hypotheses regarding the classification and / or attributes of various elements of the document located in one or more areas of the document (for example, in rectangular blocks). For example, in relation to a specific area in a document, the OCR system tests the following hypotheses: the area contains text; the area contains a drawing; the area contains a table; the area contains a diagram; and the area contains a screenshot image. Further, document analysis tools and OCR methods can be used to obtain document markup.

[00030] Computing device 100 may also perform a comparison 352 of the compared document 220 and the reference document 210 (ie, a reliable source) to determine images that correspond to each other (eg, corresponding pages). Such a comparison may be an iterative comparison in which markup and / or text documents are compared. Comparison of markings may include an analysis of the similarities and differences in terms of the number, position, parameters and / or location of the parts. For example, as shown in FIG. 3, when comparing the compared document 220 and the reference document 210, the computing device can determine that the images have the same number of graphic elements and text elements, and that these elements have similar parameters and location (location). It can also analyze the layout and determine that the graphic elements are in the same position with respect to the paragraphs, namely that they are embedded in the first and last paragraphs of the page.

[00031] Comparison 352 can also analyze the text obtained after character recognition, which was discussed above, to detect appropriate images of documents. Text analysis can be based on a technique that determines the degree of similarity between lines of characters for each image in a document, for example, by determining the number of matching lines of characters (e.g. words), and / or the number of lines that do not match. The term “matching images” may mean that the images of a document are approximately consistent without an exact match. In one example implementation, the corresponding images may have the same or similar number of elements. These elements may include words, text blocks, illustration blocks, etc. In another example implementation, the corresponding images may be represented by images in which the coefficient of matching (e.g., matching or related) words exceeds a certain value. The technique for determining matching words can include calculating and comparing the editing distance for pairs of words, and allows you to determine, for example, approximate line matching or fuzzy line matching.

[00032] The edit distance value may be an integer that quantifies the difference between the two lines. The value of the editing distance can be obtained by counting the minimum number of operations required to convert one character string to another character string. Editing distance can be calculated using any method, for example, Levenshtein metrics. The Levenshtein metric allows you to calculate the editing distance by measuring the number of single character editing cases required to change one word into another word. In one implementation example, the Levenshtein metric can only measure the following operations: deleting one character, inserting one character and replacing one character with another character without taking into account other operations, such as permuting two adjacent characters.

[00033] A pair of words can be considered a coincidence if the editing distance is less than a threshold value that can be determined based on a relative limit, an absolute limit, or a combination of the first and second. The relative limit can be based on the length of the search word, and can be represented as the ratio of the editing distance to the length of the string, for example, 1: N, where N represents the number of characters in the word (for example, 2, 3, 4, 5). In one example, the relative limit may be 1: 3 of the length of the search word, while a nine-character word may contain three errors and be considered an approximate match. In addition to the relative limit or as an alternative to it, you can use the absolute limit of differences regardless of the length of the string. For example, if there are more than X differences, then a pair of words cannot be considered a coincidence. In this case, X may have a value between 1 and 10, for example 3. This value is selected based on empirical analysis.

[00034] A computing device may use the approximate string matching discussed above to hypothesize the similarity that images of individual documents are relevant, for example, whether they are related or similar to each other (for example, if one document is a later version another document, or if both documents have a common source document). The similarity hypothesis can be based on the number of matching words and / or the number of words that do not match according to the above calculations. Similarly to the case of matching words, the correspondence of images (level of correspondence) can also have a threshold value. The threshold can set a limit value, expressed as the number or percentage of matching words used to determine that images of documents match (match). In one implementation example, a hypothesis can be based only on a part of the words in the image, for example, only on long words, and not on each word of the image. Long words can be words with a number of characters exceeding a given value (for example, 3, 4, 5 characters). Using a comparison of long words, you can eliminate the many false differences that occur more often in short words.

[00035] After the computing device 100 detects the corresponding images in the reference document, it can proceed to calculate the markup conversion function by determining reference points within each of the corresponding images. In one example implementation, the computing device may determine reference points at run time by identifying reference points in an image, such as a specific word or phrase. Words or phrases can be unique because they appear once on each of the corresponding images and, therefore, may be useful for matching the corresponding images.

[00036] When using words as reference frames, reference points may be located in the centers of words. The computing device can approximate the center of the word by determining the rectangular dimensions of the word and using the center of the rectangular dimensions as a reference point. In one example implementation, the OCR procedure may provide the computing device with the coordinates of the rectangles that include (eg, span) the words. As shown in FIG. 3, the computing device 100 can analyze the reference document 210 and determine three unique (or longest) words that will play the role of image reference frames represented by rectangles 314A-C. In the centers of the rectangles are the reference points 316A-C, respectively.

[00037] The computing device may determine that a pair of corresponding images may have several unique image frames and thus multiple variations of reference points. Reference points can be determined using absolute or relative coordinates, and a two-dimensional plane can be determined from three points not lying on one straight line. These points can define a triangle with a reference structure, for example, a reference structure 318 defined by reference points 316A-C.

[00038] When choosing between several possible reference points, the computing device can select the reference points as far as possible from each other. The use of points that are remote from each other makes it possible to increase the stability of the transformation against errors that arise due to the granularity of the coordinate system, so remote points can increase the accuracy of the conversion. The determination of the most distant points can be based on the selection of a set of reference points from among several candidates for reference points that define a reference structure 318 having a maximum size (for example, the largest area and / or largest perimeter).

[00039] The plurality of reference points 316A-C in reference document 210 may have a corresponding set of reference points in the compared document 220. Two sets of reference points can be compared to determine a linear or geometric transformation function. The transformation function may contain a system of linear and / or geometric equations that can be summed using a transformation matrix (for example, a 2 × 2 matrix).

[00040] The following is an example of a linear transformation equation:

Linear transformation equation

matrix of linear transformation;

matrix of linear transformation;

- координаты реперной точки X в изображении эталонного документа;

- coordinates of the reference point X in the image of the reference document;

- координаты реперной точки Y в изображении эталонного документа;

- coordinates of the reference point Y in the image of the reference document;

- координаты реперной точки Z в изображении эталонного документа;

- coordinates of the reference point Z in the image of the reference document;

- координаты реперной точки X′ в сравниваемом изображении;

- coordinates of the reference point X ′ in the compared image;

- координаты реперной точки Y′ в сравниваемом изображении;

- coordinates of the reference point Y ′ in the compared image;

- координаты реперной точки Z′ в сравниваемом изображении;

- coordinates of the reference point Z ′ in the compared image;

. Координаты могут измеряться в любой системе координат, которая может использоваться для идентификации точки в двумерном пространстве, например, в декартовой системе координат или в полярной системе координат.[00041] It consists of two sets of reference points, with each set comprising at least three points. Each reference point may contain at least two coordinates, for example,

. Coordinates can be measured in any coordinate system that can be used to identify a point in two-dimensional space, for example, in a Cartesian coordinate system or in a polar coordinate system.

[00042] After determining the conversion function, the computing device 100 can perform image conversion 360, in which the image conversion function can be used to change the markup, text and / or graphic elements of the compared document 220 in accordance with the image markup of the reference document. As shown in FIG. 3, the conversion of the image 360 can change the image 324b by rotating the text and graphics 5 degrees counterclockwise and moving them down, the result is an image 324C.

[00043] After applying the conversion function, the computing device 100 may perform a quality check of the image conversion results. The verification may include selecting an additional set of reference points in the reference document 210, which are different from the first set of reference points. Based on these new points, the computing device 100 can select the corresponding points in the compared document 220 and convert the image using the previous conversion function. If the resulting points in the linked image are within a threshold distance from the corresponding frame in the image, for example, inside a rectangle enclosing a word or in the immediate vicinity of a rectangle, then the function can determine that the transform function is correct. Otherwise, the computing device may determine that the conversion function is incorrect.

[00044] If it is determined that the conversion function is incorrect, then a new set of image frames can be selected. The procedures described above can be repeated using a new set of reference points obtained from new benchmarks. These may include identifying, applying, and testing several new conversion functions until the correct conversion function is found or until the number of failed attempts reaches a threshold (for example, 2, 3, 4, 5, or more attempts).

[00045] After determining that the image conversion results are correct, the computing device 100 may restart the character recognition procedure (eg, OCR) in the converted version of the compared document 220. The second character recognition procedure may produce text results that have a lower recognition error. These results can replace the results of the first character recognition procedure, and can be used to further calculate the difference 362.

[00046] A process with comparison and conversion steps may be iterative. In one example implementation, the computing device can iterate through a document consisting of several parts, sequentially processing image after image (page by page), while it is possible to select an image from each document and compare them, and then move to another image in each document and their comparison. If the images match (while talking about the corresponding images), then you can start converting the image and re-recognize the characters, if they do not match (do not match), then you can skip the steps of the conversion and the second character recognition and begin to calculate, filter and classify the differences (based on the result of character recognition in the first stage), which is described in more detail below. In another example implementation, iterative comparison may include selecting an image (e.g., a page) from the document being compared 220 or from a reference document 210, and then comparing it with several images (e.g., pages) from another document until a match is found, or an image ( for example, the pages) of another document will be exhausted, and at that moment the iterative comparison will go to the next part and perform a similar series of comparisons.

[00047] FIG. 4 is a flow chart of one illustrative example of a method 400 for processing electronic documents in accordance with one or more aspects of the present invention. Method 400 and / or each of its individual functions, procedures, subprograms or operations may be performed by one or more processors of a computer device (eg, computing device 100 in FIG. 1) and / or each of its individual functions, programs, subprograms or operations may performed by one or more processors of a computer device (eg, computing device 100 in FIG. 1) that runs this method. In some implementations of the invention, method 400 may be performed using a single processing stream. In an alternative approach, method 400 may be performed using two or more processing streams, with each processing stream performing one or more separate functions, procedures, routines, or operations of the method. In one example, the workflows or processing threads of method 400 may be synchronized (for example, using semaphores, critical sections, and / or other thread synchronization mechanisms).

[00048] In block 410, the computing device 100 may receive several electronic documents, for example, reference documents 210 and compared documents 220. Documents 210 and 220 may come from different sources and stored in different places. In one example implementation, the user can manually select documents 210 and 220, providing information about their location to computing device 100. In another example implementation, computing device 100 can automatically receive the compared document from a digitizing device (eg, a scanner), while the user does not provide information about location, and subsequently can pick up the corresponding reference document from the document repository automatically, using any known classification method. A document repository may be accessible to a computing device, it may reside in a computing device or in a remote device (for example, a company server, a third-party server).

[00049] At block 415, computing device 100 may also perform image preprocessing for one or more documents in order to improve image quality. The computing device can then analyze the documents, as shown in block 420, to determine the markup of the image of the document, for example, the location of the text blocks.

[00050] At block 425, computing device 100 may also perform a first character recognition procedure. The first character recognition procedure may be performed after the analysis of the document is completed, and use the results of the analysis of the document to locate parts of the document that may include text. The first character recognition procedure can then analyze these parts to get the results in text.

[00051] At block 430, computing device 100 may iteratively compare images of a document (eg, pages) to identify corresponding images (eg, pages). Such a result can be achieved by selecting an image (for example, a page or part of a page) from each document and comparing them, for example, based on the results of determining the markup and text for fragments of text 420 and 425. Such a comparison may include a text comparison that calculates the editing distance for a few pairs of words. Such a comparison can only take into account long words having a number of characters greater than a predetermined number and ignore shorter words.

[00052] At block 440, computing device 100 may determine whether the image of the reference document (eg, pages) matches the image of the reference document (eg, pages). Since this procedure is intended to determine differences, images of documents may correspond approximately in contrast to an exact match, as described above. If the images do not match, then the method proceeds to block 470 and checks for additional images for comparison. Otherwise, the method proceeds to block 450.

[00053] At block 450, computing device 100 may convert images (eg, pages) of the document to be compared to eliminate distortions that might have been introduced during digitization. Such a conversion may include analyzing the reference points in both documents to calculate the conversion function, and then applying this conversion function to part of the document (for example, to the page).

[00054] At block 460, the computing device may perform a second character recognition procedure (eg, re-start character recognition) in the document to be compared using special dictionaries derived from a reference document (eg, a dictionary for a piece of text or a dictionary for a page). The second character recognition may be performed after the conversion of the document being compared 220, in contrast to the first character recognition that is performed before the conversion. The results of the second character recognition may have fewer inaccuracies than the first character recognition, since the converted document may have less distortion. The computing device can replace the results of the first character recognition procedure of the image of the compared document with the results of the second character recognition procedure. Therefore, the use of these results can lead to fewer false differences.

[00055] At block 470, the computing device can calculate the differences and eliminate, filter, and classify the remaining differences. Computing device 100 can calculate the differences using a comparison method, such as the method used in diff for Unix or WinMerge for Microsoft. These differences are not final, they may also include one or more additional iterations of checking, sorting and removing minor discrepancies. Some of the differences may be due to OCR discrepancies due to inaccuracies in the OCR subsystem, rather than the actual differences between documents. OCR discrepancies can be caused by defects in the documents being compared, for example, in the form of blurry text, out of focus text, glare, or excessive noise. In some embodiments of the invention, these defects may be caused by scanning or photographing the text. In one example implementation, OCR discrepancies can be reduced by comparing strings (eg, words) that differ in the documents being compared. If the different characters are graphically similar, then it is very likely that these discrepancies were caused by the recognition features. If a mismatch consists of visually similar characters having different Unicode codes, then such a mismatch is also considered minor and may not be shown to the user. The following are examples of differences caused by the visual similarity of characters: the letter “O” and the number zero “0”; differences in alphabets, such as the letters "ABC" in the Cyrillic alphabet and similar letters "ABC" in the Latin alphabet; differences caused by different character widths, such as dashes and hyphens of various lengths; fixed interval and ordinary hieroglyphs; shares in the form of ¾ and 3/4, etc.

[00056] After resolving the differences, the computing device can filter and classify the remaining differences. Difference filtering is designed to remove false or minor differences, and classification is used to categorize and refine types of changes. Classification of differences may include, for example, identifying or clustering types of significant changes, such as “insert”, “delete”, “change text”, etc. They can be displayed to the user in order to allow the user to quickly determine visually which types of inconsistencies they receive are important to them and which are not significant. For example, a mismatch of the type “symbol X is changed to Y” may be minor for the user, while a mismatch of the type “word A changed to word B” may be significant. In some embodiments of the invention, a list of differences that have been identified as minor may be displayed to the user. Such a list can be presented in a different way (for example, displayed only after a user selects it, displayed below in the interface, etc.) than in the list of significant differences. In one example, calculating differences and resolving, filtering, and classifying remaining differences may use the method described in Russian Patent Application 2013156257 entitled “Device and Method for Finding Differences in Documents”, the contents of which are incorporated herein by reference.

[00057] At block 480, computing device 100 may analyze a document to determine if documents have additional parts (eg, images / pages) that have not yet been analyzed. If there are additional images, the computing device 100 may go to block 430 to compare a different set of pages. In the absence of additional images, it may go to block 490.

[00058] In block 490, the remaining differences can be presented to the user in the form of a modified image with annotations and / or as a text list of differences. For example, the system can be configured to determine the display mode for the user of the results of the comparison. Display of comparison results can also be provided by default in the settings. According to some embodiments of the invention, a user display function can be selected for settings to provide as much accurate details as possible about the results of the analysis containing all the differences found, including differences caused by inaccurate OCR results. In addition, the user can manually select the types of significant discrepancies that should be included in the final list of discrepancies, and those discrepancies that can be removed from this list. In some embodiments, various variations of the discrepancies found can be established.

[00059] The results of a document comparison can be displayed, for example, by displaying areas of difference on a screen or by printing. If discrepancies are found in the documents, that specific area (text) where these differences are located is highlighted. The type of differences detected can be reported to the user in a specific area color. In some embodiments, the color of the region may be provided to the user in advance at predetermined parameters. For example, yellow may indicate that “delete” differences have been detected in this area, while red may indicate “inserts,” etc. In some embodiments of the invention, if a change is found within one word, the entire word is highlighted in yellow, and the changed character in it is highlighted in red. In various embodiments of the invention, various other methods of representing differences (e.g., visual presentation) may be used.

[00060] FIG. 5 is an example illustrating a procedure for improving document quality and comparison for three or more documents. In this case, the first compared document 220, the second compared document 520 and the reference document 510 can be compared to determine the differences 570. The computing device 100 can dynamically select the reference document 510 from the document store 512 at run time. Dynamic selection can be based on the classification of one or more compared documents 220 and 520. As mentioned above, reference document 510 may be an earlier version of the document, in which case the classification can be used to determine the source document and / or the most similar version (for example, the latest). In another example, a computing device may not have access to a source document or an earlier version, and it may use classification to identify a common reference document (for example, a template). A common reference document may be loosely coupled with the compared document in the sense that the compared document cannot be obtained from the common reference document. Despite this, the general reference document may have a markup similar to the marking of the compared documents 220 and 520, for example, the general reference document may be an image of a standard contract, passport, business card, slide from a presentation, Word document or other similar document. The classification of the compared document (compared documents) can be based on any known technique, for example, on morphological analysis of images.

[00061] After receiving the reference document 510, the computing device can perform actions similar to those that are performed with the compared document 220 in FIG. 3. In addition, it can perform the same processing steps or steps, basically similar to the steps for the document 520 being compared. The computing device can perform the described actions in parallel, for example, preprocessing the image 350 and converting the image 360 can be similar and be performed in parallel with the preliminary image processing 550 and image conversion 560, respectively. Similarly, comparison 551 and calculation of differences 562 can be performed in parallel with comparison 552 and calculation of differences 564.

[00062] In another example implementation, which is not shown, the reference document 510 can be used to convert the compared documents 220 and 520 without using the calculation of differences. Thus, differences 570 may exist between the compared documents, and not between the compared document and the reference document. In this case, the reference document can be used to convert images and restart OCR recognition, but it is not required for any subsequent comparison steps.

[00063] FIG. 6 shows a more detailed diagram of an example computing device 600, within which there is a set of instructions that can cause the computing device to execute any one or more of the methods described herein. Computing device 600 may include the same components as computing device 100 in FIG. 1, as well as some additional or other components, some of which may be optional, to provide aspects of the present invention. A computing device may be connected to another computing device via a local area network, a corporate network, an extranet, or the Internet. A computing device can operate as a server or client in a client / server network environment, or as a peer computing device in a peer-to-peer (or distributed) network environment. A computing device may be provided in the form of a personal computer (PC), tablet computer, set-top box (STB), personal digital assistant (PDA), cell phone, or any computing device capable of executing a set of commands (sequentially or otherwise) defining operations that must be performed by this computing device. In addition, while only one computing device is shown, the term “computing device” also includes any combination of computing devices that individually or collectively execute a set (or several sets) of instructions for implementing any method or methods described in this description.

[00064] An example computing device 600 includes a processor 602, random access memory 604 (eg, read-only memory (ROM) or dynamic random access memory (DRAM)) and data storage device 618 that communicate with each other via bus 630.

[00065] The processor 602 may be represented by one or more universal data processing devices, such as a microprocessor, central processing unit, and the like. In particular, processor 602 may be a full instruction set microprocessor (CISC), an abbreviated instruction set microprocessor (RISC), an extra long instruction word microprocessor (VLIW), or a processor that implements other instruction sets or processors that implement combination of instruction sets. Processor 602 may also be one or more special-purpose processing devices, such as application specific integrated circuit (ASIC), field programmable gate array (FPGA), digital signal processor (DSP), network processor, or the like. The processor 602 is configured to execute instructions 622 to perform the operations and functions described in this description of the invention.

[00066] The computer device 600 may further include a network interface device 608, a video display unit 610, a character input device 612 (eg, a keyboard), a cursor control device 614, and a signal generator.

[00067] The storage device 618 may include a computer-readable storage medium 624 that stores one or more sets of instructions 622 that implements one or more of the methods or functions described herein. The instructions 622 may also be located wholly or at least partially in the main memory 604 and / or in the processor 602 when executed by the computing device 1000, the RAM 604 and the processor 602 also constituting a computer-readable storage medium. Commands 622 may additionally be transmitted or received over the network 620 through the network interface device 608.

[00068] In some implementations of the invention, instructions 622 may include instructions of a method 400 for improving the quality and comparison of documents, which may be performed using the comparison module 190 shown in FIG. 1. While the computer-readable storage medium 624 is shown in the example of FIG. 4 as a single medium, the term “machine-readable medium” should be understood as a single medium or a plurality of such media (for example, a centralized or distributed database, and / or corresponding caches and servers) that store one or more sets of instructions. The term “computer-readable storage medium” also refers to any medium capable of storing, encoding or transferring a set of machine-executable instructions upon execution by which the machine implements one or more methods of the present invention. Thus, the term “computer readable storage medium” also refers to solid state memory, as well as optical and magnetic media.

[00069] The methods, components, and functions described in this document can be implemented using discrete hardware components, or they can be integrated into the functions of other hardware components, such as ASICS (specialized custom integrated circuit), FPGA (programmable logic integrated circuit ), DSP (digital signal processor) or similar devices. In addition, methods, components, and functions can be implemented using firmware modules or function diagrams inside the device. The methods, components and functions can also be implemented using any combination of computing tools and software components, or exclusively using software.

[00070] In the above description, numerous details are set forth. However, it will be apparent to any person skilled in the art who is familiar with this description that the present invention can be practiced with little or no of these specific details. In some cases, well-known structures and devices are shown in block diagram form, and not in detail, so as not to complicate the description of the present invention.

[00071] Some parts of the description of preferred embodiments have been represented using algorithms and symbolic representations of operations on data bits in computer memory. These algorithmic descriptions and representations are the means used by specialists in the field of data processing to most effectively transfer the essence of their work to other specialists in this field of technology. In this document and in general, an algorithm is a self-consistent sequence of operations leading to the desired result. Operations require physical manipulations with physical quantities. Usually, although not necessarily, these quantities take the form of electrical or magnetic signals that can be stored, transmitted, combined, compared and subjected to other manipulations. It turned out that, first of all, for normal use, it is convenient to describe these signals in the form of bits, values, elements, symbols, members, numbers, etc.

[00072] However, it should be borne in mind that all these and similar terms should be associated with the corresponding physical quantities, and that they are simply convenient labels that apply to these quantities. Unless specifically indicated otherwise, which is obvious from the discussion below, it should be understood that throughout the description, all statements that use such terms as “definition”, “calculation”, “calculation”, “receipt”, “definition”, “Change”, etc., refers to actions and processes in a computing device or similar electronic computing device that manipulates data and converts data presented in the form of physical (eg, electronic) quantities in the registers of a computing device -keeping and memory into other data similarly represented as physical quantities within the registers or memories of the computing device, or other storage devices and information transmission or display devices.

[00073] The present invention also relates to a device for performing the operations described in this document. Such a device may be specially designed for the required purposes or it may contain a universal computer that is selectively activated or reconfigured using a computer program stored in the computer. Such a computer program may be stored on a computer-readable storage medium, for example (but not limited to): a disk of any type, including a floppy disk, optical disks, CD-ROM and magneto-optical disks, read-only memory (ROM), random access memory (RAM) ), programmable read-only memory (EPROM), electrically erasable read-only memory (EEPROM), magnetic or optical cards or any type of medium suitable for storing electronic commands.

[00074] It should be borne in mind that the above description is intended to illustrate, and that it is not restrictive. Various other implementations will become apparent to those skilled in the art after reading and understanding the above description. Therefore, the scope of disclosure should be determined with reference to the appended claims along with the full scope of equivalents to which such claims are entitled.

Claims (21)

1. A method for comparing images of documents, performed by means of a computing device containing a processor, which consists in:
obtaining an image of the first document from the reference document and the corresponding image of the second document from the compared document;
determining the markup of the received images of the first and second documents;
performing the first optical character recognition procedure of the received images of the first and second documents, and
the formation of the reference dictionary, and the reference dictionary contains words from a text block from the image of the first document;
converting the image of the second document from the compared document based on the markup of the image of the first document from the reference document;
performing repeated character recognition on the resulting image conversion of the second document, and repeated character recognition in the compared document includes the use of the generated reference dictionary. 2. The method according to claim 1, further comprising calculating the difference between the compared document and the reference document based on the results of the primary character recognition of the first document from the reference document and the repeated recognition of the characters of the second document from the compared document. 3. The method according to p. 1, characterized in that the comparison of images of documents comprises comparing at least part of the first markup and part of the first text obtained by character recognition on the first image of the document, and at least part of the second markup and part of the second text, obtained by recognizing characters in the second image of the document. 4. The method according to p. 3, characterized in that the part of the first text and part of the second text include words containing at least a pre-selected number of characters, and comparing images of documents further comprises calculating the editing distance between the corresponding words. 5. The method according to p. 1, characterized in that the image of the first document consists of a page image of the first document. 6. The method according to p. 1, characterized in that the conversion includes linear transformation of the image of the second document based on the positions of three points in the image of the first document and in the image of the second document. 7. The method according to claim 6, characterized in that the position of the three points in the image of the first document and in the image of the second document are determined based on unique words in the results of optical character recognition in the image of the first document and in the image of the second document. 8. The method of claim 1, further comprising an iterative page-by-page comparison of document images based on the results of optical character recognition and the identified markup, which are a reference document and a compared document, to identify an image of a first document from a reference document that corresponds to an image of a second document from the compared document . 9. A computer-implemented system for comparing images of documents, containing:
memory;
a processor working with this memory, and this processor provides:
obtaining the image of the first document from the reference document and the corresponding image of the second document from the compared document;
determining the markup of the received images of the first and second documents;
performing the first optical character recognition procedure of the received images of the first and second documents, and
the formation of the reference dictionary, and the reference dictionary contains words from a text block from the image of the first document;
converting the image of the second document from the compared document based on the markup of the image of the first document from the reference document;
performing repeated character recognition on the resulting image conversion of the second document, the repeated character recognition in the compared document includes the use of the generated reference dictionary. 10. The computer-implemented system according to claim 9, characterized in that the processor further provides: calculating the differences between the compared document and the reference document based on the results of the primary character recognition of the first document from the reference document and the repeated recognition of the characters of the second document from the compared document. 11. The computer-implemented system according to claim 9, characterized in that the comparison of images of documents comprises comparing at least part of the first markup and part of the first text obtained by recognizing characters in the first image of the document, and at least part of the second markup and part second text obtained by recognizing characters in the second image of the document. 12. The computer-implemented system of claim 11, wherein the part of the first text and part of the second text contain words having at least a predetermined number of characters, and comparing images of documents further comprises calculating the editing distance between the corresponding words. 13. The computer-implemented system according to claim 9, characterized in that the image of the first document consists of a page image of the first document. 14. The computer-implemented system according to claim 9, characterized in that the conversion includes a linear transformation of the image of the second document based on the positions of three points on the image of the first document and on the image of the second document. 15. The computer-implemented system according to claim 9, characterized in that the position of three points in the image of the first document and in the image of the second document are determined based on unique words in the results of optical character recognition in the image of the first document and in the image of the second document. 16. A permanent computer-readable storage medium containing executable instructions for comparing images of documents that, when executed by a computing device, lead to operations including:
obtaining the image of the first document from the reference document and the corresponding image of the second document from the compared document;
determining the markup of the received images of the first and second documents;
performing the first optical character recognition procedure of the received images of the first and second documents, and
the formation of the reference dictionary, and the reference dictionary contains words from a text block from the image of the first document;
converting the image of the second document from the compared document based on the markup of the image of the first document from the reference document;
performing repeated character recognition on the resulting image conversion of the second document, the repeated character recognition in the compared document includes the use of the generated reference dictionary. 17. A permanent computer-readable storage medium according to claim 16, further providing for calculating the differences between the compared document and the reference document based on the results of the primary recognition of the characters of the first document from the reference document and the repeated recognition of the characters of the second document from the compared document. 18. A permanent computer-readable storage medium according to claim 16, characterized in that the comparison of images of documents comprises comparing at least part of the first markup and part of the first text obtained by character recognition on the first image of the document, and at least part of the second markup and part second text obtained by recognizing characters in the second image of the document. 19. A permanent computer-readable storage medium according to claim 18, characterized in that part of the first text and part of the second text contain words having at least a predetermined number of characters, and comparing document images further involves calculating the editing distance between the corresponding words. 20. The permanent computer-readable storage medium according to claim 16, characterized in that the conversion includes linear transformation of the image of the second document based on the positions of three points on the image of the first document and on the image of the second document. 21. A permanent computer-readable storage medium according to claim 20, characterized in that the position of three points in the image of the first document and in the image of the second document are determined based on unique words in the results of optical character recognition in the image of the first document and in the image of the second document.

Images