KR20220013187A - Document recognition system and method - Google Patents

Abstract

Disclosed are a document recognition system and method capable of recognizing a document with a high recognition rate irrespective of tilt, perspective distortion, ratio, scale, and the like. According to one embodiment of the present invention, the document recognition system comprises: a preprocessing unit detecting a line in a table of a document in which one or more texts are written in a text area and removing the line in the table; a text recognition unit recognizing the texts of the document from which the line of the table is removed by optical character recognition (OCR); an outline detection unit determining the density and meaning of text on the basis of word dictionary data and detecting an outline of the text according to an entered keyword; an area determination unit determining an outer area in which the text is written among the entire area of the document through the outline of the text; a document analysis unit classifying documents on the basis of the text to determine a type of document; a tilt detection unit detecting a tilt of the document; and a document recognition unit detecting a title of the document by a preset template and recognizing the document by considering the tilt of the document.

Description

Document recognition system and method

The present invention relates to a document recognition system and method, and more particularly, to a document recognition system and method capable of recognizing a document with a high recognition rate regardless of tilt, perspective distortion, ratio, scale, and the like.

With the rapid spread of digital storage media, digitization of documents that existed previously on paper is being actively developed. Such a phenomenon is being accelerated with the development of optical character recognition (OCR), which is a technology for automatically recognizing characters included in a document.

As prior art related to such optical character recognition technology, Korean Patent Registration No. 10-1686363 (hereinafter 'Prior Art 1') and Korean Patent Publication No. 10-1114135 (hereinafter 'Prior Art 2') have been disclosed. there is a bar

On the other hand, in the prior art, after scanning a document with a scanner in consideration of the recognition rate of the document, it has been common to recognize the document using optical character recognition technology. Alternatively, a technology for recognizing a document through an image of the document captured by a device is being studied.

In the process of recognizing a document shot with such a terminal or device, after detecting the outline of the document from the image of the document, the user crops the image of the document along the outline, and then recognizing the document using optical character recognition technology will do

However, when an image of a document is abnormally captured in the process of photographing a document with a terminal or device, or when the image of a document is abnormally cropped while a user crops the image of the document, an error occurs in document recognition there was

In addition, in the prior art, since the document is recognized including the lines of the table output to the document, the recognition speed of the document is slowed, and since the document is recognized as a block divided by a preset template, tilt and perspective distortion, etc. When this occurs, there is a problem in that it is impossible to accurately recognize the document.

Republic of Korea Patent Publication No. 10-1686363 (registered on Dec. 07, 2016) Republic of Korea Patent Publication No. 10-1114135 (Registered on 02.01.2012)

Accordingly, the present invention has been devised to solve the above problems, and an object of the present invention is to provide a document recognition system and method capable of recognizing a document with a high recognition rate regardless of tilt, perspective distortion, ratio, scale, and the like.

Another object of the present invention is to provide a document recognition system and method capable of recognizing a document photographed by a terminal or device, or a document normally or abnormally scanned by a scanner, with a high recognition rate.

However, the technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned are clearly to those of ordinary skill in the art to which the present invention belongs from the description below. can be understood

As a technical means for achieving the above object, in a character recognition system, a preprocessor for detecting a line in a table of a document in which one or more texts are written in a text area, and then removing the line in the table; A text recognition unit for recognizing the text of the document from which the line of the table is removed by OCR (optical character recognition); an outline detection unit for detecting an outline of text written in a document by removing the entered keyword or meaningless text; an area determination unit that determines an outer area in which text is written among the entire area of the document through the outline of the text; a document analysis unit for classifying documents based on text to determine the types of documents; a tilt detection unit for detecting a tilt of the document; and a document recognition unit that determines whether a value is written in an item of a document by a preset template, forms a reference line connecting the item and the value of the item, and recognizes the document by substituting the gradient detected through the reference line; include

In addition, the tilt detection unit divides the document into a plurality of regions by forming a plurality of reference lines, and detects the tilt of the document by calculating the degree of inclination of the reference line.

In addition, the reference line extends in the horizontal direction to connect the text of the document, so that the document is divided in the vertical direction through a plurality of regions.

In addition, in the template, the title of the document, the text area, the item included in the text area, the text of the item, and the location of the item are stored.

The document is at least one of a document pre-stored in the terminal or device, a document normally scanned by the scanner, a document abnormally scanned by the scanner, a document photographed by a mobile phone camera, or a document photographed by another device.

Also, the keyword is text included in the word dictionary data or text collected from a document to be recognized through the document recognition unit.

In addition, the document analysis unit classifies documents based on one or more texts in the document, the position of the text, the text of the title written in the document, the size of the title text, the text size ratio of the item, and a special keyword written for each document to classify the document type to determine

In a character recognition method as a technical method for achieving the above object, the method comprising: a preprocessor detecting a line in a table of a document in which one or more texts are written in a text area, and then removing the line in the table; Recognizing, by the text recognition unit, the text of the document from which the lines of the table are removed by OCR; detecting, by the outline detecting unit, the outline of the text written in the document by removing the entered keyword or meaningless text; determining, by the area determination unit, an outer area in which text is written among the entire area of the document based on the outline of the text; classifying the document based on the text by the document analyzing unit to determine the type of the document; detecting a tilt of the document by a tilt detection unit; and after the document recognition unit determines whether a value is written in an item of the document by a preset template, forming a reference line connecting the item and the value of the item, and substituting a gradient detected through the reference line to recognize the document; includes

According to the present invention, it is possible to contribute to convenience and cost reduction by recognizing a document with a high recognition rate.

In addition, according to the present invention, it is possible to contribute to convenience and cost reduction by implementing automation of document recognition.

However, the effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those of ordinary skill in the art to which the present invention belongs from the following description. will be able

1 is a conceptual diagram of a document recognition system according to an embodiment of the present invention.
2 is a block diagram illustrating a document recognition system according to an embodiment of the present invention.
3 is a diagram illustrating an example of a document.
Fig. 4 is a diagram showing an example of a document in which the lines in the table are removed.
5 is a diagram illustrating an example of a document in which a region is divided into a plurality along a plurality of reference lines.
6 is a view showing an example of a document recognized through the document recognition unit of the present invention.
7 is a flowchart illustrating a document recognition method according to an embodiment of the present invention.

Hereinafter, with reference to the accompanying drawings, embodiments of the present invention will be described in detail so that those of ordinary skill in the art to which the present invention pertains can easily implement them. However, since the description of the present invention is merely an embodiment for structural or functional description, the scope of the present invention should not be construed as being limited by the embodiment described in the text. That is, since the embodiment may have various changes and may have various forms, it should be understood that the scope of the present invention includes equivalents capable of realizing the technical idea. In addition, since the object or effect presented in the present invention does not mean that a specific embodiment should include all of them or only such effects, it should not be understood that the scope of the present invention is limited thereby.

The meaning of the terms described in the present invention should be understood as follows.

Terms such as “first” and “second” are for distinguishing one component from another, and the scope of rights should not be limited by these terms. For example, a first component may be termed a second component, and similarly, a second component may also be termed a first component. When a component is referred to as being “connected to” another component, it may be directly connected to the other component, but it should be understood that other components may exist in between. On the other hand, when it is mentioned that a certain element is "directly connected" to another element, it should be understood that the other element does not exist in the middle. Meanwhile, other expressions describing the relationship between elements, that is, "between" and "between" or "neighboring to" and "directly adjacent to", etc., should be interpreted similarly.

The singular expression is to be understood to include the plural expression unless the context clearly dictates otherwise, and terms such as "comprise" or "have" are not intended to refer to the specified feature, number, step, action, component, part or any of them. It is intended to indicate that a combination exists, and it should be understood that it does not preclude the possibility of the existence or addition of one or more other features or numbers, steps, operations, components, parts, or combinations thereof.

All terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs, unless otherwise defined. Terms defined in the dictionary should be interpreted as being consistent with the meaning of the context of the related art, and cannot be interpreted as having an ideal or excessively formal meaning unless explicitly defined in the present invention.

document recognition system

Hereinafter, the document recognition system 1 of a preferred embodiment will be described in detail with reference to the accompanying drawings.

1 is a conceptual diagram of a document recognition system according to an embodiment of the present invention, FIG. 2 is a block diagram illustrating a document recognition system according to an embodiment of the present invention, FIG. 3 is a diagram illustrating an example of a document, and FIG. 4 is a diagram showing an example of a document in which the lines of the table have been removed, FIG. 5 is a diagram showing an example of a document in which an area is divided into a plurality according to a plurality of reference lines, and FIG. 6 is a document recognized through the document recognition unit of the present invention It is a figure which shows an example of a document.

Referring to FIG. 1 , the document recognition system 1 is capable of recognizing a document 2 with a high recognition rate regardless of tilt, perspective distortion, ratio and scale, etc., and transfers recognition information of the document 2 to the server 100 . can be forwarded to

However, the document recognition system 1 does not necessarily transmit the recognition information of the document 2 to the server 100, and transfers the recognition information of the document 2 to a device such as a user terminal rather than the server 100. it is possible

In addition, the document recognition system 1 may be implemented as one of a computer, a cloud, a server, and a network operated by a user company or a provider to replace the server 100 .

The document 2 is a document previously stored in a terminal or device in which one or more texts are written in the text area 2b, a document normally scanned by a scanner, a document scanned abnormally by a scanner, and a document photographed with a mobile phone camera or other It is at least one of the documents captured by the device, and as a specific example, a business registration certificate, real estate contract, import/export document, medical expense receipt, medical detailed statement, measurement certificate, ID, receipt certificate, copy of bankbook, invoice, bill of lating), LC (letter of credit), etc. may mean documents.

The server 100 may receive recognition information of the document 2 from the document recognition system 1 , and transmit the received recognition information of the document 2 to the database 200 .

Here, the server 100 may be a user company, a computer operated by a provider, a cloud, or a network.

The server 100 does not necessarily transmit the recognition information of the document 2 to the database 200, and it is possible to transmit the recognition information of the document 2 to a device such as a user terminal.

In addition, when the server 100 is replaced by the document recognition system 1 , as the role is replaced by the document recognition system 1 , the configuration may be omitted.

The database 200 may store recognition information of the document 2 received from the document recognition system 1 or the server 100 .

Here, the database 200 may be a database of a user company or a provider operating the document recognition system 1 or the server 100 .

The user company or the provider recognizes the document 2 through the process of storing the recognition information of the document 2 in the database 200 from the process in which the document recognition system 1 recognizes the document 2 can be implemented with automation.

On the other hand, the document recognition system 1, the server 100, and the database 200 are implemented as one system so that the process of recognizing the document 2 to the process of storing the recognition information of the document 2 is one configuration. It may be made from an element, or it may be made from multiple components.

2 to 6 , the document recognition system 1 includes a preprocessor 10 , a text recognition unit 20 , an outline detection unit 30 , an area determination unit 40 for recognition of the document 2 , It is configured to include a document analysis unit 50 , a tilt detection unit 60 , and a document recognition unit 70 .

The components of the document recognition system 1 are not limited as described above, and components may be added or omitted as necessary, and the order of the recognition process of the document 2 to be described later may be changed.

After detecting the line 2a of the table of the document 2, the pre-processing unit 10 removes the line 2a of the table of the document 2 as shown in FIG.

In this way, the pre-processing unit 10 removes the table line 2a is to read the table line 2a as an image in the optical character recognition (OCR)-based text recognition process performed in the text recognition unit 20 This is to prevent errors so that text can be easily recognized, and to prevent a decrease in the recognition speed caused by dividing the recognition into several blocks when the line is recognized without removing the line.

Also, the pre-processing unit 10 may apply a clean up option, which is a pre-processing of the document 2 , to the document 2 in order to remove the line 2a from the table of the document 2 .

Here, it is preferable that the clean-up option is performed during the pre-processing (or editing) process of the document 2, and the pre-processing unit 10 detects the line 2a of the table that is not recognized in the optical character recognition of the document 2 After that, it is an option to remove it by dragging it with a tool.

The text recognition unit 20 recognizes text written in the text area 2b of the document 2 from which the line 2a of the table has been removed through a text recognition process based on optical character recognition.

Here, the text recognition process based on optical character recognition performed by the text recognition unit 20 may include sequential processes of binarization (pre-processing), segmentation, feature extraction, and recognition (classification). Since such a text recognition process is a known technique, a detailed description thereof will be omitted for convenience.

The outline detection unit 30 detects (or reads) the outline of the text described in the document 2 by removing the entered keyword or meaningless text.

Here, the keyword means text included in the word dictionary data or text collected from the document 2 to be recognized by the document recognition unit 70 .

Also, when the outline detection unit 30 erroneously recognizes noise in the process of detecting the outline of the text described in the document 2, "?", "*", "!", "T", ">", " ^", "?Q", "??", "??" Meaningless text such as etc. may be listed and displayed in the document 2 . Through this, the user can confirm that there is an error in the recognition process of the document 2 .

The area determination unit 40 determines the outer area 2c of the document 2 based on the detected text outline.

Here, the outer area 2c is to determine the area in which text is written among the entire area of the document 2, and the area of the document 2 other than the outer area 2c is extracted ( or removed).

In addition, the area determination unit 40 extracts the area of the remaining document 2 excluding the outer area 2c, thereby preventing the document recognition unit 70 from recognizing unnecessary images except for the document 2 .

The document analysis unit 50 classifies the document 2 based on one or more texts described in the document 2 and a position in which the one or more texts are written to determine the type of the document 2 .

As a specific example, when detecting special keywords 'measurement' and 'measurement date' from the text of item 2e described in the text area of the document 2, the document analysis unit 50 detects 'measurement' and 'measurement date' If the writing position of ' is the same as the preset position, it is possible to determine the document (2) as 'weighing certificate'. Here, the special keyword means a keyword (text or value) that must be described for each document 2 .

As another example, the document analysis unit 50 may determine the document 2 as an identification card when detecting that special keywords 'resident number' and 'license number' are written in the text of the item 2e, and the item ( When it is detected that at least one of the special keywords 'corporate name (account holder)', 'bank name' and 'account number' is written in the text of 2e), the document (2) can be identified as a copy of the passbook, and the item ( If one or more of the special keywords 'corporation registration number', 'date of business' and 'place of business' are written in the text of 2e), document (2) can be identified as a business registration certificate, and in the text of item (2e), When it is detected that at least one of the special keywords 'purchase contract', 'seller' and 'buyer' is described, the document 2 may be determined as a real estate contract.

Furthermore, the document analysis unit 50 classifies the document 2 by determining the text of the title 2d described in the document 2, the size of the title 2d, and the text size ratio of the item 2e. The type of the document 2 can be determined.

As a specific example, when the document analysis unit 50 detects the special keyword 'measurement certificate' from the text of the title 2d described in the document 2, the text size of the 'measurement certificate' is the same as the preset size or If the text size ratio of the title (2d) and the other items (2e) is the same or similar, the document (2) can be determined as a 'measurement certificate'.

That is, the document analyzing unit 50 determines the ratio of the text of the title 2d, the size of the title 2d, and the text size of the item 2e, as well as the one or more texts described in the document 2 and the positions of the one or more texts. And it is preferable to classify the document 2 through a special keyword to determine the type of the document 2 .

The tilt detection unit 60 forms a plurality of reference lines 61 in the area of the document 2 to divide the document 2 into a plurality of areas 63 , and calculates the degree of inclination of the reference line 61 , respectively The slope of (2) is detected.

As a specific example, the tilt detection unit 60 is the document 2 through a plurality of reference lines (61a, 61b, 61c, 61d, 61e, 61f, 61g) extending in the horizontal direction to connect the text of the document (2) The document 2 is vertically divided into a plurality of regions 63a, 63b, 63c, 63d, 63e, 63f, 63g, 63h, and a plurality of reference lines 61a, 61b, 61c, 61d, 61e, 61f, 61g ) is calculated, and the inclination of each area of the document 2 is detected.

Here, the tilt detection unit 60 forms the plurality of reference lines 61 in the area of the document 2 according to the shooting angle and method when the document 2 is a document 2 photographed by a terminal or device. This is because the inclination of the document 2 varies for each part, and it is possible to accurately detect the inclination of the document 2 only by calculating the inclination of each of these documents 2 , so that the recognition rate of the document 2 is improved.

The document recognition unit 70 preferentially detects the item 2e of the document 2 and the value of the item 2e by a preset template.

Here, the template includes the title 2d of the document 2, the text area 2b, the item 2e included in the text area 2b, the text of the item 2e, and the item ( The location of 2e) is stored, the title 2d is an element for classifying the type of the document 2, and the item 2e refers to an element in which text required according to the type of the document 2 is described.

In addition, the document recognition unit 70 detects the item 2e and the value of the item 2e, and then substitutes the detected inclination of the document 2 to recognize the document 2 .

As a specific example, the document recognition unit 70 may set the value of the item 2e described on the right side of the item 2e by a preset template (eg, '8710' (vehicle number) in FIG. 5, '15:47:22 39,860 kg' (gross weight)), by substituting the detected slope through the reference line 61 connecting the values of item (2e) and item (2e), item (2e) and item (2e) ) can be accurately recognized, and the document 2 can be accurately recognized by recognizing the title 2d through this process.

Here, the value of the item (2e) has been described as described on the right side of the item (2e), but it may be described not only on the right side, but also on the left, bottom, upper side, diagonal direction, etc. based on the item (2e).

In addition, the document recognition unit 70 may remove unnecessary images other than the document 2 in the process of recognizing the document 2 . However, as the region determining unit 40 determines the outer region 2c of the document 2 , the unnecessary image removal process may be omitted.

In the document recognition system 1 , a convolution neural network (CNN) algorithm or an artificial neural network (ANN) algorithm for recognizing the document 2 may be pre-stored.

That is, the document recognition system 1 learns the process of recognizing the document 2 to the CNN algorithm or ANN algorithm, and then the CNN algorithm or ANN algorithm that has been trained is performed by the pre-processing unit 10 and the text recognition unit 20 ), it is preferable to recognize the document 2 by controlling each process of the outline detection unit 30, the area determination unit 40, the document analysis unit 50, the inclination detection unit 60, and the document recognition unit 70. .

How to recognize documents

Hereinafter, a document recognition method of a preferred embodiment using the document recognition system 1 will be described in detail with reference to the accompanying drawings.

7 is a flowchart illustrating a document recognition method according to an embodiment of the present invention.

First, the preprocessor 10 may detect a line 2a of a table of a document in which one or more texts are written in the text area 2b, and then remove the line 2a of the table (S1).

After the step (S1), the text recognition unit 20 may recognize the text written in the text area 2b of the document through a text recognition process based on optical character recognition (S2).

After the step (S2), the outline detection unit 30 may detect the outline of the text described in the document 2 by removing the entered keyword or meaningless text (S3).

After the step S3, the area determination unit 40 may determine the outer area 2c of the document 2 through the outline of the document 2 (S4).

After the step S4 , the document analysis unit 50 is configured to generate the text described in the document 2 , preferably the one or more texts described in the document 2 and the position of the one or more texts, the text of the title 2d, the title (2d) The type of the document 2 can be determined by classifying the document 2 through the text size, the text size ratio of the item 2e, and the special keyword (S5).

After the step (S5), the tilt detection unit 60 may detect the tilt of the document 2 (S6).

After the step S6, the document recognition unit 70 detects the item 2e of the document 2 by a preset template (S7), and then connects the item 2e and the value of the item 2e By substituting the detected slope along the reference line 61 to recognize the item 2e and the value of the item 2e, the document 2 can be recognized by recognizing the title 2d through this process (S8) ).

The detailed description of the preferred embodiments of the present invention disclosed as described above is provided to enable any person skilled in the art to make and practice the present invention. Although the above has been described with reference to preferred embodiments of the present invention, it will be understood by those skilled in the art that various modifications and changes can be made to the present invention without departing from the scope of the present invention. For example, those skilled in the art can use each configuration described in the above-described embodiments in a way in combination with each other. Accordingly, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The present invention may be embodied in other specific forms without departing from the spirit and essential characteristics of the present invention. Accordingly, the above detailed description should not be construed as restrictive in all respects but as exemplary. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the present invention are included in the scope of the present invention. The present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein. In addition, claims that are not explicitly cited in the claims may be combined to form an embodiment, or may be included as new claims by amendment after filing.

1: Document Recognition System, 2: Document,
2a: line in table, 2b: text area,
2c: Outer area, 2d: Title,
2e: item, 10: preprocessor,
20: text recognition unit, 30: outline detection unit,
40: area determination unit, 50: document analysis unit,
60: slope detection unit, 61: reference line,
63: area, 70: document recognition unit,
100: server, 200: database.

Claims (8)

In the character recognition system,
a pre-processing unit for detecting a line in a table of a document in which one or more texts are written in a text area, and then removing the line in the table;
a text recognition unit for recognizing the text of the document from which the lines of the table are removed through optical character recognition (OCR);
an outline detection unit for detecting an outline of text written in a document by removing the entered keyword or meaningless text;
an area determination unit that determines an outer area in which text is written among the entire area of the document based on the outline of the text;
a document analysis unit for classifying the document based on the text and determining the type of the document;
a tilt detection unit for detecting a tilt of the document; and
After determining whether a value is written in the item of the document by a preset template, a reference line connecting the item and the value of the item is formed, and the document is recognized by substituting a slope detected through the reference line A document recognition system comprising: a recognition unit. The method of claim 1,
The slope detection unit,
The document recognition system, characterized in that the document is divided into a plurality of regions by forming a plurality of the reference lines, and the inclination of the reference line is calculated respectively to detect the inclination of the document. 3. The method of claim 2,
The baseline is
The document recognition system, characterized in that extending in a horizontal direction to connect the text of the document, so that the document is divided in a vertical direction through the plurality of regions. The method of claim 1,
The template is
The document recognition system, characterized in that the title of the document, the text area, an item included in the text area, the text of the item and the location of the item are stored. The method of claim 1,
The document is
A document recognition system, characterized in that it is at least one of a document pre-stored in a terminal or device, a document normally scanned by a scanner, a document abnormally scanned by the scanner, a document photographed by a mobile phone camera, or a document photographed by another device . The method of claim 1,
The keyword is
A document recognition system, characterized in that it is a text included in word dictionary data or a text collected from a document to be recognized through the document recognition unit. The method of claim 1,
The document analysis unit,
Classifying the document through one or more texts described in the document, the position where the text is written, the text of the title written in the document, the size of the title text, the text size ratio of the item, and a special keyword described for each document Document recognition system, characterized in that for determining the type of. In the character recognition method,
after the preprocessing unit detects a line in a table of a document in which one or more texts are written in a text area, removing the line in the table;
Recognizing, by the text recognition unit, the text of the document from which the lines of the table are removed by OCR;
detecting, by the outline detecting unit, the outline of the text written in the document by removing the entered keyword or meaningless text;
determining, by an area determination unit, an outer area in which text is written among the entire area of the document based on the outline of the text;
classifying the document based on the text by a document analyzing unit to determine the type of the document;
detecting a tilt of the document by a tilt detection unit; and
After the document recognition unit determines whether or not a value is written in the item of the document by a preset template, a reference line is formed connecting the item and the value of the item, and the inclination detected through the reference line is substituted for the document Recognizing; document recognition method comprising the.

Images