JP7644282B1 - Information processing system, information processing method, and program - Google Patents

Abstract

A system capable of appropriately extracting information from an image is provided.
[Solution] In an information processing system, an electronic data generation device includes a second information acquisition unit that acquires second text information from a second optical character recognition device different from the first optical character recognition device that recognizes, when a first accuracy index indicating the degree of recognition accuracy for characters indicated by the first text information is determined to be lower than a first standard for the accuracy of recognition of characters included in an image, as second text information, and the second information acquisition unit recognizes at least one character included in a specific range image, which is an image of a portion of the target image, including a character corresponding to the first standard index determined to be lower than the first standard, from a first optical character recognition device that recognizes each of a plurality of characters included in a target image as first text information, and a character identification unit that identifies a character included in the target image based on the first text information and the second text information.
[Selected Figure] Figure 1

Description

The present invention relates to an information processing system, an information processing method, and a program.

A system that uses a machine learning module to extract information from document images is disclosed (Patent Document 1).

JP 2022-79439 A

The document data extraction system described in Patent Document 1 acquires image data associated with a document and extracts metadata from the image data by optical character recognition. The metadata specifies a string of text content items and text content item features associated with each text content item in the string of text content items. The document data extraction system uses a machine learning module to determine one or more text content items associated with a key based on the string of text content items and the text content item features. This enables the document data extraction system to extract information from a document image.

However, if an optical character recognition device that performs optical character recognition cannot properly generate metadata from image data, a problem arises in which information cannot be properly extracted from document images.

Therefore, in order to solve the above problems, the present invention aims to provide a system that can appropriately extract information from images.

An information processing system according to one aspect of the present invention includes a target image acquisition unit that acquires a target image including characters from a predetermined device, a first information acquisition unit that acquires, from a first optical character recognition device that recognizes each of a plurality of characters included in the target image as first text information, the first text information for each of the plurality of characters and a first accuracy index indicating the degree of recognition accuracy for the characters indicated by the first text information, and a first reference index based on at least one of the first accuracy indexes for the plurality of characters that is higher than a first reference regarding the accuracy of recognition of characters included in an image. If it is determined that the first reference indicator is lower, a specific range image is an image of a part of the target image that includes characters corresponding to the first reference indicator determined to be lower than the first reference, and at least one character included in the specific range image is recognized as second text information. A second information acquisition unit acquires the second text information about the characters included in the specific range image from a second optical character recognition device different from the first optical character recognition device, and a character identification unit identifies the characters included in the target image based on the first text information and the second text information.

In one aspect of the present invention, an information processing method includes a computer that acquires a target image including characters from a predetermined device, acquires the first text information for each of the multiple characters included in the target image from a first optical character recognition device that recognizes each of the multiple characters included in the target image as first text information, and acquires the first text information for each of the multiple characters and a first accuracy index indicating the degree of recognition accuracy for the characters indicated by the first text information from a first optical character recognition device that recognizes each of the multiple characters included in the target image as first text information, and when a first reference index based on at least one of the first accuracy indexes for the multiple characters is determined to be lower than a first reference for the accuracy of recognition of the characters included in the image, acquires the second text information for the characters included in the specific range image from a second optical character recognition device different from the first optical character recognition device that recognizes each of at least one character included in the specific range image as second text information, the second text information for the characters included in the specific range image being a specific range image that is an image of a part of the target image including the characters corresponding to the first reference index determined to be lower than the first reference, and the second text information for the characters included in the specific range image being acquired from the second optical character recognition device that is different from the first optical character recognition device, and the second text information is acquired.

A program according to one aspect of the present invention causes a computer to execute the following: acquire a target image including characters from a specified device; acquire, from a first optical character recognition device that recognizes each of a plurality of characters included in the target image as first text information, the first text information for each of the plurality of characters and a first accuracy index indicating the degree of recognition accuracy for the character indicated by the first text information; if a first reference index based on at least one of the first accuracy indexes for the plurality of characters is determined to be lower than a first reference standard for the accuracy of recognition of characters included in an image, acquire the second text information for the characters included in the specific range image from a second optical character recognition device different from the first optical character recognition device that recognizes each of at least one character included in the specific range image as second text information; and identify the characters included in the target image based on the first text information and the second text information.

The present invention provides a system that can appropriately extract information from images.

FIG. 1 is a diagram showing an overview of an electronic data generation system. 1 is a database showing an example of target image information. 1 is a database showing an example of specific range image information. FIG. 4 is a diagram showing an example of a screen displayed on a display unit. 1 is a flowchart showing a processing procedure of the electronic data generation system. 11 is a table showing validity indicators for line information that is text information of one line of a character string. FIG. 2 illustrates an example of a hardware configuration of a computer.

The electronic data generation system 10 according to one embodiment of the present invention will be described in detail below with reference to the drawings. However, the embodiment described below is merely an example, and is not intended to exclude the application of various modifications or techniques not explicitly described below. In other words, the present invention can be implemented by modifying it in various ways or combining the various embodiments without departing from the spirit of the invention. In addition, in the description of the drawings below, identical or similar parts are denoted by the same or similar reference numerals.

In addition, in this embodiment, the terms "part", "device", and "system" do not simply mean physical means, but also include cases where the functions of the "part", "device", and "system" are realized by software. Furthermore, the functions of one "part", "device", or "system" may be realized by two or more physical means or devices, and the functions of two or more "parts", "devices", or "systems" may be realized by one physical means or device. Furthermore, the various functions described below of each of the multiple devices that make up electronic data generation system 10 may be configured to be executed by other devices in the multiple devices.

Overview of Electronic Data Generation System 10
<<Configuration Overview>>
An overview of an electronic data generation system 10 will be described with reference to Fig. 1. Fig. 1 is a diagram showing an overview of the electronic data generation system 10.

The electronic data generation system 10 is a system that uses multiple optical character recognition devices to accurately generate digital documents from images. Specifically, the electronic data generation system 10 generates text information, which is digital data, from images specified in graphic formats such as JPEG (Joint Photographic Experts Group), TIFF (Tagged Image File Format), and PNG (Portable Network Graphics) and images in PDF (Portable Document Format) data (hereinafter referred to as "target images") through multiple optical character recognition devices.

The target image may be, for example, an image of various contracts or papers. For the sake of convenience, the following description will be given assuming that the target image is an image of a single page of a contract.

The electronic data generation system 10 includes, for example, an electronic data generation device 100, a first optical character recognition device 200, a second optical character recognition device 300, and a user terminal 400.

The electronic data generation device 100 is a device that outputs a digital document that is the result of character recognition of a target image based on the results of character recognition of the target image by each of two different optical character recognition devices.

The first optical character recognition device 200 is a device that performs character recognition on a target image.

The second optical character recognition device 300 is a device that performs character recognition on an image within a predetermined range of the target image obtained from the electronic data generation device 100.

The electronic data generating device 100, the first optical character recognition device 200 and the second optical character recognition device 300 may be, for example, a cloud computer, a server computer, a personal computer (e.g., a desktop, a laptop, a tablet, etc.), a media computer platform (e.g., a cable or satellite set-top box, a digital video recorder), a handheld computing device (e.g., a PDA, an email client, etc.), or other types of computers or communication platforms. At least a portion of the processing in the electronic data generating device 100, the first optical character recognition device 200 and the second optical character recognition device 300 may be realized by one or more computers (for example, but not limited to, cloud computing consisting of one or more computers).

The user terminal 400 is a device that accepts user input and displays various information.

The user terminal 400 may be, for example, a smartphone, a mobile phone (feature phone), a personal computer (e.g., desktop, laptop, tablet, etc.), a media computing platform (e.g., cable, satellite set-top box, digital video recorder), a handheld computing device (e.g., PDA (Personal Digital Assistant), email client, etc.), a wearable device (glasses-type device, watch-type device, etc.), another type of computer, or a communication platform.

<<Processing Overview>>
An overview of the processing of an electronic data generating system 10 will be described with reference to FIG.

First, in step S10, the electronic data generation device 100 transmits a target image acquired from a specified device to the first optical character recognition device 200.

In step S11, the first optical character recognition device 200 recognizes characters contained in a target image (e.g., an image of one page) and generates text information about the recognized characters (hereinafter referred to as "first text information"). The first text information may include the coordinates of the characters on the target image. At this time, the first optical character recognition device 200 generates a degree of recognition accuracy for the characters indicated by the generated first text information (hereinafter referred to as "first accuracy index"). For convenience, the first text information, first accuracy index, and coordinates may be collectively referred to as "first generated information" below.

The first optical character recognition device 200 transmits the first generation information to the electronic data generation device 100.

In step S12, if the electronic data generating device 100 determines that a reference index (hereinafter referred to as the "first reference index") based on at least one of the first accuracy indexes for multiple characters is lower than a standard (hereinafter referred to as the "first standard") regarding the accuracy of recognition of characters contained in an image, it identifies an image of a predetermined range of the target image (hereinafter referred to as the "specific range image") that contains characters corresponding to the reference index.

The first reference index may be, for example, the first accuracy index for each of the multiple characters, or the average value of the first accuracy index for the multiple characters included in a line-unit image or a block-unit image.

The first criterion is, for example, a threshold value that can be compared to the first accuracy index.

The specific range image may be, for example, an image of a single sentence (hereinafter referred to as a "block image") obtained by segmenting a portion of a page-unit target image, an image of one line (hereinafter referred to as a "line image"), or a character image, which is an image of a single character. For convenience, the specific range image will be described below as a "line image."

The electronic data generating device 100 transmits the specific range image to the second optical character recognition device 300. That is, the electronic data generating device 100 causes an optical character recognition device different from the optical character recognition device that performed character recognition on the target image to perform character recognition again on the specific range image (e.g., a line image) of, for example, a part of the target image, which includes characters determined to have low accuracy of character recognition.

In step S13, the second optical character recognition device 300 recognizes characters contained in the specific range image and generates text information about the recognized characters (hereinafter referred to as "second text information"). The second text information may include the coordinates of the characters on the target image. At this time, the second optical character recognition device 300 generates a degree of recognition accuracy for the characters indicated by the generated second text information (hereinafter referred to as "second accuracy index"). Note that, hereinafter, the second text information, second accuracy index, and coordinates may be collectively referred to as "second generated information".

The second optical character recognition device 300 transmits the second generated information to the electronic data generation device 100.

In step S14, the electronic data generation device 100 identifies characters included in the specific range image based on a comparison result (hereinafter referred to as the "first comparison result") between a reference indicator (hereinafter referred to as the "second reference indicator") based on at least one of the second accuracy indicators for each of at least one character included in the specific range image and a standard regarding the accuracy of recognition of characters included in the image (hereinafter referred to as the "second standard").

The second reference index may be, for example, a second accuracy index for each character included in the specific range image, or may be an average value of the second accuracy indexes for multiple characters included in the specific range image.

The second criterion is, for example, a threshold value that can be compared to the second accuracy index.

The electronic data generating device 100 transmits a screen including information indicating the characters contained in the identified specific range image (hereinafter referred to as the "character recognition result") to the user terminal 400.

As described above, when the accuracy of character recognition by the first optical character recognition device 200 for characters included in a target image on a page basis is low, the electronic data generation device 100 obtains the results of character recognition of a specific range image (here, as an example, a line image on a line basis) of a part of the target image including the characters using a second optical character recognition device 300 different from the first optical character recognition device 200, and identifies the characters included in the target image based on the results.

In other words, the electronic data generation device 100 uses multiple optical character recognition devices with different target ranges for character recognition (for example, the first optical character recognition device 200 performs character recognition on a page-by-page basis, and the second optical character recognition device 300 performs character recognition on a line-by-line basis) to perform character recognition on images with low character recognition accuracy, thereby making it possible to achieve highly accurate character recognition.

Moreover, in the electronic data generation system 10, instead of performing all character recognition in the target range using the second optical character recognition device 300, which is more expensive to perform character recognition than the first optical character recognition device 200, the accuracy of character recognition is low with the first optical character recognition device 200, and character recognition is performed in a narrower range than the target range, thereby making it possible to increase the accuracy of character recognition and reduce the cost of character recognition.

Note that the specific range image sent to the second optical character recognition device 300 is not limited to being a line image, but may be an image of a page, a block image, or a character image.

The electronic data generation system 10 may further include at least one optical character recognition device in addition to the first optical character recognition device 200 and the second optical character recognition device 300. In this case, the optical character recognition device may perform character recognition on an image in the same range as either the first optical character recognition device 200 or the second optical character recognition device 300, or may perform character recognition on an image in a narrower range than the range in which the first optical character recognition device 200 and the second optical character recognition device 300 perform character recognition. This enables the electronic data generation system 10 to achieve more accurate character recognition.

Electronic Data Generation Device 100
As shown in FIG. 1, the electronic data generation device 100 includes a memory unit 101, a target image acquisition unit 102, a first information acquisition unit 103, a first judgment unit 104, a specific range identification unit 105, an information transmission unit 106, a second information acquisition unit 107, a second judgment unit 108, a character identification unit 109, and a display processing unit 110.

The memory unit 101 includes, for example, target image information D101a and specific range image information D101b.

The target image information D101a will be described with reference to FIG. 2. FIG. 2 is a database showing an example of the target image information 101a. The target image information 101a is a database in which data related to the target image is stored.

2, the target image information D101a includes items such as [target image ID], [target image], [block information], [line information], [character information], [coordinates], and [first accuracy index]. [Target image ID] stores identification information that can uniquely identify the target image. [Target image] stores the target image. [Block information] stores text information of a block image that indicates a chunk of text in the target image (hereinafter referred to as "block information"). [Line information] stores text information for each line included in the block image (hereinafter referred to as "line information"). [Character information] stores text information of characters included in a line (hereinafter referred to as "character information"). [Coordinates] stores the coordinates in the target image of the characters (character information) indicated by each piece of first text information. [First accuracy index] stores a first accuracy index that indicates the degree of accuracy of the characters indicated by each piece of first text information.

The specific range image information D101b will be described with reference to FIG. 3. FIG. 3 is a database showing an example of specific range image information 101b. Specific range image information 101b is a database in which data related to specific range images is stored.

As shown in FIG. 3, the specific range image information D101b includes items such as a [specific range image ID], a [specific range image], a [second text information], a [coordinates], and a [second accuracy index]. The [specific range image ID] stores identification information that can uniquely identify the specific range image. The [specific range image] stores the specific range image. The [second text information] stores second text information of the character image included in the specific range image (line image in FIG. 3). The [coordinates] stores the coordinates in the target image or the coordinates in the specific range image of the characters indicated by the second text information. The [second accuracy index] stores a second accuracy index that indicates the degree of accuracy of the characters indicated by the second text information.

The target image acquisition unit 102 acquires a target image from a specific device. The specific device is, for example, a scanner device capable of acquiring printouts on paper as an image, or a server device that stores the target image. The target image acquisition unit 102 may transmit the acquired target image to the first optical character recognition device 200.

Returning to FIG. 1, the first information acquisition unit 103 acquires first generation information for the target image from the first optical character recognition device 200. Specifically, the electronic data generation device 100 acquires, for example, first text information (including coordinates) and a first accuracy index for each character image included in the target image from the first optical character recognition device 200. The first generation information is stored in the target image information D101a.

The first determination unit 104 determines whether the first reference index is lower than the first reference. Specifically, when the threshold value as the first reference is "0.6" and the image in a predetermined range of the target image (for example, a line image) is "ABC", if the first accuracy indexes of the first text information "A", "B", and "D" (character image "C" is recognized as "D") are "0.99", "0.99", and "0.55", respectively, the first determination unit 104 determines that the first reference index in the image in the predetermined range (here, the first accuracy index "0.55" corresponding to "D") is lower than the first reference (here, the threshold value "0.6").

In other words, if at least one (or the average value) of the first accuracy indices corresponding to the first text information in a predetermined range of images (e.g., block images, line images, or character images) included in the target image is lower than a threshold value, the electronic data generation device 100 may determine that accurate character recognition has not been performed for the image in the predetermined range.

Based on the judgment result of the first judgment unit 104 (hereinafter referred to as the "first judgment result"), the specific range identification unit 105 identifies a specific range image, which is an image of a partial range of the target image, including characters corresponding to the first reference indicator that has been judged to be lower than the first standard. In this case, the specific range identification unit 105 identifies the coordinates of each character image included in the target image acquired from the first optical character recognition device 200, and identifies a specific range image (e.g., a line image) including characters corresponding to the first reference indicator based on the coordinates.

Here, an overview of an example of a process for identifying a line image (hereinafter referred to as "line identification process") will be described. In the line identification process, first, a black dot image at the left edge of the target image is identified. Next, in the line identification process, a histogram (e.g., the horizontal axis is the distance from the left edge of the target image and the vertical axis is the number of dot images) is generated while identifying black dots in a predetermined width in the height direction from the identified dot image to the right in the horizontal direction. Next, a histogram is generated while similarly identifying black dot images in a predetermined width at an angle shifted from the horizontal. Then, in the line identification process, the coordinates of the upper left dot and the lower right dot are identified based on the histogram, thereby identifying the coordinates of the range of the line image. This makes it possible to appropriately identify a predetermined group of images containing multiple characters.

In addition, in the histogram, if the first distance range in which black dot images are identified and the second distance range in which black dot images are identified exceed a predetermined distance, the range in which black dot images are identified in the first distance range is identified as a first row image, and the range in which black dot images are identified in the second distance range is identified as a second row image that is different from the first row image. This makes it possible to identify row images that are in the same row but included in different block images as different rows.

The information transmission unit 106 transmits the identified specific range image (e.g., a line image) to the second optical character recognition device 300.

The second information acquisition unit 107 acquires second generation information about characters included in the specific range image from the second optical character recognition device 300. Specifically, the electronic data generation device 100 acquires, for example, second text information (including coordinates) and second accuracy indices (for example, "A: 0.99", "B: 0.98", "C: 0.99") about each of the character images (for example, "A", "B", "C") included in a line image (for example, "ABC") that is a specific range image from the second optical character recognition device 200.

The second determination unit 108 determines whether the second reference index is greater than or equal to the second reference. Specifically, the second determination unit 108 determines, for example, whether all of the second accuracy indices for the characters included in the specific range image are equal to or greater than a predetermined threshold. For example, when the predetermined threshold is "0.6" and the specific range image (e.g., a line image) is "ABC", if the second accuracy indices for the second text information "A", "B", and "C" are "0.99", "0.99", and "0.98", respectively, the second determination unit 108 determines that the second reference index is equal to or greater than the second reference. The second determination unit 108 may also determine whether the average value of the second accuracy indices for the characters included in the specific range image is equal to or greater than a predetermined threshold.

The second determination unit 108 may determine that the second reference index is lower than the second reference index when any one of the second accuracy indices for the specific range image is lower than a predetermined threshold. For example, when the specific range image (e.g., a line image) is "ABC", if the second accuracy indices of the second text information "A", "B", and "C" are "0.99", "0.99", and "0.50", respectively, the second determination unit 108 determines that the second reference index is lower than the second reference index.

In this way, the electronic data generation device 100 determines that the second text information is able to recognize characters more accurately for the character image in the target image than the first text information, for example, when all of the second accuracy indices for the specific range image exceed a predetermined threshold value.

The character identification unit 109 identifies characters included in the specific range image (i.e., the target image) based on the judgment result (hereinafter referred to as the "second judgment result") in the second judgment unit 108. Specifically, when the second reference index is equal to or greater than the second reference, the character identification unit 109 identifies characters indicated by the second text information corresponding to the second accuracy index as characters included in the specific range image.

For example, if the second accuracy index (second reference index) of the second text information "A", "B", and "C" are "0.99", "0.99", and "0.98", respectively, the character identification unit 109 identifies "ABC" indicated by the second text information as a character string in the specific range image because the second reference index is equal to or greater than the second reference (e.g., a threshold value of "0.6").

This makes it possible for a character string within a certain range, including characters that are recognized with low accuracy the first time by the first optical character recognition device 200, to be properly recognized the second time by the second optical character recognition device 300.

On the other hand, when the second reference index is lower than the second reference, the character identification unit 109 identifies, based on the result of determining the magnitude relationship between the first accuracy index and the second accuracy index, either the character indicated by the first text information corresponding to the first accuracy index or the character indicated by the second text information corresponding to the second accuracy index as a character in the specific range image.

Specifically, the character identification unit 109 identifies the minimum value indicating the higher value among the minimum value of the second accuracy index of each character image included in the specific range image acquired from the second optical character recognition device 300 and the minimum value of the first accuracy index corresponding to the first text information of the image corresponding to the specific range image acquired from the first optical character recognition device 200. The character identification unit 109 identifies the character indicated by the text information (the first text information or the second text information) corresponding to the accuracy index indicating the identified minimum value as the character included in the specific range image.

For example, for the line image "ABC", if the first accuracy indexes of the first text information "A", "F", and "D" (here, character image "B" is recognized as "F" and "C" is recognized as "D") are "0.99", "0.40", and "0.55", and the second accuracy indexes of the second text information "A", "B", and "E" (character image "C" is recognized as "E") are "0.99", "0.99", and "0.50", respectively, the character identification unit 109 determines that the minimum value of the second accuracy index "0.50" is higher than the minimum value of the first accuracy index "0.40". In this case, the character identification unit 109 identifies "ABE" indicated by the second text information as a character string in the specific range image.

This makes it possible to adopt a character string that is estimated to have been recognized more accurately from a predetermined range of character strings including characters with low accuracy in the first character recognition by the first optical character recognition device 200 and a character string in the second character recognition by the second optical character recognition device 300.

In the above description, the minimum value indicating the higher value between the minimum value of the second accuracy index and the minimum value of the first accuracy index is specified, but this is not limited to this. For example, the character identification unit 109 may use an average value instead of the minimum value, and in this case, the character indicated by the text information corresponding to the accuracy index (second reference index) indicating the higher average value may be specified as the character of the specific range image. For example, the electronic data generation device 100 can reduce the user's effort in making corrections by performing a process of identifying the minimum value indicating the higher value of the minimum values, while performing a process of identifying the higher average value of the average values can reduce the user's effort in making corrections, since it is possible to adopt text that is only required to be reasonably well organized overall.

As described above, when the accuracy of character recognition by the first optical character recognition device 200 for an image in a predetermined range (e.g., line units) of the target image is low, the electronic data generation device 100 obtains the result of character recognition of the image in the predetermined range by the second optical character recognition device 300, which is different from the first optical character recognition device 200. In other words, by performing character recognition using two different optical character recognition devices, the electronic data generation device 100 can achieve highly accurate character recognition for images with low character recognition accuracy.

More specifically, in the electronic data generation system 10, it is desirable to perform character recognition over a wide range (e.g., on a page-by-page basis) using the first optical character recognition device 200, which has a low cost of performing character recognition and a low character recognition accuracy, and to perform character recognition over a narrower range using the second optical character recognition device 300, which has a higher cost of performing character recognition than the first optical character recognition device 200 and a higher character recognition accuracy than the first optical character recognition device 200 (e.g., has a higher accuracy of character recognition on a line-by-line basis). This makes it possible for the electronic data generation system 10 to increase the accuracy of character recognition while reducing the cost of character recognition.

The display processing unit 110 associates the target image with the text information (character recognition result) identified by the character identification unit 109 and displays them on the screen. An example screen will be described below with reference to FIG. 4. FIG. 4 is a diagram showing an example screen displayed on the display unit.

As shown in FIG. 4, the screen T10 includes a first display area T11 and a second display area T12. The first display area T11 is an area in which the target image is displayed. The first display area T11 is, for example, half of one side of the screen. The second display area T12 is an area in which the first text information in the target image and the second text information (line information in FIG. 4) in the specific range image are displayed. The second display area T12 is, for example, half of the other side of the screen.

The display processing unit 110, for example, identifiably displays characters indicated by line information displayed in the second display area among characters included in the target image displayed in the first display area T11. Specifically, as shown in FIG. 4, the display processing unit 110, for example, displays an object OT1 that can identify a line image of the target image, and displays an object OT2 in the line information corresponding to the line image. For example, the display color of the object OT1 is the same color as the display color of the object OT2. This allows the electronic data generation device 100 to provide the user with the correspondence between the target image and the first text information and the second text information, which are the results of character recognition of a predetermined range of the target image, so that the user can easily understand misrecognition of the target image, etc.

<<Modifications>>
The character identification unit 109 may identify either the character indicated by the first text information or the character indicated by the second text information as a character included in the target image based on the number of inputs, which is the number of times a specific range image is input to the second optical character recognition device 300. Specifically, for example, when the number of inputs of a specific range image (e.g., a line image) input to the second optical character recognition device 300 exceeds a predetermined number of times, the character identification unit 10 identifies the character indicated by the first text information corresponding to the first accuracy index as a character included in the target image. As a result, the electronic data generation system 10 can achieve cost reduction by performing character recognition using an optical character recognition device with a lower cost when, for example, the cost of character recognition processing by the second optical character recognition device 300 is higher than the cost of character recognition processing by the first optical character recognition device 200, or when the cost exceeds a certain cost.

In this case, the display processing unit 110 may display the number of inputs, which is the number of times the specific range image was input to the second optical character recognition device 300, at a predetermined position on the screen T10. Specifically, the display processing unit 110 may display at a predetermined position in the second display area. Furthermore, as shown in FIG. 4, the display processing unit 110 may display the number of inputs in association with the second text information corresponding to the line image ("server" in FIG. 4) that became the target of input to the second optical character recognition device 300 after the input number was exceeded ("5 times" in FIG. 4). In this way, the electronic data generation system 10 allows the user to easily grasp the specific range images for which character recognition by the second optical character recognition device 300 could not be performed due to the number of times limit.

First Optical Character Recognition Device 200
1, a description will be given of the configuration of the first optical character recognition device 200. The first optical character recognition device 200 is a device that, for example, when a target image is input, recognizes characters included in the target image and generates, for example, first text information, a first accuracy index, and coordinates (first generated information) for each recognized character.

As shown in FIG. 1, the first optical character recognition device 200 includes, for example, a storage unit 210, a transmission/reception unit 220, and a processing unit 230. The storage unit 210 stores various information. The processing unit 230 executes processing for character recognition. The transmission/reception unit 220 transmits and receives various information to and from the electronic data generation device 100. The processing unit 230 analyzes an image using, for example, a neural network trained to distinguish between characters. The neural network includes, for example, a plurality of convolutional network layers and a recurrent network layer. The processing unit 230 segments, for example, a target image into pages, blocks, lines, or characters. Character recognition is performed on the characters included in the segmented image to generate, for example, first generated information for each character. The processing unit 230 may generate, for example, the first generated information of a segmented image (for example, a target image, a block image, or a line image) as a single group.

Second Optical Character Recognition Device 300
The configuration of the second optical character recognition device 300 will be described with reference to Fig. 1. The second optical character recognition device 300 is a device that, for example, when a specific range image is input, recognizes characters included in the specific range image and generates, for example, second text information, a second accuracy index, and coordinates (second generated information) for each recognized character.

It is desirable that the second optical character recognition device 300 is a device capable of achieving higher accuracy in character identification than the first optical character recognition device 200, for example, when the specific range image is a line image. In this case, in the electronic data generation system 10, it is possible to improve the accuracy of character recognition by using the second optical character recognition device 300, which has high character recognition accuracy for line images, for line images for which the first optical character recognition device 200 has low character recognition accuracy.

As shown in FIG. 1, the second optical character recognition device 300 includes, for example, a storage unit 310, a transmission/reception unit 320, and a processing unit 330. The storage unit 310 stores various information. The processing unit 330 executes processing for character recognition. The transmission/reception unit 320 transmits and receives various information to and from the electronic data generation device 100. The processing unit 330 analyzes an image using, for example, a neural network trained to distinguish between characters. The neural network includes, for example, a plurality of convolutional network layers and a recurrent network layer. The processing unit 330 may be the same as the processing unit 230 of the first optical character recognition device 200, or may be a functional unit that executes processing specialized for character recognition of line images. In this case, the processing unit 330 segments, for example, the line image for each character. Then, the processing unit 330 executes character recognition on the character to generate, for example, second generation information for each character.

User Terminal 400
The configuration of the user terminal 400 will be described with reference to Fig. 1. As shown in Fig. 1, the user terminal 400 includes functional units, for example, a storage unit 410, a transmission/reception unit 420, and a display processing unit 430. Each functional unit is a function realized by, for example, a processor 1001 reading out a program stored in a memory 1002.

The storage unit 410 stores various information. The transmission/reception unit 420 transmits and receives various information to and from the electronic data generation device 100. The various information acquired by the transmission/reception unit 420 is stored in the storage unit 410. The display processing unit 430 displays the screen T10 acquired from the electronic data generation device 100 on the display unit.

===Processing Procedure===
The processing procedure of the electronic data generation system 10 will be described with reference to Fig. 5 and Fig. 6. Fig. 5 is a flowchart showing the processing procedure of the electronic data generation system 10. Fig. 6 is a table showing validity indices for line information, which is text information of a line of character strings. In the following, as an example, character recognition for "100BASE-TX switch set", which is a line of character string included in a target image, will be described.

In step S100, the electronic data generation device 100 acquires a target image from a specific device. The electronic data generation device 100 stores the target image in the storage unit 101. The electronic data generation device 100 transmits the target image to the first optical character recognition device 200.

In step S101, the first optical character recognition device 200 segments the target image to generate first text information, a first accuracy indicator, and coordinates for each character contained in the target image. The first optical character recognition device 200 transmits the first generated information to the electronic data generation device 100.

In step S102, the electronic data generation device 100 associates the first generation information for each character with the target image and stores it in the target image information D101a.

In step S103, the electronic data generating device 100 determines whether the first reference index of the character contained in the target image is lower than the first reference.

If it is determined that the first reference indicator is equal to or greater than the first reference (step S103: NO), in step S104, the electronic data generation device 100 identifies the character indicated by the first text information as a character included in the target image.

If it is determined that the first reference indicator is lower than the first reference (step S103: YES), in step S105, the electronic data generation device 100 refers to the target image information D101a and identifies line information including first text information corresponding to the first reference indicator determined to be lower than the first reference. Specifically, the electronic data generation device 100 identifies the first text information and the first validity indicator shown in FIG. 6(a).

In step S106, the electronic data generation device 100 identifies a line image (specific range image) corresponding to the line information from the target image based on the coordinates included in the identified line information. The electronic data generation device 100 transmits the identified line image to the second optical character recognition device 300.

In step S107, the second optical character recognition device 300 segments the line image to generate second text information, a second accuracy indicator, and coordinates for each character contained in the line image. The second optical character recognition device 300 transmits the second generated information to the electronic data generation device 100.

In step S108, the electronic data generation device 100 stores the second generation information for each character in the specific range image information D101b in association with the specific range image.

In step S109, the electronic data generating device 100 determines whether the second reference indicator in the specific range image is larger than the second reference. Specifically, it determines whether all of the second validity indicators (second reference indicators) of the line information shown in FIG. 6(b) exceed a threshold value (second reference).

If it is determined that the second reference indicator is equal to or greater than the second reference (step S109: YES), in step S110, the electronic data generation device 100 identifies the character indicated by the second text information included in the line information as the character included in the line image.

If it is determined that the second reference index is lower than the second reference (step S109: NO), in step S111, the electronic data generating device 100 compares the first accuracy index for the line information corresponding to the specific range image with the second accuracy index. Specifically, the electronic data generating device 100 compares the smallest value of the first accuracy indexes shown in FIG. 6(a) ("0.32" in FIG. 6(a)) with the smallest value of the second accuracy indexes shown in FIG. 6(b) ("0.57" in FIG. 6(b)).

In step S112, if the electronic data generating device 100 determines that the smallest value of the first accuracy indicators is greater than the smallest value of the second accuracy indicators, it identifies the line information including the first text information (line information in FIG. 6(a)) as a character included in the line image. On the other hand, if the electronic data generating device 100 determines that the smallest value of the second accuracy indicators is greater than the smallest value of the first accuracy indicators, it identifies the line information including the second text information (line information in FIG. 6(b)) as a character included in the line image.

This makes it possible to adopt a character string that is estimated to have been recognized more accurately from a predetermined range of character strings including characters with low accuracy in the first character recognition by the first optical character recognition device 200 and a character string in the second character recognition by the second optical character recognition device 300.

In addition, in step S112, the electronic data generation device 100 may compare the average value of the first accuracy index ("Average value" in FIG. 6(a)) with the average value of the second accuracy index ("Average value" in FIG. 6(b)). In this case, the electronic data generation device 100 identifies the line information (line information in FIG. 6(b)) corresponding to the average value that indicates the larger value among the respective average values as a character included in the line image.

In step S113, the electronic data generation device 100 displays screen T10 shown in FIG. 4 on the display unit of the user terminal 400, which allows a comparison between the target image and the results of character recognition of the target image.

If multiple pieces of line information are identified in step S106, the electronic data generation system 10 repeats steps S106 to S112 the number of times corresponding to the number of pieces of line information identified.

In this way, in the electronic data generation system 10, it is desirable to perform character recognition by the second optical character recognition device 300, which can perform character recognition by line, more appropriately for lines determined to have low accuracy as a result of character recognition by the first optical character recognition device 200, which can perform character recognition by page units more appropriately. Then, the electronic data generation system 10 compares the character recognition results by the first optical character recognition device 200 and the character recognition results by the second optical character recognition device 300, and adopts the character recognition result with the higher accuracy. In other words, in the electronic data generation device 100, by performing character recognition on different character recognition ranges using two different optical character recognition devices, it is possible to achieve highly accurate character recognition for images with low character recognition accuracy.

===Hardware Configuration===
7, an example of a hardware configuration for implementing the electronic data generation device 100, the first optical character recognition device 200, the second optical character recognition device 300, and the user terminal 400 on a computer will be described. FIG. 7 is a diagram showing an example of a hardware configuration of a computer.

As shown in FIG. 7, the computer 1000 includes a processor 1001, a memory 1002, a storage device 1003, an input I/F unit 1004, a data I/F unit 1005, a communication I/F unit 1006, and a display unit 1007.

The processor 1001 is a control unit that controls various processes in the computer 1000 by executing programs stored in the memory 1002.

The memory 1002 is a storage medium such as a RAM (Random Access Memory). The memory 1002 temporarily stores the program code of the program executed by the processor 1001 and data required when the program is executed.

The storage device 1003 is a non-volatile storage medium such as a hard disk drive (HDD) or flash memory. The storage device 1003 stores an operating system and various programs for implementing the above configurations.

The input I/F unit 1004 is a device for receiving input from a user. Specific examples of the input I/F unit 1004 include a keyboard, a mouse, a touch panel, various sensors, and a wearable device. The input I/F unit 1004 may be connected to the computer 1000 via an interface such as a Universal Serial Bus (USB).

The data I/F unit 1005 is a device for inputting data from outside the computer 1000. A specific example of the data I/F unit 1005 is a drive device for reading data stored in various storage media. The data I/F unit 1005 may be provided outside the computer 1000. In that case, the data I/F unit 1005 is connected to the computer 1000 via an interface such as a USB.

The communication I/F unit 1006 is a device for performing data communication via the Internet N, either wired or wirelessly, with devices external to the computer 1000. The communication I/F unit 1006 may be provided external to the computer 1000. In that case, the communication I/F unit 1006 is connected to the computer 1000 via an interface such as a USB.

The display unit 1007 is a device for displaying various types of information. Specific examples of the display unit 1007 include a liquid crystal display, an organic EL (Electro-Luminescence) display, a display of a wearable device, and the like. The display unit 1007 may be provided outside the computer 1000. In that case, the display unit 1007 is connected to the computer 1000 via, for example, a display cable. In addition, when a touch panel is adopted as the input I/F unit 1004, the display unit 1007 can be configured as an integral part of the input I/F unit 1004.

====Summary====
<1> The electronic data generation system 10 in this embodiment includes a target image acquisition unit 102 that acquires a target image including characters from a predetermined device, a first information acquisition unit 103 that acquires, from a first optical character recognition device 200 that recognizes each of a plurality of characters included in the target image as first text information, first text information for each of the plurality of characters and a first accuracy index indicating the degree of recognition accuracy for the character indicated by the first text information, and a first reference index based on at least one of the first accuracy indexes for the plurality of characters, which is a first reference index regarding the accuracy of recognition of the character included in the image. When the first optical character recognition device 200 is judged to be lower than the first standard, the specific range image is an image of a part of the target image including a character corresponding to the first reference indicator judged to be lower than the first standard, and at least one character included in the specific range image is recognized as second text information. The electronic data generation system 10 includes: a second information acquisition unit 107 that acquires second text information about characters included in the specific range image from a second optical character recognition device 300 different from the first optical character recognition device 200, and a character identification unit 109 that identifies characters included in the target image based on the first text information and the second text information. As a result, the electronic data generation system 10 can realize highly accurate character recognition by performing character recognition on an image with low accuracy of character recognition using a plurality of optical character recognition devices with different target ranges for character recognition (for example, the first optical character recognition device 200 performs character recognition on a page basis and the second optical character recognition device 300 performs character recognition on a line basis).

<2> In addition, the second information acquisition unit 107 in the electronic data generation system 10 in this embodiment acquires from the second optical character recognition device 300 second text information for each of at least one character included in the specific range image and a second accuracy index indicating the degree of accuracy of recognition for the character indicated by the second text information, and the character identification unit 109 identifies the character included in the specific range image based on the result of determining the magnitude relationship between the second reference index based on at least one of the second accuracy indexes for each of at least one character included in the specific range image and the second reference regarding the accuracy of recognition of the character included in the image. As a result, the electronic data generation system 10 is able to appropriately recognize a character string in the second character recognition by the second optical character recognition device 300 for a character string in a predetermined range that includes a character with low accuracy in the first character recognition by the first optical character recognition device 200.

<3> Furthermore, when the character identification unit 109 of the electronic data generation system 10 in this embodiment determines that the second reference index for all characters included in the specific range image is equal to or greater than the second reference index, it identifies the characters indicated by the second text information corresponding to the second accuracy index as characters included in the specific range image. This allows the electronic data generation system 10 to properly recognize character strings in the second character recognition by the second optical character recognition device 300 for character strings in a predetermined range that include characters with low accuracy in the first character recognition by the first optical character recognition device 200.

<4> In addition, when the character identification unit 109 of the electronic data generation system 10 in this embodiment determines that the second reference index for at least one of the characters included in the specific range image is lower than the second reference, the character identification unit 109 identifies the characters included in the specific range image based on the result of determining the magnitude relationship between the first accuracy index of at least one character included in the range corresponding to the specific range image in the target image and the second accuracy index of at least one character included in the specific range image. This allows the electronic data generation system 10 to realize more appropriate character recognition using a character string that is estimated to have been recognized more accurately between a character string in a predetermined range that includes characters with low accuracy in the first character recognition by the first optical character recognition device 200 and a character string in the second character recognition by the second optical character recognition device 300.

<5> In addition, the target image acquisition unit 102 of the electronic data generation system 10 in this embodiment acquires a target image that is an image of one page unit, and the information transmission unit 106 transmits a specific range image that is an image of a line unit of text included in the target image to the second optical character recognition device 300. As a result, in the electronic data generation system 10, for example, for a line image in which the accuracy of character recognition in the first optical character recognition device 200 is low, by using the second optical character recognition device 300, which has high accuracy of character recognition for line images, it is possible to improve the accuracy of character recognition.

<6> Furthermore, in the present embodiment, the character identification unit 109 of the electronic data generation system 10 identifies the character indicated by the first text information corresponding to the first accuracy index as a character included in the specific range image when inputting the specific range image to the second optical character recognition device 300 satisfies the conditions for inputting the specific range image to the second optical character recognition device 300. As a result, the electronic data generation system 10 can achieve cost reduction by performing character recognition using a less expensive optical character recognition device when, for example, the cost of character recognition processing by the second optical character recognition device 300 is higher than the cost of character recognition processing by the first optical character recognition device 200, or when the cost exceeds a certain cost.

<7> In addition, the electronic data generation system 10 in this embodiment further includes a display processing unit 110 that displays the target image in a first display area T11 of the screen T10 and displays second text information indicating characters contained in the specific range image in a second display area T12 different from the first display area T11 on the screen T10. This allows the electronic data generation system 10 to provide the user with the correspondence between the target image and the first text information and the second text information, which are the results of character recognition of a specified range of the target image, so that the user can easily grasp misrecognition of the target image, etc.

<8> In addition, the display processing unit 110 in the electronic data generation system 10 in this embodiment displays second text information indicating characters included in the specific range image and first text information indicating characters included in the target image excluding the characters included in the specific range image in the second display area T12, and displays the second text information in an identifiable manner. As a result, the electronic data generation system 10 can display the second text information in the second optical character recognition device 300 in a manner that is easily identifiable by the user, making it possible for the user to easily grasp the degree of misrecognition of the target image, etc.

<9> In addition, the display processing unit 110 of the electronic data generation system 10 in this embodiment displays the number of inputs, which is the number of times that a specific range image has been input to the second optical character recognition device 300, in a predetermined display area of the screen. This allows the electronic data generation system 10 to allow the user to easily grasp the number of times character recognition has been performed by the second optical character recognition device 300.

10...electronic data generation system, 100...electronic data generation device, 101...storage unit, 102...target image acquisition unit, 103...first information acquisition unit, 104...first judgment unit, 105...specific range identification unit, 106...information transmission unit, 107...second information acquisition unit, 108...second judgment unit, 109...character identification unit, 110...display processing unit, 200...first optical character recognition device, 300...second optical character recognition device, 400...user terminal.

Claims (9)

a target image acquisition unit that acquires a target image including characters from a predetermined device;
a first information acquisition unit that acquires, from a first optical character recognition device that recognizes each of a plurality of characters included in the target image as first text information, the first text information for each of the plurality of characters and a first accuracy index that indicates a degree of recognition accuracy for the character indicated by the first text information;
a second information acquisition unit that acquires, when a first reference indicator based on at least one of the first accuracy indicators for the plurality of characters is determined to be lower than a first standard related to accuracy of recognition of characters included in an image, from a second optical character recognition device different from the first optical character recognition device, a specific range image that is an image of a partial range of the target image including characters corresponding to the first reference indicator determined to be lower than the first standard, and that recognizes each of at least one character included in the specific range image as second text information, the second text information for each of at least one character included in the specific range image and a second accuracy indicator indicating a degree of accuracy of recognition for the character indicated by the second text information;
a character identification unit that identifies a character included in the specific range image based on a result of determining whether a second reference indicator based on at least one of the second accuracy indicators for each of at least one character included in the specific range image is larger than a second reference indicator related to accuracy of recognition of the character included in the image; and
Equipped with
when it is determined that the second reference index for at least one character among the characters included in the specific range image is lower than the second reference, the character identification unit identifies a character included in the specific range image based on a result of determining a magnitude relationship between the first accuracy index of at least one character included in a range corresponding to the specific range image in the target image and the second accuracy index of at least one character included in the specific range image.
Information processing system. a target image acquisition unit that acquires a target image including characters from a predetermined device;
a first information acquisition unit that acquires, from a first optical character recognition device that recognizes each of a plurality of characters included in the target image as first text information, the first text information for each of the plurality of characters and a first accuracy index that indicates a degree of recognition accuracy for the character indicated by the first text information;
a second information acquisition unit that acquires the second text information about characters included in a specific range image from a second optical character recognition device different from the first optical character recognition device, the second information acquisition unit being configured to acquire, when a first reference indicator based on at least one of the first accuracy indicators for the plurality of characters is determined to be lower than a first standard related to the accuracy of recognition of characters included in an image, a specific range image that is an image of a partial range of the target image and includes characters corresponding to the first reference indicator determined to be lower than the first standard, the specific range image being input and recognizing each of at least one character included in the specific range image as second text information;
a character identification unit that identifies characters included in the target image based on the first text information and the second text information;
Equipped with
the character identification unit, when the number of times that the specific range image is input to the second optical character recognition device exceeds a predetermined number of times, identifies a character indicated by the first text information corresponding to the first accuracy index as a character included in the specific range image.
Information processing system. when it is determined that the second reference indicator for all characters included in the specific range image is equal to or greater than the second reference, the character identification unit identifies a character indicated by the second text information corresponding to the second accuracy indicator as a character included in the specific range image.
The information processing system according to claim 1 . The target image acquisition unit acquires the target image, which is an image of one page unit,
the second information acquisition unit acquires, from the second optical character recognition device, the second text information about characters included in the specific range image, which is an image of a line of a sentence included in the target image;
The information processing system according to claim 1 . Displaying the target image in a first display area of a screen;
a display processing unit that displays the second text information indicating characters included in the specific range image in a second display area different from the first display area on the screen;
The information processing system according to claim 1 further comprising: The display processing unit is
displaying, in the second display area, the second text information indicating characters included in the specific range image and the first text information indicating characters included in the target image excluding the characters included in the specific range image;
displaying the second text information in an identifiable manner;
6. The information processing system according to claim 5. the display processing unit displays, in a predetermined display area of the screen, an input count, which is the number of times the specific range image has been input to the second optical character recognition device;
6. The information processing system according to claim 5. The computer
Acquiring a target image including text from a predetermined device;
acquiring, from a first optical character recognition device that recognizes each of a plurality of characters included in the target image as first text information, the first text information for each of the plurality of characters and a first accuracy index indicating a degree of recognition accuracy for the character indicated by the first text information;
When a first reference indicator based on at least one of the first accuracy indicators for the plurality of characters is determined to be lower than a first reference regarding accuracy of recognition of characters included in an image, a specific range image is an image of a partial range of the target image including a character corresponding to the first reference indicator determined to be lower than the first reference, and each of at least one character included in the specific range image is recognized as second text information. The specific range image is obtained from a second optical character recognition device different from the first optical character recognition device, and the second text information and a second accuracy indicator indicating a degree of accuracy of recognition for the character indicated by the second text information are obtained for each of at least one character included in the specific range image;
identifying a character included in the specific range image based on a result of determining whether a second reference indicator based on at least one of the second accuracy indicators for each of at least one character included in the specific range image is larger than a second reference indicator related to accuracy of recognition of the character included in the image;
when it is determined that the second reference index for at least one character among the characters included in the specific range image is lower than the second reference, identifying characters included in the specific range image based on a result of determining a magnitude relationship between the first accuracy index of at least one character included in a range corresponding to the specific range image in the target image and the second accuracy index of at least one character included in the specific range image;
An information processing method for performing the above. On the computer,
Acquiring a target image including text from a predetermined device;
acquiring, from a first optical character recognition device that recognizes each of a plurality of characters included in the target image as first text information, the first text information for each of the plurality of characters and a first accuracy index indicating a degree of recognition accuracy for the character indicated by the first text information;
When a first reference indicator based on at least one of the first accuracy indicators for the plurality of characters is determined to be lower than a first reference regarding accuracy of recognition of characters included in an image, a specific range image is an image of a partial range of the target image including a character corresponding to the first reference indicator determined to be lower than the first reference, and each of at least one character included in the specific range image is recognized as second text information. The specific range image is obtained from a second optical character recognition device different from the first optical character recognition device, and the second text information and a second accuracy indicator indicating a degree of accuracy of recognition for the character indicated by the second text information are obtained for each of at least one character included in the specific range image;
identifying a character included in the specific range image based on a result of determining whether a second reference indicator based on at least one of the second accuracy indicators for each of at least one character included in the specific range image is larger than a second reference indicator related to accuracy of recognition of the character included in the image;
when it is determined that the second reference index for at least one character among the characters included in the specific range image is lower than the second reference, identifying characters included in the specific range image based on a result of determining a magnitude relationship between the first accuracy index of at least one character included in a range corresponding to the specific range image in the target image and the second accuracy index of at least one character included in the specific range image;
A program that executes the following.

Images