JP7107138B2 - Information processing device, information processing method and program - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and a program.

2. Description of the Related Art In document management, it is common practice to mask a portion of a document that should be kept secret so that it cannot be visually recognized, and then manage the document. For example, Japanese Patent Application Laid-Open No. 2002-200001 discloses a document management system that saves a scanned image of a document after masking it. A document management system is disclosed that generates images of .

JP 2007-140958 A

However, the invention according to Patent Document 1 masks a predetermined area within a document. Therefore, when characters or the like are handwritten in a document, there is a possibility that the handwritten characters or the like may be masked depending on the position where the characters or the like are written.

An object of one aspect of the present invention is to provide an information processing apparatus and the like capable of appropriately managing documents.

In one aspect, an information processing apparatus includes an acquisition unit that acquires a form image obtained by scanning a form having pre-printed content and a handwritten object that is handwritten, and identifies the content from the form image. a content specifying unit, an extracting unit that extracts the handwritten object from the paper image, and a masking region that excludes the handwritten object from a content region, which is a peripheral region that includes the content, to specify a masking region so that the masking region cannot be visually recognized. and a generation unit for generating the paper image subjected to image processing, and the content identification unit identifies the content according to text extracted by character recognition of the paper image .

In one aspect, document management can be properly performed.

1 is a schematic diagram showing a configuration example of a document management system; FIG. It is a block diagram which shows the structural example of PC. FIG. 10 is an explanatory diagram relating to specifying processing of a content area; FIG. 10 is an explanatory diagram relating to identifying processing of a handwritten entry area; FIG. 10 is an explanatory diagram regarding mask processing; 4 is a flow chart showing an example of a processing procedure executed by a PC; FIG. 11 is an explanatory diagram of specifying processing of a content area according to Embodiment 2; FIG. 10 is a flow chart showing an example of a processing procedure executed by a PC according to Embodiment 2; FIG. 4 is a functional block diagram showing the operation of the PC of the form described above;

Hereinafter, the present invention will be described in detail based on the drawings showing its embodiments.
(Embodiment 1)
FIG. 1 is a schematic diagram showing a configuration example of a document management system. In this embodiment, as an example of a document to be managed, a test form (fill-in form) in which a respondent (student) answers a question in a school test by handwriting is assumed. This section describes a document management system that stores documents after masking them. The document management system includes an information processing device 1 , a server 2 and a scanner 3 . The information processing device 1 and the server 2 are connected for communication via a network N such as the Internet.

The information processing device 1 is an information processing device capable of various types of information processing and transmission/reception of information, and is, for example, a personal computer, a multifunction device, a multifunction terminal, or the like. In the present embodiment, the information processing apparatus 1 is assumed to be a personal computer, and for the sake of brevity, it will be replaced with PC1 in the following description. A PC 1 according to this embodiment is a terminal device operated by a school teacher, and is connected to a scanner 3 for optically reading a paper medium. The PC 1 uploads the image of the test paper scanned by the scanner 3 to the server 2 and stores it. In this case, the PC 1 performs masking on the content area in which content such as sentences, pictures, photographs, etc., which is pre-printed on the test paper and which may correspond to the copyrighted work is printed. Upload your image.

The server 2 is a so-called cloud server, and stores and saves the image of the test paper uploaded from the PC 1 in a database. Although detailed explanation is omitted because it is irrelevant to the present embodiment, the server 2 scores, tabulates, and analyzes the answers written on the test paper, notifies the students and teachers of the analysis results, and gives individual students Provide appropriate teaching materials.

Note that the image storage location is not limited to the server 2 on the cloud, and may be stored locally in the PC 1, for example.

In the following description, the PC 1, which is a local terminal, performs the masking process, but the server 2, which is the upload destination of the image, may perform the masking process.

When the test sheet is scanned and the image is saved as described above, if the test sheet contains a copyrighted work, saving the scanned image as it is may constitute an illegal copy of the copyrighted work. In order to avoid such a situation, it is conceivable to mask the part that may correspond to the copyrighted work.

On the other hand, for example, when a respondent writes an answer overlapping a question sentence, depending on the position where the answer is written, there is a possibility that the answer cannot be identified by masking (see FIGS. 4, 5, etc.). Therefore, in the present embodiment, the PC 1 identifies an area in which a handwritten answer (object) is entered from the image, and excludes the identified area from the masking target, thereby preventing the answer from being unidentifiable.

FIG. 2 is a block diagram showing a configuration example of the PC1. The PC 1 includes a control section 11 , a main storage section 12 , a communication section 13 , a display section 14 , an input section 15 and an auxiliary storage section 16 .
The control unit 11 has an arithmetic processing unit such as one or more CPU (Central Processing Unit), MPU (Micro-Processing Unit), etc. By reading and executing the program P stored in the auxiliary storage unit 16, Various information processing, control processing, and the like relating to the PC 1 are performed. The main storage unit 12 is a temporary storage area such as a RAM, and temporarily stores data necessary for the control unit 11 to execute arithmetic processing. The communication unit 13 is a communication module that transmits and receives information to and from the outside. The display unit 14 is a display device such as a liquid crystal display or an organic EL (Electro Luminescence) display, and displays an image given from the control unit 11 . The input unit 15 is an operation interface such as a keyboard and a mouse, and inputs operation contents to the control unit 11 . The auxiliary storage unit 16 is a non-volatile storage area such as a hard disk or a large-capacity memory, and stores programs P and other data necessary for the control unit 11 to execute processing.

FIG. 3 is an explanatory diagram of content area specifying processing. FIG. 3 conceptually illustrates how a region in which a content is printed is specified from an image obtained by scanning a test sheet. The outline of this embodiment will be described below.

From the scanner 3, the PC 1 scans a test sheet (answer sheet) in which a respondent fills in a handwritten answer (handwritten object), and on which contents that may correspond to a copyrighted work are printed. to get The test form is an entry form for filling in answers to questions. For example, as shown in FIG. 3, question sentences, question contents, answer columns, and the like are printed with characters, symbols, and the like. In addition to characters and symbols, the printed content (content) of the test paper may include, for example, figures, photographs, pictures, etc., and the content is not particularly limited.

The respondent fills in the answers by handwriting on the test paper. In addition to characters, the content to be entered may be strokes, symbols, graphics, or the like, and may be any object that is handwritten.

From the sheet image, the PC 1 identifies locations corresponding to question sentences, question content, answer columns, etc., that is, contents, and identifies a content area, which is a peripheral area containing the contents. In FIG. 3, the hatched portion corresponds to the content area. For example, the PC 1 identifies the content area from the paper image of the test paper on which no answers have been entered. The PC 1 identifies a content area that may correspond to the copyrighted work, and masks the area.

Various methods are conceivable for specifying the content area. In FIG. 3, as an example, three methods are conceptually illustrated.

As a first method, the PC 1 performs character recognition on the paper image, extracts the text printed on the test paper, identifies the extracted text as content, and identifies the area surrounding the text as the content area. . The upper right portion of FIG. 3 illustrates how character recognition is performed to identify a content area. The PC 1 performs character recognition on the paper image and extracts individual texts printed on the test paper. The PC 1 identifies each text as content, and sets (identifies) a hatched rectangular area as the content area so that each text is covered.

As a second method, the PC 1 identifies the content area according to the pixel value distribution (pixel density) in the paper image. The center of the right side of FIG. 3 illustrates how the content area is specified based on the pixel value distribution. The PC 1 refers to the pixel value of each pixel in the paper image, discriminates the distribution of the pixel values, and finds parts different from the background of the test paper, such as characters, symbols, strokes, etc. printed on the test paper. identify. For example, when focusing on a certain pixel in the paper image, the PC 1 counts the number of pixels whose RGB value difference from the background color is equal to or greater than a threshold in the pixel and its surrounding pixels (for example, 3×3 pixels). When the number of counted pixels is equal to or greater than a predetermined number (for example, 3 pixels), the PC 1 identifies the portion including the pixels as content.

The PC 1 sets (specifies) a content area so as to cover the specified portion. As a result, symbols and strokes that are difficult to identify by character recognition can be masked.

The pixel value referred to by PC1 may be a color-related value such as an RGB value, or may be a brightness-related value such as luminance.

As a third method, the PC 1 identifies content from the paper image using a discriminator (learned model) that has been constructed in advance by machine learning such as deep learning. The lower part of FIG. 3 illustrates how a paper image is input to the discriminator and the discrimination result of the content area is obtained as an output. For example, the PC 1 is a classifier related to a CNN (Convolution Neural Network) or the like. A discriminator that has learned features is used. Note that the machine learning method is not limited to CNN, and other neural networks, SVMs (Support Vector Machines), Bayesian networks, decision trees, and the like may also be used. The PC 1 inputs the paper image to the discriminator, extracts the image feature amount, and acquires the discrimination result of discriminating the content as an output value. The PC 1 sets (specifies) a content area according to the identification result output from the identifier.

PC1 identifies the content area using any of the methods exemplified above. Alternatively, the PC 1 may identify the content area by combining a plurality of techniques.

4A and 4B are explanatory diagrams related to the specifying process of the handwritten entry area. FIG. 4 conceptually illustrates how handwritten answers (objects) are extracted from a paper image of a test sheet on which answers have already been written, and a handwritten entry area surrounding the extracted answers is specified.

In the image example shown in the upper part of FIG. 4, the answer written by the respondent protrudes from the range in which the answer should be written, and partially overlaps the question sentence (content). As already mentioned, if the content area is masked as it is in such a state, part of the answer will not be visible. Therefore, the PC 1 extracts the handwritten answers from the paper image, and specifies a handwritten entry area to be excluded from the mask target so as to enclose the extracted answers.

For example, the PC 1 extracts a handwritten answer by comparing a form image in which answers have been written and a form image in which no answers have been written, and extracting the difference between the two. For example, the PC 1 compares the pixel values in the form image in which the answers have been filled in with the pixel values in the form image in which the answers have not yet been filled in, which are the same pixels as the pixels in the filled form image. . The PC1 compares the pixel values of each pixel and extracts the difference between the two.

An image example showing the extracted answers is shown in the middle of FIG. As shown in FIG. 4, the PC 1 extracts objects corresponding to handwritten answers. The PC 1 sets (identifies), for example, a rectangular frame area as a handwritten entry area so as to surround the object.

FIG. 5 is an explanatory diagram relating to mask processing. FIG. 5 shows how masking is performed on the image of the test sheet on which the answers are written.

The PC 1 performs image processing, that is, mask processing, on the form image in which the answers have been written so that the area excluding the handwritten entry area from the above-identified content area is invisible. The process is conceptually illustrated on the right side of FIG. On the right side of FIG. 5, the black rectangular frame represents the content area to be masked, and the white rectangular frame represents the handwritten entry area. The PC 1 compares the content area shown in the upper right part of FIG. 5 with the handwritten entry area shown in the middle part of the right part of FIG. 5, and specifies the area shown in the lower right part of FIG. 5 excluding the handwritten entry area. . As a result, as indicated by the dotted-line rectangular frame in the lower right part of FIG. 5, the overlapping portion between the content area and the handwritten entry area, that is, the overlapping portion of the question sentence and the answer, is excluded from the masking target.

As shown on the left side of FIG. 5, the PC 1 performs masking to make the specified area invisible, and generates a mask image for storage. The PC 1 transmits the generated mask image to the server 2 and stores it on the database.

FIG. 6 is a flowchart showing an example of a processing procedure executed by the PC1. Based on FIG. 6, the contents of processing executed by the PC 1 will be described.
The control unit 11 of the PC 1 acquires from the scanner 3 a paper image obtained by scanning a test paper having pre-printed content and handwritten objects (answers) (step S11). Contents are sentences, pictures, photographs, etc. that may correspond to copyrighted works, but the specific contents thereof are not particularly limited.

The control unit 11 identifies the content from the paper image (step S12). Specifically, the control unit 11 identifies the content from the blank image of the answer sheet. For example, the control unit 11 may extract text printed on the test paper by character recognition on the paper image, and specify the extracted text as content. Further, for example, the control unit 11 may specify the content according to the pixel value distribution (pixel density) in the paper image. Further, for example, the control unit 11 may specify content by using a learned classifier using a sheet image in which no object is written as training data for the content.

The control unit 11 extracts an object (answer) handwritten on the test form from the form image (step S13). Specifically, the control unit 11 compares the test sheet image on which the answers have been written and the test sheet image on which the answers have not been written, and extracts handwritten characters and the like. Control unit 11 specifies a handwritten entry area so as to surround the extracted object (step S14).

The control unit 11 identifies a masking area excluding the handwritten entry area identified in step S14 from the content area, which is a peripheral area containing the content identified in step S12, in the paper image, and makes the identified masking area invisible. A mask image is generated by image processing (step S15). The control unit 11 transmits the generated mask image to the server 2 (step S16), and ends the series of processes.

In the above description, a test sheet is given as an example of an entry form to be masked, but a questionnaire form in which answers to predetermined questions (questions) are entered may be used.

Also, in the above description, the content area is made invisible by masking, but the present embodiment is not limited to this. For example, the content area is cut (removed) from the paper image. may Even in this case, the portion corresponding to the content (copyrighted work) can be appropriately processed, and the same effect as above can be obtained. In this way, the PC 1 only needs to be able to generate a paper image in which the content area is invisible, and the image processing method is not limited to masking.

In addition, in the above description, content is identified from a form image with no answers written, but the present embodiment is not limited to this, and content may be identified from a form image with answers already written. For example, the PC 1 may specify the content printed on the test paper by extracting characters, symbols, graphics, etc. that commonly appear in each image from the paper images of many respondents. Further, for example, the PC 1 may refer to a database storing predetermined image data as content corresponding to the copyrighted work, and specify the content from the paper image. In this way, the PC 1 only needs to be able to identify the content from the scanned image of the test form, and whether or not the answer is entered is not particularly limited.

Further, in the above description, a handwritten answer (object) is extracted by taking the difference between the blank form image and the filled form image, but the present embodiment is not limited to this. For example, the PC 1 may use a trained model (classifier) that has learned the characteristics of human handwritten characters to evaluate the handwriting-likeness of the characters included in the paper image, and extract handwritten characters (objects). good. In this way, the PC 1 only needs to be able to extract handwritten objects from the form image, and the method is not limited to comparison between the blank form image and the completed form image.

As described above, according to the first embodiment, it is possible to appropriately perform document management by performing masking so that an object such as a handwritten character is not covered.

Further, according to the first embodiment, it is possible to appropriately identify the handwritten entry area by comparing the filled-in sheet image and the unfilled-in sheet image.

Further, according to the first embodiment, it is possible to appropriately specify the content area by character recognition on the paper image.

Further, according to the first embodiment, it is possible to appropriately specify the content area by determining the distribution (pixel density) of the pixel values in the paper image.

Further, according to the first embodiment, by using a classifier that has already learned the features of the content in the paper image, it is possible to identify the content region with higher accuracy.

(Embodiment 2)
In this embodiment, a form of specifying a content area to be masked according to text content recognized from a test sheet will be described. In addition, the same code|symbol is attached|subjected about the content which overlaps with Embodiment 1, and description is abbreviate|omitted.
7A and 7B are explanatory diagrams of content area specifying processing according to the second embodiment. Based on FIG. 7, the outline of this embodiment will be described.

An example of a paper image (test paper) is shown in the upper part of FIG. In this embodiment, the PC 1 performs character recognition on the paper image, extracts the text printed on the test paper, and determines the content area to be masked according to the contents of the extracted text.

For example, the PC 1 identifies the question text from the text of the test paper, and excludes the area where the question text is printed from the content areas to be masked. The question text is a sentence that instructs the respondent to answer, and the possibility that the question text itself contains a copyrighted work is low. Therefore, the PC 1 excludes the question text from being masked, thereby preventing unnecessary parts from being masked.

Specifically, the PC 1 recognizes a specific keyword representing a question from the text of the test paper, and identifies the sentence following the keyword as the question sentence. The keyword is, for example, a character string representing a question number. In FIG. 7, "Q4" and "Q1" correspond to the question numbers. For example, the PC 1 refers to a table storing templates (rules) of character strings representing question numbers, and identifies keywords representing question numbers from the text of the test paper. The PC 1 identifies the identified keyword (question number) and the sentence following the keyword as the question sentence. The PC 1 excludes the area where the question text is printed from the content area so that the question text is not masked.

In the above description, the question text is specified based on rules, but the PC 1 may use a technique such as semantic analysis to specify (recognize) the question text directly from the text of the test paper. For example, the PC 1 parses the text of the test paper, and identifies the syntax of predetermined expressions used in question sentences such as imperative sentences and request sentences. For example, in the test sheet of FIG. 7, PC1 specifies commands such as "answer the question" and "choose the correct one". The PC 1 identifies a series of sentences including the identified imperative sentences as question sentences and excludes them from the content area.

As described above, the PC 1 identifies the question sentence from the text extracted by character recognition, and excludes the area in which the question sentence is printed from the content area. As a result, portions that are unlikely to correspond to copyrighted works are excluded from the masking target, and only portions that are highly likely to correspond to copyrighted works are masked.

In the above, a method of indirectly specifying areas that are likely to correspond to copyrighted works is adopted by specifying areas to be excluded from masking. On the other hand, the PC 1 may directly identify a content area that is highly likely to correspond to a copyrighted work according to the content of the text on the test sheet.

For example, the PC 1 identifies a sentence of a predetermined expression representing the position where the content (copyrighted work) is printed from the text, and identifies the content area to be masked according to the content of the sentence. The sentence is a sentence suggesting a description part of the content, and is a sentence that can determine the question format, such as whether or not the question is a reading comprehension question. To explain using the example of FIG. 7, for example, the PC 1 discriminates (specifies) the sentence "Read the following sentences A and B" appearing at the beginning of the question sentence following "Question 4", and from the sentence, Judge that it is a reading comprehension problem. In this case, the PC 1 identifies the paragraph following the sentence as content, and sets (identifies) a content area so as to cover the paragraph. In this way, the PC 1 identifies the position where the content is written according to the content of the text extracted from the paper image. For example, the PC 1 stores a table of sentences that suggest the position of the content, determines the presence or absence of the corresponding sentence while referring to the table, and identifies the content area.

Alternatively, for example, the PC 1 may directly identify the content area by identifying the keyword of the text corresponding to the content (copyrighted work) from the text extracted from the paper image. For example, the PC 1 makes a database of keywords of sentences corresponding to copyrighted works and stores them in advance, searches (specifies) sentences containing the keyword from the text, and specifies only the sentences containing the keyword as contents.

As described above, the PC 1 identifies areas to be masked and areas not to be masked from the text extracted by character recognition, and specifies content areas to be masked. In the lower part of FIG. 7, the specified content area is indicated by hatching. As in the first embodiment, the PC 1 performs mask processing on the content area and generates a mask image. As described above, masking is applied only to portions that are likely to contain copyrighted material, and unnecessary portions are prevented from being masked.

FIG. 8 is a flow chart showing an example of a processing procedure executed by the PC 1 according to the second embodiment.
After acquiring a paper image obtained by scanning the test paper (step S11), the control section 11 of the PC 1 executes the following processes. The control unit 11 performs character recognition on the paper image and extracts the text printed on the test paper (step S201). The control unit 11 identifies content according to the extracted text (step S202). For example, the control unit 11 identifies a specific keyword representing a question from the text extracted in step S201, identifies sentences containing the keyword as question sentences, and excludes them from the content. Further, for example, the control unit 11 performs semantic analysis on the text extracted in step S201, specifies syntax such as an imperative sentence and a request sentence, specifies sentences containing the syntax as question sentences, and excludes them from the content. Further, for example, the control unit 11 identifies a sentence of a predetermined expression representing the position where the content is printed from the text extracted in step S201, and identifies the content to be masked according to the content of the sentence. Further, for example, the control unit 11 identifies keywords of sentences corresponding to content from the text extracted in step S201, and identifies sentences including the keywords as content. The control unit 11 shifts the process to step S13.

As described above, according to the second embodiment, the question sentence is specified from the text printed on the test sheet and excluded from the content area, thereby excluding the part that is unlikely to contain the content from the mask target. can be done.

Further, according to the second embodiment, by estimating (specifying) the content area from the text printed on the test paper, it is possible to mask only the portion that is highly likely to contain the content.

(Embodiment 3)
FIG. 9 is a functional block diagram showing the operation of the PC 1 of the form described above. When the control unit 11 executes the program P, the PC 1 operates as follows.
Acquisition unit 91 acquires a form image obtained by scanning an entry form having pre-printed content and handwritten objects. The content specifying unit 92 specifies the content from the paper image. The extraction unit 93 extracts the handwritten object from the paper image. The generation unit 94 identifies a masking area obtained by excluding the handwritten object from a content area, which is a peripheral area including the content, and generates the paper image by performing image processing so that the masking area cannot be visually recognized.

The third embodiment is as described above, and other aspects are the same as those of the first and second embodiments, so corresponding parts are given the same reference numerals and detailed description thereof will be omitted.

The embodiments disclosed this time are illustrative in all respects and should not be considered restrictive. The scope of the present invention is indicated by the scope of the claims rather than the above-described meaning, and is intended to include all modifications within the scope and meaning equivalent to the scope of the claims.

1 PC (information processing device)
11 Control Unit 12 Main Storage Unit 13 Communication Unit 14 Display Unit 15 Input Unit 16 Auxiliary Storage Unit P Program 2 Server 3 Scanner

Claims (10)

an acquisition unit for acquiring a scanned form image of a form having pre-printed content and handwritten handwritten objects;
a content identification unit that identifies the content from the paper image;
an extraction unit that extracts the handwritten object from the paper image;
a generation unit that identifies a masking area excluding the handwritten object from a content area, which is a peripheral area including the content, and generates the paper image that has undergone image processing so that the masking area is invisible ;
The information processing apparatus , wherein the content specifying unit specifies the content according to text extracted by character recognition on the paper image . An entry specifying unit that specifies a handwritten entry area surrounding the extracted handwritten object,
The information processing apparatus according to claim 1, wherein the generation unit specifies a masking area obtained by removing the handwritten entry area from the content area. 3. The extracting unit according to claim 1, wherein the extracting unit extracts the handwritten object based on the paper image on which the handwritten object has been written and the paper image on which the handwritten object has not been written. Information processing equipment. The content identification unit
Identifying a specific keyword from the extracted text,
4. The information processing apparatus according to any one of claims 1 to 3, wherein sentences containing said keyword are excluded from said content. The content identification unit
Identifying a specific keyword from the extracted text,
4. The information processing apparatus according to any one of claims 1 to 3, wherein only sentences containing said keyword are set as said contents. The information processing apparatus according to any one of claims 1 to 5 , wherein the content specifying unit specifies a position where the content is printed according to the content of the extracted text. The information processing apparatus according to any one of claims 1 to 6 , wherein the content specifying unit specifies the content from a pixel density of the paper image or predetermined image data. wherein the content identification unit identifies the content from the form image acquired by the acquisition unit, using a plurality of form images in which the handwritten object is not written as training data for the content, and using a learned classifier. The information processing apparatus according to any one of claims 1 to 7 . obtaining a scanned form image of a form having pre-printed content and a handwritten object filled in by hand;
identifying the content from the paper image;
extracting the handwritten object from the paper image;
A process of specifying a masking area excluding the handwritten object from a content area, which is a peripheral area including the content, and generating the paper image by performing image processing so that the masking area cannot be visually recognized ,
The content is specified according to the text extracted by character recognition on the paper image.
An information processing method characterized in that a computer executes processing. obtaining a scanned form image of a form having pre-printed content and a handwritten object filled in by hand;
identifying the content from the paper image;
extracting the handwritten object from the paper image;
A process of specifying a masking area excluding the handwritten object from a content area, which is a peripheral area including the content, and generating the paper image by performing image processing so that the masking area cannot be visually recognized ,
The content is specified according to the text extracted by character recognition on the paper image.
A program characterized by causing a computer to execute processing.

Images