CN103154974B - Character recognition device, character recognition method and character recognition system - Google Patents

Abstract

The character recognition device (1) has: an image reading unit (101), which inputs an image; a character region detection unit (103), which detects a character region of the image; a character region segmentation unit (104), which divides the A character area; a character recognition unit (105) that performs character recognition for each individual character on characters present in the divided area, and outputs one or more candidates for character recognition processing results for each individual character; generation of first character string transfer data a part (106), which is input to the candidate, calculates a weight value for transition to the candidate, and generates first character string transition data based on a set of the candidate and the weight value; and a WFST processing part (109), which according to the first The character string transition data performs state transitions in sequence, accumulates the weight values in each state transition and calculates the cumulative weight value of each state transition, and outputs the result of more than one state transition according to the cumulative weight value.

Description

Character recognition device, character recognition method and character recognition system

technical field

The present invention relates to a character recognition device, a character recognition method, a character recognition system and a character recognition program.

Background technique

Conventionally, as exemplified in the following Patent Documents 1 to 2 and Non-Patent Documents 1 to 3, techniques for recognizing characters existing in a three-dimensional space such as billboards from scene images are known. In these technologies, recognition accuracy is improved by using external data such as knowledge of words and location information of shooting locations in order to cope with problems specific to scene images such as changes in brightness or distortion of characters.

For example, in Patent Document 1, a character string extracted from a billboard in an image is compared with a phonebook database, and by judging the degree of coincidence with the advertiser data or phone number data contained in the phonebook database, the extraction is judged. Whether the string of is associated with the advertiser.

In addition, in Patent Document 2, the location information acquisition unit and the orientation information acquisition unit equipped in the character recognition device are used to specify the location and direction of the photograph, and the determined location and orientation are compared with the map database, and the matching store name is used. or place name information as word knowledge, thus improving the recognition accuracy.

In addition, a weighted finite state transducer (Weighted Finite State Transducer, hereinafter referred to as "WFST") is used as a high-speed method with high versatility and scalability in the field of speech recognition or language processing. The device represents the set of symbol string transitions and weights as state transitions. In connection with this WFST, in the field of character recognition, as exemplified in Non-Patent Document 1 and Non-Patent Document 2, it has been proposed to obtain character recognition based on the premise of obtaining character strings written in separate words like English. method of results. In addition, Non-Patent Document 3 proposes a method of performing error correction using WFST after outputting a character recognition result in Japanese.

prior art literature

patent documents

Patent Document 1: Japanese Patent No. 3360030

Patent Document 2: Japanese Patent No. 4591353

non-patent literature

Non-Patent Document 1: "A Weighted Finite-State Framework for Correcting Errors in NaturalScene OCR", ICDAR 2007Vol.2, pp.889-893

Non-Patent Document 2: "The image Text Recognition Graph (iTRG)", ICME 2009, pp.266-269

Non-Patent Document 3: Revision of Finite State Transu ヂ ュー ス い た Text Mistakes, Proceedings C2-5 of the Annual Conference of the Society for Speech Processing, pp.332-335, 2009

Contents of the invention

The problem to be solved by the invention

However, in the method described in Patent Document 1, it is necessary to collate with a large amount of word knowledge contained in a telephone directory database or the like, and it may not be possible to sufficiently speed up character recognition processing. In addition, in the method described in Patent Document 2, a position information acquisition unit or an orientation information acquisition unit is separately required, and the device configuration may become complicated.

In addition, in Non-Patent Document 1 and Non-Patent Document 2, it is presupposed that there is a partition written language in which there is a partition between words. That is, the premise is that the vocabulary involved in WFST processing has already been cut out in advance. In addition, in Non-Patent Document 2, in character-by-character cutting out of character recognition, character recognition is performed using repeated cutting positions, and WFST is used to express, but misrecognition may occur in the result of character recognition may not be able to cope.

In addition, in Non-Patent Document 1 and Non-Patent Document 3, the problem of misrecognition caused by repeated cutout positions is dealt with by character fusion/separation, but there are many types of Japanese characters, and there are various characters in the actual environment. There are a variety of character designs, so it is necessary to collect a huge combination. In addition, in Non-Patent Document 3, the result of performing character recognition is temporarily used, and the premise is that the result of character recognition is obtained with high accuracy to a certain extent. Therefore, when many corrections based on language processing are performed in original character recognition, it may become difficult to perform corrections based on character shapes. In addition, it may not be possible to cope with the correction of the detection omission of the character region in the original character recognition.

Therefore, the present invention has been made in view of the above-mentioned problems, and its object is to provide a device capable of performing character recognition from scene images with high precision and high speed using a simplified device configuration without using an external database such as a phone book. A character recognition device, a character recognition method, a character recognition system, and a character recognition program.

means of solving problems

In order to solve the above-mentioned problems, the character recognition device of the present invention is characterized in that it includes: an image input unit that inputs an image including a character to be recognized; and a character region detection unit that detects a region in the image where the character exists. That is, a character area; a character area segmentation unit that divides the character area in units of individual characters; a character recognition unit that performs each individual character on the characters that exist in the segmented area divided by the character area segmentation unit. The character recognition process of outputting one or more candidates of character recognition processing results for a single character; the first character string transition data generation unit, which is input to the candidate, calculates a weight value for transition to the candidate, and generates a value based on The character string transition data of the group of the candidate and the weight value is the first character string transition data; and a finite state conversion unit, which sequentially performs state transition according to the first character string transition data, and converts each state transition The weight values are accumulated to calculate the cumulative weight value of each state transition, and more than one result of the state transition is output according to the cumulative weight value.

In addition, the character recognition method of the present invention is characterized in that it has: an image input step in which the image input unit inputs an image including a character to be recognized; a character region detection step in which the character region detection unit detects the existence of the character in the image. The area of the character area is the character area; the character area segmentation step, the character area segmentation unit divides the character area with a single character as a unit; The character recognition processing of each individual character is carried out, and the candidate of more than one character recognition processing result is output to a single character; the first character string transfer data generation step, the first character string transfer data generation unit inputs the candidate, and calculates For the weight value transferred to the candidate, generating character string transfer data based on the set of the candidate and the weight value, that is, the first character string transfer data; and a finite state conversion step, the finite state conversion unit according to the first character string transfer data 1 The character string transition data sequentially performs state transitions, accumulates the weight values in each state transition to calculate the cumulative weight value of each state transition, and outputs one or more results of the state transition according to the cumulative weight value.

In addition, the character recognition system of the present invention includes a terminal and a server, and the character recognition system is characterized in that the terminal has: an image input unit that inputs an image including a character to be recognized; a character area detection unit that detects the image Among them, the area where the character exists is the character area; the character area segmentation unit divides the character area in units of a single character; the character recognition unit exists in the segmented area divided by the character area segmentation unit Performing character recognition processing for each single character of the characters, and outputting more than one candidate for the character recognition processing result for a single character, the server has: a first character string transfer data generation unit, which is input to the candidate, calculating a weight value for a transition to the candidate, generating first character string transition data based on a set of the candidate and the weight value; and a finite state conversion unit configured based on the first character The serial transition data sequentially performs state transitions, accumulates the weight values in each state transition to calculate the cumulative weight value of each state transition, and outputs one or more results of the state transition according to the cumulative weight value.

In addition, the character recognition program of the present invention is characterized in that the computer operates as the following means: an image input means that inputs an image including a character to be recognized; a character area detection means that detects the The area in which characters exist is a character area; a character area division unit that divides the character area in units of individual characters; a character recognition unit that performs an operation on the characters existing in the division area divided by the character area division unit. Character recognition processing for each single character, outputting one or more candidates for character recognition processing results for a single character; a first character string transition data generation unit that inputs the candidates and calculates weight values for transitions to the candidates , generating character string transition data based on the group of the candidate and the weight value, that is, the first character string transition data; and a finite state conversion unit, which sequentially performs state transition according to the first character string transition data, and converts each state The weight values in the transitions are accumulated to calculate the cumulative weight value of each state transition, and more than one result of the state transition is output according to the cumulative weight value.

According to the character recognition device, character recognition method, character recognition system, and character recognition program of the present invention, since no database such as an external telephone directory is used, it is not necessary to check with a large amount of word knowledge contained in the telephone directory database or the like. , speeding up character recognition processing can be realized. In addition, since a position information acquisition unit, an azimuth information acquisition unit, and the like are not required, the device configuration can be simplified. With such an apparatus configuration, character recognition from scene images can be performed with high precision and high speed.

In addition, in the present invention, the character recognition device may further include a second character string transition data generating unit that inputs a keyword from the user, and the second character string transition data generating unit generates a character string of the keyword. The transition data is the second character string transition data, and the finite state conversion unit performs a composite operation on the first character string transition data and the second character string transition data, thereby determining whether the keywords.

According to the present invention, the character recognition device of the present invention can be effectively applied as a device for judging whether or not a user-input keyword exists in an image by using the first character string transfer data itself of the character recognition candidate group as a search table for an image.

In addition, in the present invention, the character recognition device may further include a third character string transition data generating unit that generates character string transition data for each vocabulary existing in the vocabulary database. That is, the third character string transition data, and the finite state conversion unit performs a synthesis operation on the first character string transition data and the third character string transition data, thereby detecting words existing in the image.

According to the present invention, the character recognition device of the present invention can be effectively applied as a vocabulary detection device by performing a composite operation on the first character string transition data of the character recognition candidate group and the third character string transition data in the vocabulary database.

In addition, in the present invention, the character recognition unit assigns priority to each of the plurality of candidates and outputs them, and the first character string transition data generation unit calculates the weight value according to the priority. .

According to the present invention, a specific method for calculating the weight value by the first character string transition data generation unit can be provided.

In addition, in the present invention, the character recognition unit uses at least two different recognition methods to perform the character recognition processing, and the first character string transfer data generating unit The weight value is calculated based on the output quantity of the candidates and the priority order.

According to the present invention, a specific method for calculating the weight value by the first character string transition data generation unit can be provided.

In addition, in the present invention, the first character string transition data generation unit may calculate the weight value in consideration of character string transitions of words registered in a language database.

According to the present invention, a specific method for calculating the weight value by the first character string transition data generation unit can be provided.

Furthermore, in the present invention, the first character string transition data generation unit may correct the weight value according to the position of the candidate in the image or the character size of the candidate.

According to the present invention, it is possible to provide a specific method for the first character string transfer data generation unit to correct the weight value. In addition, the accuracy of vocabulary detection can be improved by modifying the weight value.

In addition, in the present invention, in the case where the character region dividing unit divides the character region using a plurality of division patterns to generate multiple kinds of the divided regions, the character recognition unit may performing the character recognition process on each of the plurality of divided regions, the first character string transition data generation unit generates the first character string transition data for each of the candidates of the plurality of divided regions, and the finite state conversion unit A result of state transition in which the cumulative weight value is higher in all of the plurality of divided regions is output as the result.

According to the present invention, it is possible to appropriately deal with even the case where the character region segmentation unit has performed over-segmentation (Over-segmentation).

In addition, in the present invention, the first character string transition data generating unit may transition from the initial state of the character string transition to the first null transition of the candidate, that is, the first null transition, and transition from the candidate to the character string. The first character string transition data is generated including the second null transition which is the final state of the null transition and the third null transition which is the candidate null transition for skipping the candidate in units of single characters.

According to the present invention, the first string transition data includes the first dummy transition, the second dummy transition, and the third dummy transition, thereby improving the efficiency of the first string transition data and the second string transition data or the third string transition data. The precision of the compositing operation on the data.

In addition, in the present invention, when the character recognition unit outputs the candidate of the character recognition processing result, it may also output the recognition information indicating the separation between words, and the first character string transition data The generation unit adds the identification information to generate the first character string transition data, and the finite state transition unit performs the state transition in units of parts separated by two pieces of the identification information when performing the state transition. transfer.

According to the present invention, character recognition can be performed with high accuracy even for languages written with partitions by using identification information indicating partitions.

In addition, in the present invention, when the character recognition unit outputs the candidate of the character recognition processing result, it may also output the position information of the candidate in the image, and the first character string The transition data generation unit adds the position information to generate the first character string transition data, and the finite state conversion unit adds the position information to output the result.

According to the present invention, by using positional information, it is possible to specify at which position in an image the result of character recognition is located.

In addition, in the present invention, the vocabulary database has classification information for vocabulary, and the second character string transition data generation unit or the third character string transition data generation unit adds the classification information to generate For the second character string transfer data or the third character string transfer data, the finite state conversion unit adds the classification information to output the result.

According to the present invention, by using classification information, it is possible to determine which category a result of character recognition belongs to.

In addition, in the present invention, the character recognition device may include a vocabulary classification correlation vector storage unit that stores a vocabulary classification correlation vector indicating the correlation between a vocabulary and the classification information. , the first character string transfer data generating unit adds the candidate and the weight value in the first character string transfer data to the value of the vocabulary classification relevance vector, and uses the classification information with the largest value as Classification information corresponding to the candidate, and modifying the weight value for the candidate based on the classification information.

According to the present invention, it is possible to provide a specific method for the first character string transfer data generation unit to correct the weight value. In addition, the accuracy of vocabulary detection can be improved by modifying the weight value.

The effect of the invention

According to the present invention, it is possible to provide a character recognition device, character recognition method, character recognition system, and character recognition program.

Description of drawings

FIG. 1 is a schematic configuration diagram showing functional components of a character recognition device 1 .

FIG. 2 is a hardware configuration diagram of the character recognition device 1 .

FIG. 3 is a flowchart showing the overall processing flow of the present embodiment.

FIG. 4 is a diagram for explaining the operation of the character region dividing unit 104 .

FIG. 5 is a diagram showing an example of the first WFST data generated by the first character string transition data generating unit 106 .

FIG. 6 is a diagram showing processing when the character region dividing unit 104 performs over-segmentation.

FIG. 7 is a diagram for explaining that the first character string transition data generation unit 106 adjusts weight values according to the size and position of characters.

FIG. 8 is a flowchart showing WFST calculation processing.

FIG. 9 is a diagram showing a map of a WFST synthesis operation.

FIG. 10 shows an example of processing in Modification 1 of the WFST synthesis calculation.

FIG. 11 shows an example of processing in Modification 1 of the WFST synthesis calculation.

FIG. 12 shows an example of processing in Modification 1 of the WFST synthesis calculation.

FIG. 13 is a schematic configuration diagram showing functional components of the character recognition device 1 in Variation 2 of the WFST synthesis calculation.

FIG. 14 shows an example of processing in Modification 2 of the WFST synthesis calculation.

FIG. 15 shows an example of processing in Modification 2 of the WFST synthesis calculation.

FIG. 16 shows an example of processing in Modification 3 of the WFST synthesis calculation.

FIG. 17 shows an example of processing in Variation 4 of the WFST synthesis calculation.

FIG. 18 shows an example of processing in Variation 4 of the WFST synthesis calculation.

FIG. 19 is a schematic configuration diagram showing functional components of the character recognition system 100 .

detailed description

Preferred embodiments of the character recognition device, character recognition method, character recognition system, and character recognition program of the present invention will be described in detail below with reference to the drawings. In addition, in the description of the drawings, the same reference numerals are attached to the same elements, and overlapping descriptions are omitted.

(Overall structure of the character recognition device 1)

The character recognition device 1 according to the embodiment of the present invention detects a character region from a scene image and performs character recognition (for example, keyword detection, generation of a search table, etc.). FIG. 1 is a schematic configuration diagram showing functional components of a character recognition device 1 , and FIG. 2 is a hardware configuration diagram of the character recognition device 1 . As shown in Figure 2, the character recognition device 1 is constituted as a common computer system, which physically includes a camera, etc., in addition to main storage devices such as CPU11, ROM12, and RAM13, a keyboard, and a mouse, as well as a camera for reading in images. input device 14, output device 15 such as a display, communication module 16 such as a network card for sending and receiving data with other devices, auxiliary storage device 17 such as a hard disk, etc. . The image read by the input device 14 may be an image taken by the own device or an image taken by another device. By reading predetermined computer software on hardware such as CPU11, ROM12, RAM13, under the control of CPU11, input device 14, output device 15, communication module 16 are operated, and main storage device 12, 13 or auxiliary storage device 17 The reading and writing of data in the device realizes various functions of the character recognition device 1 described later.

As shown in FIG. 1 , the character recognition device 1 has the following parts as functional structural elements: an image reading unit 101 (equivalent to an “image input unit” in the claims), an image binarization unit 102, and a character area detection unit. Section 103 (corresponding to "character region detection unit" in the claims), character region segmentation unit 104 (corresponding to "character region segmentation unit" in the claims), character recognition section 105 (corresponding to the "character region segmentation unit" in the claims "character recognition unit"), the first character string transition data generation unit 106 (equivalent to the "first character string transition data generation unit" in the claims), the second character string transition data generation unit 107 (equivalent to the claims "the second character string transition data generating unit" in the claims), the third character string transition data generating unit 108 (equivalent to the "third character string transition data generating unit" in the claims), the WFST processing unit 109 ( Equivalent to the "finite state conversion unit" in the claims), character string detection unit 110 (corresponding to the "character string detection unit" in the claims), and vocabulary DB111 (corresponding to the "vocabulary database" in the claims ). Hereinafter, the operation of each component of the character recognition device 1 will be described with reference to the flowchart of FIG. 3 .

(1) Image reading

The image reading unit 101 inputs an image including characters to be recognized (step S1, which corresponds to the "image input step" in the claims). Document images such as printed documents are captured by scanners, and high-speed and high-accuracy recognition is possible, so character recognition as document images is performed by an existing document OCR engine (step S2). Then, the first character string transition data generation unit 106 generates data represented by WFST (hereinafter referred to as "first WFST data". Corresponds to "first character string transition data" in the claims) based on the candidate group of the recognition result ( Step S3, "equivalent to the first character string transition data generating step" in the claims). In addition, when the number of characters of the recognition result obtained by the existing document OCR engine is more than a predetermined number and the recognition accuracy is more than a predetermined value, it is determined to be a document, and the WFST calculation process of step S10 is not performed. For images that are too small or too large in resolution, resize them to be suitable for character recognition.

(2) Image binarization

If the image input in step S1 is not a document image, the image binarization unit 102 performs image binarization (step S4). Image binarization is performed according to local light and shade, and can also deal with low-contrast situations. The detection of black characters on a white background can also reverse the light and shade of the original image to detect white characters on a black background. In addition, noise removal is performed on regions other than conspicuous characters through occlusion processing such as dilation and contraction.

(3) Character area detection

The character area detection unit 103 detects a character area (step S5, which corresponds to the "character area detection step" in the claims). The "character area" refers to an area in which a character to be recognized exists in the image input in step S1, or an area in which there is a possibility. As for the detection of the character area, there is known a method of performing detection by statistically learning the features of the shape as described in Reference 1 below. In this device, a mark is added to each region by carrying out the marking process, and according to the shape characteristics of each region (circularity, number of holes, number of composed regions, size/aspect ratio of the outer perimeter rectangle, area of the marked region and non-marked region Ratio, etc.) to determine whether it is a character area and detect it.

<Reference 1> "A learning-based method to detect and segment text from scene images", JOURNAL OF ZHEJIANG UNIVERSITY-SCIENCE A Volume 8, Number4, pp.568-574

In this embodiment, in order to perform filtering by the WFST process described later, it is preferable to detect as many regions that may be characters as possible in advance rather than not detecting non-character noise regions from the beginning so that no omissions will occur. . Therefore, a method of connecting adjacent regions by expansion and contraction processing, a method of decomposing connected regions, and a method of removing noise around characters are also methods of detecting character regions. In addition, various methods (methods using edges or hues, high-level character region connection processing, etc.) can be added to this detection method.

(4) String candidate detection, cut out in units of single characters

The character area segmentation unit 104 detects the character string candidates in the character area, and divides the character area in units of individual characters (hereinafter referred to as "cutting out") (step S6, which corresponds to the "character area segmentation step" in the claims) ). Specifically, the character region dividing unit 104 first detects character lines. Assuming that a character line is composed of three or more characters, detection is performed based on the change of the area size/interval/angle of the character area. Marking is performed on each detected character line, and the character line is reduced based on the median value, average value, mode value, etc. of the angles assigned to each marked area. FIG. 4 is a diagram for explaining the operation of the character region dividing unit 104 . As shown in FIG. 4 , horizontal/vertical shear deformation is performed by performing a search based on the angle of the character line for each character line L, and deformation distortion of both shearing and rotation of the character is corrected. In FIG. 4 , image A1 shows an image before correction in which a rotated character string is written, and image A2 shows an image after correction of inclination in the character string direction by shearing and deforming character lines in the vertical direction.

The character area dividing unit 104 removes noise from the distortion-corrected image A2, and then obtains the character interval in the character row direction, and cuts out each character. Cutting out in units of individual characters uses a histogram obtained by adding pixels perpendicular to the direction of the character string to find candidates between characters and the size of the area obtained when character line detection is performed. Based on the median value, average value, and mode value, etc., multiple overlapping cutout positions are determined. 4 shows that the character string M in the corrected image A2 is sheared and deformed in the horizontal direction while changing the angle little by little, thereby generating a plurality of character strings M1, M2, M3, and these The character strings M1, M2, and M3 are cut out in units of single characters. The character string Y2 shows a character string obtained by cutting out the character string M2 in units of characters, and the number of blank areas in this case is four. A "blank area" refers to an area between characters, which is indicated by a symbol K in FIG. 4 . In addition, reference numeral Y3 indicates a character string obtained by cutting out the character string M3 in units of individual characters, and the number of blank areas in this case is seven. In this embodiment, the character region dividing unit 104 adopts the case where the number and area of blank regions are the largest as the character region dividing result. In the example of FIG. 4 , the character string Y3 is a character string obtained by dividing the finally selected character region. In addition, character position detection/character recognition processing based on multiple methods/parameters, such as character recognition of a single character line, is carried out through the existing OCR engine, and a single character is cut out at each possible character cut-out position, A state transition that becomes over-segmented such that overlapping of positions is allowed is obtained.

(5) Character recognition

The character recognition unit 105 performs character recognition processing for each character for each character present in the divided region (shown by a symbol D in FIG. 4 ) after the character region division unit 104 divides in step S6, and outputs one character for each character. Candidates for the above character recognition processing results (hereinafter referred to as "character recognition candidate group" or simply "candidates") (step S7 corresponds to "character recognition step" in the claims). Character recognition is performed in units of single characters by obtaining character recognition results from multiple engines.

(6) WFST data generation

The first character string transition data generation unit 106 collects duplicate candidates from the candidate group of the recognition result obtained in step S7, and generates WFST data (hereinafter also referred to as "first WFST data". It corresponds to "first WFST data" in the claims. Character string transfer data") (step S8 corresponds to the "first character string transfer data generating step" in the claims). That is, the first character string transition data generation unit 106 inputs candidates (one or more candidates for each character) of character recognition processing results from the character recognition unit 105, calculates a weight value for transition to the candidate, and generates a character string based on these candidates and The first WFST data of the group of weight values.

When the character recognition unit 105 assigns priority to and outputs a plurality of candidates of the character recognition processing results, the first character string transition data generation unit 106 calculates the weight value based on the priority. In addition, when the character recognition unit 105 has performed character recognition processing using at least two different recognition methods, the first character string transition data generation unit 106 outputs candidates based on the character recognition processing results in the different recognition methods Quantity and the above priority order to calculate the above weight value. Here, the weight values of duplicate candidates are synthesized by product/sum, so that the more the same candidate appears in each character recognition result, the smaller the weight value is. That is, in this embodiment, it can be said that the smaller the weight value is, the closer the candidate is to the actual correct result. In addition, the first character string transition data generation unit 106 may calculate the above-mentioned weight value in consideration of character string transitions of words registered in the language database.

FIG. 5 shows an example of the first WFST data generated by the first character string transition data generating unit 106 . As shown in FIG. 5 , the first WFST data is a state transition to which a plurality of candidates and their weight values are assigned. When there are multiple character recognition results, state transitions with the same initial state are arranged in parallel. In the example of FIG. 5, for example, when the actual correct result of the character recognition process is "ドコモ", a plurality of candidates in the character recognition process are, for example, "ド", "ト", "人", "コ". , "two", "mouth", "モ", "mao", "t", etc., and the respective weight values are "0.2", "0.4", "0.6", "0.2", "0.5", "0.6 ", "0.2", "0.4", "0.5".

In order to detect the keyword in the middle of the article, the first WFST data generated from the character recognition candidate group includes the ε transition from the initial state of the character string transition to each character candidate (empty transition without input and output, which corresponds to the "first empty transition"), the ε transition of the final state from each character candidate to the character string (corresponding to the "second empty transition" in the claims), weighting value to avoid capturing noise as a character And the ε transition of each character candidate is skipped in units of a single character (equivalent to "the third empty transition" in the claims). In FIG. 5 , the first idle transition is indicated by symbol E1 , the second idle transition is indicated by symbol E2 , the third idle transition is indicated by symbol E3 , and the weight value of the third idle transition is shown as "2.0", for example. In addition, in order to be able to perform calculations with an optimum processing size, the first WFST data can be divided into units of multiple lines or units of a fixed number of characters for calculations, and the results are combined for use.

Here, in the case of over-segmentation in step S6, as shown in FIG. 6, character recognition is performed in units of individual characters for each repeated position, and the transition of the cut-out position of repeated characters is expressed as a first WFST data. In other words, when the character region dividing unit 104 divides the character region using a plurality of division patterns and generates multiple kinds of divided regions (that is, in the case of over-segmentation), the character recognition unit 105 characterizes the various divided regions respectively. In the recognition process, the first character string transition data generating unit 106 generates first WFST data for character candidates in each of the plurality of types of divided regions.

The example of FIG. 6 shows that when the actual correct result of the character recognition process is, for example, "Forum" ((A) of FIG. 6 ), the division position is determined by a plurality of methods and cutout parameters, and is performed at a plurality of division positions. The results of the individual character recognition processing are shown ((B) and (C) of FIG. 6 ). The recognition result of "fbnim" was obtained in the result shown in (B) of FIG. 6 , and the recognition result of "rurn" was obtained in the result shown in (C) of FIG. 6 . In addition, in the part of "b" in the result of (B) of FIG. 6 , the first candidate is "b" and the second candidate is "o" due to noise. It is believed that the noise is due to the entry of a part of the upper right of "F" when cutting out. Regarding these two results, the first character string transition data generating unit 106 generates one first WFST data as shown in (D) of FIG. 6 . In addition, in the example of FIG. 6 , the ε transition from the initial state to the intermediate state, the ε transition from the intermediate state to the final state, and the weighted ε transition for skipping characters are omitted. In addition, the generated first WFST data is later used in the WFST synthesis calculation with the vocabulary data (see (E) and (F) in FIG. 6 ), and the WFST processing unit 109 outputs the accumulated weights in the entirety of the various divided regions. The upper value ("forum" matching the vocabulary data in the example of FIG. 6) is the result, which will be described later.

In addition, in order to further improve the accuracy of detecting meaningful words from scene images, etc., the first character string transition data generating unit 106 corrects Weights. In the example of FIG. 7 , (A) of FIG. 7 shows an image A3 in which characters enter. (B) of FIG. 7 shows the weight value firstly calculated by the first character string transition data generation unit 106 . For the character candidate "この先", the weight value "0.13" is calculated. Similarly, weight values "0.15", "0.15", "0.20", and "0.21" are calculated for "Co., Ltd.", "10km", "Kiyomizu Temple", and "hotel", respectively.

Here, the first character string transition data generation unit 106 adjusts the initially calculated weight value using the information shown in (C) and (D) of FIG. 7 . (C) of FIG. 7 is information showing the statistical spatial distribution of the value of a keyword. In this example, the center, upper left, and lower right of the image are parts with high value as keywords, and in (C) of FIG. 7 , the values as keywords are displayed in shades of color. In the part where the color is displayed darker, "1" is assigned as a weight coefficient because of a higher value as a keyword. In the portion where the color is displayed lighter, "2.5" is assigned as a weight factor because of its low value as a keyword. (D) of FIG. 7 shows a weight coefficient table corresponding to character sizes. A character with a size of "24" is assumed to have a higher value as a keyword due to its larger size, and "1" is assigned as a weight coefficient. A character with a size of "8" is assumed to be less valuable as a keyword due to its smaller size, and "2.2" is assigned as a weight factor.

(E) of FIG. 7 shows the result obtained by the first character string transition data generation unit 106 adjusting the weight value calculated initially using the information shown in (C) and (D) of FIG. 7 . Weighting is carried out by multiplying the initially calculated weight value with the weight coefficients of (C) and (D) of Figure 7 to increase the priority of words located in larger character regions or as words located in positions with higher value order. For example, for the word "Kiyomizu Temple", the weight value "0.20" initially calculated is multiplied by the weight value "1.5" of the spatial distribution and the weight value "1.0" of the character size in (C) of FIG. 7, and "0.3" is given as Adjusted weight value. Through the above processing, before the weight value adjustment, for example, the vocabulary "この先" has a smaller weight value than the vocabulary "Kiyomizu Temple", but through the weight value adjustment, the weight value of the vocabulary "この先" is greater than the weight value of the vocabulary "Kiyomizu Temple". That is, it can be said that through weight value adjustment, words that actually have value as keywords are adjusted to have smaller weight values.

(7) WFST operation processing

(WFST calculation process overall flow)

The WFST processing part 109 and the character string detection part 110 summarize the 1st WFST data generated in steps S3 and S8 into a 1st WFST data (step S9), and then perform WFST calculation processing (step S10, which is equivalent to "" in the claims finite state transition steps"). The "WFST calculation processing" performed by the WFST processing unit 109 and the character string detection unit 110 includes WFST synthesis calculation (equivalent to the "synthesis calculation" in the claims), and includes the following series of processing: WFST processing unit 109 State transitions are carried out sequentially, and the weight values in each state transition are accumulated to calculate the cumulative weight value of each state transition. When the result of more than one state transition is output to the string detection unit 110 according to the cumulative weight value, the string detection The unit 110 detects one or more character strings as character string recognition results based on the cumulative weight value. FIG. 8 is a flowchart showing WFST calculation processing. The first WFST data generated from the character recognition candidate group is not only used for word detection in WFST arithmetic processing with the vocabulary DB 111 (see FIG. 1 ), but also the first WFST data itself of the character recognition candidate group can be used as a search table for images.

In FIG. 8 , the processing flow consisting of steps S10-1, S10-2, S10-3, and S10-4 uses the first WFST data itself of the character recognition candidate group as a search table for images to determine whether The processing flow when there is a case where the user enters a keyword. In this case, the WFST processing unit 109 compares the first WFST data generated through the series of processes of steps S1 to S9 and the WFST data corresponding to the keyword input by the user (corresponding to the "second character string transfer data" in the claims, Hereinafter, it is referred to as "second WFST data.") WFST arithmetic processing is performed to determine whether or not a keyword exists in the image.

Specifically, first, a keyword is input from the user, and the second character string transition data generation unit 107 generates second WFST data for the keyword (step S10-1). In (A) of FIG. 8 , the second WFST data generated for the keyword (search vocabulary) input by the user is mapped. Next, the WFST processing unit 109 performs a WFST composition calculation using the second WFST data generated in step S10-1 and the first WFST data generated by a series of processing in steps S1 to S9 (step S10-2). Next, the WFST processing unit 109 performs calculations to obtain an optimal route based on the result of the WFST combination calculation in step S10-2 (step S10-3). Finally, the character string detection unit 110 outputs the determination result of whether or not the keyword is input by the user based on the calculation result of the optimal route, or the weight in the determination result (step S10-4).

In addition, in FIG. 8, the processing flow comprised by steps S10-5, S10-6, S10-7, S10-8, and S10-9 is a processing flow in the case of vocabulary detection of the WFST calculation process related to vocabulary DB111. In this case, the WFST processing unit 109 compares the first WFST data generated through the series of processes of steps S1 to S9 and the WFST data of each vocabulary existing in the vocabulary DB 111 (corresponding to the "third character string transfer data" in the claims). ", hereinafter referred to as "third WFST data".) WFST arithmetic processing is performed to detect vocabulary existing in the image.

Specifically, first, the third character string transition data generation unit 108 generates third WFST data for each vocabulary existing in the vocabulary DB 111 (step S10-5). Next, the WFST processing unit 109 performs WFST synthesis calculation using the third WFST data generated in step S10-5 and the first WFST data generated by a series of processing in steps S1 to S9 (step S10-6). Next, the WFST processing unit 109 performs calculations to obtain an optimal route based on the result of the WFST combination calculation in step S10-6 (step S10-7). Finally, the character string detection unit 110 outputs vocabulary in order of weight value in the optimal path (step S10-8). And, when the dictionary is classified into categories, or when there is a category information dictionary, category information is output (step S10-9).

(WFST synthesis operation)

FIG. 9 shows a map of the WFST synthesis operation (steps S10-2 and 10-6 in FIG. 8). The WFST synthesis operation is an operation for comparing state transitions represented by two pieces of WFST data, and extracting transitions of words in units of common single characters. As for the result of the WFST synthesis operation, the weight value of each transition is recalculated according to the weight values of the two transitions synthesized. The result of the WFST synthesis operation is to calculate the best path (transition with a small weight) based on the weight value of the state transition. The results obtained. In addition, in the case of over-segmentation, the WFST processing unit 109 outputs the result of state transition in which the cumulative weight value is higher in the entirety of the plurality of divisional regions as the result of the WFST synthesis calculation.

In vocabulary detection (processing flow consisting of steps S10-5 to S10-9), the first WFST data (the same as that shown in FIG. 5 ) and the first WFST data (the same as shown in FIG. The WFST synthesis operation of the third WFST data of the vocabulary data in the vocabulary DB 111 shown in (B) of FIG. The sequence of values detects the vocabulary. (C) of FIG. 9 shows that "ドコモ", "human hair", and "population" are obtained as the result of the WFST synthesis operation, and the respective weight values are "0.2+0.2+0.2=0.6", "0.6+2.0+ 0.4=2.8", "0.6+0.6=1.2". Therefore, "docomo" having the smallest weight value is detected as an optimal route, and the character string detection unit 110 outputs "docomo" as a result of vocabulary detection. In addition, since there is an ε transition for skipping characters, it is also possible to perform abbreviation detection such as detecting "天ぷらおむすび" as "天むす". In addition, when the vocabulary DB 111 has a large-scale structure, even if there is no completely matching vocabulary, it may be possible to extract a part of the vocabulary as a correct result vocabulary.

In the case of a search table (processing flow consisting of steps S10-1 to S10-4), the search keyword to be searched in the image is represented by the second WFST data, and the first WFST data of the character recognition candidate group is performed. The WFST synthesis operation. It is judged whether or not the transition from the initial state to the final state is obtained by the transition of the combined two WFST data in this WFST compositing operation. Thereby, it can be determined whether or not a search keyword exists in the character recognition candidate group, that is, whether or not a keyword input by the user exists in the image. In addition, it is also possible to assign an order to a plurality of images according to transferred weight values. In the example of Figure 9, when the keyword input by the user is any one of "ドコモ", "human hair", and "population", the transition from the initial state to the final state can be obtained through the transfer of the two synthesized WFST data Therefore, it is determined that the keyword input by the user exists in the image. However, the weight values of "ドコモ", "human hair", and "population" are "0.2+0.2+0.2=0.6", "0.6+2.0+0.4=2.8", and "0.6+0.6=1.2", respectively, so it is detected "ドコモ" with the smallest weight value is taken as the best path. When the keyword input by the user is "ドコモ", the character string detection unit 110 outputs the smallest weight value as a result of the vocabulary search.

As shown in (C) of FIG. 9 , the vocabulary data exemplified in (B) of FIG. 9 is represented by transition of vocabulary character by character with the initial state set to the same. In addition, weights may be assigned using frequency information obtained by separate statistical processing, learning information input by the user, character string lengths of vocabulary, and the like. In addition, the 1st WFST data and the 2nd WFST data and the 1st WFST data and the 3rd WFST data that are the objects of comparison do not necessarily need to be data in the same format, as long as they are data that represent the state transition of characters and are comparable data format.

(WFST synthesis operation, deformation 1)

In this embodiment, various modifications are assumed in the WFST synthesis calculation, and a modification 1 will be described below. In Variation 1, the vocabulary DB 111 has classification information on vocabulary, and the second character string transition data generating unit 107 or the third character string transition data generating unit 108 adds the classification information to generate the second WFST data or the third WFST data, and the WFST processing unit 109 adds the classification information to perform WFST composition calculation, and outputs the result. That is, in Variation 1, as the vocabulary DB 111, by synthetically computing the WFST data whose input is vocabulary and the output is classification information, and the WFST data of the vocabulary DB 111 to which classification information has been added, it is possible to obtain keywords used for detecting keywords at the same time. classified information or add classified information to keywords. In this case, in order to allow the same vocabulary to have a plurality of classification information, the serial number of the classification information is added to the input of the final state of the vocabulary DB 111, and the classification information (that is, the content of the classification information) is added to the output. In addition, in the final state of the first WFST data generated by character recognition, transitions to the maximum number of consecutive numbers of classification information used in the same vocabulary on the vocabulary DB 111 are added.

FIG. 10 shows an example of processing in Variation 1. (A) of FIG. 10 shows an example of vocabulary data to which classification information is added. In the case of a search table, (A) of FIG. 10 shows the second WFST data with classification information generated by the second character string transition data generation unit 107 . In the case of vocabulary detection, (A) of FIG. 10 shows the third WFST data with classification information generated by the third character string transition data generation unit 108 . Classification information is information for identifying a plurality of categories of the same vocabulary. For example, to the word "つばめ", two classification information of consecutive numbers "0000" and "0001", that is, "shinkansen" and "birds", are added. In addition, "<eps>" in (A) of FIG. case output. (B) of FIG. 10 shows a case where a serial number of classification information is added to the result of character recognition and converted into first WFST data. For example, in the character recognition result "つばめ", in the final state of its WFST data, the maximum number of consecutive numbers (in the example of FIG. For the transition of the two consecutive numbers "0000" and "0001"). Combination calculation of the second WFST data or the third WFST data shown in (A) of FIG. 10 and the first WFST data shown in FIG. 10(B) is performed, and (C) of FIG. 10 shows the result of the synthesis calculation. After comparing the two WFST data, only two matched paths are taken out, however, only classification information is shown as a result of (C) of FIG. 10 by the <eps> null transition.

FIG. 11 shows the same situation as that of FIG. 10 , but differs in that the result of character recognition is "すずめ". (C) of FIG. 11 shows the result of the synthesis operation. After comparing the two WFST data, only one matching path is taken out. However, as in (C) of FIG. As a result only classification information is shown.

Fig. 12 shows the same situation as that of Fig. 10, except that there is no <eps> transition. The result of the synthesis operation is shown in (C) of Figure 12. After comparing the two WFST data, only two matching paths are taken out, however, since there is no <eps> transition, the vocabulary and Categorize information on both sides.

(WFST synthesis operation, deformation 2)

Next, modification 2 will be described. In Variation 2, as shown in FIG. 13 , the character recognition device 1 further includes a lexical-category-relevance-vector storage unit 112 (corresponding to "vocabulary-category-relevance vector storage means" in the claims). The vocabulary classification correlation vector storage unit 112 is used to store a vocabulary classification correlation vector showing the correlation between vocabulary and classification information. The first character string transition data generation unit 106 adds the candidate of the character recognition processing result in the first WFST data generated by itself and the weight value of the candidate to the value of the vocabulary classification correlation vector. Next, the first character string transition data generation unit 106 uses the classification information having the largest value as the classification information corresponding to the candidate, and corrects the weight value for the candidate based on the classification information. Then, the WFST processing unit 109 performs WFST composition calculations based on the corrected weight values.

That is, in Modification 2, the relationship between classification information and vocabulary is prepared in advance as a table using the synonym relational database, whereby the priority of classification information can be changed. For example, as shown in FIG. 14, if there is a food menu as a vocabulary and a vector representing the relationship between food categories ((B) in FIG. 14 , vocabulary classification correlation vector) as classification information, the vectors of the detected vocabulary are compared to In addition, the food category with the largest vector can be detected as the food category of the detection vocabulary. Conversely, it is also possible to change the priority of the detected food menu by re-determining the weight of the vocabulary based on the acquired category order or vector value.

(A) of FIG. 14 shows vocabulary detected in character recognition (“dumpling”, “スープ”, etc., each food menu), and FIG. 14 (B) shows a correspondence table of each food menu and food category (vocabulary categorical relevance vector). (C) of FIG. 14 shows the example which calculated the vector value corresponding to each food menu of FIG. 14 (A) with reference to the correspondence table of FIG. 14 (B). In this example, since the highest vector value is calculated for "Chinese food", the category of the vocabulary shown in (A) of FIG. 14 is determined to be "Chinese food". Finally, (D) of FIG. 14 shows the situation after the weight value of each food menu in (A) of FIG. 14 is corrected reflecting the category “Chinese food” judged in (C) of FIG. 14 .

FIG. 15 is a diagram showing a calculation procedure of a correction weight value shown in (D) of FIG. 14 . The correction weight value shown in (D) of FIG. 14 is calculated through the steps of (A) to (D) of FIG. 15 . (A) of FIG. 15 shows the sum of detected class weight values, and corresponds to (B) and (C) of FIG. 14 . (B) of FIG. 15 shows the case where the reciprocal of the vocabulary weight value shown in FIG. 14 (A), that is, (1/vocabulary weight value) is taken and multiplied by the category weight value of each vocabulary. For example, for "dumplings", take the reciprocal of the vocabulary weight value shown in (A) of Figure 14, that is, "0.3", that is, take "1/0.3", and respectively compare with the category weight value shown in Figure 15 (A) "0, 1.0, 0" are multiplied together to obtain the calculation result of "0, 3.33, 0". Similarly, for "スープ", take the reciprocal of the vocabulary weight value shown in (A) of Figure 14, that is, "0.45", that is, "1/0.45", and multiply it by the category weight shown in (A) of Figure 15 Values "0, 0.3, 0.7", resulting in calculation results of "0, 0.67, 1.56".

(C) of FIG. 15 shows a case where the result of (B) of FIG. 15 is multiplied by the sum of (A) of FIG. 15 . For example, for "dumplings", the result of (B) in Figure 15, that is, "0, 3.33, 0" is multiplied with the sum of (A) in Figure 15, that is, "0.5, 2.8, 0.7", thereby obtaining "0 ,9.33,0" calculation result. Similarly, for "スープ", the result of (B) in Figure 15, that is, "0, 0.67, 1.56" is multiplied by the sum of (A) in Figure 15, that is, "0.5, 2.8, 0.7", thereby obtaining " 0, 1.87, 1.09" calculation results.

Finally, (D) of FIG. 15 shows a case where the calculation results of (C) of FIG. 15 are added for each vocabulary for each category, and the reciprocal of the added value is used as a correction weight value. For example, for "dumplings", the calculation result of "9.33" is obtained by adding all the numerical values of each category to the result of (C) in FIG. 15 , namely "0, 9.33, 0". Then, the reciprocal thereof is taken to obtain a calculation result of the corrected weight value "0.11". Similarly, for "スープ", the calculation result of "2.96" is obtained by adding all the numerical values of each category to "0, 1.87, 1.09", which is the result of (C) in FIG. 15 . Then, the inverse number thereof is taken to obtain a calculation result of the corrected weight value "0.34".

(WFST synthesis operation, deformation 3)

Next, modification 3 will be described. In Variation 3, "position information" is added to the WFST data. That is, when outputting a candidate of the character recognition processing result, the character recognition unit 105 also outputs the position information of the candidate within the image. A separate means for acquiring positional information within the image may be provided, or the acquisition may be performed by the character recognition unit 105 . Then, the first character string transition data generation unit 106 adds the position information to generate first WFST data, and the WFST processing unit 109 and the character string detection unit 110 add the position information to perform WFST composition calculation and output the result. In Variation 3, since the position of the detected vocabulary cannot be determined by the WFST synthesis operation itself, a separate table (see (C) of FIG. 16 below) is prepared to store the original position information, and the table number is added to the state Transferring. In this way, the table number is included in the detected result, so that the original position information can be identified (see (D) and (E) of FIG. 16 ).

FIG. 16 is a diagram illustrating the operation of each functional element in Variation 3. FIG. (C) of FIG. 16 shows position information output by the character recognition unit 105 . The positional information is output as a positional information table, in which x1 and y1 indicate the coordinates of the upper left of each character, and x2 and y2 indicate the coordinates of the lower right of each character. In addition, each piece of position information is identified by serial numbers such as "0000" and "0001". When outputting the candidates of the character recognition processing results, the character recognition unit 105 also outputs serial numbers as shown in (C) of FIG. 16 . (B) of FIG. 16 shows the first WFST data with position information generated by the first character string transition data generation unit 106 . Compared with the first WFST data of FIG. 5 , the difference lies in the addition of serial numbers for position information. In addition, the weighted ε transfer for skipping characters is omitted.

(A) of FIG. 16 shows an example of vocabulary data to which position information (more precisely, a serial number of position information) is added. In the case of a search table, (A) of FIG. 16 shows the second WFST data with position information generated by the second character string transition data generation unit 107 . In the case of vocabulary detection, (A) of FIG. 16 shows the third WFST data with position information generated by the third character string transition data generation unit 108 . As shown in (A) of FIG. 16 , serial numbers of position information are added at the beginning and end of the transition, and the number of consecutive numbers added is the same as the maximum number of position information shown in (C) of FIG. 16 . In this example, the maximum number of pieces of location information is 10,000 from "0000" to "9999".

(D) of FIG. 16 shows the result of the synthesis calculation of the 2nd WFST data or the 3rd WFST data of FIG. 16(A) and the 1st WFST data of FIG. 16(B). The serial number of the position information is appended to the result of the composite operation. Furthermore, as shown in (E) of FIG. 16 , the results of character recognition such as "Rain" and "スソ" can be specified by comparing the serial number added to the result of the synthesis operation with the position information table in (C) of FIG. 16 where in the image.

In the case of using only one vocabulary at an overlapping position, an array for determination in which character positions have already been detected is generated separately, and the array position corresponding to the character position of the detected vocabulary from the top of the optimal path is already detected. If the sequence has already been detected, it is determined that vocabulary overlaps, and only a vocabulary with a high priority of a keyword detected from the same position is used. In this way, by arranging the vocabulary so as to fill in the gaps, correction of the character recognition result can be performed using the vocabulary with a high priority.

(WFST compositing operation, variant 4)

Next, modification 4 will be described. In the case of a language that presupposes separated writing, there are fewer deformations of the characters than Japanese, so in the case of using the WFST data of the character recognition candidate group as shown in FIG. Only part of the word is detected like "pen" in part of ". Therefore, in Variation 4, "separation identification information (corresponding to "identification information" in the claims)" is added to the WFST data.

That is, when outputting the candidates of the character recognition processing result, the character recognition unit 105 also outputs the separation identification information indicating the separation between words. In character recognition, when a character recognized as a delimiter, that is, when the recognized character is a symbol such as a space, a period, or a parenthesis, delimiter identification information is output. Then, the first character string transition data generating unit 106 adds the segment identification information to generate the first WFST data with segment identification information. In addition, the second character string transition data generating unit 107 and the third character string transition data generating unit 108 also add the delimiter identification information to generate the second WFST data and the third WFST data with delimiter identification information, respectively. In addition to adding delimiter identification information in the case of symbols such as spaces, periods, and parentheses described above, it is also added at the beginning and end of words. Then, when performing a state transition for WFST processing, the WFST processing unit 109 performs the state transition in units of parts separated by two pieces of separation identification information.

FIG. 17 is a diagram illustrating the operation of each functional element in Variation 4. FIG. (B) of FIG. 17 shows the first WFST data with separator identification information generated by the first character string transition data generation unit 106 when the character string in the image is "{two pens}". The character recognition section 105 that recognizes the symbol "{" outputs the result of the character recognition processing, and outputs delimited recognition information. After acquiring these pieces of information, the first character string transition data generation unit 106 generates first WFST data with identification information, which takes the symbol "{" as input and separator identification information "<sp>" as output. The same applies to the symbol "}". Regarding the space between "two" and "pens", when the character recognition unit 105 outputs that the character recognition result is a space, the first character string transition data generation unit 106 generates a space in which delimiter identification information is assigned to the space. 1st WFST data with delimited identification information. In addition, in FIG. 17 , the ε transition from the initial state to the intermediate state, the ε transition from the intermediate state to the final state, and the weighted ε transition for skipping characters are omitted.

(A) of FIG. 17 shows an example of vocabulary data for partition writing to which partition identification information is added. In the case of a search table, (A) of FIG. 17 shows the second WFST data with segment identification information generated by the second character string transition data generation unit 107 . In the case of vocabulary detection, (A) of FIG. 17 shows the third WFST data with separator identification information generated by the third character string transition data generation unit 108 . As shown in (A) of FIG. 17 , partition identification information "<sp>" is appended at the beginning and end of a word. Also, make the output "<esp>" for "s" which means plural in English. Accordingly, it is possible to prevent the complex number "s" from having an influence on the result of the synthesis operation.

(C) of FIG. 17 shows the result of the synthesis calculation of the first WFST data shown in (B) of FIG. 17 and the second WFST data or third WFST data shown in (A) of FIG. 17 . When the WFST processing unit 109 performs a state transition, the state transition is performed in units of parts separated by two separation identification information, that is, the state is performed in units of "two" or "pens" shown in (B) of FIG. In the branch, the synthesis operation with the first WFST data in (A) of FIG. 17 is performed, and "pen" is output as a result.

On the other hand, (D) of FIG. 17 shows the first WFST data with separator identification information generated by the first character string transition data generation unit 106 when the character string in the image is "pencil." The character recognition unit 105 recognizes the symbol ".", outputs the result of character recognition processing, and outputs delimited recognition information. After receiving these information, the first character string transition data generation unit 106 generates the first WFST data with identification information, which takes the symbol "." as input and the separator identification information "<sp>" as output. (E) of FIG. 17 shows the result of the synthesis calculation of the first WFST data shown in (D) of FIG. 17 and the second or third WFST data shown in (A) of FIG. 17 . When the WFST processing unit 109 performs a state transition, the state transition is performed in units of parts separated by two separation identification information, that is, the state transition is performed in units of "pencils" shown in (D) of FIG. Since the synthesis operation of the first WFST data in (A) of FIG. 17 does not detect a matching vocabulary. Accordingly, it is possible to prevent the detection of only partial spelling of a word such as "pen" from a part of "pencil".

FIG. 18 is a diagram illustrating the operation of each functional element in Variation 4 when a vocabulary of a partitioned written language and a vocabulary of a non-separated written language are combined. In the method described below, the transfer of the divided identification information and the transfer of ε are added to the transfer between alphabets and characters other than alphabets. Thus, even when alphabets and characters other than alphabets exist without separation, detection of vocabulary consisting of alphabets and detection of vocabulary combining alphabets and characters other than alphabets can be performed simultaneously.

(A) of FIG. 18 shows the first WFST data with separator identification information generated first by the first character string transition data generation unit 106 . The first WFST data is generated in the same manner as in FIG. 17 , and separator identification information "<sp>" is added to the beginning and end of a word. (B) of FIG. 18 shows the first WFST data with segment identification information generated by the first character string transition data generation unit 106 by correcting (A) of FIG. 18 . In the transfer between alphabets and characters other than alphabets, that is, in the transfer between words of a language written in a partition and words in a language not written in a partition, partition identification information is added.

In addition, by adding the epsilon shift "<eps>" to the segment identification information, it is also possible to correspond to a vocabulary combining alphabets and characters other than alphabets. That is, it is considered that transitions made of letters and transitions made of characters other than letters are arranged in parallel, and state transitions as shown in (C) of FIG. 18 are added to transitions between characters. Thereby, a transition of partition identification information "<sp>" is added to the transition between alphabets and characters other than alphabets. In addition, in FIG. 18 , the ε transition from the initial state to the intermediate state, the ε transition from the intermediate state to the final state, and the weighted ε transition for skipping characters are omitted.

(As a configuration example of the character recognition system 100)

Next, another structural example of this embodiment will be described. Above, the case where the present invention is configured as the character recognition device 1 has been described, but the present invention is not limited thereto, and the present invention may be configured as the character recognition system 100 including the terminal 200 and the server 300 as shown in FIG. 19 . FIG. 19 is a schematic configuration diagram in this case, and the terminal 200 and the server 300 are communicably connectable to each other via a communication network.

The terminal 200 has the following parts as functional components: an image reading unit 101 (equivalent to “image input unit” in the claims), an image binarization unit 102, and a character region detection unit 103 (equivalent to the “image input unit” in the claims). The "character region detection unit" in the claims), the character region segmentation unit 104 (equivalent to the "character region segmentation unit" in the claims), the character recognition unit 105 (equivalent to the "character recognition unit" in the claims), The first character string transition data generation unit 106 (equivalent to the "first character string transition data generation unit" in the claims) and the second character string transition data generation unit 107 (corresponding to the "second character string transition data generation unit" in the claims Serial Transfer Data Generation Unit").

The server 300 has the following parts as functional structural elements: a third character string transfer data generation unit 108 (equivalent to “the third character string transfer data generation unit” in the claims), a WFST processing unit 109 (equivalent to the “third character string transfer data generation unit” in the claims The "finite state conversion unit" in the text), the character string detection unit 110 (corresponding to the "character string detection unit" in the claims), and the vocabulary DB 111 (corresponding to the "vocabulary database" in the claims).

The description of each of the functional components of the terminal 200 and the server 300 overlaps with that of the character recognition device 1 , so the description is omitted here. In addition, in this embodiment, the first character string transition data generation unit 106 and the second character string transition data generation unit 107 exist in the terminal 200, and the third character string transition data generation unit 108 exists in the server 300. , but not limited thereto, the first character string transition data generation unit 106, the second character string transition data generation unit 107, and the third character string transition data generation unit 108 may exist in either the terminal 200 or the server 300 middle.

FIG. 2 can be referred to as a hardware configuration diagram of the terminal 200 . As shown in FIG. 2 , the terminal 200 is constituted as a common computer system, which physically includes not only main storage devices such as CPU21, ROM22 and RAM23, keyboard, and mouse, but also devices for reading images such as cameras or An input device 24 for reading data from an external device, an output device 25 such as a display, a communication module 26 such as a network card for transmitting and receiving data with other devices, and an auxiliary storage device 27 such as a hard disk. The image read by the input device 24 may be an image taken by the own device, or an image taken by another device. By reading predetermined computer software on hardware such as CPU21, ROM22, RAM23, under the control of CPU21, input device 24, output device 25, communication module 26 are operated, and main storage device 22, 23 or auxiliary storage device 27 The functions of the above-mentioned terminal 200 are realized by reading and writing data in the terminal 200.

FIG. 2 can be referred to as a hardware configuration diagram of the server 300 . As shown in Figure 2, the server 300 is constituted as a common computer system. The computer system physically includes not only main storage devices such as CPU31, ROM32 and RAM33, a keyboard, and a mouse, but also a device for reading data from an external device, that is, an input device. A device 34 , an output device 35 such as a display, a communication module 36 such as a network card for transmitting and receiving data with other devices, an auxiliary storage device 37 such as a hard disk, and the like. By reading predetermined computer software on hardware such as CPU31, ROM32, RAM33, under the control of CPU31, input device 34, output device 35, communication module 36 are operated, and main storage device 32, 33 or auxiliary storage device 37 The functions of the above-mentioned server 300 are realized by reading and writing data in the server.

(Structure example as character recognition program)

The present invention can also be configured as a character recognition program, and the above description of the character recognition device 1 can be understood as a description of a character recognition program that causes a computer to operate as the character recognition device 1 . Although repeated description is omitted, the character recognition program uses the computer as the image reading unit 101, the image binarization unit 102, the character area detection unit 103, the character area division unit 104, the character recognition unit 105, and the first character string described above. The transition data generation unit 106 , the second character string transition data generation unit 107 , the third character string transition data generation unit 108 , the WFST processing unit 109 , and the character string detection unit 110 function. For example, a character recognition program is stored in a recording medium and provided. Moreover, as a recording medium, recording media, such as a flexible disk, CD, and DVD, recording media, such as a ROM, or a semiconductor memory, etc. can be illustrated.

(Function and effect of the present embodiment)

Next, operations and effects of the character recognition device 1 of this embodiment will be described. According to the character recognition device 1 of the present embodiment, since an external database such as a telephone directory is not used, it is not necessary to collate with a large amount of word knowledge contained in the telephone directory database, etc., and speed-up of the character recognition process can be realized. That is, according to the present embodiment, instead of detecting a word based on the character recognition result and performing comparison with an external word database, the word/category information database existing in the character recognition device 1 is expressed using a finite state transformer (WFST). Candidate groups are recognized with characters and combined with WFST, so that word extraction/category information extraction/character position extraction processing can be performed at high speed. In addition, since the position information acquisition means, the orientation information acquisition means, etc. are not required, the device configuration can be simplified. That is, character recognition can be performed using only the information in the character recognition device 1 without using the position information acquisition device or the orientation information acquisition device. With such an apparatus configuration, character recognition from scene images can be performed with high precision and high speed.

In addition, in the present embodiment, even if the character region is extracted in such a manner that noise may appear in the image, it is possible to impose strict language restrictions by WFST arithmetic processing. In this way, in addition to removing the noise, it is also possible to lower the priority. Therefore, it is possible to improve recognition accuracy even when there are problems specific to scene images such as changes in brightness and distortion of characters.

In addition, by applying the vocabulary detected by the present embodiment to the character recognition result obtained by the conventional method, the character recognition device 1 etc. of the present embodiment can be used as a means for correcting the character recognition result of the conventional method. device.

In addition, according to the present embodiment, the first WFST data itself of the character recognition candidate group can be used as a search table for images, and the character recognition device 1 etc. devices etc.

In addition, according to the present embodiment, the character recognition device 1 and the like of the present embodiment can be effectively applied as a vocabulary detection device and the like by performing a synthetic operation on the first WFST data of the character recognition candidate group and the third WFST data in the vocabulary DB 111 .

In addition, according to the present embodiment, a specific method for calculating the weight value by the first character string transition data generation unit 106 is provided.

Furthermore, according to the present embodiment, a specific method for correcting the weight value by the first character string transition data generation unit 106 is provided. In addition, the accuracy of vocabulary detection can be improved by modifying the weight value.

In addition, according to the present embodiment, even when the character region dividing unit 104 has over-divided, it can be appropriately dealt with.

Furthermore, according to the present embodiment, the first WFST data includes the first dummy jump, the second dummy jump, and the third dummy jump, thereby improving the accuracy of the synthesis calculation of the first WFST data and the second WFST data or the third WFST data.

Furthermore, according to the present embodiment, character recognition can be performed with high accuracy even for languages written with partitions by using identification information indicating partitions. In addition, it is possible to perform common processing on vocabulary for the dictionaries of a language of divided writing such as English and a language of non-divided writing such as Japanese.

Furthermore, according to the present embodiment, by using the position information, it is possible to specify where in the image the character recognition result is located.

Furthermore, according to the present embodiment, by using classification information, it is possible to specify which category the result of character recognition belongs to.

Label description

1... character recognition device, 100... character recognition system, 101... image reading part, 102... image binarization part, 103... character area detection part, 104... character area division part, 105... character recognition part, 106... No. 1 character string transition data generation unit, 107...the second character string transition data generation unit, 108...the third character string transition data generation unit, 109...processing unit, 110...character string detection unit, 111...vocabulary DB, 200...terminal , 300...server.

Industrial Availability

The present invention provides a character recognition device, a character recognition method, a character recognition system, and a character recognition program capable of performing character recognition with high precision and high speed using a simplified device configuration without using an external database such as a telephone directory.

Claims (16)

1. A character recognition device, characterized in that it has: an image input unit that inputs an image including a recognition object character; a character area detection unit that detects a character area in the image where the character exists; a character area dividing unit that divides the character area in units of individual characters; a character recognition unit that performs character recognition processing for each individual character on the characters existing in the segmented regions divided by the character region segmenting unit, and outputs one or more candidates for character recognition processing results for a single character; A first character string transition data generation unit that inputs the candidate, calculates a weight value for transition to the candidate, and generates first character string transition data that is character string transition data based on a combination of the candidate and the weight value data; and A finite state transition unit, which sequentially performs state transitions according to the first character string transition data, accumulates the weight values in each state transition to calculate the cumulative weight value of each state transition, and outputs the state transition with the small cumulative weight value The result of is used as the result of character recognition. 2. The character recognition device according to claim 1, wherein: The character recognition device further includes a second character string transition data generation unit configured to input a keyword from a user and generate second character string transition data that is character string transition data of the keyword, The finite state conversion unit performs a synthesis operation on the first character string transition data and the second character string transition data, thereby determining whether the keyword exists in the image. 3. The character recognition device according to claim 1, wherein: The character recognition device further includes a third character string transition data generating unit that generates third character string transition data that is character string transition data for each vocabulary existing in the vocabulary database, The finite state conversion unit detects vocabulary existing in the image by performing a synthetic operation on the first character string transition data and the third character string transition data. 4. The character recognition device according to any one of claims 1 to 3, wherein: The character recognition unit assigns priority to a plurality of the candidates and outputs them, The first character string transfer data generating unit calculates the weight value according to the priority order. 5. The character recognition device according to claim 4, wherein: The character recognition unit uses at least two different recognition methods to perform the character recognition process, The first character string transition data generation unit calculates the weight value based on the output number of the candidates in the different identification methods and the priority order. 6. The character recognition device according to any one of claims 1 to 3, wherein: The first character string transition data generating unit calculates the weighting value in consideration of character string transitions of words registered in a language database. 7. The character recognition device according to any one of claims 1 to 3, wherein: The first character string transition data generating unit corrects the weight value according to the position of the candidate in the image or the character size of the candidate. 8. The character recognition device according to any one of claims 1 to 3, wherein: In the case where the character region segmentation unit divides the character region using a plurality of segmentation patterns to generate multiple kinds of the segmentation regions, The character recognition unit performs the character recognition processing on the plurality of divided regions respectively, the first character string transition data generating unit generates the first character string transition data for each of the candidates of the plurality of divided regions, The finite state transition unit outputs, as the result, a state transition result in which the cumulative weight value is smaller in all of the plurality of divided regions. 9. The character recognition device according to any one of claims 1 to 3, wherein: The first character string transition data generating unit performs a first null transition from an initial state of the character string transition to the candidate null transition, and a second null transition that is a final state transition from the candidate to the character string transition. , for generating the first character string transition data by skipping the candidate null transition in units of single characters, that is, including the third null transition. 10. The character recognition device according to any one of claims 1 to 3, wherein: when the character recognition unit outputs the candidate of the result of the character recognition processing, it also outputs recognition information representing a separation between words, the first character string transition data generating unit adds the identification information to generate the first character string transition data, When the finite state transition unit performs the state transition, the state transition is performed in units of parts separated by two pieces of the identification information. 11. The character recognition device according to any one of claims 1 to 3, wherein: When the character recognition unit outputs the candidate of the character recognition processing result, it also outputs the position information of the candidate in the image, the first character string transfer data generating unit adds the position information to generate the first character string transfer data, The finite state transition unit appends the position information to output the result. 12. The character recognition device according to claim 2, wherein: The vocabulary database has classification information for vocabulary, the second character string transfer data generating unit adds the classification information to generate the second character string transfer data, The finite state conversion unit appends the classification information to output the result. 13. The character recognition device according to claim 3, wherein: The vocabulary database has classification information on vocabulary, the third character string transfer data generating unit adds the classification information to generate the third character string transfer data, The finite state conversion unit appends the classification information to output the result. 14. The character recognition device according to claim 12 or 13, characterized in that, The character recognition device has a vocabulary classification correlation vector storage unit, which stores a vocabulary classification correlation vector representing the correlation between vocabulary and the classification information, The first character string transfer data generation unit adds the candidates in the first character string transfer data and the weight value to the value of the vocabulary classification correlation vector, and uses the classification information with the largest value as the The classification information corresponding to the candidate, and modifying the weight value for the candidate based on the classification information. 15. A character recognition method, characterized in that it has: In an image input step, the image input unit inputs an image containing characters to be recognized; The character area detection step, the character area detection unit detects the area where the character exists in the image, that is, the character area; In the character region segmentation step, the character region segmentation unit divides the character region in units of single characters; A character recognition step, wherein the character recognition unit performs character recognition processing for each individual character on the characters present in the divided regions divided by the character region dividing unit, and outputs one or more candidates for character recognition processing results for a single character ; In a first character string transition data generating step, the first character string transition data generating unit inputs the candidate, calculates a weight value for transition to the candidate, and generates a character string transition based on a combination of the candidate and the weight value. The data is the first character string transfer data; and In the finite state transition step, the finite state transition unit sequentially performs state transitions according to the first character string transition data, accumulates the weight values in each state transition to calculate the cumulative weight value of each state transition, and outputs the cumulative weight value small The result of the state transition is as the result of character recognition. 16. A character recognition system comprising a terminal and a server, the character recognition system is characterized in that, The terminal has: an image input unit which inputs an image containing characters as recognition objects; a character area detection unit that detects a character area in the image where the character exists; a character area dividing unit that divides the character area in units of individual characters; a character recognition unit that performs character recognition processing for each individual character on the characters present in the segmented regions divided by the character region segmenting unit, and outputs one or more candidates for character recognition processing results for the individual characters; and A first character string transition data generation unit that inputs the candidate, calculates a weight value for transition to the candidate, and generates first character string transition data that is character string transition data based on a combination of the candidate and the weight value data, Said server has: A finite state transition unit, which sequentially performs state transitions according to the first character string transition data, accumulates the weight values in each state transition to calculate the cumulative weight value of each state transition, and outputs the state transition with the small cumulative weight value The result of is used as the result of character recognition.