JP5495657B2 - Character recognition device, character recognition program - Google Patents

Description

  The present invention relates to a character recognition device and a character recognition program, and more particularly to a character recognition device and a character recognition program applied to an input device that recognizes handwritten characters and inputs the recognized characters to a computer or the like.

  An example of a conventional character recognition device of this type is disclosed in Patent Document 1. According to Patent Literature 1, the personal authentication device includes a handwritten data input device and a signature verification device. The user writes a predetermined character in the air by holding the imaging means provided in the handwritten data input device by hand and moving it in the air. The trajectory calculation means provided in the handwritten data input device obtains a movement trajectory that is a change in position of the imaging means in the air as handwritten data (signature data). The signature verification device compares the handwritten data with reference data (personal signature data) stored in advance in the storage unit to determine whether the signature data is from the same person.

  Another example of the character recognition device is disclosed in Patent Document 2. According to Patent Document 2, it is determined whether or not writing is being performed by turning on / off the switch means in order to recognize a character written in the air, and handwritten characters are determined based on the respective moving directions when writing and not writing. Therefore, the character is recognized regardless of the number of strokes of the character.

JP 2002-259045 A JP 2007-200196 A

  However, in the personal authentication device of Patent Document 1, since the movement trajectory itself is calculated by the trajectory calculation means and collated with the reference data, the information amount of the movement trajectory and the reference data is increased, and the amount of calculation of the comparison collating means is increased. There was a problem that time became long. In addition, data input and personal authentication are basically performed with a single stroke, and characters consisting of a plurality of strokes cannot be recognized.

  In this regard, with the technique disclosed in Patent Document 2, even characters with two or more strokes written in the air can be recognized. However, it is necessary for the user to turn on / off the switch in order to indicate whether writing is in progress or not, and it is impossible to quickly write characters or the timing of writing operation and switch operation is shifted. There was a problem.

  In view of the above-mentioned problems, the present invention reduces the amount of computation and processing time, can write characters consisting of a plurality of strokes continuously, and does not limit the place, size or speed of writing. An object is to provide a device and a character recognition program.

The character recognition device of the present invention includes an imaging unit that images a user's finger, an input period detection unit that detects an input period for writing a character and a transition period for moving to the next character according to a predetermined operation of the user, The movement detection means that detects the movement of the finger as a movement vector, and the movement vector detected by the movement detection means during the input period, including the transition to the next picture, are obtained and the written code data compressed in time series is obtained. writing code data acquisition means, codes, including the transition section of the motion vector that occurs when noted brush each character to Tsugiga, when the dictionary code data storing means for storing the dictionary code data compressed in accordance with sequences of If, by comparing the writing code data and the dictionary code data and a specific means for the user to identify handwritten characters, writing code data obtaining means, movement vector When the movement amount of the camera is less than or equal to a predetermined threshold value, it is determined to be in a stationary state. When the movement amount is larger than the predetermined threshold value, a movement vector is encoded according to the movement direction. The predetermined threshold value is changed based on the movement amount history. It shall be the feature.

Further, the character recognition program of the present invention, a computer, according to a predetermined operation of the user, the input period detecting means for detecting a transition period to move to input period and the next character to be written the character, move the movement of the finger vector A movement detection means for detecting, a writing code data acquisition means for encoding the movement vector detected by the movement detection means including the transition part to the next image during the input period, and acquiring the writing code data compressed in time series, writing the code data and to function as a pre-specified hand stage prepared dictionary code data and user by comparing identifies a handwritten character, writing code data acquisition means, the movement amount of the moving vector is the predetermined threshold value It is determined that it is in a stationary state at the following time, and when it is larger than a predetermined threshold, the movement vector is coded according to the moving direction, and the predetermined threshold is based on the movement amount history. It shall be the said to be more.

The character recognition device of the present invention includes an imaging unit that images a user's finger, an input period detection unit that detects an input period for writing a character and a transition period for moving to the next character according to a predetermined operation of the user, The movement detection means that detects the movement of the finger as a movement vector, and the movement vector detected by the movement detection means during the input period, including the transition to the next picture, are obtained and the written code data compressed in time series is obtained. writing code data acquisition means, codes, including the transition section of the motion vector that occurs when noted brush each character to Tsugiga, when the dictionary code data storing means for storing the dictionary code data compressed in accordance with sequences of If, by comparing the writing code data and the dictionary code data and a specific means for the user to identify handwritten characters, writing code data obtaining means, movement vector When the movement amount of the camera is less than or equal to a predetermined threshold value, it is determined to be in a stationary state. When the movement amount is larger than the predetermined threshold value, a movement vector is coded according to the movement direction. It shall be the feature. As a result, the user can write characters consisting of a plurality of strokes continuously, the amount of calculation and processing time are reduced, and the characters can be recognized regardless of the place, size and speed of writing. .

Further, the character recognition program of the present invention is an input period detecting means for detecting an input period for writing a character and a transition period for moving to the next character in accordance with a predetermined operation of the user, and detects the shape of the captured finger. Shape detection means for performing movement detection means for detecting the movement of a finger as a movement vector, and writing in which the movement vector detected by the movement detection means is coded including the transition to the next image during the input period and is compressed in time series writing the code data acquiring means for acquiring the code data, writing code data and compares prepared the dictionary code data is made to function as a specific hand stage the user to identify handwritten characters previously, writing code data acquiring means Is determined to be stationary when the movement amount of the movement vector is less than or equal to a predetermined threshold, and when the movement vector is larger than the predetermined threshold, the movement vector is coded according to the movement direction. However, it and changes based on the amount of movement of the history a predetermined threshold. As a result, the user can write characters consisting of a plurality of strokes continuously, the amount of calculation and processing time are reduced, and the characters can be recognized regardless of the place, size and speed of writing. .

1 is a configuration diagram of an input system according to Embodiment 1. FIG. 1 is a configuration diagram of a character input device according to Embodiment 1. FIG. It is a figure explaining direction code data. It is a figure explaining writing code data. It is a figure explaining the derivation process of the similarity degree of writing code data and dictionary code data. It is a figure which shows the memory map of RAM. It is a figure which shows the table data for evaluation of RAM. It is a flowchart which shows operation | movement of an input system. FIG. 6 is a configuration diagram of an input system according to a second embodiment. FIG. 6 is a configuration diagram of a character input device according to a second embodiment. It is a figure which shows the method of detecting the position and shape of a hand.

(Embodiment 1)
<Configuration>
FIG. 1 is a configuration diagram of an input system 10 according to the present embodiment. Referring to FIG. 1, an input system 10 includes a character input device 12. The character input device 12 stores a character recognition program and also functions as a character recognition device. The character input device 12 includes a pen-shaped housing (case) 14 and a switch 16 disposed so that the operation unit is exposed from a part of the case 14. The input system 10 includes a computer 20, and the character input device 12 and the computer 20 are connected using a cable 18 (such as a USB cable or an IEEE1394 cable).

  In the present embodiment, the character input device 12 and the computer 20 are connected using the cable 18, but the present invention is not limited to this. It can also be configured as an input system 10 in which the device 12 and the computer 20 are integrally formed. Further, as the computer 20, not only general-purpose computers (desktops, laptops, PDAs, etc.), but also any products having a CPU in the interior from mobile phones, in-vehicle navigation, AV products to electronic dictionaries are applicable.

  Therefore, there are various ways of dividing the hardware and software of the character input device 12 and the computer 20 constituting the input system 10 including an integrated type, but in the following, as an example, the character input device 12 uses handwritten characters. The computer 20 has a function of judging in character units and transmitting the candidate or result to the computer 20, and the computer 20 has a kana character conversion function, a prediction function, etc., and controls the whole including the operation. It will be described as a configuration.

  FIG. 2 is a block diagram showing an electrical configuration of the character input device 12. Referring to FIG. 2, the character input device 12 includes a CPU 30, and the CPU 30 is connected to a ROM 34, a RAM 36 and an input / output (I / O) interface (hereinafter simply referred to as “interface”) 38 via an internal bus 32. Connected. The character input device 12 includes the camera 40 and the switch 16 described above, and the camera 40 and the switch 16 are connected to the CPU 30 via the interface 38 and the internal bus 32. Such a circuit component is accommodated in the case 14. Although detailed illustration is omitted, the camera 40 is disposed in the case 14 so that the lens portion thereof is exposed at the distal end portion 14a of the case 14 (the distal end portion of the character input device 12). In FIG. 2, the switch 16 is described as being housed in the case 14. However, as described above, the switch 16 is disposed such that the operation unit is exposed from the case 14. Further, although omitted in FIG. 2, one end of the above-described cable 18 is connected to the interface 38 and the other end is connected to the computer 20.

<Operation>
For example, the user holds the character input device 12 with his / her finger so as to hold the pen, and writes a character (in this embodiment, hiragana). Then, the character input device 12 recognizes the character written by the user and inputs the recognized character to the computer 20. Hereinafter, a character recognition method in the character input device 12 will be described in detail.

<Character recognition method>
In the character input device 12, image data from the camera 40 is given to the CPU 30 via the interface 38 and the bus 32. In the present embodiment, the CPU 30 detects the image data from the camera 40 every predetermined time (1 frame: 1/30 second), and a motion vector between frames (between the previous frame and the current frame). Calculate Since the method of calculating the motion vector is already well known, detailed description thereof is omitted here.

  CPU30 produces | generates the information (movement information) regarding the movement of the character input device 12, using the calculated motion vector. This information is stored in the data storage area of the RAM 36 shown in FIG. In the present embodiment, the movement information is code data (written code data 524) for characters written by the user. Specifically, the CPU 30 detects the on / off state of the switch 16 and determines whether writing is in units of characters. When the switch 16 is pressed (on state), the CPU 30 determines that the input period is in writing in character units, and continues coding. When the switch 16 is not pressed (off state), writing is performed in character units. Is completed and the transition to the next character is determined, the encoding is stopped and the input character specifying process is started. Thus, the CPU 30 operates as an input period detection unit that detects the input period and the transition period.

  Here, when the switch 16 is on, the CPU 30 operates as a movement detecting unit that detects a motion vector calculated based on image data from the camera 40 as a direction during writing. The information used here is not based on the distance and direction from the origin at the position where the switch 16 is turned on, but on the motion vector information itself between frames, and the writing code describing them in time series Data 524 (movement information) is generated. In the present embodiment, direction code data 522 as shown in FIG. 3 is prepared, and referring to this, encoding (coding) is performed so that the direction code is added only when the direction code data changes. .

  The direction code data 522 shown in FIG. 3 is data for dividing a two-dimensional direction into eight directions (direction with motion) including up / down / left / right and diagonal directions, and encoding motion vectors into alphabets according to the directions. It is.

  Specifically, “h” is assigned to the upward direction, and “a” is assigned to the upper right direction rotated 45 degrees clockwise from this upward direction. Further, “b” is assigned to the right direction, and “c” is assigned to the lower right side rotated 45 degrees clockwise from the right direction. Furthermore, “d” is assigned to the downward direction, and “e” is assigned to the diagonally downward left direction rotated 45 degrees clockwise from the downward direction. Then, “f” is assigned to the left direction, and “g” is assigned to the upper left diagonally rotated 45 degrees clockwise from the left direction.

  Also, “S” is assigned to a motion vector with little motion as no motion. Here, “there is little motion” means that the scalar of the motion vector is less than a predetermined threshold (predetermined distance), and the threshold is set by the developer or designer of the character input device 12. However, since the motion vector is calculated based on the image data of the adjacent frame and the distance that the human moves the finger in a short period such as one frame is considered to be very short, the predetermined value is set to a relatively small value. The In the present embodiment, not only the motion is small, but also when it is determined that the reliability of the motion vector detected from the image data is low, encoding is performed to “S”.

  In general, there is a slight difference in signal level even on a single color wall or floor. Therefore, even when characters are written with the character input device 12 (the lens of the camera 40) facing them, the motion vector is accurately detected. can do. However, there is a possibility that a motion vector is erroneously detected when it is extremely dark or too bright, or when image data has a regular repetitive pattern. However, the reliability of the detected motion vector can be generally determined in the process of calculation for the detection.

  Here, a method for generating the writing code data 524 corresponding to the character written by the user will be described. The method of generating the dictionary code data 520 is the same as the method of generating the writing code data 524 that is generated when a character is written by carrying the correct writing order and writing (in this embodiment, the character input device 12). Therefore, a duplicate description is omitted. However, the dictionary code data 520 may be generated by arranging the codes of the writing middle direction code data 522 in accordance with the correct writing order and brush stroke without actually writing characters.

  For example, as shown in FIG. 4, a case where the user writes “a” in hiragana with the correct writing order and brush stroke will be described. “A” is originally composed of three strokes (1), (3) and (5), but in order to eliminate the need for brush up / down operations, (2) and (4) Brush strokes are added. First, with the switch 16 turned on, a horizontal bar is written as shown in (1). At this time, the horizontal bar is in the right direction or almost right direction from the start of writing to the end of writing, and is encoded into “b” by referring to the writing middle direction code data 522. Therefore, when the user finishes writing the horizontal bar, the portion at which the writing speed at the beginning and end of the horizontal bar is reduced is encoded as “S”, and the data of the code “SbS” is stored in the RAM 36 as the writing code data 524. Remembered. Here, the code (alphabet) is not additionally stored in the RAM 36 unless the code changes, that is, unless the moving direction (motion vector) or moving speed changes.

  Next, when the brush is carried from the end point of the stroke (1) shown in (2) to the start point of the stroke (3), the moving direction is diagonally left upward, and “g” is obtained by referring to the moving direction code data 522. ”And the code“ S ”is added when the brush stops at the start point of the stroke (3). Therefore, the data of the code string“ SbSgS ”is stored in the RAM 36 as the writing code data 524. The

  Next, when writing the vertical bar of the stroke (3), it is downward or almost downward from the beginning of writing of the vertical bar to the end of writing, and therefore it is encoded to “d” by referring to the direction code data 522. For this reason, at the time when the writing of the vertical bar is completed, the data of the code string “SbSgSdS” is stored in the RAM 36 as the writing code data 524.

  Next, in carrying the brush of (4) from the end point of the stroke (3) of the vertical bar to the start point of the stroke (5), the direction is diagonally upward to the right, and referring to the movement direction code data 522, It is encoded as “a”. Therefore, since the code “S” is added when the brush stops at the start point of the stroke (5), the data of the code string “SbSgSdSaS” is stored in the RAM 36 as the writing code data 524.

  Finally, while the user is writing the stroke (5), the moving direction is diagonally downward left, leftward, diagonally upward left, upward, diagonally upward right, diagonally downward right, downward, diagonally downward left. It changes in the order. Therefore, by referring to the direction code data 522, the codes are sequentially added in the order of “e”, “f”, “g”, “h”, “a”, “b”, “c”, “d”, “e”. It becomes. Therefore, when the stroke (5) has been written, the data of the code string “SbSgSdSaSephghabdeS” is stored in the RAM 36 as the writing code data 524. In this way, the CPU 30 and the RAM 36 function as a writing code data acquisition unit that acquires the writing code data 524 obtained by encoding the movement vector and compressing the movement vector in time series.

  As described above, the writing code data 524 is generated, and when the user turns off the switch 16 when the input of one character is completed, the writing code data 524 at that time is the dictionary code data 520 as the final code data of one character. It compares with each of the code data (dictionary code data 520a, 520b, 520c, ...) about the several character contained in (refer FIG. 6), As a result, it specifies to the character which the most similar dictionary code data 520 shows. In this way, characters written by the user are recognized.

  Here, the dictionary code data 520 includes a plurality of dictionary code data 520a, 520b, 520c,. Hereinafter, in this specification, when “dictionary code data 520” is simply referred to, the dictionary code data 520a, 520b, 520c,... And the dictionary code data 520a, 520b, 520c,. Sometimes it points. Each of the dictionary code data 520a, 520b, 520c,... Has a moving direction code data 522 that is detected according to time series when each character is written in the correct writing order and stroke using the character input device 12. Is generated by being referred to.

  However, in the present embodiment, the dictionary code data 520 stores one dictionary code data 520 for one character as will be described later, but a plurality of dictionary code data 520 is stored for one character. You may make it memorize | store. This is also for the purpose of increasing the character recognition rate, for example, corresponding to characters whose writing order varies depending on the user, such as “M”. Further, for example, the dictionary code data 520 may be registered by the user before use, or may be configured to learn. Furthermore, the dictionary code data of characters that are very similar, such as “I” and “Ri”, “Ko” and “Te”, “U”, “Chi”, and “Ra”, emphasize the differences so that they can be easily distinguished. You may modify the code. Thus, the CPU 30 and the RAM 36 function as dictionary code data storage means for creating and storing dictionary code data.

<Similarity evaluation method>
Next, a method for evaluating the similarity between the writing code data 524 and the dictionary code data 520a, 520b, 520c,... In the present embodiment will be described with reference to FIG. For the evaluation of the similarity, for example, DP matching can be used as a method suitable for matching including time expansion and contraction with respect to time series data.

  As an example of DP matching, a case where data of a code string “SdcSaScdS” is obtained as writing code data 524 by writing a character by the user will be described. For simplicity, the dictionary code data “I” is described as “SdSaScdS” and the dictionary code data “SH” is described as “SdcbaS”.

  First, when the writing code data 524 shown in FIG. 5 (A) is obtained, a local matrix indicating the similarity between the writing code data 524 and each dictionary code data 520 from the writing code data 524 as shown in FIG. 5 (B). Create Here, the degree of similarity means that “0” is “match”, and the smaller the value, the more similar. In general, in the case of simple DP matching, the local matrix is given, for example, “3” when they do not match and “0” when they match. However, since the direction difference between “a” and “c” is larger and the degree of similarity is smaller than the direction difference between direction codes “a” and “b”, the direction code is not simply expressed as a match / mismatch. It is considered that more accurate matching can be realized by giving similarity according to the combination.

  Next, according to the equation shown in FIG. 5C, a cumulative matrix (FIG. 5D) indicating the degree of similarity including the path (continuity) is sequentially calculated from P (0, 0) and accumulated. The last value of the matrix calculation (FIG. 5E) is obtained as the similarity with each dictionary code data. Since the similarity to “I” is “3” and the similarity to “I” is “18”, the input pattern is more similar to “I” than “I”. . Similarly, it is possible to specify an input character by calculating and evaluating the similarity to all character candidates. In this way, the CPU 30 functions as a specifying unit that specifies the character written by the user by comparing the written code data and the dictionary code data.

  That is, the character recognition device according to the first embodiment includes an input period detecting unit that detects an input period for writing a character and a transition period for moving to the next character according to a predetermined operation of the user, and movement of a finger as a movement vector. Movement detection means for detecting, movement information acquisition means for acquiring handwritten code data 524 encoded in a time series in which the movement vector detected by the movement detection means is encoded including the transition portion to the next image during the input period. A dictionary code data storage means for storing the dictionary code data which is coded in accordance with the time series by encoding the movement vector generated when writing each character in the correct writing order and stroke, including the transition part to the next picture And a specifying means for specifying the character written by the user by comparing the written code data acquired by the movement information acquiring means with the dictionary code data. By coding the movement vector, the amount of calculation and the processing time can be reduced. In addition, by coding including the transition to the next image, it is possible to write characters consisting of a plurality of strokes continuously, and the character recognition device has no restrictions on the place, size or speed of writing.

<Program>
FIG. 6 is an illustrative view showing a memory map of the RAM 36 shown in FIG. However, programs and some data stored in the RAM 36 are loaded from the ROM 34. As shown in FIG. 6, the RAM 36 includes a program storage area 50 and a data storage area 52. The program storage area 50 stores a character recognition program and a character input program 508. The character recognition program includes a main processing program 500, a finger movement detection program 502, a coding program 504, and a character identification program 506.

  The main processing program 500 is a program for processing the main routine for character recognition according to the present embodiment. The finger motion detection program 502 detects an image from the camera 40 every predetermined time (for example, one frame), compares the image of the previous frame with the current frame image, and detects (calculates) a finger motion vector. ) A coding program 504 is a program for coding a movement vector according to the movement direction of the user's finger when the user writes a character in the air using the character input device 12 and arranging the codes in time series. It is.

  The character specifying program 506 is a program for specifying a character written by the user. Briefly, code data (written code data 524) for characters written by the user and code data (dictionary code data 520a, 520b, 520c,...) For each character registered in advance as a dictionary. In comparison, the character corresponding to the dictionary code data 520 that is most similar to the written code data 524 (having the smallest evaluation value) is specified (determined). Thereby, the character written by the user is recognized. The character input program 508 is a program for inputting (transmitting) the character specified (recognized) by the character specifying program 506 to the computer 20.

  The data storage area 52 stores data such as dictionary code data 520, direction data 522, writing code data 524, and evaluation table data 526, and a flag such as a writing flag 528.

  The dictionary code data 520 includes a plurality of dictionary code data 520a, 520b, 520c,. The dictionary code data 520a is code data when the hiragana “a” is written in the correct writing order and brush stroke, and is represented, for example, by data of a code string “SbSgSdSaSefghabdedeS”. The dictionary code data 520b is code data when “i” of hiragana is written in the correct writing order and brush stroke, and is represented by data of a code string “SdSaScdS”, for example. The dictionary code data 520c is code data when writing “u” of hiragana with the correct writing order and stroke stroke, and the following is omitted, but dictionary code data 520 for other hiragana is also stored. . However, in addition to hiragana, dictionary code data 520 for numbers (0-9), alphabets (az, AZ) or both may be stored. Also, patterns corresponding to operations such as “backspace”, “enter key”, and “direction key” other than characters may be registered and stored.

  The direction data 522 is, for example, the direction code data 522 shown in FIG. As described above, for example, when the switch 16 is turned on and the user writes characters with a single stroke, the movement information is encoded with reference to the direction code data 522.

  The written code data 524 is code data in which the moving direction of the character input device 12 itself is encoded according to a time series when the user writes a character with a single stroke. Although detailed explanation is omitted, when the computer 20 determines that the input of one character is completed (inputs the next character) or the character is canceled (corrected), the command from the computer 20 is Accordingly, the written code data 524 is reset (erased).

  The evaluation table data 526 is data about a table configured so that an evaluation value can be described for each character registered in the dictionary code data 520, and is used for specifying (recognizing) a character written by a user. It is done. Specifically, as shown in FIG. 7, candidate characters are described one by one, and when the evaluation values of the writing code data 524 and the dictionary code data 520 corresponding to each candidate character are calculated, the calculated evaluation is performed. A value is described corresponding to the corresponding candidate character. However, the evaluation table data 526 is reset every time a character is specified, and all evaluation values are deleted.

  Returning to FIG. 6, the writing flag 528 is a flag for determining whether or not writing is in progress, and is constituted by, for example, a 1-bit register. In the present embodiment, an example has been described in which writing is performed when the switch 16 is on, while writing is not performed when the switch 16 is off. However, the switch is toggled or an on / off switch is provided separately. It is possible to replace the switch by a gesture operation or the like. In any case, it is determined whether writing is in progress, and the period (input period) from the start to the end of input in character units is indicated.

  As a specific example, the CPU 30 executes a main routine (overall processing) shown in FIG. When starting the entire process, the CPU 30 generates a motion vector (step S1). That is, the image data input from the camera 40 is detected for each frame, and the movement (movement direction and movement amount) between the immediately preceding frame image and the current frame image is detected.

  Next, when the writing flag 528 is on, the moving direction of the character input device 12 is detected from the motion vector detected in step S1 to create writing code data 524 (coding process), and the writing code when the code changes A code is added to the data 524 (step S2). Then, a character specifying process for specifying the input character by comparing the similarity between the obtained writing code data 524 and each dictionary code data 520 is executed (step S3). The character specified in step S3 is input to the computer 20 (step S4). Note that the entire process, that is, the scan time of steps S1 to S4 is one frame. By repeatedly executing this entire process, a motion vector is calculated (detected) for each frame.

  In the present embodiment, the movement of the character input device 12 is detected by arranging the camera 40 at the tip 14a of the case 14 and detecting a motion vector from the image of the camera 40. A movement of the character input device 12 may be detected by mounting a sensor or an angular velocity sensor.

  That is, the character recognition program according to the first embodiment causes the computer to detect the input period detecting means for detecting the input period for writing a character and the transition period for moving to the next character in accordance with a predetermined operation of the user. Movement detection means for detecting as a movement vector, movement information acquisition means for acquiring movement information compressed in time series by coding the movement vector detected by the movement detection means including a transition part to the next picture during the input period Then, the movement information acquired by the movement information acquisition means is compared with the dictionary code data prepared in advance to function as specifying means for specifying the character written by the user. By coding the movement vector, the amount of calculation and the processing time can be reduced. In addition, by coding including the transition to the next image, it is possible to write characters consisting of a plurality of strokes continuously, and the character recognition program has no restrictions on the place, size and speed of writing.

<Improved code data detection accuracy>
Next, improvement in detection accuracy of the written code data 524 will be described. When the movement amount (scalar) of the motion vector is less than a predetermined value, it is encoded as a code “S” and determined to be in a stationary state (the amount of motion is small), but this predetermined value (threshold value) can be appropriately controlled. This is the first method for improving the detection accuracy. When writing characters in the air, the size and speed of the characters may vary greatly depending on the situation and the person. When writing large characters or writing quickly, it is determined that the brush is moving even if it is intended to be stationary, but when writing small characters or when writing slowly, it is determined that the brush is moving even if it is intended to move. It may be erroneously detected as a state. Therefore, when there is no large change in the situation, it is considered that the speed of writing and the size of characters by the user are generally close to the speed and the size and speed of characters written immediately before. Therefore, calculate the maximum value, minimum value, and average value of the writing speed (moving amount between frames) for the last few characters, and change the threshold value for whether or not the camera is stationary based on the values. Thus, the above-described erroneous detection can be reduced. Specifically, the control is adaptively performed so that the threshold value is increased when the character writing speed is high and the threshold value is decreased when the character writing speed is low.

  That is, the handwritten code data acquisition means determines that the movement vector is stationary when the movement amount of the movement vector is equal to or less than a predetermined threshold value, and encodes the movement vector according to the movement direction when the movement vector is larger than the predetermined threshold value. Changed based on volume history. Accordingly, the detection accuracy of the writing code data 524 is improved in consideration of the writing speed and the individual differences in character size.

  The second improvement in detection accuracy is to avoid hunting at the boundary point of the direction vector. Since the CPU 30 does not add a new code when there is no change in the written code data 524, for example, when the user writes a horizontal bar in the right direction, the same direction code as “bbb. As a result, only “b” is encoded. On the other hand, if the horizontal bar is tilted slightly, the direction code will be near the boundary between b and c and will be detected as “bcbcbc”, and since it is not a continuation of the same direction code, it will be encoded as “bcbcbc” as it is. Will be. In this case, although the user tries to write the same horizontal bar, the result of encoding is greatly different, and erroneous detection occurs. Therefore, even if “c” is detected after “bcb”, processing that does not add a code (hunting avoidance) is performed. If “a” is detected at this time, a code is added as usual, so that the S-shaped trajectory included in “fu”, “mu”, etc. can be accurately expressed.

  That is, the handwritten code data acquisition means performs coding by deleting the repeated portion when the movement vector repeats two directions. Thereby, hunting can be avoided and the detection accuracy of the written code data 524 is improved.

<Effect>
According to the character recognition device 12 of the first embodiment, the following effects can be obtained as described above. That is, the character recognition device includes an input period detecting unit that detects an input period for writing a character and a transition period for moving to the next character, and a movement detecting unit that detects a movement of a finger as a movement vector according to a predetermined operation of the user. And during the input period, the movement information detecting means for encoding the movement vector detected by the movement detecting means including the transition part to the next picture and acquiring the written code data 524 compressed according to the time series, and each character is correct Dictionary code data storage means for storing dictionary code data that is coded in accordance with the time series by coding a movement vector generated when writing in stroke order and stroke, including a transition part to the next picture, and movement information acquisition means Specifying means for comparing the written code data acquired by the user and the dictionary code data to specify the character written by the user. By coding the movement vector, the amount of calculation and the processing time can be reduced. In addition, by coding including the transition to the next image, it is possible to write characters consisting of a plurality of strokes continuously, and the character recognition device has no restrictions on the place, size or speed of writing.

  The handwritten code data acquisition means determines that the movement vector is stationary when the movement amount of the movement vector is less than or equal to a predetermined threshold value, and encodes the movement vector according to the movement direction when the movement vector is larger than the predetermined threshold value. Changed based on volume history. Accordingly, the detection accuracy of the writing code data 524 is improved in consideration of the writing speed and the individual differences in character size.

  Further, the handwritten code data acquisition means performs coding by deleting the repeated portion when the movement vector repeats two directions. Thereby, hunting can be avoided and the detection accuracy of the written code data 524 is improved.

  Further, according to the character recognition program of the first embodiment, the following effects can be obtained as already described. That is, the character recognition program causes the computer to detect an input period for writing a character and a transition period for moving to the next character according to a predetermined operation of the user, and a movement for detecting movement of a finger as a movement vector. A movement information acquisition means and a movement information acquisition means for acquiring movement information that is encoded in a time-series manner by encoding the movement vector detected by the movement detection means during the input period, including a transition portion to the next image. The acquired movement information is compared with dictionary code data prepared in advance to function as specifying means for specifying a character written by the user. By coding the movement vector, the amount of calculation and the processing time can be reduced. In addition, by coding including the transition to the next image, it is possible to write characters consisting of a plurality of strokes continuously, and the character recognition program has no restrictions on the place, size and speed of writing.

(Embodiment 2)
<Configuration>
In the first embodiment, the user writes a character in the air while holding the pen-shaped casing 14 of the character input device 12 and the character input device 12 recognizes this, but in the second embodiment, the user recognizes this. Writes a character in the air with hands free, and the character input device recognizes the character.

  FIG. 9 is a diagram illustrating a usage example of the input system 90 according to the second embodiment. Referring to FIG. 9, the input system 90 includes a character input device 92 and a display unit 94. The character input device 92 is incorporated in the input system 90, and is, for example, incorporated in the same casing and integrated. The character input device 92 images the user's hand with a camera, and recognizes the character written by the user in the air based on the movement and shape of the hand. The recognition result of the character input device 92 is transmitted to the input system 90, and the input system 90 having a kana character conversion function and a prediction function controls the whole including display and operation of the display unit 94.

  In the present embodiment, the character input device 92 is built in and integrated with the input system 90. However, the character input device 92 is not limited to this, and is separated as in the first embodiment to be wired or wireless. You may comprise so that it may communicate. In any case, the character input device 92 is not held in the hand, but is installed in the environment to detect the movement and shape of the hand from the outside. The input system 90 includes not only general-purpose computers (desktops, laptops, PDAs, etc.), but also mobile phones, in-vehicle navigation systems, and all products having an internal CPU from AV products to electronic dictionaries. The input system 90 and the lower-level component character input device 92 can be separated and configured depending on the product.

  FIG. 10 is a block diagram showing an electrical configuration of the character input device 92. Referring to FIG. 10, character input device 92 includes a CPU 30, and CPU 30 is connected to ROM 34, RAM 36 and input / output (I / O) interface (hereinafter simply referred to as “interface”) 38 via internal bus 32. Connected. The character input device 92 includes a camera 40, and the camera 40 is connected to the CPU 30 via the interface 38 and the internal bus 32. However, the camera 40 is arranged so that the lens portion faces the user and images the user's hand (detailed illustration is omitted).

<Operation>
For example, a user puts his hand in front of the character input device 92 and starts inputting characters by grasping his hand, writing characters with a single stroke in the air while holding his hand and opening his hand when one character is written. Perform the action. Then, the character input device 92 detects the start and end timings of the input period for writing one character from the shape of the user's hand imaged by the camera 40, and further detects the handwriting from the position of the user's hand during writing. That is, in the first embodiment, detection of the start and end of the input period in character units is detected by turning on / off the switch 16, but in the second embodiment, this is detected from the shape of the finger imaged by the camera 40. To detect. In the first embodiment, the writing direction is detected from the motion vector of the entire image captured by the camera 40. However, in the second embodiment, the position of the hand is detected from the image captured by the camera. The direction of writing is detected from the movement of the. Regarding other processes, since characters can be recognized by the same process as in the first embodiment, a method for detecting the shape and movement of the hand from the captured image of the camera 40 will be described next.

<Detection of hand shape and movement>
First, as a method for detecting the hand that has been put out before, there is a method using skin color information. In general, a region close to skin color is extracted from the captured image and binarized based on the premise that the hand is skin color. In principle, there is a problem of false detection when the user wears gloves and the hand color is not skin color, or there is something close to skin color such as a human face behind, but the hand moves fast Even if it can be detected.

  As another method, using the infrared LED emission control and the fact that the infrared reflection level increases as the distance from the subject is closer, the hand area that is put forward from the distance image is detected as a binarized image. I can do it. This method is not good at detecting fast hand movements, but has the advantage that the hand can be detected correctly even when there is an object close to skin color such as a human face in the background.

  FIG. 11 is a binarized image in which the hand that has been put out previously is detected using the above-described method. When a binarized image as shown in FIG. 11A is detected, palm position detection processing is performed in order to detect the direction of writing, and hand input is detected in order to detect an input period from the start to the end of writing. Perform shape determination processing. Hereinafter, these processes will be described.

  The center of gravity is often used to detect the position of the palm, but if the arm is exposed, the center of gravity moves in the direction in which the arm extends, so the exact palm position can be detected. Absent. Therefore, it is generally used that the binarized region of the palm that has been put forward is wider than the width of the binarized region of the arm. For example, as shown in FIG. 11B, a point having the maximum distance from the end of the binarized area is obtained, and this position is set as the center of the palm. Then, the movement vector at the center of the palm is encoded to be written code data 524. In this way, the CPU 30 functions as a position detection unit that detects the position of the finger.

  In the hand shape determination process, for example, a state in which the hand is held and a state in which the hand is opened are determined. Using the palm center detected by the above-described method, as shown in FIG. 11C, an evaluation parameter is obtained for a binarized region located above the palm center position. The evaluation parameters are the area of the binarized area, the area of a rectangle or circle circumscribing the binarized area, the perimeter of the binarized area, and the like. When the hand is held, (circumscribed area / binarized area) and (peripheral length / binarized area) are smaller than when the hand is opened. Therefore, the shape can be determined by evaluating these parameters. . As described above, the CPU 30 also has a function as a shape detection unit that detects the shape of the finger, and detects the input period and the transition period from the shape of the finger.

  That is, the character recognition device according to the second embodiment further includes an imaging unit (camera 40) that images the user's finger, and the input period detection unit includes a shape detection unit that detects the shape of the finger captured by the imaging unit. An input period and the transition period are detected based on a detection result of the shape detection unit, and the movement detection unit includes a position detection unit that detects a position of a finger imaged by the imaging unit, and the position detection unit detects The movement vector is detected from the position change of the finger to be performed. The user writes the characters in the air without holding the device in his hand and recognizes the characters using the camera installed on the environment side, so the handwriting input in the air is more flexible and convenient than with a device. Can be realized.

  Note that the character recognition devices of the first and second embodiments can be applied to various uses. For example, when the present invention is applied to an in-vehicle device such as a navigation system, character input is realized by detecting the movement of the driver who draws a stroke in the air with his hand next to the handle, and for destination setting and name search Available. For this reason, conventionally, it has been necessary to direct the line of sight to the touch panel to press the touch panel displayed with 50 sounds, but character input can be performed easily and safely without diverting the line of sight. In addition, when the present invention is applied to an AV device, character input can be realized by detecting a stroke drawn in the air with a camera, and it can be used for reserved program search, recorded content search, and Internet search. Therefore, compared to the conventional method of setting icons with 50 sounds displayed by operating the up / down / left / right buttons and the input method used on mobile phones, it is possible to input characters intuitively and easily by writing with a single stroke. it can. In addition, if the present invention is applied to a remote control, it is possible to input by moving the hand in front of the remote control with a built-in camera on the desk, etc., or by writing with one hand holding the remote control itself It is. These character inputs are more effective when combined with an inference function using past history and context.

<Effect>
According to the character recognition device of the second embodiment, the following effects can be obtained as described above. That is, the character recognition device further includes an imaging unit (camera 40) that images the user's finger, and the input period detection unit includes a shape detection unit that detects the shape of the finger imaged by the imaging unit. The movement detection means includes a position detection means for detecting the position of the finger imaged by the imaging means, and the position change of the finger detected by the position detection means is detected based on the detection result of The movement vector is detected from The user writes the characters in the air without holding the device in his hand and recognizes the characters using the camera installed on the environment side, so the handwriting input in the air is more flexible and convenient than with a device. Can be realized.

  10,90 input system, 12,92 character input device, 14 housing, 16 switch, 30 CPU, 36 RAM, 40 camera, 500 main processing program, 502 finger motion detection program, 504 encoding program, 506 character identification program 508, character input program, 520 dictionary code data, 522 direction data, 524 writing code data, 526 evaluation table data, 528 writing flag.

Claims (7)

An imaging means for imaging a user's finger;
An input period detecting means for detecting an input period for writing a character and a transition period for moving to the next character according to a predetermined operation of the user;
Movement detection means for detecting movement of a finger as a movement vector;
During the input period, the movement vector detected by the movement detection unit is coded including a transition part to the next image, and the writing code data acquisition unit acquires the writing code data compressed according to time series;
Dictionary code data storage means for storing the dictionary code data encoded according to the time series, the movement vector generated when each character is written, including a transition portion to the next picture,
A specifying means for specifying a character written by a user by comparing the writing code data and the dictionary code data;
The writing code data acquisition means determines a stationary state when the movement amount of the movement vector is equal to or less than a predetermined threshold, and encodes the movement vector according to the movement direction when the movement vector is larger than the predetermined threshold. It is changed based on the movement amount history,
Character recognition device. The input period detection means comprises a shape detection means for detecting the shape of a finger imaged by the imaging means,
The character recognition device according to claim 1. The input period detection means detects the input period and the transition period based on the detection result of the shape detection means,
The movement detection unit includes a position detection unit that detects a position of a finger imaged by the imaging unit, and detects the movement vector from a change in a finger position detected by the position detection unit. 2. The character recognition device according to 2. The dictionary code data storage means includes a plurality of the dictionary code data corresponding to each character, and in addition to the case of writing with the correct stroke order and stroke, the dictionary corresponding to the case of writing with the incorrect stroke order and stroke Comprising at least one code data,
The character recognition apparatus in any one of Claims 1-3.   The character recognition apparatus according to claim 1, wherein the writing code data obtaining unit performs coding by deleting repeated portions when the movement vector repeats two directions. Computer
An input period detecting means for detecting an input period for writing a character and a transition period for moving to the next character in accordance with a predetermined operation of the user;
Movement detection means for detecting movement of a finger as a movement vector;
During the input period, the movement vector detected by the movement detection unit is coded including a transition part to the next picture, and the writing code data acquisition unit acquires the writing code data compressed according to time series,
The writing code data and the dictionary code data prepared in advance are compared to function as a specifying means for specifying the characters written by the user,
The handwritten code data acquisition means determines that the movement vector is stationary when the movement amount of the movement vector is equal to or less than a predetermined threshold, and encodes the movement vector according to the movement direction when the movement vector is larger than the predetermined threshold, and moves the predetermined threshold. Change based on the history of quantity,
Character recognition program. The input period detection means includes a shape detection means for detecting the shape of a finger imaged by the imaging means ,
The character recognition program according to claim 6.

Images