CN103080878B - One or more groups of method and apparatus are divided into for hand-written stroke will be overlapped - Google Patents

Abstract

Provide a method that, device and computer program, so that a series of geometrical property based on strokes determines and the associated plurality of feature of current stroke, and the feature for being then based on being associated with these strokes will be divided into one or more groups of strokes including a series of Chong Die hand-written strokes.The segmentation can it is any be divided into one or more characters the step of before perform, and in fact every group of stroke can be a part for character or character, but whether how not have one group of stroke to include the stroke from more than one character.

Description

Method and apparatus for segmenting overlapping handwritten strokes into one or more groups

technical field

Exemplary embodiments of the present invention relate to the processing of overlapping handwriting, and more particularly, to segmenting a series of strokes including overlapping handwriting into one or more groups of strokes.

Background technique

Various devices capture or receive handwritten input through touch screens or other input devices. For example, many computers, tablets, cell phones, personal digital assistants (PDAs), and other types of electronic devices include touch screens that allow users to provide tactile input, such as in the form of handwriting. Handwriting input may be captured by the electronic device and processed in an attempt to recognize handwritten characters to inform future actions of the electronic device, including, for example, storage or transmission of representations of the handwriting input.

To make character recognition easier, handwritten input is usually processed to segment the handwritten input into different characters, such as letters, numbers or other characters. Several different techniques exist for segmenting handwritten input. One technique exploits the pause between each character input. According to this technique, a user interface, such as a touch screen, only needs to include a writing area. The user can write a character in the writing area. Then pause to receive the character and clear the writing area to prepare the writing area for the reception of the next character. Thus, pauses between each character input are used for handwriting segmentation.

Another technique employs a user interface with two or more writing areas, such as two or more touch screens. The user can write a character in one of these writing areas, and then can switch to the other writing area to write the next character. While the user is writing characters in one of these writing areas, characters previously written in the other writing area can be received and processed, and the other writing area can then be cleared to prepare the other writing area for reception of the next character . Therefore, switching between different writing regions is used for handwriting segmentation.

Yet another technique employs a relatively large single writing area. Within the writing area, the user can write several characters in succession one after the other in a manner comparable to writing characters on a piece of paper. The technique exploits the spatial separation between written characters and the underlying intelligence to properly segment handwritten input.

In yet another technique, the user interface provides a single writing area in which the user can write several characters consecutively in such a way that the characters overlap each other with no space between them. By exploiting the underlying intelligence, overlapping characters can be segmented to separate one character from the other. However, this procedure is more complicated than the above technique using commonly employed recognition processing. Under maximum likelihood estimation, the recognition process can erroneously segment a series of strokes into characters. This recognition process is computationally intensive and challenging to operate in real time, especially for small devices.

A variety of techniques that utilize overlapping continuous handwriting to allow electronic devices with relatively small user interfaces to receive continuous handwritten input using a user's finger or other stylus so that handwritten input can be provided relatively quickly compared to input one character at a time . However, the reliance on overlapping continuous handwriting can pose challenges with regard to segmentation and user interaction. In this regard, overlapping continuous handwriting can add to the difficulties associated with efficiently segmenting overlapping handwriting to separate each character from the others, especially compared to the more common continuous handwriting with spatial separations between written characters. Additionally, the overlapping of multiple characters can create difficulties for the user, as the user may not be able to clearly see what the user is currently writing due to the noisy and complex background of other overlapping characters.

Contents of the invention

According to an exemplary embodiment, a method, apparatus and computer program product are provided to segment a series of strokes including overlapping handwriting into one or more groups of strokes. This segmentation may be performed prior to any segmentation into one or more characters, and indeed each set of strokes may be a character or part of a character, but in any event no set of strokes includes strokes from more than one character. By segmenting a series of strokes into one or more groups of strokes, subsequent segmentation of the series of strokes into one or more characters becomes more efficient. Additionally, the previous set or sets of strokes may be displayed in a less noticeably different manner while continuously receiving overlapping handwriting to allow the user to view the latest strokes more clearly.

In one embodiment, a method is provided, the method comprising: receiving a series of strokes comprising overlapping handwriting, and for each stroke of the plurality of strokes, determining, by a processor, a relationship with Multiple features associated with the current stroke. The method of this embodiment also segments a series of strokes into one or more groups of strokes based on features associated with the strokes. As mentioned above, each set of strokes is a character or part of a character, but no set of strokes includes strokes from more than one character.

The method of one embodiment may also segment the series of strokes into one or more characters based on one or more groups of strokes into which the series of strokes have been segmented, thereby increasing the efficiency of segmenting a series of strokes into characters. According to an example embodiment, the method may determine the plurality of features associated with the current stroke by determining the plurality of features associated with the current stroke based solely on geometric properties of the series of strokes. In one embodiment, the method may also cause display of at least some of the groups such that at least one group is displayed in a distinct manner from at least one other group. Therefore, the displayed image of overlapping handwriting can be simplified to facilitate the user to view, for example, the latest strokes.

In one embodiment, the method also normalizes a number of features associated with the current stroke. In this regard, the plurality of features may be normalized based on the total size of a series of strokes. When determining the plurality of features associated with the current stroke, the plurality of features can be selected from the end point of the current stroke, the geometric center of the current stroke, the starting point of the next stroke, the geometric center of the next stroke, the smallest Select from the group consisting of the rectangle, the smallest rectangle that contains the next stroke, and the smallest rectangle that contains the previous stroke.

The method can be executed in incremental or batch mode. In incremental mode, the steps of determination of features and segmentation of a series of strokes are repeated after each successive stroke is received. In batch mode, the steps of determination of multiple features and segmentation of a series of strokes are performed repeatedly after receiving multiple strokes.

In another embodiment, an apparatus is provided comprising at least one processor and at least one memory comprising computer program code. The at least one memory and computer program code of the apparatus of this embodiment are configured, with at least one processor, to cause the apparatus to at least receive a series of strokes comprising overlapping handwriting, and for each of the plurality of strokes, based on the The geometric properties of the series of strokes determine a number of characteristics associated with the current stroke. The at least one memory and the computer program code of the apparatus of this embodiment are further configured, with at least one processor, to cause the apparatus to segment a series of strokes into one or more groups of strokes based on features associated with the strokes. As mentioned above, each set of strokes is a character or part of a character, but no set of strokes includes strokes from more than one character.

The at least one memory and the computer program code of the apparatus of an embodiment may also be configured to, with at least one processor, cause the apparatus to segment the series of strokes into one or more groups of strokes based on the group or groups of strokes into which the series of strokes has been segmented. or multiple characters, thereby increasing the efficiency of segmenting a series of strokes into characters. The at least one memory and the computer program code of the apparatus of an exemplary embodiment are further configured to, with at least one processor, cause the apparatus to determine, by determining a plurality of features associated with a current stroke based solely on geometric properties of the series of strokes, to determine A number of features associated with the current stroke. In one embodiment, the at least one memory and the computer program code of the apparatus are further configured to, with at least one processor, cause the apparatus to cause the display of at least some of the groups such that at least one group ends with Significantly different patterns from at least one other group are shown. Therefore, the displayed image of overlapping handwriting can be simplified to facilitate the user to view, for example, the latest strokes.

In one embodiment, the at least one memory and the computer program code of the apparatus are further configured to, with at least one processor, cause the apparatus to normalize the plurality of features associated with the current stroke. In this regard, multiple features may be normalized based on the total size of a series of strokes. When determining a plurality of features associated with the current stroke, the plurality of features can be selected from the endpoint of the current stroke, the geometric center of the current stroke, the starting point of the next stroke, the geometric center of the next stroke, the smallest rectangle containing the current stroke, Select from the group consisting of the smallest rectangle containing the next stroke and the smallest rectangle containing the previous stroke.

Analysis of strokes can be performed in incremental or batch mode. In incremental mode, the steps of determination of features and segmentation of a series of strokes are repeated after each successive stroke is received. In batch mode, the steps of determination of multiple features and segmentation of a series of strokes are performed repeatedly after receiving multiple strokes.

In yet another embodiment, an apparatus is provided that includes means for receiving a series of strokes including overlapping handwriting, and for each stroke in the plurality of strokes, determining a relationship with the current A device for associating multiple features with strokes. The apparatus of this embodiment may also include means for segmenting a series of strokes into one or more groups of strokes based on features associated with the strokes. As mentioned above, each set of strokes is a character or part of a character, but no set of strokes includes strokes from more than one character.

The apparatus of an embodiment may further comprise means for segmenting the series of strokes into one or more characters based on the set or groups of strokes into which the series has been segmented, thereby increasing the efficiency of segmenting the series of strokes into characters . In an exemplary embodiment, the means for determining the plurality of features associated with the current stroke includes means for determining the plurality of features associated with the current stroke based solely on geometric properties of the series of strokes. In one embodiment, the apparatus may further comprise means for causing display of at least some of said groups such that at least one group is displayed in a manner distinct from at least another group. Therefore, the displayed image of overlapping handwriting can be simplified to facilitate the user to view, for example, the latest strokes.

In one embodiment, the device further comprises means for normalizing the plurality of features associated with the current stroke. In this regard, multiple features may be normalized based on the total size of a series of strokes. When determining a plurality of features associated with the current stroke, the plurality of features can be selected from the endpoint of the current stroke, the geometric center of the current stroke, the starting point of the next stroke, the geometric center of the next stroke, the smallest rectangle containing the current stroke, Select from the group consisting of the smallest rectangle containing the next stroke and the smallest rectangle containing the previous stroke.

The device can analyze strokes in incremental or batch mode. In incremental mode, the steps of determination of features and segmentation of a series of strokes are repeated after each successive stroke is received. In batch mode, the steps of determination of multiple features and segmentation of a series of strokes are performed repeatedly after receiving multiple strokes.

In yet another embodiment, a computer program product comprising at least one computer readable memory having computer executable code portions stored therein is provided. The computer-executable code portion includes program code instructions for receiving a series of strokes including overlapping handwriting and for, for each of the plurality of strokes, determining a plurality of strokes associated with the current stroke based on geometric characteristics of the series of strokes. characteristic program code instructions. The computer-executable code portion of this embodiment also includes program code instructions for segmenting a series of strokes into one or more groups of strokes based on features associated with the strokes. As mentioned above, each set of strokes is a character or part of a character, but no set of strokes includes strokes from more than one character.

The computer-executable code portion of an embodiment may also include program code instructions for segmenting a series of strokes into one or more characters based on one or more groups of strokes into which the series has been segmented, thereby adding a Efficiency with which series of strokes are segmented into characters. The computer-executable code portion of an exemplary embodiment may also include program code instructions for determining a number of features associated with a current stroke by determining the number of features associated with the current stroke based only on geometric properties of the series of strokes . In one embodiment, the computer-executable code portions may further include program code instructions for causing display of at least some of the groups such that at least one group is displayed in a manner that is distinct from at least another group. show. Therefore, the displayed image of overlapping handwriting can be simplified to facilitate the user to view, for example, the latest strokes.

In one embodiment, the computer-executable code portion further includes program code instructions for normalizing the plurality of features associated with the current stroke. In this regard, multiple features may be normalized based on the total size of a series of strokes. When determining a plurality of features associated with the current stroke, the plurality of features can be selected from the endpoint of the current stroke, the geometric center of the current stroke, the starting point of the next stroke, the geometric center of the next stroke, the smallest rectangle containing the current stroke , the smallest rectangle containing the next stroke, and the smallest rectangle containing the previous stroke.

The computer program product can analyze strokes in incremental mode or batch mode. In incremental mode, the steps of determination of features and segmentation of a series of strokes are repeated after each successive stroke is received. In batch mode, the steps of determination of features and segmentation of the series of strokes are repeated after receiving a plurality of strokes.

Description of drawings

Having described certain exemplary embodiments of the present disclosure in general terms, reference will now be made to the accompanying drawings, which are not necessarily drawn to scale, in which:

Figure 1 is an illustration of overlapping handwriting in which four characters overlap;

Figure 2 is a block diagram of an apparatus according to an exemplary embodiment of the present invention;

Figure 3 is a functional block diagram of operations performed in accordance with an exemplary embodiment of the invention;

FIG. 4 is a flowchart illustrating operations performed in accordance with an exemplary embodiment of the present invention;

5 is an illustration of overlapping handwriting in which features of three consecutive strokes are recognized, according to an exemplary embodiment of the invention;

6 is an illustration of the manner in which the overlapping handwriting of FIG. 1 may be processed in order to recognize four characters, according to an exemplary embodiment of the invention;

7 is a flowchart illustrating operations performed in incremental mode according to an exemplary embodiment of the present invention; and

FIG. 8 is a flowchart illustrating operations performed in a batch mode according to another exemplary embodiment of the present invention.

detailed description

Some embodiments of the present invention now will be described more fully hereinafter with reference to the accompanying drawings, in which some, but not all embodiments of the invention are shown. Indeed, various embodiments of the invention may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this description will satisfy applicable legal requirements. Like reference numerals refer to like elements throughout. As used herein, the terms "data," "content," "information" and similar terms may be used interchangeably to refer to data capable of being transmitted, received and/or stored in accordance with various embodiments of the invention. Thus, use of any such terms should not be taken to limit the spirit and scope of the various embodiments of the invention.

In addition, as used herein, the term "circuitry" refers to: (a) a hardware-only circuit implementation (e.g., implemented in analog and/or digital circuits); (b) circuits and a combination of computer program product(s) of software and/or firmware instructions on a readable memory, which run together to cause the apparatus to perform one or more of the functions described herein; and (c) circuitry, such as a microprocessor or microprocessor The part of a processor that requires software or firmware to operate, even if that software or firmware does not physically exist. This definition of 'circuitry' applies to all uses of this term herein, including in any claims. As a further example, as used herein, the term 'circuitry' also includes an implementation comprising one or more processors and/or portion(s) thereof and accompanying software and/or firmware. As another example, as used herein, the term "circuitry" also includes, for example, baseband integrated circuits or application processor integrated circuits used in mobile phones, or in servers, cellular network devices, other network devices, and/or other computing devices similar integrated circuits.

As defined herein, "computer-readable storage media," which refer to permanent, physical storage media (e.g., volatile or non-volatile storage devices), are distinguishable from "computer-readable transmission media," which refer to electromagnetic signals .

As shown in FIG. 1 , handwriting can be input via a user interface, such as a touch screen, in an overlapping manner in which characters are continuously overlapped without space or time intervals between the characters. In this regard, the example of FIG. 1 shows that the four characters respectively listed and indicated at 100 in the lower part of the figure overlap in sequence. In this regard, display 102 shows input of a first character, display 104 shows overlapping input of first and second characters, display 106 shows overlapping input of first, second and third characters, and display 108 shows Overlapping input of all four characters.

Overlaying handwriting allows users to take full advantage of the user interface, which is particularly useful for users providing handwriting input with their fingers, eg, on relatively small and low resolution touch screens. Additionally, the use of overlapping handwriting provides a relatively natural and fast technique for receiving handwritten input. Referring to the progressively crowded display of FIG. 1 , it will be appreciated that reliance on overlapping handwriting to provide handwritten input presents challenges for segmentation of different characters as additional characters overlap previous characters. In addition, overlapping handwriting can also make it quite difficult for a user to see the character the user is currently writing or the previous character due to the simultaneous display of many other overlapping characters. Therefore, an exemplary embodiment according to the present invention provides a technique for pre-segmenting overlapping handwriting.

The overlapping handwriting, which may be pre-segmented, may be received via any of a variety of input devices, such as a user interface, eg, a touch screen, or the like. Not only can overlapping handwriting be received via a variety of different input devices, but these input devices can be authenticated and can include a portion of a variety of different types of electronic devices. For example, FIG. 2 shows a block diagram of a mobile terminal 10 that may embody an exemplary embodiment of the present invention. It should be understood, however, that the mobile terminal 10 shown and hereinafter described is merely an example of one type of device that may benefit from exemplary embodiments of the present invention, and that various types of mobile terminals, such as portable digital assistants (PDAs), cell phones, pagers, mobile TVs, gaming devices, laptop computers, cameras, video recorders, audio/video players, radios, positioning devices such as Global Positioning System (GPS) devices, or the above and other types of sound and text Combinations of communication systems can easily use the exemplary embodiments of the present invention.

Mobile terminal 10 may include antenna 12 or multiple antennas in operable communication with transmitter 14 and receiver 16 . The mobile terminal 10 may further include means, such as a processor 20, for providing signals to and receiving signals from the transmitter 14 and receiver 16, respectively. The signal includes signal information according to the air interface standard of the applicable cellular system and also includes user speech, received data and/or user generated data. In this regard, the mobile terminal 10 is capable of operating according to any of a variety of first, second, third and/or fourth generation communication protocols, and the like. For example, the mobile terminal 10 can be based on second generation (2G) wireless communication protocols IS-136, Time Division Multiple Access (TDMA), Global System for Mobile Communications (GSM) and IS-95 Code Division Multiple Access (CDMA), or according to third Generation (3G) wireless communication protocols such as Universal Mobile Telecommunications System (UMTS), CDMA2000, Wideband CDMA (WCDMA) and Time Division Synchronous CDMA (TD-CDMA), according to 3.9G wireless communication protocols such as Evolved UMTS Terrestrial Radio Access Network (E- UTRAN), operating according to the fourth generation (4G) wireless communication protocol, etc. Alternatively or additionally, the mobile terminal 10 is capable of operating according to non-cellular communication mechanisms. For example, the mobile terminal 10 is capable of communicating in a wireless local area network (WLAN) or other communication network.

In some embodiments, the processor 20 may include circuitry desired to implement the audio and logic functions of the mobile terminal 10 . For example, processor 20 includes one or more digital signal processors and/or one or more microprocessors. The processor may further include one or more analog-to-digital converters, one or more digital-to-analog converters, and/or other support circuits. The control and signal processing functions of the mobile terminal 10 are distributed between these devices according to their respective capabilities. Accordingly, processor 20 may also include functionality to modulate and transmit previously convolutionally encoded and interleaved messages and data. Processor 20 may additionally include an internal speech coder, and may include an internal data modem. Additionally, processor 20 may include functionality to operate one or more software programs, which may be stored in memory. For example, processor 20 is capable of operating a connectivity program, such as a conventional web browser. The connectivity program may then allow the mobile terminal 10 to transmit and receive web content, such as location-based content and/or other web content, according to, for example, Wireless Application Protocol (WAP), Hypertext Transfer Protocol (HTTP), and/or the like.

Mobile terminal 10 can also comprise user interface, and this user interface comprises output device and input device, and output device is for example traditional earphone or loudspeaker 24, ringer 22, microphone 26, display 28, and input device is for example user input interface, these All coupled to processor 20 . The user input interface that allows mobile terminal 10 to receive data may include any of a number of devices that allow mobile terminal 10 to receive data, such as keypad 30, a touch screen display (as represented by display 28), or other input devices. In embodiments that include a keypad 30 , the keypad may include conventional numeric (0-9) keys and relative keys (#, *) as well as other hard and soft keys for operating the mobile terminal 10 . Alternatively, keypad 30 may also include various conventional QWERTY keyboard arrangements. Keypad 30 may also include soft keys with associated functions. Additionally or alternatively, the mobile terminal 10 may include an interface device, such as a joystick or other user input interface. The mobile terminal 10 may further include a battery 34, such as a vibrating battery pack, for powering various circuits required to operate the mobile terminal 10, and optionally for providing mechanical vibrations as a detectable output.

As noted above, the user input interface may include a touch screen display 28, which may embody any known touch screen display. Thus, for example, touch screen display 28 may be configured to implement touch recognition by any suitable technique, such as resistive, capacitive, infrared, strain gauge, surface wave, optical imaging, discrete signal technology, acoustic pulse recognition, and the like. Touch screen display 28 may be configured to receive indications of user input and provide a representation of the user input to processor 20 .

The mobile terminal 10 may further include a User Identity Module (UIM) 38 . UIM 38 is typically a memory device with a built-in processor. UIM 38 may include, for example, a Subscriber Identity Module (SIM), Universal Integrated Circuit Card (UICC), Universal Subscriber Identity Module (USIM), Removable Subscriber Identity Module (R-UIM), and the like. UIM 38 typically stores information elements related to mobile users. In addition to the UIM 38, the mobile terminal 10 may also be equipped with memory. For example, the mobile terminal 10 may include volatile memory 40 such as volatile Random Access Memory (RAM) including a cache area for temporary storage of data. The mobile terminal 10 may also include other non-volatile memory 42, which may be embedded and/or may be removable. The memory may store various pieces of information and data used by the mobile terminal 10 to perform various functions of the mobile terminal 10 . For example, the memory may include an identifier capable of uniquely identifying the mobile terminal 10, such as an International Mobile Equipment Identity (IMEI) code. Additionally, these memories may store instructions for determining cell id information. Specifically, these memories can store applications for execution by the processor 20, which determine the identity of the current cell with which the mobile terminal 10 is in communication, such as cell id identity or cell id information.

Regardless of the type of input device and the type of electronic device including the input device, overlapping handwriting input may be analyzed according to an exemplary embodiment as shown in FIG. 3 . In this embodiment, overlaid handwriting may be received via a touch screen 110, such as touch screen display 28 of FIG. 2 or the like. Each stroke of the handwriting input may then be subjected to feature extraction 112, wherein one or more features associated with the stroke are determined as described below. A determination is then made, eg by classifier 114, whether the current stroke is part of the same group of strokes as the previous stroke, or whether the current stroke is part of another group of strokes. Although various classifiers 114 may be used to analyze features associated with the current stroke to determine the group to which the current stroke belongs, commonly used statistical classifiers such as support vector machines or artificial neural networks may be employed.

As shown in FIG. 3 , classifier 114 analyzes the current stroke by utilizing segmentation rules 116 . Segmentation rules 116 may be determined in various ways. However, in the illustrated embodiment, a handwriting database 118 is provided that includes a plurality of different series of handwritten strokes. Each series of strokes of the handwriting database 118 may then undergo feature extraction 120, wherein each series of strokes of the handwriting database is analyzed to identify features associated with the series of strokes and the individual strokes comprising the series. In one embodiment, these features that have been extracted and associated with the current strokes of the handwriting database 118 are trained 122 to refine the segmentation rules. In this regard, handwriting database 118 may contain many samples of different words and/or sentences written by different people. These words and/or sentences can be manually separated into characters. Based on this separation into characters, each stroke may be classified as the last stroke of the character or a stroke other than the last stroke of the character. Accordingly, training 122 may learn from strokes in handwriting database 118 to establish segmentation rules that define parameters to be considered in determining whether a stroke is or is not the last stroke of a character. As described below, a stroke may be represented by a feature vector based on various parameters of the stroke, and the value of the feature vector may then be determined by the classifier 114 based on the segmentation rule 116 . Next, the classifier may compare the value of the feature vector representing the stroke with a predefined threshold to determine whether the stroke is or is not the last stroke of the character. Accordingly, training 122 may be used to determine a set of parameters that provides the greatest success rate in properly classifying strokes for all of the various strokes in handwriting database 118 . Based on training 122, segmentation rules 116 are defined, which in turn define a set of parameters that can be utilized by classifier 114 to classify strokes based on the different series of strokes stored in handwriting database 118, such as according to the features and /or include individual strokes for each series. By comparing the value of a feature vector based on features that have been extracted from the current stroke with a predefined threshold, the classifier 114 can thus determine whether the current stroke is part of the same stroke group as the previous stroke or whether the current stroke begins with a different stroke group.

Since the purpose is to segment strokes into complete characters, errors may occur if the segmented group includes only a part of the character or if the segmented group includes strokes from more than one character. Since the method and apparatus of an exemplary embodiment will more easily accommodate and correct any errors associated with a segmented group comprising only a portion of a character, as the segmented group may be combined with the following pre-segmentation, in one embodiment, The predefined threshold may be adjusted, for example by increasing it, in order to reduce the likelihood that a segmented group will include strokes from more than one character.

In the exemplary embodiment of FIG. 3 , operations above the dashed line include a training phase 126 . As such, these operations may be performed in advance and do not need to be repeated when overlapping handwriting input is received by the touch screen 110 . However, operations below the dashed line include execution stage 124 and are performed upon or after receipt of overlapping handwriting input, such as after entering one or more strokes on touch screen 110 .

For further explanation, reference is now made to FIG. 4, in which operations performed by an apparatus according to an exemplary embodiment of the present invention are shown. The device can be applied on the mobile terminal 10, for example. Alternatively, however, the apparatus may be embodied on various other devices, including both mobile and stationary devices, eg, any of the devices listed above. The device may include means for receiving a sequence of strokes, such as a processor 20, a user input interface (eg, touch screen display 28), or the like. See operation 130 of FIG. 4 . As described above, the received plurality of strokes includes overlapping handwriting, in which a plurality of characters are continuously written one after the other. For each stroke, a number of features may be determined. See operation 132 . In this regard, the device may include means, such as processor 20, for determining a plurality of features associated with each stroke. Various features may be determined according to exemplary embodiments of the present invention. However, in one embodiment, the plurality of features determined for the current stroke includes features related to the current stroke as well as features related to previous and next strokes. In this regard, FIG. 5 illustrates a plurality of overlapping handwritten characters including a current stroke 200 , a previous stroke 210 and a next stroke 220 .

For example, the characteristics determined for the current stroke may include the end point of the current stroke, the geometric center of the current stroke, the starting point of the next stroke, the geometric center of the next stroke, the smallest rectangle containing the current stroke, the smallest rectangle containing the next stroke, Rectangle and the smallest rectangle containing the previous stroke. With respect to those features that define a particular point or location, those features are typically defined by a pair of coordinates, such as x, y coordinates. Similarly, for those features defined by rectangles or other two-dimensional shapes, each shape, such as each rectangle, can be defined with four features, such as the coordinates of the left and right sides of the shape and the coordinates of the top and bottom of the shape. Referring to FIG. 5 , according to an exemplary embodiment, the features extracted from the current stroke may include the x, y coordinates (endX, endY) of the end point of the current stroke indicated at 204, the geometric center (currentGCX, currentGCY) of the current stroke indicated at 206 ), the starting point of the next stroke marked by 222 (startX, startY), the geometric center of the next stroke marked by 226 (nextGCX, nextGCY) and the smallest rectangle including the current stroke, the next stroke and the previous stroke. Referring to FIG. 5, each rectangle may be defined by four features, ie, coordinates related to left, right, upper, and lower. In this regard, in Figure 5, the left, right, upper, and lower coordinates of the smallest rectangle associated with the current stroke, the next stroke, and the previous stroke are prefixed with current (current), next (next) and previous (previous a) marked. For example, the features related to the smallest rectangle of the current stroke are indicated by currentLeft (current left), currentRight (current right), currentTop (current top) and currentBottom (current bottom) in FIG. 5 . Other points marked in FIG. 5 for orientation include the start point of the current stroke at 202, the end point of the next stroke at 224, and the start and end points of the previous stroke at 212 and 214, respectively, although these other points are not included in this example. Non-existent implementations were not extracted as features.

Although the geometric center of a stroke can be defined in various ways, according to one exemplary embodiment, the geometric center of a stroke is defined as the average point of all points in the stroke. For example, a stroke containing an end point ( _xi , y _i ) (i=0...n-1), has a geometric center (GCX, GCY), which is defined as follows:

GCX=sum(x _i )/n, GCY=sum(y _i )/n; i=0...n-1

Multiple overlapping characters and corresponding strokes comprising these characters may be of different sizes, as the characters may be written smaller or larger while still attempting to convey the same meaning. As such, the apparatus may comprise means, such as the processor 20, for normalizing the features extracted from the current stroke to account for (eg by removing) any effect of differences in the size of the same stroke. See operation 134 of FIG. 4 . In one embodiment, these features are normalized based on the total size of a series of strokes. In this regard, the smallest rectangle containing each handwritten stroke can be defined as shown in FIG. 5 . In this regard, the smallest rectangle containing all strokes may be defined by coordinates relative to the top, bottom, left and right of overlapping handwritten characters. In an exemplary embodiment where the smallest rectangle containing the entire series of strokes is positioned so as to have a corner at the origin of the coordinate system and have sides extending along the coordinate axis, the smallest rectangle containing the entire series of strokes can be defined by its width such as totalWidth (total width) and its height as totalHeight (total height) in simplified form.

In one example, the features already described above can be normalized as follows, where the prefixes Current (current), Next (next) and Pre (previous) refer to the current stroke, next stroke, and previous stroke respectively relevant features.

Thus, CurrentStrokeEndX and CurrentStrokeEndY are the normalized x and y coordinates of the end point of the current stroke. NextStrokeStartX and NextStrokeStartY are the normalized x and y coordinates of the start point of the next stroke. CurrentGCX and CurrentGCY are the normalized x and y coordinates of the geometric center of the current stroke. NextGCX and NextGCY are the normalized x and y coordinates of the geometric center of the next stroke. CurrentLeft, CurrentRight, CurrentTop, and CurrentBottom are the left, right, top, and bottom coordinates of the current stroke, respectively. NextLeft, NextRight, NextTop, and NextBottom are the left, right, top, and bottom coordinates of the next stroke, respectively. Finally, PreLeft, PreRight, PreTop, and PreBottom are the left, right, top, and bottom coordinates of the previous stroke, respectively.

In one embodiment, the plurality of features extracted from the current stroke may then be combined, eg, by the processor 20, into a feature vector. In instances where the current stroke being analyzed is the initial stroke, there will be no previous stroke. As such, features related to the previous stroke may be set to a predefined value, eg -1.

Once features have been determined for the current stroke, and in one example, the strokes have been normalized, the series of strokes may be segmented into one or more groups of strokes based on features associated with the strokes. See operation 136 of FIG. 4 . As such, the apparatus may include means, such as the processor 20, for segmenting the plurality of series of strokes into one or more groups of strokes based on these features. As noted above, these strokes may be segmented by classifier 114 into one or more groups of strokes according to techniques described in more detail below. The apparatus may also include means, such as processor 20, for segmenting the series of strokes into one or more characters based on the set or groups of strokes into which the series of strokes have been segmented. See operation 138. Segmentation into characters based on these groups can be performed in various ways, including by utilizing segmentation rules 116, etc., which define parameters to be utilized by classifier 114 in order to properly estimate the feature vectors of the strokes. As such, processor 20 of an exemplary embodiment may include or implement feature extraction 112 , classifier 114 , and segmentation rules 116 , as described above in connection with FIG. 3 . By initially defining groups of strokes and then defining characters based on these groups, segmentation of strokes to characters can be performed in a more computationally efficient manner than in the case of segmenting multiple overlapping characters without intermediate grouping of strokes .

Referring to FIG. 6 , a plurality of overlapping characters are shown at 230 . Multiple groups may be defined as shown at 232 by dividing multiple series of strokes into stroke groups. Thereafter, the series of strokes may be segmented into characters based on the groups shown at 234 . The resulting characters can then be recognized, such as by pattern recognition or the like, as shown at 236, so that multiple overlapping handwritten characters can be very efficiently segmented and recognized as series of characters. In the exemplary embodiment of FIG. 6 , based on the segmentation of strokes into characters by these groups, multiple possible combinations of these groups, eg, all possible combinations of these groups, can be identified, as shown at 234 and 236 . In this regard, each character represented by a different combination of these groups may be recognized by a handwriting recognition engine, which may be embodied by processor 20 or another computing device in communication with the processor. In one embodiment, the handwriting recognition engine can also determine the similarity between each combination of these groups and the current character set. Each character set represented by a combination of these groups may then be analyzed, such as by a language model embodied by processor 20 or another computing device in communication with the processor, to determine whether each character set is meaningful. The device (such as the processor 20), the handwriting recognition engine and/or the language model may then determine whether the current character set determined by the language model makes sense based on the measure of similarity determined by the handwriting recognition engine. Each possible set of characters is measured and in some embodiments assigned a score for a combination of measurements of predefined geometric properties for each set. In this regard, each group may be analyzed, for example, by processor 20, to determine whether the group satisfies one or more predefined geometric properties, for example, a geometric property may be related to the size of the group, a group smaller than a predefined threshold being considered too large. Small rather than full characters. As another example of a geometric characteristic, groups located along the sides of the writing area, such as along the leftmost or rightmost portions of the touch screen display 28, may not be considered complete characters. The character set with the highest score as determined by processor 20, for example based on one or more of similarity, meaningfulness, and geometric properties, may be identified as the character set that best represents the plurality of overlapping characters, as shown at 238 .

In one embodiment, one or more constraints may be imposed on the process of identifying multiple possible combinations of these groups to increase the efficiency of identifying and subsequently processing these combinations. For example, these possible combinations can be limited by limiting any potential combinations or characters to a maximum of four groups.

In addition to increasing the efficiency with which multiple overlapping characters can be segmented, the segmentation of a series of strokes into one or more groups of strokes also facilitates the display of handwritten input. In this regard, the apparatus may also include means, such as processor 20, display 28, etc., for causing at least some of the groups to be displayed such that at least one group is displayed in a manner that is distinct from at least another group. See operation 140 of FIG. 4 . For example, only a subset of the groups may be displayed such that at least one group is not displayed. In this regard, a relatively small number of the most recent groups may be displayed with all previous groups removed from the display 28 . In this way, the display 28 is less cluttered and the user can more easily view the stroke currently being entered and the strokes immediately preceding it. In other embodiments, groups of strokes may be displayed in such a way that there is a clear distinction between the groups of strokes. For example, the groups of strokes may be displayed in different colors and/or brightness levels, such as different colors or brightness levels that vary based on the order in which the strokes were received. In one exemplary embodiment, the newest stroke group may be displayed with the darkest color (and/or brightness), the immediately preceding stroke group may be displayed with a slightly lighter color (and/or brightness), and so on until the stroke group is displayed with the darkest color (and/or brightness). The brightest color (and/or brightness) for the initial group of strokes shown. Alternatively, different stroke groups may be represented by different types of lines, for example, the latest stroke group is represented by a solid line, the previous stroke group is represented by a dot-dash line, and so on. In each embodiment, stroke groups are displayed such that different stroke groups are distinct, and in some embodiments newer stroke groups are more apparent.

In one embodiment, the determination of features associated with a stroke and the segmentation of that stroke into the current group may be performed after each stroke is input, for example as shown in FIG. 7 . With regard to subsequent descriptions of FIG. 7, the current stroke and the next stroke (as discussed above) are denoted stroke k−1 and stroke k, respectively. Referring to operations 250, 252, and 254 of FIG. 7, after a stroke k is input, it is determined whether the stroke is an initial stroke, ie, whether k=0. In instances where the strokes that have been input are initial strokes, an actual writing area may be initialized. See operation 256. In this regard, the actual writing area may be initialized to be the smallest rectangle enclosing the initial stroke, and may use the total width and total height of the rectangle oriented and positioned relative to the coordinate system as shown in FIG. The top, bottom, left, and right coordinates of the definition. Thereafter, counter k may be incremented before waiting for the input of the next stroke. See operation 258. After the next stroke is input, the actual writing area may be recalculated so that the recalculated actual writing area represents the smallest rectangle enclosing each stroke. See operation 260 . Thereafter, the characteristics of the current stroke k-1 can be determined. See operation 262 . Based on the determined features for stroke k-1, classifier 114 may provide a value _Spre based on the features of stroke k-1. In this regard, the classifier 114 may provide a value S _pre based on the feature vector of the stroke k−1 and a segmentation rule 116 defining parameters utilized by the classifier to estimate the feature vector of the stroke. See operation 264 of FIG. 7 . The value S _pre provided by the classifier 114 may then be compared with a predefined threshold T _pre for classification. See operation 266. Stroke k−1 and stroke k are considered to belong to different groups if the value S _pre provided by the classifier 114 exceeds a predefined threshold T _pre for classification. See operation 268 . Conversely, stroke k−1 and stroke k are considered to belong to the same group if the value S _pre provided by the classifier 114 is not greater than the predefined threshold T _pre for classification. See operation 270 . The process described above and shown in Figure 7 may be repeated incrementally for each stroke entered in order to properly segment the series of strokes into groups.

As an alternative to analyzing each stroke individually after its input in order to group the strokes appropriately, multiple strokes may be analyzed in a batch process, such as shown in FIG. 8 . In this regard, the actual writing area can be determined, such as the smallest rectangle enclosing a plurality of strokes 0, 1, . . . , M-1. See operation 280 . Then, a counter k can be initialized, eg k=1, and it can be determined whether all strokes of the batch have been considered, for example by comparing this counter with the number of strokes M. See operations 282 and 284 . In instances where all strokes have not been considered, features of stroke k−1 are determined and then a value S _pre for stroke k−1 may be determined based on the features of stroke k−1 , for example by classifier 114 . See operations 286 and 288. As before, a comparison of the value S _pre with the threshold T _pre can be done, wherein stroke k−1 and stroke k belong to different groups or the same group, respectively, depending on whether the value S _pre is greater than the threshold T _pre for classification. See operations 290, 292 and 294. This process may be repeated for each stroke of the batch indicated by the increment of counter k in operation 296, until each stroke has been accounted for and properly sorted into groups.

4, 7 and 8 are flowcharts of methods and program products according to exemplary embodiments of the invention. It will be understood that each block of the flow diagrams, and combinations of blocks in the flow diagrams, can be implemented in various ways, such as hardware, firmware, processors, circuits, and/or in combination with one or more computer program instructions other devices related to the execution of the software. For example, one or more of the procedures described above may be embodied by computer program instructions. In this regard, the computer program instructions embodying the processes described above may be stored in the storage device of the mobile terminal 10 and executed by the processor 20 in the mobile terminal. It will be understood that any such computer program instructions may be loaded onto a computer or other programmable apparatus (e.g., hardware) to produce a machine such that execution of these instructions on the computer or other programmable apparatus generates for execution ( Each) means of the function specified in the flowchart block. These computer program instructions may also be stored in a computer-readable memory that directs a computer or other programmable apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce a sequence of events including execution of the flowchart block(s). A product of a command device for a given function. These computer program instructions can also be loaded into a computer or other programmable device to cause a series of operations to be performed on the computer or other programmable device to generate a computer-executed process, so that these instructions executed on the computer or other programmable device can be executed The function specified in the flowchart block.

Accordingly, blocks of the flowchart support combinations of means for performing the specified functions, combinations of operations for performing the specified functions and program instructions for performing the specified functions. It will also be understood that one or more blocks of the flowcharts, and combinations of blocks in the flowcharts, can be implemented by special purpose hardware-based computer systems which perform the specified functions, or combinations of special purpose hardware and computer instructions.

Many modifications and other embodiments of the inventions set forth herein will come to mind to one skilled in the art to which these inventions pertain having the benefit of the teachings presented in the foregoing descriptions and the associated drawings. Therefore, it is to be understood that the inventions are not to be limited to the particular embodiments disclosed and that modifications and other embodiments are to be included within the scope of the appended claims. Additionally, although the foregoing description and associated drawings have described exemplary embodiments in the context of certain exemplary combinations of elements and/or functions, it should be appreciated that different combinations of elements and/or functions may also be produced by Alternative embodiments are provided without departing from the scope of the appended claims. In this regard, for example, different combinations of elements and/or functions than those explicitly described above are also contemplated and may be set forth in some of the appended claims. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for limitation.

Claims (27)

1. a kind of method for processing handwriting input, including：

Reception is included in a series of completely overlapped hand-written strokes between hand-written character；

For each stroke in multiple strokes, determined and current stroke using characteristic of the processor based on a series of strokes Associated plurality of feature；

A series of strokes are divided into by multigroup stroke based on the feature being associated with the plurality of stroke, wherein, often Group stroke is a part for character or character, but does not have one group of stroke to include the stroke from more than one character；And

Cause in multigroup stroke at least some group display so that the least one set in described at least some group with At least another group in described at least some group it is completely overlapped and by differ markedly from it is described at least another group in the way of carry out Show.

2. the method for claim 1, is included wherein determining with the associated plurality of feature of current stroke：By described one it is Each stroke representation in row stroke is into a corresponding characteristic vector；And wherein splitting a series of strokes includes：It is logical Cross and the corresponding characteristic vector of each stroke in a series of strokes is compared with a predefined threshold value, will be described A series of strokes are divided into multigroup stroke.

3. the method for claim 1, wherein splitting a series of strokes includes：Based on segmentation rule to determine State last stroke whether a series of each stroke in strokes is character.

4. the method for claim 1, further includes：By the plurality of feature mark being associated with the current stroke Standardization.

5. method as claimed in claim 4, wherein, the plurality of feature normalization is included：Based on a series of strokes Overall size by the plurality of feature normalization.

6. the method for claim 1, wherein determine that the plurality of feature includes determining from the group being made up of the following Multiple features of middle selection：The terminal of the current stroke, the geometric center of the current stroke, the starting point of next stroke, under The geometric center of unicursal, the minimum rectangle comprising the current stroke, the minimum rectangle comprising next stroke and include The minimum rectangle of previous adjacent stroke.

7. the method as described in any one of claim 1-6, wherein it is determined that the plurality of feature and segmentation are described a series of Stroke repeats after each continuous stroke is received.

8. the method as described in any one of claim 1-6, wherein it is determined that the plurality of feature and segmentation are described a series of Stroke is carried out after multiple strokes are received.

9. the method as described in any one of claim 1-6, wherein it is determined that what is be associated with the current stroke is described more Individual feature includes：The geometrical property for being based only upon a series of strokes determines the plurality of spy being associated with the current stroke Levy.

10. a kind of device for processing handwriting input, including at least one processor and including computer program code extremely A few memorizer, at least one memorizer and the computer program code are configured to and at least one process Device causes described device at least together：

Reception is included in a series of completely overlapped hand-written strokes between hand-written character；

For each stroke in multiple strokes, the geometrical property determination based on a series of strokes is associated with current stroke Multiple features；

A series of strokes are divided into by multigroup stroke based on the feature being associated with the plurality of stroke, wherein, often Group stroke is a part for character or character, but does not have one group of stroke to include the stroke from more than one character；And

Cause in multigroup stroke at least some group display so that the least one set in described at least some group with At least another group in described at least some group it is completely overlapped and by differ markedly from it is described at least another group in the way of carry out Show.

11. devices as claimed in claim 10, wherein, at least one memorizer and computer program code are further It is configured to：Cause together with least one processor described device by each stroke representation in a series of strokes into One corresponding characteristic vector；And by by the corresponding characteristic vector of each stroke in a series of strokes with one Predefined threshold value is compared, and a series of strokes are divided into multigroup stroke.

12. devices as claimed in claim 10, wherein, at least one memorizer and computer program code are further It is configured to together with least one processor so that described device is determined in a series of strokes based on segmentation rule Each stroke be whether character last stroke.

13. devices as claimed in claim 10, wherein, at least one memorizer and computer program code are further It is configured to together with least one processor so that the plurality of spy that will be associated with the current stroke of described device Levy standardization.

14. devices as claimed in claim 13, wherein, at least one memorizer and computer program code are further It is configured to cause described device together with least one processor by the overall size based on a series of strokes by institute State multiple feature normalizations and by the plurality of feature normalization.

15. devices as claimed in claim 10, wherein, at least one memorizer and computer program code are further It is configured to together with least one processor so that described device is selected from the group being made up of the following by determining Multiple features and determine the plurality of feature：The terminal of the current stroke, the geometric center of the current stroke, next record The starting point of picture, the geometric center of next stroke, the minimum rectangle comprising the current stroke, the minimum comprising next stroke Rectangle and the minimum rectangle comprising previous adjacent stroke.

16. devices as described in any one of claim 10-15, wherein, at least one memorizer and computer journey Sequence code be further configured to cause together with least one processor described device receive each continuous stroke it The plurality of feature and a series of segmentation strokes are repeatedly determined afterwards.

17. devices as described in any one of claim 10-15, wherein, at least one memorizer and computer journey Sequence code is further configured to together with least one processor, cause described device true after multiple strokes are received Fixed the plurality of feature and a series of segmentation strokes.

18. devices as described in any one of claim 10-15, wherein, at least one memorizer and computer journey Sequence code is further configured to together with least one processor so that described device is described a series of by being based only upon The geometrical property of stroke determines and the associated plurality of feature of the current stroke, so that it is determined that being associated with the current stroke The plurality of feature.

A kind of 19. equipment for processing handwriting input, including：

For receiving the device for being included in a series of completely overlapped hand-written strokes between hand-written character；

For for each stroke in multiple strokes, the geometrical property based on a series of strokes determines and current stroke phase The device of multiple features of association；

For a series of strokes to be divided into the dress of multigroup stroke based on the feature being associated with the plurality of stroke Put, wherein, every group of stroke is a part for character or character, but does not have one group of stroke to include the pen from more than one character Draw；And

For cause in multigroup stroke at least some group show so that in described at least some group at least one At least another group in group and described at least some group it is completely overlapped and by differ markedly from it is described at least another group in the way of The device for being shown.

20. equipment as claimed in claim 19, wherein for determining the device bag with the associated plurality of feature of current stroke Include：For by each stroke representation in a series of strokes into a corresponding characteristic vector device；And wherein use Include in the device for splitting a series of strokes：For by by the corresponding spy of each stroke in a series of strokes Levy vector to be compared with a predefined threshold value, a series of strokes are divided into into the device of multigroup stroke.

21. equipment as claimed in claim 19, wherein, the device for splitting a series of strokes is included for base Determine that in segmentation rule whether each stroke in a series of strokes is the device of last stroke of character.

22. equipment as claimed in claim 19, further include：For will be the plurality of with what the current stroke was associated The device of feature normalization.

23. equipment as claimed in claim 22, wherein, it is described for by the device of the plurality of feature normalization include for Overall size based on a series of strokes is by the device of the plurality of feature normalization.

24. equipment as claimed in claim 19, wherein, the device for determining the plurality of feature is included for determining The device of the plurality of feature selected from the group being made up of the following：The terminal of the current stroke, the current pen The geometric center of picture, the starting point of next stroke, the geometric center of next stroke, the minimum rectangle comprising the current stroke, bag Minimum rectangle containing next stroke and the minimum rectangle comprising previous adjacent stroke.

25. equipment as described in any one of claim 19-24, wherein, the dress for determining the plurality of feature Put and repeatedly operate after each continuous stroke is received for splitting a series of device of strokes with described.

26. equipment as described in any one of claim 19-24, wherein, the dress for determining the plurality of feature Put and operate after multiple strokes are received for splitting a series of device of strokes with described.

27. equipment as described in any one of claim 19-24, wherein, it is described for determination and the current stroke phase The device of multiple features of association includes determining and the current stroke for the geometrical property for being based only upon a series of strokes The device of associated the plurality of feature.