JP2017146690A - Data processing apparatus and control program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a data processing apparatus capable of easily merging separated multiple pieces of stroke data, and a control program.SOLUTION: Stroke data corresponding to handwriting on a paper sheet 121 of a paper medium 100 are generated and transmitted to a smartphone 19 by a reader. A file of the stroke data is separated for each writing definition operation. Text data are generated and stored for each file by the smartphone 19. A user images the paper sheet 121 of a page where the file of the stroke data is separated by postscript by using a camera of the smartphone 19. The smartphone 19 generates imaging text data from imaging data of the paper sheet 121. The smartphone 19 compares the created imaging text data with the text data stored for each file and extracts a file including a matched portion. The smartphone 19 merges stroke data of the extracted file with each other.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a data processing apparatus and a control program.

  A handwriting input system that can generate data including coordinate information indicating the position of a writing instrument is known (see, for example, Patent Document 1). In the handwriting input system disclosed in Patent Document 1, a reading device reads a handwriting written on a notebook by a user. The user checks a check box provided on the page in order to confirm the written page. When detecting the check put in the check box, the reading device generates stroke data including coordinate information indicating a plurality of positions in the locus of the writing instrument. The reading device transmits the generated stroke data to an information terminal such as a personal computer. The information terminal generates image data indicating a line drawing written by a writing instrument based on the received stroke data. The information terminal displays an image based on the image data on the display.

JP 2014-86049 A

  In some cases, the user adds a check to a check box, once the page is confirmed and saved, and then added to the same note page. In this case, there is a problem that the stroke data is separated as a separate page. When the user tries to combine the stroke data into one, it is necessary for the user to select the corresponding page and perform the merge process, which is troublesome.

  An object of the present invention is to provide a data processing apparatus and a control program capable of easily merging a plurality of separated stroke data.

  The data processing apparatus according to the first aspect of the present invention provides a stroke for storing stroke data obtained by separating a handwriting written on a recording medium as data including coordinate information and time information for each confirmation operation for determining the handwriting. Data storage means, text data creation means for creating text data corresponding to the stroke data stored by the stroke data storage means, and first storage means for storing the text data created by the text data creation means Imaging means for imaging, and imaging text data creation means for recognizing character information from imaging data imaged by the imaging means and creating imaging text data that is text data corresponding to the recognized character information And the imaging text data created by the imaging text data creation means. Text data extracting means for comparing text data with the text data stored by the first storage means and extracting the text data including a matching portion; and the text data extracted by the text data extracting means And a first synthesis means for synthesizing the stroke data corresponding to.

  According to the first aspect, the stroke data can be automatically merged only by imaging the recording medium corresponding to the stroke data to be merged (combined) with the imaging means, so that it does not take time and effort for the user. Since the captured text data is compared with the text data, it is easy to match whether or not they match. Since the stroke data is merged, it is easy to convert the text into a page unit of the recording medium.

  The data processing device according to the first aspect stores image data creation means for creating image data corresponding to the stroke data stored by the stroke data storage means, and the image data created by the image data creation means. Image data extraction for comparing the second storage means, the imaging data imaged by the imaging means, and the image data stored by the second storage means, and extracting the image data including a matching portion And the first synthesizing unit corresponds to the image data extracted by the image data extracting unit when the text data extracting unit cannot extract the text data including the matching portion. The stroke data may be combined. Even if the text data extraction unit cannot extract the text data including the matching part, the data processing device extracts the image data including the matching part and corresponds to the extracted image data. Stroke data can be combined. Since the data processing apparatus compares the captured image data and the image data, it is easier to perform matching whether or not they match. Thereby, the data processing apparatus can improve the extraction accuracy of stroke data to be merged.

  The data processing apparatus according to the first aspect includes first selection means for selecting any one of the stroke data synthesized by the first synthesis means from the stroke data stored by the stroke data storage means. May be. Before comparing text data, the user can select any of the stroke data to merge. Thereby, the data processing apparatus can perform matching processing of text data correctly and promptly.

  The data processing device according to the second aspect is a stroke data storage unit that stores handwriting written on a recording medium as data including coordinate information and time information as stroke data separated for each confirmation operation for determining the handwriting. Image data creation means for creating image data corresponding to the stroke data stored by the stroke data storage means, third storage means for storing the image data created by the image data creation means, and imaging Image data extraction means for comparing the image data captured by the image capture means with the image data stored by the third storage means and extracting the image data including a matching portion; The stroke data corresponding to the image data extracted by the image data extraction means. Characterized by comprising a second combining means for combining each other.

  According to the second aspect, the stroke data can be automatically merged only by imaging the recording medium corresponding to the stroke data to be merged (combined) with the imaging means, so that it does not take time and effort for the user. Since the imaging data and the image data are compared, it is easy to match whether or not they match. Since the stroke data is merged, it is easy to convert the text into a page unit of the recording medium.

  The data processing apparatus according to the second aspect includes second selection means for selecting any one of the stroke data synthesized by the second synthesis means from the stroke data stored by the stroke data storage means. May be. Before comparing the captured image data and the image data, the user can select any of the stroke data to be merged. As a result, the data processing apparatus can accurately and promptly perform the matching process between the imaging data and the image data.

  The control program according to the third aspect of the present invention is a stroke data that stores stroke data obtained by separating a handwriting written on a recording medium as data including coordinate information and time information for each confirmation operation for determining the handwriting. A storage unit; a text data creation unit that creates text data corresponding to the stroke data stored by the stroke data storage unit; a storage unit that stores the text data created by the text data creation unit; A computer of a data processing apparatus including an imaging unit that recognizes character information from the imaging data captured by the imaging unit and creates captured text data that is text data corresponding to the recognized character information Imaging text data creation step, and imaging text data creation step A text data extraction step for comparing the captured text data created in the storage unit with the text data stored in the storage unit, and extracting the text data including a matching portion; and extracting in the text data extraction step And a combining step of combining the stroke data corresponding to the text data. According to the 3rd aspect, there can exist an effect similar to a 1st aspect.

  In addition, you may combine the structure of a part of said 1st aspect arbitrarily.

It is a figure which shows the outline | summary of the handwriting input system. 2 is a block diagram showing an electrical configuration of the handwriting input system 1. FIG. It is explanatory drawing which showed notionally the method to merge the separated stroke data. It is a flowchart of a data generation process. It is a flowchart of a merge control process. It is a flowchart of an initial process. It is a flowchart of the object file 1 determination process. It is a flowchart of the object file 2 determination process.

  Embodiments of the present invention will be described with reference to the drawings. The drawings to be referred to are used for explaining the technical features that can be adopted by the present invention, and the configuration of the apparatus described is not intended to be limited to this, but merely an illustrative example.

  The outline of the handwriting input system 1 will be described with reference to FIG. In the following description, the upper left side, lower right side, upper side, lower side, upper right side, and lower left side of FIG. 1 are respectively referred to as the left side, right side, front side, rear side, upper side, and lower side of the reading device 2 and the paper medium 100. And

  As shown in FIG. 1, the handwriting input system 1 includes a reading device 2, an electronic pen 3, a smartphone 19, and the like. The reading device 2 is a thin and light handwriting input device that can be folded and carried. In the handwriting input system 1, the user uses the electronic pen 3 to enter a line drawing on the paper 120 of the paper medium 100 attached to the reading device 2. The line drawing includes characters (letters, numbers, symbols, etc.), figures, and the like. The reading device 2 detects the position of the electronic pen 3. The reading device 2 identifies the locus of the electronic pen 3 based on the plurality of positions of the electronic pen 3 detected over time. The smartphone 19 generates and stores text data or image data obtained by digitizing a line drawing written on the paper 120 based on the locus data of the electronic pen 3 specified by the reading device 2.

  The reading device 2 includes a pair of left and right left reading devices 2L and 2R, and a cover 4. Each of the left reading device 2L and the right reading device 2R has a rectangular thin plate shape. The left reading device 2L and the right reading device 2R are arranged on the front surface of the cover 4 so as to be spread in the left-right direction. The left reading device 2L and the right reading device 2R are electrically connected by a flat cable (not shown). On the left side of the cover 4, a bag-like bag portion 4A is provided. The left reading device 2L is detachably attached to the cover 4 by being inserted into the bag portion 4A. The right reading device 2R is detachably attached to the right front surface of the cover 4 with, for example, a double-sided tape or an adhesive resin film.

  The paper medium 100 is a booklet that can be spread in the left-right direction. In the paper medium 100, a pair of front covers (a front cover 110 </ b> L and a back cover 110 </ b> R) and a plurality of papers 120 are bound at a part of each edge. As an example, the paper medium 100 is an A5 size notebook. The paper medium 100 is mounted on the reading device 2 so that the front cover 110L is placed on the front surface of the left reading device 2L and the back cover sheet 110R is placed on the front surface of the right reading device 2R. The paper medium 100 is detachably attached to the reading device 2 with, for example, a double-sided tape or an adhesive resin film. With the paper medium 100 positioned in the reading device 2, the paper medium 100 is loaded into the reading device 2. The left reading device 2L and the right reading device 2R can move integrally with the front cover 110L and the back cover 110R, respectively.

  Note that the reading device 2 may include only one of the left reading device 2L and the right reading device 2R. In this case, the paper medium 100 that can be loaded into the reading device 2 is, for example, an A5 size report sheet.

  The user can use the electronic pen 3 to enter a line drawing on the paper 120 of the paper medium 100 attached to the reading device 2. The electronic pen 3 is a known electromagnetic induction type electronic pen. The electronic pen 3 mainly includes a cylindrical body 30, a core body 31, a coil 32, a variable capacitor 33, a substrate 34, a capacitor 35, and an ink storage unit 36.

  The cylindrical body 30 has a substantially cylindrical shape, and houses a part of the core body 31, the coil 32, the variable capacitor 33, the substrate 34, the capacitor 35, and the ink storage unit 36 therein. The core body 31 is provided at the tip of the electronic pen 3. The core body 31 is urged toward the distal end side of the electronic pen 3 by an elastic member not shown. The distal end portion of the core body 31 protrudes outside the cylindrical body 30. The rear end side of the core body 31 is connected to the ink storage unit 36. The ink storage unit 36 stores ink and supplies the ink to the core 31. When the user fills in the sheet 120 using the electronic pen 3, a line drawing is formed on the sheet 120 by ink.

  The coil 32 is held between the core body 31 and the variable capacitor 33 while being wound around the ink storage portion 36. The variable capacitor 33 is fixed inside the electronic pen 3 by the substrate 34. A capacitor 35 is mounted on the substrate 34. The capacitor 35 and the variable capacitor 33 are connected in parallel to the coil 32 and constitute a known resonance (tuning) circuit.

  The electronic pen 3 further includes a detection switch (not shown). The detection switch is mounted on the tip of the substrate 34. In a state in which a line drawing is written on the paper 120 using the electronic pen 3, the core body 31 is slightly retracted into the electronic pen 3. In this state, the rear end portion of the core body 31 pushes the detection switch, and the detection switch is in the ON state. The CPU 21 (see FIG. 2) can detect whether or not a line drawing has been entered on the paper 120 based on whether or not the detection switch is in the ON state.

  The smartphone 19 is an information terminal that includes a touch panel 191 and a display 192. The touch panel 191 is used for inputting various instructions. The display 192 can display an image corresponding to the image data, characters corresponding to the text data, and the like. Instead of the smartphone 19, a terminal device such as a general-purpose PC or a tablet PC may be used.

  The electrical configuration of the handwriting input system 1 will be described with reference to FIG. The reading device 2 includes sensor boards 7L and 7R, a main board 20, sensor control boards 28 and 29, and an input unit 25. The sensor substrates 7L and 7R are provided inside the left reading device 2L and the right reading device 2R, respectively. The input unit 25 is provided in the right reading device 2R.

  The main board 20 includes a CPU 21, a RAM 22, a flash ROM 23, and a wireless communication unit 24. The RAM 22, flash ROM 23, and wireless communication unit 24 are electrically connected to the CPU 21. The CPU 21 controls the reading device 2. The RAM 22 temporarily stores various data such as calculation data.

  The flash ROM 23 stores various programs that the CPU 21 executes to control the reading device 2. The flash ROM 23 stores stroke data indicating the locus of the electronic pen 3. The flash ROM 23 stores a position specifying table described later. The flash ROM 23 stores information related to the reading device 2 (hereinafter referred to as internal information). The internal information includes, for example, firmware version information of the reading device 2, the number of times stroke data is saved, the number of times the power is turned on, the number of times that an error has occurred, the type of paper medium 100 loaded in the reading device 2, date, time, and various settings Including values. The flash ROM 23 stores internal information as numerical data. The wireless communication unit 24 is a controller for executing short-range wireless communication with an external electronic device.

  The input unit 25 is electrically connected to the CPU 21. The input unit 25 includes a switch for inputting an instruction to the reading device 2. Specifically, the input unit 25 includes a power switch. The power switch is a switch for inputting a power ON / OFF instruction to the reading device 2.

  On the sensor boards 7L and 7R, a large number of elongated loop coils are arranged in each of the left-right direction (X-axis direction) and the vertical direction (Y-axis direction). The sensor board 7L is electrically connected to the ASIC 28A of the sensor control board 28. The ASIC 28A detects the position of the electronic pen 3 as coordinate information when the writing operation by the electronic pen 3 is performed on the sensor substrate 7L. The sensor board 7R is electrically connected to the ASIC 29A of the sensor control board 29. The ASIC 29A detects the position of the electronic pen 3 as coordinate information when the writing operation by the electronic pen 3 is performed on the sensor substrate 7R. Of the ASICs 28A and 29A, the master-side ASIC 28A is directly connected to the CPU 21, and the slave-side ASIC 29A is connected to the CPU 21 via the ASIC 28A.

  Each of the sensor substrates 7L and 7R includes a detection region. The detection area is an area corresponding to a portion where the loop coils are arranged in each of the sensor substrates 7L and 7R. Therefore, when the paper medium 100 is mounted on the reading device 2 at an appropriate position, the paper medium 100 is placed on the detection area. The detection region of the sensor substrate 7L and the detection region of the sensor substrate 7R have the same configuration. Therefore, in the following description, only the detection region of the sensor substrate 7L will be described, and the description of the detection region of the sensor substrate 7R will be omitted.

  The principle of acquiring line drawing data will be described. The CPU 21 controls the ASICs 28A and 29A to flow a current (excitation transmission current) having a specific frequency to each of the loop coils of the sensor boards 7L and 7R. Thereby, a magnetic field is generated from each loop coil of the sensor substrates 7L and 7R. In this state, for example, when the user performs an operation of writing a line drawing on the paper 120 of the paper medium 100 fixed to the reading device 2 using the electronic pen 3, the electronic pen 3 comes close to the sensor boards 7L and 7R. Therefore, the resonance circuit of the electronic pen 3 resonates by electromagnetic induction and generates an induction magnetic field.

  Next, the CPU 21 controls the ASICs 28A and 29A to stop the generation of magnetic fields from the loop coils of the sensor substrates 7L and 7R. Each of the loop coils of the sensor substrates 7L and 7R receives an induced magnetic field generated from the resonance circuit of the electronic pen 3. The CPU 21 controls the ASICs 28A and 29A to detect signal currents (reception currents) flowing through the loop coils of the sensor boards 7L and 7R. The ASICs 28A and 29A execute this operation one by one for all the loop coils, so that the position of the electronic pen 3 is detected as coordinate information based on the received current.

  The position specifying table is a table that defines correspondence between all coordinate points of the electronic pen 3 that can be detected by the reading device 2 and coordinate points on the paper 120 fixed to the reading device 2. The CPU 21 can specify the line drawing entry position on the paper 120 based on the detected coordinate point of the electronic pen 3 by referring to the position specifying table.

  In the present embodiment, the CPU 21 can determine which of the entry area 122 and the check box 130 in the paper 121 of the paper medium 100 shown in FIG. 3 is based on the detected coordinate point of the electronic pen 3. When a line drawing is written in the writing area 122, the CPU 21 acquires line drawing data for each continuous line drawing and temporarily stores it in the RAM 22. The line drawing data includes a plurality of coordinate information indicating the trajectory of the electronic pen 3 in one entry operation until the electronic pen 3 contacts the paper 121 and leaves. The coordinate information may indicate either absolute coordinates or relative coordinates.

  When a line drawing is entered in the check box 130, the CPU 21 generates stroke data based on the line drawing data temporarily stored in the RAM 22 and stores it in the flash ROM 23. The stroke data includes coordinate information indicating a part or the whole of the line drawing entered in the entry area 122, time information indicating the order relationship between strokes, and the like. The detection area includes an instruction area. When the paper medium 100 is loaded in the reading device 2 at an appropriate position, the indication area and the check box 130 correspond to each other in position.

  The smartphone 19 includes a CPU 41, a RAM 42, a flash ROM 43, a wireless communication unit 44, an input circuit 45, an output circuit 46, a drive circuit 47, a touch panel 191, a display 192, and a camera 193. The CPU 41 controls the smartphone 19. The CPU 41 is electrically connected to the RAM 42, the flash ROM 43, the wireless communication unit 44, the input circuit 45, the output circuit 46, and the drive circuit 47.

  The RAM 42 stores various temporary data. The wireless communication unit 44 is a controller for executing short-range wireless communication with an external electronic device. The input circuit 45 performs control to send an instruction from the touch panel 191 to the CPU 41. The output circuit 46 performs control to display an image on the display 192 in accordance with an instruction from the CPU 41. The drive circuit 47 drives the camera 193 in accordance with an instruction from the CPU 41 and performs control to transmit image data captured by the camera 193 to the CPU 41.

  The flash ROM 43 stores various programs executed by the CPU 41 and stroke data received from the reading device 2. The smartphone 19 includes a medium reading device (for example, a memory card slot) not shown. The smartphone 19 can read a program stored in a storage medium (for example, a memory card) with a medium reader and install it in the flash ROM 43. The smartphone 19 may receive a program from an external device (not shown) connected to the smartphone 19 or a network and install the program in the flash ROM 43.

  With reference to FIG. 3, a sheet 121 which is an example of the sheet 120 will be described. The sheet 121 has a rectangular shape that extends in the vertical direction and the horizontal direction. An entry area 122 and a check box 130 are provided on both the front and rear sides of the sheet 121 (that is, two sheets). The entry area 122 and the check box 130 are areas independent of each other on the paper surface of the paper 121. The paper surface of the paper 121 corresponds to one page of the paper medium 100.

  The entry area 122 is an area for entering a line drawing indicating information using the electronic pen 3. Specifically, the entry area 122 has a substantially rectangular shape extending in the vertical direction and the horizontal direction along the edge of the paper surface of the paper 121. The entry area 122 is an area on the paper 121 excluding the check box 130. For example, the user uses the electronic pen 3 to freely enter various information (characters, characters, figures, etc.) to be stored electronically in the entry area 122.

  The check box 130 is an area in which a line drawing for instructing generation of stroke data based on the line drawing data temporarily stored in the RAM 22 can be entered using the electronic pen 3. In the present embodiment, the check box 130 is provided at the lower right corner of the paper surface of the paper 121. For example, after entering a line drawing such as characters in the entry area 122, the user enters a check mark in the check box 130 when he / she wants to save the entered line drawing.

  The data generation process will be described with reference to FIG. Data processing is processing for generating stroke data based on line drawing data. As shown in FIG. 1, the user attaches the paper medium 100 to the reading device 2 at an appropriate position. The user attaches the paper medium 100 to the reading device 2 such that the front cover 110L is placed on the front surface of the left reading device 2L and the back cover 110R is placed on the front surface of the right reading device 2R. In this state, when the user presses the power switch, the CPU 21 of the reading device 2 detects an operation of turning on the power of the reading device 2 via the input unit 25. In this case, the CPU 21 starts data generation processing by operating based on the program stored in the flash ROM 23.

  As shown in FIG. 4, the CPU 21 determines whether or not a line drawing has been entered on the paper 121 based on whether or not the detection switch of the electronic pen 3 is in an ON state (S1). When the line drawing is on the form 121 (S1: YES), the CPU 21 determines whether or not the line drawing entry position is in the check box 130 (S2).

  When the line drawing entry position is not in the check box 130 (S2: NO), the line drawing entry position is in the entry area 122. Therefore, the CPU 21 acquires line drawing data for each continuous line drawing (S4). The CPU 21 temporarily stores the acquired line drawing data in the RAM 22. When the line drawing entry position is in the check box 130 (S2: YES), the CPU 21 generates stroke data based on the line drawing data temporarily stored in the RAM 22 (S3) and stores it in the flash ROM 23 (S5). ).

  When the line drawing is not in a state of being written on the sheet 121 (S1: NO), or after executing S4 or S5, the CPU 21 has received a data request command from the smartphone 19 via the wireless communication unit 24. (S6). The data request command is a command for the smartphone 19 to request transmission of stroke data stored in the flash ROM 23. The data request command includes information for designating transmission target stroke data among the stroke data stored in the flash ROM 23.

  When the data request command has not been received (S6: NO), the CPU 21 returns to S1 and repeats the process. For example, when the user enters a line drawing with the electronic pen 3 in the entry area 122 of the paper 121 on the next page (S1: YES, S2: NO), the CPU 21 acquires the line drawing data (S4) and temporarily stores it in the RAM 22. Remember. After the line drawing is entered, if the user checks the check box 130 (S6: NO, S1: YES, S2: YES), the CPU 21 generates stroke data based on the line drawing data temporarily stored in the RAM 22. (S3). The CPU 41 stores the generated stroke data in the flash ROM 23 as a separate file separated from the previously stored stroke data file (S5). That is, in this embodiment, every time the user checks the check box 130 with the electronic pen 3, the flash ROM 23 sequentially stores the stroke data file generated in S3.

  When the data request command is received from the smartphone 19 (S6: YES), the CPU 21 transmits the file specified as the transmission target of the data request command among the stroke data files stored in the flash ROM 23 via the wireless communication unit 24. To the smartphone 19 (S7). On the other hand, the CPU 41 of the smartphone 19 receives the file transmitted from the reading device 2 and stores it in the flash ROM 43. The CPU 21 returns to S1 and repeats the process. When the CPU 21 detects an operation of turning off the power of the reading device 2 via the input unit 25, the data generation process is terminated.

  The flash ROM 43 of the smartphone 19 stores a stroke data file in the same manner as the flash ROM 23 of the reading device 2. When browsing the text written on the paper 121 on the display 192 of the smartphone 19, the user selects a file to be browsed on the touch panel 191, and performs, for example, a text display operation. When the CPU 41 detects a text display operation, the CPU 41 reads the stroke data of the file selected by the user from the files stored in the flash ROM 43, converts the data into text data, and displays the text data on the display 192. The smartphone 19 can perform image display in addition to text display. When the CPU 41 detects an image display operation, the CPU 41 converts the stroke data read from the flash ROM 43 into image data and then displays the image data on the display 192.

  With reference to FIG. 3, a case where stroke data is separately stored when added to the sheet 121 will be described with a specific example. As described above, in the present embodiment, each time the user checks the check box 130 with the electronic pen 3, a file of stroke data is sequentially stored in the flash ROM 23. For example, the user writes “1. Purpose...” In the entry area 122 of the paper 121 with the electronic pen 3 and puts a check in the check box 130. In this case, the stroke data corresponding to the written portion is stored in the flash ROM 23 as the n = 1st file.

  In some cases, the user additionally writes a sentence or the like on the paper 121 on the same page. For example, the user adds a sentence following “2. Method ...” after the document “1. Purpose...” Written earlier, and puts a check in the check box 130. In this case, the flash ROM 23 stores the added stroke data as the n = 2nd file. That is, in the flash ROM 23, although it is a sentence written on the paper 121 of the same page, the stroke data corresponding to these sentences is separated into two (n = 1, 2) and stored as separate files. The The CPU 21 transmits these two files based on the data request command from the smartphone 19. Therefore, similarly to the reading device 2, two stroke data files are stored in the flash ROM 43 of the smartphone 19.

  When the user browses the text written on the paper 121 on the display 192 of the smartphone 19, the flash ROM 43 stores the stroke data separately in two files. Alternatively, it is necessary to display it after converting it into image data. In the present embodiment, the user can easily merge the stroke data separated into two by using the camera 193 of the smartphone 19, for example.

  The merge control process will be described with reference to FIG. The user uses the touch panel 191 of the smartphone 19 to input a start instruction for merge control processing. When detecting the input of the start instruction, the CPU 41 of the smartphone 19 starts this process based on the program stored in the flash ROM 43. The CPU 41 executes initial processing (S10).

  The initial process will be described with reference to FIG. The CPU 41 sets 1 to n (S21). n is a file number assigned to the stroke data file stored in the flash ROM 43. The CPU 41 generates text data corresponding to the stroke data of the n = 1st file (S22). For example, the CPU 41 may convert the stroke data into text data by performing matching processing with text determination data stored in advance in the flash ROM 43. Further, it may be converted into text data while maintaining the character shape indicated by the stroke data. The CPU 41 stores the generated text data in the RAM 42.

  Subsequently, the CPU 41 generates image data corresponding to the stroke data of the n = 1st file (S23). For example, the CPU 41 generates line drawing data from coordinate information included in the stroke data, and converts a set of the generated line drawing data into image data. The CPU 41 stores the generated image data in the RAM 42. The CPU 41 adds 1 to n (S24). The CPU 41 determines whether n is larger than N (S25). N is the total number of files stored in the flash ROM 43 and is set for each paper medium 100. When n is N or less (S25: NO), the CPU 41 returns to S22, and repeats the above processing for n = 2 and subsequent (S22, S23). The CPU 41 generates text data and image data for each stroke data file and stores them in the RAM 42. As a result of adding 1 to n (S24), if n is larger than N (S25: YES), the CPU 41 ends the initial process and proceeds to S11 of the merge control process shown in FIG.

  As shown in FIG. 5, the CPU 41 displays a file selection screen on the display 192 (S11). The file selection scene is a screen that allows the user to select one target file to be merged. When the user recognizes any one of the stroke data files to be merged, it is preferable to select a file to be merged from the files stored in the flash ROM 43 on the file selection screen.

  The user selects a target file by inputting a file number and performing a confirmation operation. By selecting the target file, the CPU 41 can omit the target file 1 determination process described later. Therefore, the CPU 41 can perform the merge control process promptly and accurately. When the target file is selected (S12: YES), the CPU 41 determines the selected file as the target file 1 (S13), and executes the next imaging process (S14). The CPU 41 stores a flag indicating that the target file 1 has been determined in the RAM 42. In the file selection screen, when the user performs a confirming operation without inputting a file number (S12: NO), the CPU 41 performs an imaging process without doing anything (S14).

  In the imaging process, the CPU 41 activates a camera application built in the smartphone 19 and displays a guide message or the like on the display 192, for example. The guide message is a message that prompts the user to image the paper 121 after the additional recording. In accordance with the guide message, the user images the sheet 121 after the addition with the camera 193. The drive circuit 47 transmits imaging data of the camera 193 to the CPU 41. The CPU 41 stores the received imaging data in the RAM 42. The imaging data is image data captured by the camera 193. The CPU 41 extracts text data from the imaging data stored in the RAM 42 (S15). For example, the CPU 41 may extract text data of the imaging data by specifying a character or the like by matching with a pattern stored in advance in the flash ROM 43. The CPU 41 stores text data extracted from the imaging data (hereinafter referred to as imaging text data) in the RAM 42.

  The CPU 41 refers to the flag stored in the RAM 42 and determines whether the target file 1 has been determined (S16). When the target file 1 has not been determined (S16: NO), the CPU 41 executes target file 1 determination processing (S17). When the target file 1 has been determined (S16: YES), the CPU 41 executes target file 2 determination processing (S18).

  The target file 1 determination process will be described with reference to FIG. The CPU 41 sets 1 to n (S31). The CPU 41 compares the captured text data stored in the RAM 42 with the text data of the n = 1st file stored in the RAM 42 (S32). The CPU 41 determines whether or not there is a portion in the captured text data that matches the text data of the n = 1st file (S33). If there is no matching part (S33: NO), the CPU 41 adds 1 to n (S34). The CPU 41 determines whether n is larger than N (S35). When n is N or less (S35: NO), the CPU 41 returns to S32 and repeats the process. If there is a matching part (S33: YES), the CPU 41 determines the n-th file as the target file 1 (S42). The CPU 41 ends the target file 1 determination process and executes the target file 2 determination process (S18 in FIG. 5).

  If there is no matching part (S33: NO) and 1 is added to n (S34). If n is larger than N (S35: YES), the comparison is completed for all the files N, so the CPU 41 again sets n = 1 is set (S37). Since no matching part was found in the comparison with the text data, the CPU 41 compares the image data stored in the RAM 42 with the image data of the n = 1st file stored in the RAM 42 (S38).

  The CPU 41 determines whether or not there is a portion in the captured image data that matches the image data of the n = 1st file (S39). If there is no matching part (S39: NO), the CPU 41 adds 1 to n (S40). The CPU 41 determines whether n is greater than N (S41). When n is N or less (S41: NO), the CPU 41 returns to S38 and repeats the process. If there is a matching part (S39: YES), the CPU 41 determines the n-th file as the target file 1 (S42). The CPU 41 ends the target file 1 determination process and executes the target file 2 determination process (S18 in FIG. 5).

  If there is no matching part (S39: NO) and 1 is added to n (S40). If n is larger than N (S41: YES), the CPU 41 completes the comparison for all the files N. There is no part of the stored file that matches the image data. Therefore, the CPU 41 ends the merge control process with an error. At the end of the error, for example, the CPU 41 may display an error message on the display 192 indicating that the target file to be merged was not found. In this way, in the target file 1 extraction process, even if a portion that matches the image data is not found in the comparison between the text data, by comparing again with the image data, the stroke data corresponding to the portion that matches the image data is obtained. Can be extracted with high accuracy.

  The target file 2 extraction process will be described with reference to FIG. The CPU 41 deletes the text data of the target file 1 from the captured text data stored in the RAM 42 (S50). As a result, the CPU 41 can prevent the file already determined as the target file 1 from being redundantly determined as the target file 2 in this processing. The CPU 41 sets 1 to n (S51). The CPU 41 compares the captured text data stored in the RAM 42 with the text data of the n = 1st file stored in the RAM 42 (S52). The CPU 41 determines whether there is a portion in the captured text data that matches the text data of the n = 1st file (S53). If there is no matching part (S53: NO), the CPU 41 adds 1 to n (S54). The CPU 41 determines whether n is larger than N (S55). When n is N or less (S55: NO), the CPU 41 returns to S52 and repeats the process. If there is a matching part (S53: YES), the CPU 41 determines the n-th file as the target file 2 (S62). CPU41 complete | finishes the object file 2 determination process, and advances a process to S19 of FIG.

  If there is no matching part (S53: NO) and 1 is added to n (S54). If n is larger than N (S55: YES), the comparison is completed for all the files N. The image data of the target file 1 is deleted from the stored imaging data (S56). As a result, the CPU 41 can prevent the file already determined as the target file 1 from being redundantly determined as the target file 2 in this processing.

  The CPU 41 sets n = 1 again (S57). Since no matching portion was found in the comparison with the text data, the CPU 41 compares the imaging data stored in the RAM 42 with the image data of the n = 1st file stored in the RAM 42 (S58).

  The CPU 41 determines whether or not there is a portion in the captured image data that matches the image data of the n = 1st file (S59). If there is no matching part (S59: NO), the CPU 41 adds 1 to n (S60). The CPU 41 determines whether n is larger than N (S61). When n is N or less (S61: NO), the CPU 41 returns to S58 and repeats the process. If there is a matching part (S59: YES), the CPU 41 determines the n-th file as the target file 2 (S62). CPU41 complete | finishes the object file 2 determination process, and advances a process to S19 of FIG.

  If there is no matching part (S59: NO) and 1 is added to n (S60). If n is larger than N (S61: YES), the CPU 41 completes the comparison for all the files N, so the flash ROM 43 There is no file that matches the captured image data among the files stored in. Therefore, the CPU 41 ends the merge control process with an error. As described above, even in the target file 2 extraction process, even if a portion that matches the image data is not found in the comparison between the text data, the stroke corresponding to the portion that matches the image data is compared with the image data again. Data can be extracted with high accuracy.

  Referring back to FIG. 5, the CPU 41 reads the file determined as the target file 1 and the file determined as the target file 2 from the stroke data files stored in the flash ROM 43, and performs a stroke data merge process ( S19). For example, as shown in FIG. 3, when n = 1 is determined to be the target file 1 and n = 2 is the target file 2, for example, by merging the target file 2 with the target file 1, The n = 1st file is stroke data of a handwriting in which an additional portion “2. Method ...” is added after “1. Purpose”. The merged stroke data file is stored in the flash ROM 43. Note that after the merge process, the n = 2nd file may be deleted. Further, the CPU 41 may newly generate stroke data obtained by merging the n = 1 and second files as another file, and leave the n = 1 and second files separated into two.

  As described above, in this embodiment, since the separated stroke data can be automatically merged simply by capturing the paper 121 of the corresponding page in which the stroke data is separated by the camera 193 of the smartphone 19, the user's trouble is reduced. It does not take. Furthermore, in the present embodiment, the portion added to the paper 121 can be saved so that the portion added to the same file is added to the same file instead of a separate file. Therefore, this embodiment is easy to handle for the user, and can improve workability in organizing data. Furthermore, in this embodiment, since the stroke data is merged, the merged stroke data can be easily converted into text data or image data in units of pages.

  As described above, the handwriting input system of this embodiment includes the reading device 2, the electronic pen 3, and the smartphone 19. The reading device 2 generates stroke data corresponding to the handwriting written on the paper 121 of the paper medium 100 and stores it in the flash ROM 23. The stroke data file is separated for each confirmation operation for confirming writing. The confirmation operation for confirming writing is, for example, an operation in which the user checks in the check box 130 of the paper 121. The reading device 2 transmits a stroke data file to the smartphone 19. The smartphone 19 stores the received stroke data in the flash ROM 43 for each file. The smartphone 19 creates text data corresponding to the stroke data for each file and stores it in the RAM 42. The smartphone 19 includes a camera 193. The user images the sheet 121 of the page on which the stroke data file is separated and saved by adding a sentence or the like with the camera 193. The smartphone 19 recognizes characters and the like from the image data captured by the camera 193, creates imaged text data corresponding to the recognized characters and stores it in the RAM 42.

  The smartphone 19 compares the captured text data stored in the RAM 42 with a text data file that is stored in a plurality of pieces, and extracts a file including a matching portion. The smartphone 19 merges the stroke data corresponding to the extracted files. As a result, the handwriting input system 1 captures, with the camera 193, the sheet 121 that is the corresponding page of the paper medium 100 corresponding to the stroke data to be merged even if the stroke data is separated and stored for each writing confirmation operation. As a result, the separated stroke data can be automatically merged. In addition, since the captured text data is compared with the text data, it is easy to perform matching whether or not they match. In addition, since the stroke data is merged, it is easy to convert the text into pages from there.

  The smartphone 19 of the above embodiment creates image data corresponding to the stroke data stored in the flash ROM 43 for each file, and stores it in the RAM 42. Even if the smartphone 19 cannot extract text data including a portion that matches the captured text data, the smartphone 19 captures image data captured by the camera 193 and a file of image data to be stored separately. Can be extracted and files containing matching parts can be extracted. The smartphone 19 merges the stroke data corresponding to the extracted files. Since the smartphone 19 compares the imaged data with the image data, it is easier to match whether or not they match. Thereby, the precision which extracts the stroke data which wants to merge can be improved.

  The smartphone 19 of the above embodiment can select any one of the stroke data to be merged from the stroke data file stored in the flash ROM 43. Thereby, when comparing text data, the user can select any of the stroke data to be combined. Thereby, the matching process of text data can be performed correctly and rapidly.

  In the above embodiment, the handwriting input system 1 is an example of the “data processing apparatus” of the present invention. The CPU 21 that executes the process of S5 is an example of the “stroke data storage means” in the present invention. The CPU 41 that executes the process of S22 is an example of the “text data creating means” and the “first storage means” in the present invention. The camera 193 of the smartphone 19 is an example of the “imaging unit” in the present invention. The CPU 41 that executes the process of S15 is an example of the “captured text data creation unit” of the present invention. The CPU 41 that executes the processes of S17 and S18 is an example of the “text data extracting means” in the present invention. The CPU 41 that executes the process of S19 is an example of the “first synthesis means” in the present invention. The CPU 41 that executes the processing of S23 is an example of the “image data creation means” and “second storage means” in the present invention. The CPU 41 that executes the processes of S38, S39, and S42 and the processes of S58, S59, and S62 is an example of the “image data extraction unit” in the present invention. The CPU 41 that executes the processes of S11 to S13 is an example of the “first selection unit” and the “second selection unit” of the present invention.

  The present invention is not limited to the above embodiment, and various modifications can be made. In the target file 1 determination process (see FIG. 7) and the target file 2 determination process (see FIG. 8) of the above embodiment, the captured text data is compared with the text data first, and if no matching part is found, Although the imaged data and the image data are compared to extract a matching portion, the latter comparison between the imaged data and the image data may be omitted. In that case, if no matching part is found in the comparison between the text data, the process may end with an error. Contrary to the above embodiment, the imaging data and the image data are compared first, and if the matching part is not found, the imaging text data and the text data are compared to extract the matching part. Good.

  Further, only the comparison between the imaged data and the image data is performed, and if no matching part is found, the process may end in error. In addition, in the form in which the captured data and the image data are compared and the stroke data corresponding to the image data from which the matching portion is extracted are combined, the stroke data corresponding to the image data from which the matching portion is extracted is combined The CPU that executes is an example of the “second synthesis means” in the present invention. Even in these modified examples, it is preferable to select the target file in S12 of the merge control process shown in FIG.

  In the above embodiment, the CPU 41 of the smartphone 19 executes the merge control process shown in FIG. 5, but the CPU 21 of the reading device 2 may execute it. In this case, the merged stroke data file may be transmitted to the smartphone 19.

  In the imaging process of S14 of the merge control process (see FIG. 5) of the above embodiment, the camera 193 of the smartphone 19 is used, but other imaging means may be used. For example, the imaging data is obtained using a scanner or the like. You may make it acquire.

  In the merge control process of the above embodiment, two target files 1 and 2 to be merged are extracted, but three or more target files may be extracted. If there are at least two target files to be merged, they can be merged. If three or more target files are found, they may be merged together.

  In the above embodiment, the positions, shapes, sizes, etc. of the entry area 122 and the check box 130 on the sheet 121 shown in FIG. 3 are not limited to the above embodiment. For example, the check box 130 may be provided in any of the upper left corner, the lower left corner, and the upper right corner of the paper 121.

  The data generation process (see FIG. 4) and the merge control process (see FIG. 5) of the above embodiment may be executed by electronic components (for example, ASIC) other than the CPU 21 and CPU 41, respectively. Each of the data generation process and the merge control process may be distributed by a plurality of electronic devices (that is, a plurality of CPUs).

  In the above embodiment, page information from the reading device 2 may be read by the smartphone 19.

  In the merge control process (see FIG. 5) of the above-described embodiment, the process is made accurate by selecting one file to be merged and performing a comparison. At this time, for example, the acquired file is stored on the smartphone 19. A plurality of files may be displayed on the display 192, and the files to be merged may be merged by drag and drop while confirming by the user.

1 Handwriting Input System 2 Reading Device 3 Electronic Pen 19 Smartphone 21 CPU
22 RAM
23 Flash ROM
41 CPU
42 RAM
43 Flash ROM
100 paper medium 120 paper 122 entry area 130 check box 193 camera

Claims (6)

Stroke data storage means for storing stroke data separated for each confirmation operation for determining the handwriting as data including coordinate information and time information, and the handwriting written on the recording medium;
Text data creation means for creating text data corresponding to the stroke data stored by the stroke data storage means;
First storage means for storing the text data created by the text data creation means;
Imaging means for imaging;
Imaging text data creation means for recognizing character information from imaging data captured by the imaging means, and creating imaging text data that is text data corresponding to the recognized character information;
Text data extraction means for comparing the captured text data created by the captured text data creation means with the text data stored by the first storage means and extracting the text data including a matching portion;
A data processing apparatus comprising: first combining means for combining the stroke data corresponding to the text data extracted by the text data extracting means. Image data creation means for creating image data corresponding to the stroke data stored by the stroke data storage means;
Second storage means for storing the image data created by the image data creation means;
Image data extraction means for comparing the imaging data captured by the imaging means with the image data stored by the second storage means and extracting the image data including a matching portion;
The first synthesis means includes
The stroke data corresponding to the image data extracted by the image data extraction unit is synthesized when the text data extraction unit cannot extract the text data including the matching portion. Item 4. The data processing device according to Item 1. The first selecting means for selecting any one of the stroke data synthesized by the first synthesizing means from the stroke data stored by the stroke data storing means. 2. A data processing apparatus according to 2. Stroke data storage means for storing stroke data separated for each confirmation operation for determining the handwriting as data including coordinate information and time information, and the handwriting written on the recording medium;
Image data creation means for creating image data corresponding to the stroke data stored by the stroke data storage means;
Third storage means for storing the image data created by the image data creation means;
Imaging means for imaging;
Image data extraction means for comparing the image data captured by the imaging means with the image data stored by the third storage means, and extracting the image data including a matching portion;
A data processing apparatus comprising: second combining means for combining the stroke data corresponding to the image data extracted by the image data extracting means. 5. The apparatus according to claim 4, further comprising second selection means for selecting any one of the stroke data synthesized by the second synthesis means from the stroke data stored by the stroke data storage means. The data processing apparatus described. A stroke data storage unit for storing stroke data separated for each confirmation operation for determining the handwriting as data including coordinate information and time information, and the handwriting written on the recording medium;
A text data creation unit that creates text data corresponding to the stroke data stored by the stroke data storage unit;
A storage unit for storing the text data created by the text data creation unit;
In a computer of a data processing device provided with an imaging unit for imaging,
Imaging text data creation step of recognizing character information from imaging data imaged by the imaging unit and creating imaging text data that is text data corresponding to the recognized character information;
A text data extraction step of comparing the captured text data created in the captured text data creation step with the text data stored in the storage unit and extracting the text data including a matching portion;
A control program for executing a combining step of combining the stroke data corresponding to the text data extracted in the text data extracting step.

Images