JPH10254989A - Image processing system and method - Google Patents

Abstract

(57) [Summary] [PROBLEMS] To improve the input user interface and promote the miniaturization of portable terminals. A handwritten memo or the like is written on a sheet of paper, and a closed loop pattern is drawn so as to surround the memo. The image on the paper is read using the image input unit 3. Image processing unit 4
Detects a closed loop pattern from the input image data, extracts the image data in the area inside the closed loop pattern, and sends it to the host device 6. The image pattern analysis / recognition unit 7 performs an image recognition process on the difference image data received from the mobile terminal 1 and outputs the recognition result to the mobile terminal 1.
Return to.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a system for processing image data input using a certain terminal device by another information processing device. The present invention also relates to a technique for removing unnecessary patterns from an image pattern read using an electronic camera or the like.

[0002]

2. Description of the Related Art In the information society in recent years, various documents, drawings, and the like have been digitized, especially for business purposes. In some cases, even a note-sized content,
It is required to be stored electronically.

[0003] When digitizing a document, the most common method is to start an application program for document creation on a personal computer and input the characters one by one using a keyboard by a document creator. In general, when digitizing a drawing or the like, a method of activating an application program for drawing creation or the like and inputting the drawing by using a keyboard or a mouse device is generally used.

Recently, electronic cameras have become widespread. The electronic camera stores image data read by photographing a subject as digital data. Then, by connecting to a personal computer or the like via a predetermined interface, image data read by the electronic camera can be processed on the personal computer. Therefore, for example, if a document written on paper is photographed by an electronic camera, and image data read by the electronic camera is input to a personal computer or the like and character recognition processing is performed, the document written on the paper can be obtained. Can be digitized.

[0005] As described above, when a document or drawing is digitized, it is generally performed using a keyboard or a mouse. In a specific case, the document or drawing is captured using an electronic camera (or an image scanner). For example, image recognition processing is performed on image data.

[0006]

However, the operation using a keyboard or a mouse device is not always easy for all users, and some users are not good at the operation. Also, when digitizing the contents of a memo,
While handwriting can be performed in a short time, key input requires an operation to be started after activating an application program for document creation each time, which is troublesome.

On the other hand, with a method of reading the contents written on paper or the like using an electronic camera or the like, it is difficult to photograph only a necessary range, and there is a possibility that a document other than a document whose image recognition is desired may be read. high. In this case, it is necessary to specify the image recognition area after displaying the image data read by the camera shooting on the display of the personal computer, or to correct the document obtained as a result of the image recognition, which is also troublesome. .

Further, in the above-described method using an electronic camera, if a notebook computer or the like for executing image recognition processing is not carried, the contents photographed at a travel destination (such as a business trip destination) are instantly obtained on the spot. It cannot be digitized. In addition, high-performance character recognition software generally has a large program scale, and it is not desirable to mount such a large-scale program on a simple portable terminal or the like.

It is an object of the present invention to improve the input user interface and promote the miniaturization of a portable terminal.

[0010]

An image processing system according to the present invention has a configuration in which an image read by a first terminal device is recognized by a second terminal device, and an area including a handwritten memo or the like is predetermined. Reading means for reading an image on a recording surface designated as a recognition target area in accordance with rules, detecting means and extraction means for extracting the recognition target area from the image read by the reading means in accordance with the predetermined rule, A recognition unit that recognizes a character or a specific pattern included in the image data of the area extracted by the detection unit and the extraction unit. The reading means is provided in a first terminal device, and the recognition means is provided in a second terminal device. Further, the detection means and the extraction means may be provided first, or may be provided in the second terminal device.

The predetermined rule is, for example,
A method in which a region inside the closed loop pattern is set as a recognition target region, a method in which a region at a predetermined position with respect to a predetermined identification instruction pattern (for example, an underline) is set as a recognition target region, or a method in which a predetermined specific For example, there is a method of setting a region including a color as a recognition target region.

According to the above method, only the image data of a necessary area is extracted and recognized from the image read by the reading means. Therefore, only the desired pattern is recognized, and unnecessary images are removed without being recognized.

Further, since the function of reading image data and the function of executing image recognition are provided in independent devices, the size of the first terminal device can be reduced while the size of the first terminal device is reduced. The advanced functions of the second terminal device can be used.

[0014]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a system configuration diagram of the present embodiment. The mobile terminal 1 has a communication function, requests data processing from another device connected to the network 5, receives the result, and displays the result on the image display unit 2 composed of, for example, a liquid crystal display. The portable terminal 1 transmits and receives data wirelessly via a nearest base station, transmits and receives data to and from a device (optical link unit) having a communication function by Ir (infrared) communication, or uses a wired connection. Is connected to the network 5 by a method of transmitting and receiving data. Further, the mobile terminal 1 includes an image input unit 3 including, for example, an electronic camera, and transmits image data captured using the image input unit 3 to another device (for example, a host device 6) connected to the network 5 to perform image processing. Has the function of requesting Furthermore, the mobile terminal 1
The image processing unit 4 can process image data captured using the image input unit 3.

Network 5 is a public telephone network, PHS
It is a network or a LAN, and accommodates the host device 6. The host device 6 is a server machine, and the mobile terminal 1
Executes data processing according to the request transferred from. Further, the host device 6 includes an image pattern analysis / recognition unit 7. The image pattern analysis / recognition unit 7 performs a pattern recognition process or a character recognition process on the image data sent from the mobile terminal 1 and detects a predetermined pattern and a character string included in the received image data. I do.

In the above configuration, when the user of the portable terminal 1 wants to digitize a memo handwritten on paper or the like, a closed loop pattern is drawn so as to surround the area where the memo or the like is written on the paper, and an image of the paper is written. Reading is performed using the image input unit 3. The image processing unit 4 analyzes the image data read by the image input unit 3 and, when detecting a closed loop pattern, extracts image data of an area inside the closed loop pattern. The extracted image data consists only of a memo or the like written on the paper by the user, and unnecessary areas have been deleted. The mobile terminal 1 sends the image pattern data extracted by the image processing unit 4 to the host device 6 via the network 5.

Upon receiving the image pattern data from the portable terminal 1, the host device 6 receives the image pattern data from the image pattern analysis / recognition unit 7.
To analyze the image pattern. Here, the image pattern analyzed / recognized by the image pattern analysis / recognition unit 7 is the content of the memo written by the user on paper or the like, and unnecessary images are removed. Therefore, the host device 6 can obtain only the content written by the user on the paper or the like as a recognition result.

The processing of the image processing unit 4 is performed by the host device 6.
May be executed. In this case, the mobile terminal 1 sends the image read using the image input unit 3 to the host device 6 as it is, and the image pattern analysis / recognition unit 7 extracts the image data of the area inside the closed loop pattern from the image data. Then, the extracted image pattern is analyzed.

The recognition result by the image pattern analysis / recognition unit 7 may be stored in the host device 6 or may be transferred to the portable terminal 1 automatically or as needed. In this case, the mobile terminal 1 displays the recognition result received from the host device 6 on the image display unit 2. In addition, image pattern analysis
The recognition result by the recognition unit 7 may be transferred to a predetermined terminal device specified in advance, a search process may be performed using the recognition result as a keyword, or the recognition result may be converted into a predetermined language. May be translated.

The image pattern analysis / recognition unit 7 may be realized as a function obtained by executing a predetermined program by the host device 6, or may be realized by a computer independent of the host device 6. .

As described above, in the system of this embodiment,
When it is desired to digitize a character or a figure written on a memo on a sheet of paper, a closed loop pattern is drawn so as to surround an area where the memo is written on the sheet of paper, and the closed loop pattern is photographed by a camera or the like, and the image data (or predetermined It is only necessary to transmit the image data subjected to the image processing) to the host device 6. In other words, the operation of the user is very simple because it is only necessary to surround the memo or the like in a closed loop pattern and read it with a camera or the like. In addition, the portable terminal 1 does not need a character recognition function and only has a function of reading an image and a communication function as a component requirement for recognizing the above memo or the like (depending on the configuration, a closed loop pattern Function is also required), so that weight and cost can be reduced.

FIG. 2 is an external view of the portable terminal 1. The mobile terminal 1 includes an LCD display unit 11, a camera 12, an IC card slot 13, and a wireless communication antenna 14. Further, in order to support other communication methods, a socket for Ir communication, a socket for wired communication, and the like are provided. The LCD display unit 11 has a touch panel configuration, and the mobile terminal 1 can process handwritten input information.

FIG. 3 is a configuration diagram of the portable terminal 1. CP
U21 executes a program stored in the storage device 22 (ROM and RAM). The CPU 21 and the storage device 22 are connected to each other via a bus 23.

The storage device 22 is composed of a semiconductor memory, a magnetic recording medium, or an optical recording medium, and stores programs and data. The storage device 22 may be fixedly provided in the mobile terminal 1 or may be detachably mounted.

The recording medium driver 24 is connected to the bus 23 and reads data stored in a portable recording medium (including a semiconductor memory, a magnetic disk, an optical disk, and a magneto-optical disk) 25, or reads out data stored in a portable recording medium. This is a device for writing data to the recording medium 25. As an example of the portable recording medium 25, an IC card is assumed. CPU
The program 21 can also execute a program stored in the portable recording medium 25.

The programs and data to be recorded in the storage device 22 may be received and recorded from another device connected via a communication line or the like. The programs and data stored in the provided storage device may be used via a communication line or the like.

The unit corresponding to the LCD display unit 11 is:
Liquid crystal display (LCD) 31, Liquid crystal display 3
1, an LCD driver 3 for outputting information stored in the memory 32 to the liquid crystal display 31 under the control of the LCD controller 34.
3. L controlling the memory 32 and the LCD driver 33
It comprises a CD control unit 34, a touch panel unit 35, and a touch panel control unit 36 that notifies the CPU 21 of input information detected by the touch panel unit 35.

The camera 12 is, for example, a CCD camera, and its output is converted into bitmap digital data by an A / D converter 37 and written into a memory 38. The camera control unit 39 outputs the image data stored in the memory 38 to the bus 23 according to an instruction from the CPU 21.

When transmitting data from the portable terminal 1, the communication control unit 40 generates a transmission packet according to the instruction of the CPU 21 and passes it to the wireless transceivers 41 and 42 or the wired transceiver 43. When data is received, data stored in a packet received via the wireless transceiver 41 or 42 or the wired transceiver 43 is output onto the bus 24. Wireless transceiver 41
Are connected to the wireless communication antenna 14 shown in FIG. 3 and exchange wireless data with the wireless base station 4. The wireless transceiver 42 is a transceiver for performing Ir communication, and the wired transceiver 43 is, for example, a modem. Wireless transceiver 42 and wired transceiver 4
3 is optionally attached and detached. In addition, the mobile terminal 1
Further, a clock 44 is provided.

FIG. 4 is a diagram for explaining the concept of image processing of the system according to the present embodiment, and is an example of an image when a user reads a memo handwritten on a sheet of paper with a camera. In this example, “meeting with Mr. XX / XX / K” in an empty space on the paper in which alphabets A to Z are written in advance
And writing a memo and shooting the memo with the camera 12. The user draws a closed loop on the paper surrounding a memo sentence to be character-recognized in order to distinguish a part to be character-recognized from a part not to be character-recognized. This closed loop can be drawn on the paper surface with an ordinary pen or pencil.

The user photographs at least the memo text to be recognized and the closed loop pattern surrounding the memo text. At this time, in the example shown in FIG. 4, alphabets Q to Z, which are unnecessary images, are also read as image data together with the memo.

Even in such a state, in the image processing of the present embodiment, since the image recognition processing is performed only on the area inside the closed loop pattern, the alphabets located in the area outside the closed loop pattern are Q to Z are not recognized, and only the memo “meeting with Mr. XX / XX / K” is recognized. As described above, it is possible to recognize only a desired pattern while removing unnecessary patterns with a simple user interface.

FIG. 5 is a flowchart of a process for transferring image data read by camera photographing in the portable terminal 1 to the host device 6 and recognizing the image. The program that realizes each function shown in this flowchart is
The program is stored in the storage device 22 in the form of a program code readable by the CPU 21.

FIG. 5 is a flowchart showing a process after the portable terminal 1 detects some input in an input waiting state. The following description refers to the example of FIG. That is, an example will be described in which a character or the like written by a user is read by photographing with a camera.

In step S1, it is checked whether the detected input is an input from the camera 12. If the input is from the camera 12, the processing from step S2 is executed. If the input is another input, the processing proceeds to step S21. Perform other processing corresponding to the input. As described above, when the user shoots using the camera 12 of the mobile terminal 1, the processing after step S2 is started. When there is a camera input,
Time information output from the clock 44 is held.

In step S2, the image data captured by the camera 12 is displayed on the LCD display unit 11.
That is, the image data captured by the camera 12 is written into the memory 32 of the LCD display unit 11, and the image data is displayed on the liquid crystal display 31. Subsequently, in step S3, a user instruction input screen is displayed. The user instruction input screen is a screen for sending the image read by the camera shooting to the host device 6 so as to allow the user to specify whether or not to recognize the image, and overwrites the image read by the camera shooting. That is, the image data of the user instruction input screen stored in the storage device 22 is overwritten on the memory 32 in which the image data input from the camera 12 is written by the processing of step S3. The user instruction input screen includes, for example, buttons of “recognition”, “save”, and “cancel”. Therefore, in steps S2 and S3, the LCD display unit 11
Shows a state in which the above three buttons are provided on an image within the camera shooting range surrounded by a dotted line in FIG.

When the user inputs an instruction on the user instruction input screen, that is, when the user presses one of the displayed buttons, the processing from step S4 is executed. First, in step S4, an instruction from the user is recognized. This processing detects which button the user has pressed. In step S5, it is determined whether the user's instruction is "recognition". If it is detected that the recognition button has been pressed, it is determined that the user's instruction is an image recognition service, and the process proceeds to step S6.

In step S6, the image data input by the camera 12 is temporarily stored in the RAM area of the storage device 22. In step S7, recognition target image data is extracted from the image data. That is, for example, in the example shown in FIG. 4, an area inside the closed loop pattern is regarded as a recognition target area, and image data of the area is extracted. The processing in step S7 will be described later in detail. Subsequently, in step S8, a packet for transferring the image data extracted in step S7 to the host device 6 is created. The processing in step S8 will also be described later. Then, in step S9, the packet created in step S8 is transmitted to the network 5 shown in FIG.

On the other hand, if it is determined in step S5 that the user's instruction is not "recognition", it is determined in step S10 whether the user has pressed the save button. If the user presses the save button, the portable terminal 1 saves the image data in step S11. On the other hand, if the user has not pressed the save button, it is assumed that the user has pressed the cancel button, and the image data is discarded in step S12.

As described above, when the portable terminal 1 reads an image in which the recognition target area is represented using the closed loop pattern, it extracts only the area inside the closed loop pattern as the recognition target area, and extracts the extracted area. The image data of the recognition target area is sent to the host device 6. Note that, as described above, the extracted image data includes only an area (an area where a memo is written) that the user actually wants to recognize the image, and unnecessary images are removed.

FIG. 6 is a detailed flowchart of the process in step S7 shown in FIG. 5, and shows a process for extracting image data to be recognized. In step S31, image data read by shooting with the camera 12 is read from the RAM area of the storage device 22, the image data is subjected to preprocessing for removing noise of the image data, and then the line detection process is performed. Execute Line detection processing is an existing technology. For example, when examining the density distribution or color distribution of image data, a “line” can be detected as a series of points having a large change rate. The rate of change of the density distribution or the color distribution also increases at the boundary between a certain area and another area, but the existing technology is used to discriminate "lines" and "boundaries between areas". . As described above, by examining the density distribution or the color distribution in the entire region of the preprocessed image data, "lines" included in the image are detected.

In step S32, a line connection process for connecting the breaks of the "line" detected in step S31 is executed. For example, in the example shown in FIG. 4, although the closed loop pattern generally surrounds the handwritten memo, a part thereof is broken, so-called "line break" occurs. The wire connection process is a process for repairing such a "cut wire",
This is also an existing technology. For example, when a line segment of a predetermined length is extended from the end of the line where the "line break" has occurred in the direction in which the line extends, and the line segment is turned within a predetermined angle range around the end of the line. Then, a process is performed to connect these two lines if they come into contact with another line. By this line connection processing, a closed loop pattern (substantially closed loop pattern) as shown in FIG. 4 which is partially interrupted is repaired to become a closed loop pattern.

In step S33, among the closed loop patterns included in the image data, those whose area of the area surrounded by the closed loop pattern is larger than a predetermined value are detected. That is, for example, the letter “Q” in FIG. 4 may be regarded as a closed loop pattern, and therefore, by executing step S33, the closed loop pattern other than the closed loop pattern for specifying such a recognition target area may be used. Is excluded. Note that the number of closed loop patterns detected in this process does not need to be one, but may be plural. Then, in step S34, image data of an area surrounded by the closed loop pattern detected in step S33 is output.

FIG. 7 is a diagram showing an application example of the image processing. As described above, the boundary between regions can be detected by examining the density distribution or color distribution of the image data. However, the example shown in FIG. 7 uses this technique. That is, memos and the like are written on paper, but nothing is written on the paper except for the memo to be image-recognized. When the paper on which the memo is written is placed on a desk or the like of a different color from the paper and the camera 12 is used to photograph the paper,
The entire surface of the paper is shown, and the top of the desk is shown so as to surround the paper.

If step S7 in FIG. 5 is executed on the image data thus captured, the color of the paper surface is different from the color of the upper surface of the desk, so that the outer peripheral edge of the paper surface is detected as a closed loop pattern, Are extracted as recognition target image data. Thus, by changing the photographing method, a necessary area can be extracted without drawing a closed loop pattern surrounding the memo on the paper surface with a pen or a pencil.

By the way, in the example shown in FIG. 4 or FIG.
Although the area for image recognition is identified from unnecessary areas by surrounding the handwritten memo with a closed loop pattern, other methods may be used. For example, a rule may be provided in which a character string to be recognized is underlined, and only a character string with an underline is extracted from image data captured using the camera 12. In the example shown in FIG. 8A, only “meeting with Mr. XX / XX / K” is underlined. In this case, the alphabet Q
~ Z is not extracted and "meeting with Mr. K on XX / XX / day"
Only those are extracted.

FIG. 9A is a flowchart for explaining a recognition target image data extraction process in a case where a character string to be recognized is identified using an underline. This process is executed as step S7 in the flowchart of FIG.

Steps S41 and S42 are the same as steps S31 and S32 in FIG.
Is detected, and the "line" is repaired. In step S43, from among the "lines" obtained by the above processing, the "lines" extending longer than a predetermined length in the horizontal
Is detected. The number of "lines" detected here may be plural, and two lines are detected in the example shown in FIG. In step S44, each "line" detected in step S43
A region above each of these “lines” is cut out along the line with a predetermined width, and each cut-out region is output as a recognition target region.

Color may be used as a method for identifying an image recognition target area. For example, a rule that “the character string to be recognized is written in red” is provided, and the camera 1
2 may be used to extract only a character string written in red from image data captured using 2. FIG.
In the example shown in (b), only “meeting with Mr. XX / XX / K” is written in red. In this case, the alphabet Q
~ Z is not extracted and "meeting with Mr. K on XX / XX / day"
Only those are extracted.

FIG. 9 (b) is a flowchart for explaining the recognition target image data extraction processing in the case where a character string to be recognized is written in a color designated in advance and identified. This processing is also shown in FIG.
Is executed as step S7. In step S51, an area including pixels of a color designated in advance and having a predetermined density or more is cut out from the image data read by photographing with the camera 12, and the image data in the cut out area is output as a recognition target area.

FIG. 10A is a diagram showing the structure of a packet transmitted from the portable terminal 1. This packet is shown in FIG.
Is created in step S8 shown in FIG. Each packet is composed of a header part and a data part. The header section stores a source address, a destination address, and the like. What address system is stored as the source address and the destination address depends on the network configuration to which the present embodiment is applied. For example, in TCP / IP communication, an IP address is stored.

The data section stores application identification information, terminal ID, image data, time information, and the like. The application identification information is information for identifying an application program to be started in the terminal (the host device 6 in this case) of the destination. That is, it is a command for activating a predetermined program in the destination terminal. In the present embodiment, information for identifying the image data analysis / recognition program is set. In addition,
The application identification information may be specified as a port number in TCP / IP communication.

The terminal ID is an identification number of the portable terminal 1. The image data is difference image data obtained by the processing in step S8 in FIG. 5, and is compressed when stored in a packet. The time information is information indicating the time when the image data transmitted to the host device 6 was photographed by the camera.

FIG. 10B is a detailed flowchart of the packet creation process in step S8 shown in FIG. In step S61, the image data to be transferred to the host device 6 is compressed and stored in the data section. The compression method is, for example, JP
EG. In step S62, time information indicating the time at which the image data was captured by the camera is set. In step S63, information for identifying the image data analysis / recognition program is set as application identification information.
In step S64, a terminal ID is set as information for identifying the mobile terminal 1 (information for identifying the own device). further,
In step S65, a header section is created. In the header part, at least the address of the portable terminal 1 (the address of the own device) is set as the source address, and the address of the host device 6 is set as the destination address.

The packet created as described above is
It is sent to the network 5. The network 5 transfers the packet to the host device 6 according to the destination address of the packet. Hereinafter, the host device 6 that receives and processes this packet will be described.

FIG. 11 is a configuration diagram of the host device 6.
The storage device 51 is configured by a semiconductor memory, a magnetic recording medium, or an optical recording medium, and stores programs, data, and the like. The storage device 51 may be fixedly provided in the host device 6 or may be detachably mounted.

The recording medium driver 52 reads data stored in a portable recording medium (including a semiconductor memory, a magnetic disk, an optical disk, a magneto-optical disk, etc.) 53 or writes data into the portable recording medium 53. Device. The communication control unit 54 is a unit that controls transmission and reception of data with the network. Transmission and reception of packets with the mobile terminal 1 are also controlled here.

The CPU 55 loads a program or the like from the storage device 51 or the portable recording medium 53 into the memory 56 and executes it. Note that the program and data to be recorded in the storage device 51 may be those in which what has been stored in the portable recording medium 53 may be written, or may be transmitted from another device on the network via a communication line or the like. It may be configured to receive and record. Further, the CPU 55 may use programs, data, and the like stored in another storage device provided on the network via a communication line or the like.

FIG. 12 is a flowchart for explaining the processing of the host device 6. Here, the operation centered on the host device 6 receiving the packet (the packet created by the processing of the flowchart of FIG. 5) transmitted from the mobile terminal 1 and processing the image data according to the information set in the data part thereof is mainly described. Next, an operation when the host device 6 receives a packet from the network 5 will be described.

When the packet is received in step S71,
The terminal I set in the packet at step S72
By examining D, the terminal that transmitted the packet is recognized. In step S73, the application specified by the application identification information set in the received packet is started. Step S74
Then, it is determined whether the started application includes an image recognition process. If the application does not include the image recognition processing, the processing of the application is executed in step S80.

On the other hand, if the application includes an image recognition process, in step S75, the image data stored in the received packet is decompressed, and the process proceeds to step S7.
6, the image data is subjected to pattern recognition processing.
Perform character recognition processing. The image data on which the recognition process is performed at this time is the image data extracted in step S7 of FIG. That is, in the example of FIG. 4 or FIG.
Image recognition processing is performed on the image data of the area including only “meeting with Mr. XX / XX / K”. Therefore, as a result of this image recognition processing, only the contents that the user has written down are recognized.

In step S77, if the recognition result in step S76 includes a character, the character is converted into a character code (for example, an ASCII code or a JIS code). In step S78, a packet for returning the recognition result to the mobile terminal 1 is created. That is, the transmission destination address of this packet is the portable terminal 1, and the recognition result is stored in the data portion thereof. Then, in step S79, the created packet is transmitted to the network 5.

The packet sent to the network 5 in step S79 is transferred to the portable terminal 1. Upon receiving this packet, the portable terminal 1 displays the result of the image recognition processing by the host device 6 on the LCD display unit 11. That is, in the case of the above example, the image of FIG. 4C is displayed. The recognition result is stored in the mobile terminal 1 according to a user's instruction.

As described above, the image recognition result may be stored in the host device 6. In this case, the recognition result is stored in association with the terminal ID of the terminal device (that is, the mobile terminal 1) that has transmitted the difference image data. The recognition result may be transferred to a predetermined terminal device specified in advance. Further, a search process may be executed using a character string included in the recognition result as a keyword, or the character string may be translated into a predetermined language. In this case, the search result or the translation result is sent to the mobile terminal 1.

By the way, there is a strong demand for reducing the size, weight, and cost of a portable terminal. One way to respond to these requests is to move part of the processing being executed on the mobile terminal side to the host device side in the usage method as in the present embodiment. For example, the processing of step S7 in the flowchart of FIG. Step S7 is, as described above,
Including image processing, in general, if such processing is to be realized with high accuracy or in a short time, a processor having a high processing capability and a large-scale program are required. Therefore, causing the host device 6 to execute the process of step S7 is effective in reducing the size and weight of the mobile terminal 1 or reducing the cost.

When causing the host device 6 to execute the process of step S7, the portable terminal 1 sends the image data read using the camera 12 to the host device 6 as it is. In the flowchart of FIG. 12, the host device 6 executes step S7 after the processing of step S75.

In the above embodiment, a processing program executed by the host device 6, that is, a program for realizing each function shown in the flowchart of FIG. 12, and a program for interpreting and processing information transferred via the network And the like are stored in the storage device 51 or the portable recording medium 53 in the form of a program code readable by the CPU 55. Alternatively, one stored in another device connected via a network is used.

In the above embodiment, the configuration in which image data is captured using a camera has been described, but the present invention is not limited to this configuration. For example, image data read using an image scanner is transferred to a host device. You may make it.

As described above, the image processing system according to the present embodiment uses a portable terminal to photograph an image, such as a page on which a memo has been written, with a camera or the like, and allows the host device to perform processing only on a necessary area in the image. And performs a recognition process.
For this reason, the mobile terminal only has a function of reading an image and transferring the image to the host device and a function of requesting the host device to perform processing (depending on the configuration, a function of extracting a necessary area). The contents of the memo can be digitized with a simple user interface that takes a picture of the memo.

[0070]

According to the system of the present invention, a recording surface including a desired handwritten memo or the like is captured as image data by a camera or the like provided in a portable terminal, and the handwritten memo or the like can be digitized simply by requesting recognition from a host device. . Thus, the present invention provides a user interface with improved operability. At this time, only the area inside the closed loop pattern, the area containing the underlined character string, or the area containing the pattern drawn in a specific color is extracted and the image is recognized, so the necessary Only information is accurately recognized.

Since the portable terminal does not execute image data processing, particularly pattern recognition processing and character recognition, which require high-speed processing, the portable terminal can provide advanced image data without providing a large-capacity memory or a high-performance processor. Processing results can be obtained.

[Brief description of the drawings]

FIG. 1 is a system configuration diagram of an embodiment.

FIG. 2 is an external view of a portable terminal.

FIG. 3 is a configuration diagram of a mobile terminal.

FIG. 4 is a diagram for explaining the concept of image processing of the system according to the embodiment.

FIG. 5 is a flowchart of a process in which image data captured by camera shooting in a portable terminal is transferred to a host device to perform image recognition.

FIG. 6 is a flowchart of a process of extracting recognition target image data.

FIG. 7 is a diagram illustrating an application example of the image processing illustrated in FIG. 4;

FIG. 8 is a diagram illustrating an entry example for identifying a recognition target image.

9 is a flowchart illustrating a process for extracting recognition target image data when a memo or the like is entered by the method shown in FIG. 8;

FIG. 10A is a diagram illustrating a structure of a packet transmitted from a mobile terminal, and FIG. 10B is a detailed flowchart of a packet creation process.

FIG. 11 is a configuration diagram of a host device.

FIG. 12 is a flowchart illustrating processing of a host device.

[Explanation of symbols]

 Reference Signs List 1 mobile terminal 2 image display unit 3 image input unit 4 image processing unit 5 network 6 host device 7 image pattern analysis / recognition unit 12 camera 21 CPU 22 storage device 24 recording medium driver 25 portable recording medium 31 liquid crystal display 40 communication control unit Reference Signs List 51 storage device 52 recording medium driver 53 portable recording medium 54 communication control unit 55 CPU 56 memory

Claims (4)

[Claims] 1. A reading means for reading an image, a detecting means for detecting a closed loop pattern or a substantially closed loop pattern from an image read by the reading means, and an inner part of the closed loop pattern or the substantially closed loop pattern detected by the detecting means. Extracting means for extracting image data of the area; and recognizing means for recognizing a character or a specific pattern included in the image data of the area extracted by the extracting means. An image processing system, wherein the recognition means is provided in the second terminal device, and the detection means and the extraction means are provided in the first or second terminal device. 2. A reading means for reading an image, a detecting means for detecting a predetermined identification instruction pattern from the image read by the reading means, and a predetermined identification instruction pattern detected by the detection means. Extracting means for extracting image data of an area at a position, and recognizing means for recognizing a character or a specific pattern included in the image data of the area extracted by the extracting means; An image processing system provided in the terminal device of (1), wherein the recognizing means is provided in the second terminal device, and the detecting means and the extracting means are provided in the first or second terminal device. Reading means for reading the image; extracting means for extracting image data of an image drawn in a predetermined color from the image read by the reading means; image data extracted by the extracting means Recognizing means for recognizing a character or a specific pattern included in the above, wherein the reading means is provided in a first terminal device, the recognizing means is provided in a second terminal device, and the detecting means and the extracting means An image processing system provided in the first or second terminal device. 4. A method for using a camera to capture a recording surface including a handwritten memo or the like as image data and recognizing the image data in another device, wherein the area including the handwritten memo or the like is determined according to a predetermined rule. Designating as a recognition target area; reading a recording surface including the handwritten memo or the like using a camera; extracting the recognition target area in accordance with the predetermined rule; Performing a recognition process on the image data of the region.